LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget

LongStraw addresses the growing gap between inference context lengths and RL post-training, where inference systems handle million-token contexts but post-training often stays at 256K tokens or below. The method evaluates shared prompts without autograd, retains only model-specific state for later tokens, and replays short response branches one at a time to reduce the live training graph under a fixed GPU budget. On eight H20 GPUs, LongStraw completes grouped Qwen scoring and response backward at 2.1M positions for groups of 2 and 8, with a stress test reaching 4.46M positions, while on 32 H20 GPUs it validates the execution path for a 2.1M-token prompt across all 78 layers of GLM-5.2. This matters because it enables million-token RL post-training for AI agents with long trajectories under fixed GPU budgets, though the abstract does not disclose complete training correctness.