{"id":"8e998cac-100e-40f9-a2c0-ad3fbbabeb1a","arxiv_id":"2505.17667","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"QwenLong-L1-32B, trained with warm-up SFT plus curriculum-based long-context RL, reaches 70.7 average accuracy on seven DocQA benchmarks, though the evaluation gives it longer context than its proprietary baselines.","lead":"This paper trains a 32B model to reason over very long documents by first fine-tuning it on short examples, then using progressive reinforcement learning stages that gradually increase context length. The authors report that the resulting model, QwenLong-L1-32B, matches or beats several larger and proprietary reasoning models on seven document question-answering benchmarks, but the comparison is weakened by an uneven evaluation setup.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Training data includes 200 Musique and 600 DocMath problems whose disjointness from the Table 4 eval sets is not demonstrated; removing Musique alone flips the claimed edge over Claude-3.7 and o3-mini.","rationale":"The reader's rejection is correct, but the most load-bearing defect is not the 50K context cap. The context cap is explicitly acknowledged in Section 3.3 and is a real confound, yet most average document lengths in Table 3 are 5K-30K, so the cap mostly bites on long tails; it weakens but does not by itself explain a 70.7 average. The benchmark-overlap issue is sharper. Section 3.1 states that 600 DocMath training problems are drawn from the same valid split later used for DocMath evaluation and that 200 Musique examples are sampled from Musique while Musique is also a Table 4 benchmark. A single-column removal calculation shows that the headline comparison flips when Musique is excluded. The paper provides no ID-level disjointness check, no train/test contamination analysis, and no error bars, so the 0.1-0.4 point margins over o3-mini and Claude are within the range that one 200-example benchmark can decide. I credit the released models and data and the internal ablations showing phased RL and retrospective sampling help, so the engineering recipe may well be sound; the problem is the evidence for the SOTA claim. The correct disposition remains rejection pending a clean evaluation, consistent with the reader's verdict.","tokens_in":30848,"tokens_out":7608,"duration_ms":56054,"concrete_test":"Download the released DOCQA-RL-1.6K and the exact Table 4 evaluation instances (LongBench Musique and DocMath eval splits). Normalize whitespace and tokenization, then compare question IDs and context hashes between train and eval for these two benchmarks, and also check the remaining five benchmarks against both the RL and SFT training sets. If any near-duplicate question-context pair exists, recompute Table 4 averages excluding that benchmark; if the recomputed QwenLong-32B average no longer exceeds o3-mini and Claude, the headline result depends on leaked training data. If the sets are clean, the concern does not land and the context-cap comparison should then be tested separately.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.1 builds DOCQA-RL-1.6K from 600 DocMath problems and 200 Musique examples, then Section 3.3 evaluates seven DocQA benchmarks including DocMath and Musique. No statement anywhere demonstrates that the 200 Musique RL examples are disjoint from the 200 Musique evaluation examples (taken from LongBench), and DocMath RL/eval are explicitly carved from the same valid split (75%/25%, Section 3.1). The central claim in Section 4.1 is the 70.7 average on Table 4; that average is fragile to split hygiene. Removing the Musique column alone changes QwenLong-L1-32B-DAPO to 70.93, Claude-3.7-Sonnet-Thinking to 71.13, OpenAI-o3-mini to 71.00, and Qwen3-235B-A22B to 71.83, so the headline 'exceeds o3-mini and matches Claude' disappears. This is a load-bearing unverified assumption, not a stylistic issue: one potentially leaked 200-example benchmark controls the central comparison. The asymmetric 50K context cap in Section 3.3 is also concerning, but less decisive because most benchmark documents average well under 50K; the leakage risk is the single check that can flip the headline.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes QwenLong-L1, a reinforcement learning framework for adapting short-context reasoning models to long-context document question answering. The framework combines warm-up SFT, curriculum-guided phased RL with progressive context scaling, difficulty-aware retrospective sampling, and a hybrid rule/LLM-judge reward. Experiments on seven DocQA benchmarks report that the 32B model achieves an average score of 70.7, outperforming OpenAI-o3-mini and Qwen3-235B-A22B and matching Claude-3.7-Sonnet-Thinking. The paper also presents ablations and analyses of reasoning behavior.","tokens_in":31132,"tokens_out":7503,"duration_ms":64685,"significance":"If the reported results are reliable, the paper would provide a useful recipe for long-context reasoning RL, with a clear formalization of the problem and an open release of code and model weights. The ablations separating the contributions of SFT, phased RL, and retrospective sampling are informative, and the analysis of emergent reasoning behaviors is a nice addition. However, the headline comparative claims rest on an evaluation protocol that is currently not sufficiently rigorous: potential training/evaluation overlap in the Musique benchmark, an asymmetric input-length cap for proprietary baselines, a metric that directly matches the training reward, and no uncertainty estimates. The significance of the contribution hinges on resolving these confounds.","major_comments":[{"comment":"The RL training data includes 200 examples sampled from Musique, while the evaluation set is 200 Musique examples taken from LongBench; the paper does not demonstrate that these two sets are disjoint, or provide any safeguard against overlap. Removing the Musique column from Table 4 changes the average of QwenLong-L1-32B-DAPO from 70.7 to 70.9, while Claude-3.7-Sonnet-Thinking becomes 71.1, OpenAI-o3-mini 71.0, and Qwen3-235B-A22B 71.8. The headline claim that QwenLong-L1 exceeds o3-mini and Qwen3-235B and matches Claude disappears. The authors must either state and empirically verify the disjointness of the 200 RL examples from the 200 LongBench evaluation examples, or re-run the evaluation on a clearly disjoint held-out set.","section":"Section 3.1 and Section 3.3 (Table 4)"},{"comment":"All proprietary baselines are capped at a maximum input length of 50K tokens, while QwenLong-L1 is evaluated with up to 120K tokens. Since several documents in the benchmarks exceed 50K (DocMath max 176,285; Frames max 117,131; NarrativeQA max 65,357), the baselines are evaluated on truncated inputs, whereas QwenLong-L1 can use the full document. This confounds model quality with input access. Please re-evaluate baselines with their full supported context lengths (or at least the same maximum as QwenLong-L1) and report how the results change.","section":"Section 3.3 (Configurations)"},{"comment":"The reported metric is max(exact match, LLM-judged accuracy) and is explicitly stated to be 'aligned with the reward function' in Section 2.4. Because the model is trained with the same max(rule, LLM-judge) reward, the evaluation metric is not independent of the training objective. The training judge (Qwen2.5-1.5B) and evaluation judge (DeepSeek-V3) differ, but the max operation counts a correct answer under either criterion, so the model is directly optimized for the reported metric. The paper should also report standard task metrics (e.g., EM and F1 separately) or an evaluation metric not used during training, to permit comparison with published baselines.","section":"Section 3.3 and Section 2.4"},{"comment":"All results are point estimates without error bars, confidence intervals, or significance tests. With 200 examples per benchmark, the standard error of a 70% accuracy is about 3.2 percentage points; the differences between QwenLong-L1-32B-DAPO (70.7) and Claude-3.7-Sonnet-Thinking (70.7), OpenAI-o3-mini (70.4), and Qwen3-235B-A22B (70.6) are within one standard error. Please report uncertainty estimates, e.g., via multiple evaluation runs with different sampling temperatures, or bootstrap confidence intervals, and state which differences are statistically meaningful.","section":"Table 4"}],"minor_comments":[{"comment":"The prompt template contains the typo 'quesiton' instead of 'question' in the introductory line.","section":"Table 2"},{"comment":"The word 'mathcing' should be 'matching'.","section":"Section 4.1"},{"comment":"The PDF appears to contain corrupted character sequences (e.g., '/uni00000027/uni00000052/...') in the author list, after Figure 1, in Section 4.2, and elsewhere; the source file should be recompiled so that these artifacts do not appear in the published version.","section":"Throughout the manuscript"},{"comment":"Please clarify whether the 200 MultiHopRAG and 200 Musique examples are drawn from the training splits or test splits of those datasets, and report the sampling method so readers can assess independence from the evaluation sets.","section":"Section 3.1"},{"comment":"The paper does not report the total number of RL steps or the compute budget; including this information would help readers assess the practical cost of the method.","section":"Section 3.2 and Table 4"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about Musique leakage is well-founded and is the single most important check: removing the Musique column flips the headline comparison. If the authors cannot prove disjointness of the 200 RL training examples from the 200 LongBench evaluation examples, the central claim of outperforming o3-mini and matching Claude-3.7-Sonnet-Thinking is unsupported. The asymmetric context cap and the reward-aligned metric compound the issue, so I recommend that the editor require concrete evidence of disjointness and a re-evaluation with symmetric context lengths before any acceptance decision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is a serious engineering contribution, but the headline result is not trustworthy as reported. The one thing to know: the claim that QwenLong-L1-32B beats o3-mini and matches Claude-3.7 hinges on a possible overlap between the 200 Musique examples used for RL training and the 200 Musique questions used for evaluation. The paper never says the two sets are disjoint, and removing that one column from Table 4 flips the ordering. That's a load-bearing check that the authors need to address.\n\nWhat's genuinely useful here is the recipe. Progressive context scaling with curriculum phases, warm-up SFT, difficulty-aware retrospective sampling, and a hybrid rule+judge reward is a sensible combination, and the ablations (Figure 5) suggest each piece pulls its weight. The models are released, and the training dynamics analysis (entropy collapse, KL spikes, behavior counts) is informative. The claim to be \"the first RL framework designed for long-context LRMs\" is overstated—each component has prior art, and the novelty is in the combination, not the ingredients.\n\nThe soft spots are real but not all equal. The Musique overlap question is the critical one. DocMath at least has an explicit 75/25 split of the valid set, so that part looks disjoint. Musique is silent. The metric is also explicitly the max of EM and LLM-judged accuracy, the same objective the reward function optimizes; that's not inherently unfair, but it means the eval score is the training objective, and any judge leniency is baked in. The asymmetric context cap (50K for proprietary baselines vs 120K for this model) is a minor confound because most benchmark documents average well below 50K, but the DocMath max length of 176K means some individual examples do get truncated. No error bars anywhere, so the 0.3-point differences are noise-level.\n\nWho should read it: anyone building RL training pipelines for long-context reasoning. The recipe and ablations are worth stealing even if the headline numbers don't hold. It deserves a serious referee: the framework is sensible, the artifacts are real, and the evaluation concerns are fixable with clear disjointness statements and re-analyzed tables. I'd send it to review, but ask for a detailed data hygiene pass before accepting.\n\nFor me: I'd bring it to a reading group to discuss the evaluation pitfall, but I wouldn't cite the main result until the overlap is resolved.","headline":"A credible engineering recipe for long-context RL, but the headline comparison is built on a train/eval overlap that no one has ruled out.","tokens_in":31730,"tokens_out":4532,"would_cite":true,"duration_ms":30945,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Reinforcement learning with progressive context scaling turns a short-context 32B model into a long-context reasoner that beats several larger proprietary models on document QA.","keywords":["long-context reasoning","reinforcement learning","large reasoning models","progressive context scaling","document question answering","curriculum RL","retrospective sampling","QwenLong-L1"],"falsifier":"Run OpenAI-o3-mini, Qwen3-235B-A22B, and Claude-3.7-Sonnet-Thinking on the same seven benchmarks with the same 120K input allowance QwenLong-L1-32B receives; if o3-mini or Qwen3-235B-A22B then surpasses 70.7, the paper's superiority claim rests on the asymmetric context cap rather than model quality.","tokens_in":30620,"feed_emoji":"🧠","tokens_out":13987,"duration_ms":93306,"temperature":0.7,"pith_summary":"Long-context reasoning models are hard to train with reinforcement learning because long inputs shrink output entropy and inflate KL-divergence variance, which makes optimization slow and unstable. This paper argues that both problems are manageable if the context window is scaled gradually rather than opened all at once. The proposed QwenLong-L1 recipe starts from a short-context reasoning model, adds a warm-up supervised fine-tuning stage at 20K tokens, then runs group-relative RL in two length phases, 20K and 60K, while re-sampling the hardest examples from the first phase into the second. On seven long-document question-answering benchmarks, the 32B version reaches an average of 70.7, which the paper reports as above OpenAI-o3-mini and Qwen3-235B-A22B and on par with Claude-3.7-Sonnet-Thinking. If the recipe holds up, it gives open 32B-class models a practical route to deep-research-style tasks that today seem to require much larger or proprietary systems.","feed_headline":"32B model trained with long-context RL beats o3-mini and Qwen3-235B","feed_subtitle":"A 5.1-point gain over its starting model lets a 32B system surpass several proprietary reasoners.","key_machinery":"QwenLong-L1 is a progressive context scaling framework for long-context reasoning RL. Its load-bearing mechanism is the curriculum over input length: a warm-up SFT stage at 20K tokens initializes the policy, then group-relative RL runs in two phases with context caps of 20K and 60K, and a difficulty-aware retrospective sampler re-injects the hardest zero-reward examples from the earlier phase into the later one. The reward is a hybrid of rule-based exact-match verification and an LLM-as-a-judge semantic-equivalence check, combined by taking the maximum of the two signals. Because long contexts make a learned value network computationally prohibitive, the framework uses group-relative advantages from GRPO or DAPO instead, with DAPO adding decoupled clip thresholds and dynamic sampling that the paper finds most effective.","core_discovery":"On the paper's own terms, the central discovery is that long-context reasoning can be taught to a short-context reasoning model with RL, provided the context length is increased through a curriculum. Starting from R1-Distill-Qwen-14B and R1-Distill-Qwen-32B, QwenLong-L1 gains about 4 and 5 average points over its base checkpoints; the 32B model reaches 70.7, exceeding OpenAI-o3-mini (70.4), Qwen3-235B-A22B (70.6), QwQ-Plus (70.0), and Qwen3-Plus (70.3), and matching Claude-3.7-Sonnet-Thinking (70.7). The paper also reports that warm-up SFT alone yields only small gains, that phased RL keeps entropy higher and KL divergence more stable than single-stage RL, and that RL training progressively increases grounding, subgoal-setting, backtracking, and verification behaviors in the generated reasoning traces.","pith_inferences":["The paper's evaluation gives its own model up to 120K input tokens while capping proprietary baselines at 50K; a natural follow-up is to equalize context length, which would clarify how much of the reported margin is model capability and how much is the asymmetric context cap.","The finding that SFT increases reasoning-behavior counts without improving scores suggests those behaviors are correlates of successful training rather than its cause; distilling RL-generated successful traces back into an SFT model would test whether imitation can recover any of the RL gain.","The difficulty signal in retrospective sampling is computed once from base-model rewards, so as the policy improves the same examples become easier; re-scoring difficulty periodically during later phases is a direct, testable refinement the paper does not explore.","The hybrid reward's max operation is not ablated, so a purely rule-based reward on open-ended DocQA answers is a testable prediction implied by the paper's motivation for combining the two signals."],"forward_implications":["Open-source 32B-class models can reach the long-document reasoning level of much larger proprietary systems, lowering the compute and API cost barriers for deep-research-style applications.","Long-context reasoning RL should be scheduled as a curriculum over input length rather than run in a single stage, because single-stage training shows entropy collapse and KL spikes in the paper's training-dynamics analysis.","Warm-up SFT is best treated as an initialization step, not the final training stage, since the ablations show SFT-only models trail RL models by several average points despite generating similar reasoning behaviors.","Difficulty-aware retrospective sampling is a cheap mechanism for maintaining exploration, because hard examples from earlier phases keep reward low and policy entropy high during later training.","Test-time scaling pays off on this policy: with 16 samples, the 14B model's Pass@2 exceeds both DeepSeek-R1 and OpenAI-o1-preview, indicating the RL-trained policy retains useful exploration diversity."],"supporting_citations":[{"why":"Supplies the base checkpoints R1-Distill-Qwen-14B and R1-Distill-Qwen-32B, the SFT teacher DeepSeek-R1, and the open-source baseline DeepSeek-R1.","marker":"[11]"},{"why":"Provides the GRPO group-relative advantage estimator used as one of the two RL algorithms.","marker":"[34]"},{"why":"Provides the DAPO algorithm, including decoupled clip thresholds, dynamic sampling, and token-level loss, used in the best-performing configuration.","marker":"[54]"},{"why":"Supplies five of the seven evaluation test sets via LongBench (2WikiMultihopQA, HotpotQA, Musique, NarrativeQA, Qasper).","marker":"[2]"},{"why":"Provides the DocMath training and evaluation items for mathematical reasoning over long documents.","marker":"[57]"},{"why":"Provides the MultiHopRAG examples used to build the multi-hop portion of the DOCQA-RL-1.6K training set.","marker":"[36]"},{"why":"Provides the Musique examples used alongside MultiHopRAG for multi-hop training.","marker":"[44]"},{"why":"Serves as the DeepSeek-V3 judge model that decides final answer equivalence in evaluation.","marker":"[21]"}],"fun_headline_variants":["Long-context RL curriculum: 32B matches Claude-3.7-Sonnet","Curriculum RL lifts 32B to 70.7 on long-context benchmarks","32B with curriculum RL beats Qwen3-235B and o3-mini","Progressive RL scaling for long-context reasoning in 32B"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that capping all proprietary baselines at 50K input tokens while letting QwenLong-L1 read up to 120K is a fair comparison; if that cap hands the model extra evidence the baselines cannot see, the reported wins are not wins.","fun_headline_variants_meta":{"raw":{"variants":["Long-context RL curriculum: 32B matches Claude-3.7-Sonnet","Curriculum RL lifts 32B to 70.7 on long-context benchmarks","32B with curriculum RL beats Qwen3-235B and o3-mini","Progressive RL scaling for long-context reasoning in 32B"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001223,"raw_usage":{"total_tokens":5059,"prompt_tokens":1003,"completion_tokens":4056,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":619,"completion_tokens_details":{"reasoning_tokens":3969}},"tokens_in":619,"tokens_out":4056,"duration_ms":34351,"temperature":1.0,"reasoning_tokens":3969,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:42:05.957089+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run OpenAI-o3-mini, Qwen3-235B-A22B, and Claude-3.7-Sonnet-Thinking on the same seven benchmarks with the same 120K input allowance QwenLong-L1-32B receives; if o3-mini or Qwen3-235B-A22B then surpasses 70.7, the paper's superiority claim rests on the asymmetric context cap rather than model quality.","supporting_citations":[{"cited_title":"Docmath-eval: Evaluating math reasoning capabilities of llms in understanding long and specialized documents","cited_arxiv_id":null,"evidence_quote":"Provides the DocMath training and evaluation items for mathematical reasoning over long documents."},{"cited_title":"Multihop-rag: Benchmarking retrieval-augmented generation for multi-hop queries","cited_arxiv_id":null,"evidence_quote":"Provides the MultiHopRAG examples used to build the multi-hop portion of the DOCQA-RL-1.6K training set."}],"review_version":1}