{"id":"6c6a0d9b-c2b2-45a5-8527-ae150d838441","arxiv_id":"2507.04391","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Long chain-of-thought and RL training on math problems improves general reasoning benchmarks, while short chain-of-thought math fine-tuning often degrades performance.","lead":"This paper tests whether training large language models on mathematical problem-solving improves broader reasoning abilities. It finds that long chain-of-thought and rule-based reinforcement learning approaches generalize, while traditional short-response fine-tuning often impairs performance.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The s1.1-short ablation does not isolate response length as the causal variable; GPT-4o rewriting changes content, self-reflection, and format, so the claim that long CoT itself drives generalization is not firmly established.","rationale":"I agree with the reader that the most load-bearing assumption is the isolation of response length in the s1.1-short ablation. The paper otherwise provides a useful empirical mapping: LIMO and s1.1 improve on several general reasoning benchmarks, the short-CoT SFT models on Mistral degrade, and the RL models show gains. These are real observations, and the paper honestly notes the base-model differences. However, the central causal language in the abstract—'by extending the model's reasoning processes'—requires length to be the active ingredient. The GPT-4o rewriting confounding is real, and a matched comparison on the same base model and same data provenance would settle it. I do not think this invalidates the paper; it makes the conclusion conditional on future work, which matches the reader's CONDITIONAL verdict.","tokens_in":15549,"tokens_out":5744,"duration_ms":57842,"concrete_test":"Train a matched short-CoT counterpart of LIMO's 817 MPS examples using concise but expert-written solutions from the source datasets (or the original short-reference versions) rather than GPT-4o rewrites, keeping the base model Qwen2.5-32B-Instruct and training recipe identical. Additionally, decode the long-CoT model with a max-token cap of ~450 tokens on LogiQA/GPQA to see whether performance collapses to base level. If a matched short model matches LIMO, or the capped long model retains gains, response length is not the causal driver; if the capped model drops and the matched short model stays low, length is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—that MPS training with long, self-reflective CoT generalizes better than short CoT—rests on the s1.1 vs s1.1-short ablation in §3.3. In that ablation, GPT-4o rewrites the original long answers into short answers with average length ~419 tokens, and s1.1-short underperforms the base model. However, the rewriting simultaneously changes content quality, removes self-correction/verification steps, and alters formatting/prompt structure; any of these, not length per se, could cause the drop. The ablation therefore does not isolate reasoning length. The claim is also cross-checked against short-CoT SFT models (Math-CoT, Math-PoT) that use a different base model (Mistral-7B) than the long-CoT models (Qwen2.5-32B), introducing a model-size/alignment confound into the headline comparison. Even the RL results in Table 3 lack a same-base SFT-only control column, making the 'RL generalizes' leg of the claim harder to attribute to RL rather than to the SFT warmup. These gaps do not refute the directional pattern, but they prevent a causal interpretation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper reports an empirical study of whether training LLMs on mathematical problem-solving (MPS) transfers to general reasoning tasks. The authors compare five strategies: continual pretraining on math text (DeepSeekMath, RhO-Math), SFT on STEM QA (MAmmoTH2), SFT on short-CoT math data (Math-CoT, Math-PoT), SFT on long self-reflective CoT (LIMO, s1.1), and rule-based RL on math queries (SimpleRL family). They evaluate on five MPS and eight general-reasoning benchmarks. They report that continual pretraining generalizes modestly, short-CoT SFT generalizes poorly and often hurts, while long-CoT SFT and rule-based RL on math queries improve general reasoning, with gains of 10–12% relative on GPQA and LogiQA for LIMO. A preliminary study of non-MPS SFT datasets (Magicoder, Magpie, OpenOrca) finds only localized gains.","tokens_in":15752,"tokens_out":9404,"duration_ms":90150,"significance":"The question addressed here is timely and practically important given the current emphasis on long-reasoning models and the widespread use of math SFT in post-training. The paper's strengths include the breadth of the evaluation (13 benchmarks), the use of multiple independent checkpoints per strategy, and the inclusion of a same-base ablation (s1.1 vs s1.1-short). If the central pattern is confirmed, the results would provide useful guidance for data selection in post-training. However, the causal interpretation is not yet fully supported: the same-base ablation changes content as well as length, the cross-strategy comparisons mix base-model families, and the RL results lack an SFT-only control. The claims are directionally plausible and worth publishing after these controls are added or the causal language is softened.","major_comments":[{"comment":"Section 3.3, Table 2: The sentence \"the performance of s1.1-short is consistently lower than that of the base model\" is not supported by the reported numbers: on GPQA-diamond, s1.1-short achieves 53.5 versus 50.0 for Qwen2.5-32B-Instruct. Moreover, the proposed ablation does not isolate response length as the causal variable; GPT-4o rewriting to produce 418.9-token answers also removes self-reflection and verification steps and changes content and formatting. The drop relative to s1.1 (or to the base model on most tasks) could therefore be due to these co-varying factors rather than to length itself. A control that compresses the same reasoning steps while preserving self-correction behavior, or a training study that varies only length, is needed before the conclusion that longer reasoning paths are beneficial can be drawn.","section":"3.3"},{"comment":"The headline comparison in Tables 1 and 2 between short-CoT SFT models (Math-CoT, Math-PoT, based on Mistral-7B) and long-CoT models (LIMO, s1.1, based on Qwen2.5-32B-Instruct) confounds the training strategy with base-model capacity, size, and alignment state. Statements such as \"conventional SFT on short reasoning chains generalizes poorly\" are supported by only one same-base control, s1.1-short, which has the content-rewriting confound noted above. To make the central claim robust, the paper needs same-base short-CoT SFT models on Qwen2.5-32B-Instruct (or an equivalent controlled comparison), or the conclusions must be explicitly limited to the specific models and configurations studied.","section":"3.2 and 3.3"},{"comment":"Table 3 does not include the SFT-only warmup checkpoint as a control for the SimpleRL models. The paper attributes the improvements of Qwen2.5-Math-SimpleRL and Qwen2.5-32B-SimpleRL to rule-based RL, but these models begin with an SFT stage, and the reported baseline (e.g., Qwen2.5-Math-7B) is not that SFT-stage checkpoint. Without reporting the SFT-only model's generalization performance, it is not possible to determine how much of the gain comes from RL versus the SFT warmup. I request that the SFT-only control be added for each SimpleRL variant, or that the attribution to RL be substantially softened.","section":"3.4"}],"minor_comments":[{"comment":"There is a typo in the abstract: \"In constrast\" should be \"In contrast\".","section":"Abstract"},{"comment":"The claim that Math-PoT SFT \"declines in eight tasks\" is not supported by Table 1: relative to Mistral-7B, Math-PoT improves on ZebraLogic, ProofWriter, and DocMath, and declines on seven of the listed non-MPS benchmarks. Please correct the count or clarify which task subset is being referenced.","section":"3.2"},{"comment":"The benchmark name is rendered inconsistently: the text and Figure 1 use \"GPQA\" while Table 2 uses \"GPQAdiamond\". Please specify the exact evaluation set (e.g., GPQA diamond subset) and the prompting/evaluation protocol in the main text or appendix.","section":"3.3"},{"comment":"There is a typo in the ablation description: \"using the same question from s1 1 but with a shorter CoT answer\" should refer to \"s1.1\" consistently.","section":"3.3"},{"comment":"The data-coverage analysis in Figure 3 is qualitative and based on PCA projections. Reporting a quantitative overlap metric (e.g., nearest-neighbor distribution distance or retrieval-based overlap) would strengthen the claim that WebInstruct and OpenWebMath cover general reasoning topics better than MetaMath.","section":"3.2"}],"recommendation":"major_revision","confidential_remarks":"The paper evaluates models the authors created (SimpleRL variants, Math-CoT/Math-PoT SFT) and open checkpoints, all on independent benchmarks, so there is no circularity concern. The main risk is causal overclaiming from confounded ablations and missing controls; the revision should either add the requested controls or soften the causal framing. The paper is within the journal's scope and the central direction is worth reporting."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth your time if you think about post-training data choices. It compares five ways to train on math—continual pretraining, STEM SFT, short-CoT SFT, long-CoT SFT, and rule-based RL—and measures transfer to eight general reasoning benchmarks. The main takeaway is that short-CoT SFT generalizes poorly and often hurts, while long-CoT SFT and RL help, and CPT helps somewhat. That pattern is consistent with prior work, but the systematic side-by-side is new and useful.\n\nWhat it does well: the evaluation is broad (GPQA, LogiQA, BBH, etc.), the models are mostly open or reproducible, and the paper is honest about using existing checkpoints. The s1.1 vs s1.1-short ablation is a genuine attempt to isolate length, even if imperfect. The response-length data across tasks is also a nice piece of evidence. Credit is given to the relevant prior work (Huang et al., Chu et al.), and the paper doesn't oversell its novelty.\n\nThe soft spots are real but not deal-breakers. The main one is the s1.1-short ablation: rewriting the long answers with GPT-4o changes content quality, removes self-correction/verification, and alters format, so it does not isolate response length as the causal variable. The claim that long CoT itself drives generalization is not fully established by this experiment. Also, the cross-strategy comparison mixes base models and sizes—short-CoT SFT is on Mistral-7B while long-CoT SFT is on Qwen2.5-32B—so some of the difference could come from model scale or alignment rather than training type. The RL results lack a same-base SFT-only control, making the \"RL generalizes\" claim a bit underdetermined. No error bars or significance tests are reported, which is minor but worth noting.\n\nAre these problems fatal? No. The directional pattern is credible and supported by other recent work. But the paper should be more careful in its wording: \"response length\" as a mechanism is not established; \"long-CoT training\" is a safer description. The short-CoT-hurts result should be flagged as preliminary given the different base model.\n\nMy recommendation: send it to peer review. A good referee will ask for tighter claims and perhaps a same-base control for the RL leg or a less confounded way to vary response length. The question matters, the evidence is useful, and the issues are addressable.","headline":"A useful, well-run empirical comparison of five MPS training strategies, with a real but addressable confound in the long-vs-short CoT ablation.","tokens_in":16349,"tokens_out":3377,"would_cite":true,"duration_ms":32721,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that supervised fine-tuning on mathematical problem-solving with long, self-reflective reasoning chains, and rule-based reinforcement learning on math queries, generalize to broader reasoning tasks, whereas conventional…","keywords":["mathematical problem-solving","reasoning generalization","long chain-of-thought","rule-based reinforcement learning","instruction tuning","continual pretraining","self-reflection","large language models"],"falsifier":"Train two models from the same base on the same long-CoT math solutions: one with answers truncated to short length without rewriting content, and one with the original long answers. If the truncated-length model keeps the cross-domain gains, then length is not the active ingredient; if it loses them while the original retains them, the conclusion is supported. Alternatively, a length-capped decoding evaluation of LIMO or s1.1 (max tokens set to base-model levels) that still showed the gains would contradict the 'long reasoning mode' mechanism.","tokens_in":15327,"feed_emoji":"🧠","tokens_out":7382,"duration_ms":67602,"temperature":0.7,"pith_summary":"The paper asks whether teaching a large language model to solve math problems builds general reasoning skills or only raises math scores. It compares five training strategies—continual pretraining on math text, instruction tuning on short and on long chain-of-thought solutions, fine-tuning on STEM question-answer data, and rule-based reinforcement learning on math queries—across five math and eight general reasoning benchmarks. The central finding is that long, self-reflective reasoning chains and rule-based reinforcement learning on math problems transfer to other reasoning domains, while ordinary short chain-of-thought math instruction tuning usually does not and often degrades non-math performance. This matters because math is one of the few domains where large volumes of verifiable, long reasoning traces can be generated, so it could be a practical route to stronger general reasoning in language models.","feed_headline":"Long math reasoning chains generalize; short ones don't","feed_subtitle":"Training on long, self-correcting math solutions lifts non-math reasoning, while short chain-of-thought math tuning often hurts.","key_machinery":"The carrying mechanism is the long, self-reflective chain-of-thought response—a solution trace that includes intermediate verification, self-checks, and alternative paths—used as supervised fine-tuning data, plus rule-based reinforcement learning with format and correctness rewards on math queries. The paper's behavioral evidence for the mechanism is response length: after long-CoT math training, the model produces substantially longer outputs on non-math benchmarks, and the short-answer ablation (rewriting s1.1's responses to about 419 tokens) erases the generalization gains. The rule-based RL component works by using a $\\boxed{}$ answer-format constraint and ground-truth verification as rewards, which lets the model discover extended reasoning without demonstration data.","core_discovery":"On the paper's own terms, the discovery is an empirical asymmetry: the way a model learns math determines whether that learning travels. Fine-tuning on a few hundred math problems solved with long, self-correcting chains—the LIMO recipe, for instance—improved Qwen2.5-32B-Instruct on every general reasoning task tested, with relative gains of 10.2% on GPQA and 11.8% on LogiQA, while also lengthening the model's responses in non-math domains. Fine-tuning on the same kind of problems with short step-by-step answers, by contrast, often lowered general reasoning scores; in the paper's control experiment, rewriting s1.1's long answers into short ones removed the cross-domain benefit. Rule-based reinforcement learning on math queries produced similar transfer, beating the base model and a math-data SFT model on most benchmarks. The paper concludes that short reasoning chains acquire math-specific behavior, whereas long, self-reflective chains activate a general 'long reasoning mode' that extends into other domains.","pith_inferences":["If long-chain length is the real driver, then methods that extend inference-time budgets or induce self-correction in any domain could reproduce the transfer without math-specific data; a length-capped evaluation would test this.","The paper's mechanism suggests the math domain's value is its verifiable long CoT traces, not math content per se; training on long CoT in other verifiable domains, such as code execution, should transfer similarly if the hypothesis is right.","The GPT-4o rewriting ablation leaves open that content quality, not length, caused the drop; a cleaner test would compress long answers by removing only self-correction and verification phrases while preserving content."],"forward_implications":["Conventional short-CoT math instruction tuning should not be assumed to strengthen general reasoning; math-benchmark gains can coexist with losses elsewhere.","Long-CoT math fine-tuning can transfer across domains with very few examples, suggesting that data quality and format may matter more than scale.","Rule-based RL on math queries is a viable alternative path to general reasoning improvement, and an SFT warmup before RL helps.","Continual pretraining on math text gives modest broad improvements, but they are less consistent than those from long-CoT SFT.","Because diverse non-math SFT datasets failed to generalize broadly, improving general reasoning may require new training objectives rather than more instruction-tuning data."],"supporting_citations":[{"why":"Supplies the LIMO model, trained on 817 math problems with long self-reflective chains, the paper's main evidence that long-CoT MPS training generalizes to non-math tasks.","marker":"Ye et al. 2025"},{"why":"Supplies the s1 and s1.1 models and the long-CoT training recipe that the paper ablates into short-CoT form.","marker":"Muennighoff et al. 2025"},{"why":"Supplies the SimpleRL-Zero and SimpleRL models, the rule-based RL evidence for cross-domain generalization from math queries.","marker":"Zeng et al. 2025b"},{"why":"Supplies DeepSeekMath, the continual-pretraining checkpoint used to measure generalization from math text.","marker":"Shao et al. 2024"},{"why":"Supplies RhO-Math, the second continual-pretraining checkpoint used to measure generalization from math text.","marker":"Lin et al. 2024"},{"why":"Supplies MetaMath, the short-CoT MPS dataset used for the Math-CoT SFT model that generalizes poorly.","marker":"Yu et al. 2023"},{"why":"Supplies NuminaMath-TIR, the program-of-thought MPS dataset used for the Math-PoT SFT model.","marker":"Li et al. 2024"},{"why":"Supplies MAmmoTH2, the STEM instruction-tuned model used as a diverse-data comparison point.","marker":"Yue et al. 2024"},{"why":"Prior result the paper extends: distilling long math reasoning chains increases response length and self-reflection in other domains.","marker":"Huang et al. 2024"}],"fun_headline_variants":["Long math reasoning chains generalize; short ones backfire","Self-correcting math training lifts non-math reasoning","Short math answers hurt general reasoning; long ones help","Learning math with long CoT improves other reasoning domains","Why short math tuning impairs reasoning while long chains transfer"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the s1.1-short ablation changed only the length of the reasoning chain; if rewriting the long answers with GPT-4o also changed answer quality, self-correction behavior, or format, the causal link between chain length and cross-domain generalization is not established.","fun_headline_variants_meta":{"raw":{"variants":["Long math reasoning chains generalize; short ones backfire","Self-correcting math training lifts non-math reasoning","Short math answers hurt general reasoning; long ones help","Learning math with long CoT improves other reasoning domains","Why short math tuning impairs reasoning while long chains transfer"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000735,"raw_usage":{"total_tokens":3307,"prompt_tokens":987,"completion_tokens":2320,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":603,"completion_tokens_details":{"reasoning_tokens":2242}},"tokens_in":603,"tokens_out":2320,"duration_ms":17268,"temperature":1.0,"reasoning_tokens":2242,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:48:34.039721+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train two models from the same base on the same long-CoT math solutions: one with answers truncated to short length without rewriting content, and one with the original long answers. If the truncated-length model keeps the cross-domain gains, then length is not the active ingredient; if it loses them while the original retains them, the conclusion is supported. Alternatively, a length-capped decoding evaluation of LIMO or s1.1 (max tokens set to base-model levels) that still showed the gains would contradict the 'long reasoning mode' mechanism.","supporting_citations":[],"review_version":1}