{"id":"377b0b4c-1086-46e4-a40d-559ae085dbf8","arxiv_id":"2505.21067","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A 920-example distillation from DeepSeek R1 outperforms zero-RL models on the same Qwen2.5-32B base across several math and science benchmarks.","lead":"Using only 920 solved math problems from a strong AI model, the authors fine-tuned a 32-billion-parameter language model and found it beats models trained with far more data through reinforcement learning. The paper suggests the fine-tuned model reasons more flexibly, shifting approaches and checking its own work like a human solver.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central outperform claim rests on a single uncontrolled evaluation of third-party zero-RL checkpoints; the documented answer-extraction adaptation (Appendix C.1) covers DAPO but not ORZ's <answer> format, so the margin may be an artifact.","rationale":"The reader's weakest_assumption is the same evaluation-fairness risk I would stress: the distill-vs-zero-RL comparison depends on third-party checkpoints scored under an answer-extraction protocol that may disadvantage them. I agree with the CONDITIONAL verdict: the paper has a real and interesting result, but the headline claim should not be accepted as established until the evaluation asymmetry is resolved. I did not find an independent fatal flaw; the token-frequency and cognitive-behavior analyses are correlational and length-confounded, but they are secondary to the empirical claim. The manuscript itself acknowledges that the training samples are not directly comparable (Section 3) and that answer extraction can significantly affect results (Appendix C.1), which strengthens the need for the concrete test above. Verdict should remain CONDITIONAL pending that check.","tokens_in":18469,"tokens_out":5835,"duration_ms":71190,"concrete_test":"Re-run ORZ-32B, DAPO-32B, SimpleRL-32B, and the authors' distilled model on AIME2024, AIME2025, HMMT Feb 2025, GPQA Diamond, and MATH500 using each model's official generation prompt and native answer extraction (ORZ: parse <answer>; DAPO: parse final 'Answer:'; SimpleRL: parse boxed), and report per-problem extraction agreement plus bootstrap 95% confidence intervals over the 30-problem sets. If any zero-RL model's corrected score exceeds the Table 1 value by more than 3 points on AIME/HMMT, or if the confidence intervals overlap, the central outperform claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table 1 is the entire empirical basis for the abstract's central claim that 920 distilled examples clearly outperform zero-RL. The control arm is not trained or scored by the authors: it consists of three externally released checkpoints (DAPO-32B, ORZ-32B, SimpleRL-32B), evaluated with a single harness whose extraction prioritizes boxed answers. Appendix C.1 states that extraction was adapted for DAPO ('for example, for DAPO-32B, we extract the text following \"Answer:\"'), but no corresponding adaptation is documented for ORZ-32B, whose official template requires answers inside <answer> tags. If ORZ's AIME2024 (41.9 vs 61.2), AIME2025 (33.3 vs 50.0), and HMMT (20.9 vs 34.6) scores are depressed by a handful of extraction failures, the 'clearly outperform' claim collapses on exactly the benchmarks used to advertise it. With only 30 problems per AIME/HMMT set and no confidence intervals, a 5-10 point gap can be a few responses, so the absent extraction audit is the load-bearing vulnerability. The mechanistic chapters cannot compensate if the headline comparison itself is unfair.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports that supervised fine-tuning of Qwen2.5-32B on 920 DeepSeek-R1-generated solutions to historical AIME problems (with no correctness filtering) produces a model that outperforms three released zero-RL checkpoints (DAPO-32B, ORZ-32B, SimpleRL-32B) on AIME2024, AIME2025, HMMT, GPQA Diamond, and MATH500. The authors attribute the gap to 'flexible reasoning', operationalized through token-frequency differences (anthropomorphic tokens and logical connectors) and two GPT-4o-counted cognitive behaviors (Multi-Perspective Thinking or Attempting; Metacognitive Awareness), and they support this with a token-blocking decoding ablation. The paper also discusses reward hacking and overfitting in zero-RL outputs and proposes that distilled models are better starting points for subsequent RL.","tokens_in":18706,"tokens_out":8828,"duration_ms":93045,"significance":"If the comparison is fair, the headline result is practically significant: it suggests that a small amount of teacher-distilled SFT can match or beat large-scale RL on a 32B base model, with much lower compute. The paper's strengths include a direct benchmark against public checkpoints, detailed evaluation settings (Appendix B and C), a temperature-0.6 robustness check, and a falsifiable token-blocking probe. However, the zero-RL arm relies on third-party checkpoints evaluated with a single answer-extraction harness, and the cognitive-behavior analyses are confounded by response length. These issues currently prevent the paper from establishing its strongest claims as stated.","major_comments":[{"comment":"The comparison against zero-RL is the load-bearing evidence for the abstract, but it rests on an incomplete answer-extraction audit. Appendix C.1 documents that the Qwen2.5-Math harness, which prioritizes answers inside \\boxed, was adapted for DAPO-32B by extracting the text following 'Answer:', yet no corresponding adaptation is described for ORZ-32B, whose official prompt (Table 11) requires the answer inside <answer> tags. If ORZ-32B outputs are not boxed, even a modest extraction failure rate on the 30-problem AIME/HMMT sets would move its scores by several points and erode the margins shown in Table 1 (e.g., 41.9 vs 61.2 on AIME2024). Please report per-model raw extraction success rates and, if needed, use each model's native answer format with a matching parser.","section":"Section 3, Appendix C.1, Table 1"},{"comment":"The headline 'clearly outperform' is supported only by point estimates on 30-problem benchmarks, with no confidence intervals or significance tests. With only 30 problems, the 61.2 vs 50.6 gap on AIME2024 corresponds to roughly three to four additional solved problems, and the observed difference is within the range of sampling variability. Please report bootstrap confidence intervals (or per-problem pass rates) for Avg@32 and Pass@8(40), and temper the abstract's 'clearly' wording accordingly.","section":"Section 3, Table 1"},{"comment":"Table 3 reports a baseline AIME2025 Avg@32 of 52.9 for the distilled model, while Table 1 reports 50.0 for the same model and same evaluation setting. This discrepancy is not explained. Because the token-restriction delta (-14.9) is computed against this 52.9 baseline, the internal consistency of the ablation depends on resolving which number is correct; if the baseline differs by evaluation seed or sampling, the delta must be recomputed on a matched baseline.","section":"Section 4.1, Tables 1 and 3"},{"comment":"The cognitive-behavior counts are reported per response, but the distilled model's responses are substantially longer than those of the zero-RL models (e.g., AIME2024 average length 13,975 vs 7,916 for DAPO and 10,174 for ORZ). Without normalizing by output length, the higher counts in Figure 4 may simply reflect the fact that the distilled model writes more tokens. Please report per-token densities (or a length-controlled analysis) before concluding that distillation 'enhances' these behaviors.","section":"Section 4.2, Figure 4, Table 1"},{"comment":"The explanatory claim that the two cognitive behaviors 'give rise to' flexible reasoning is partly circular and partly correlational. The behaviors are operationalized through the same linguistic markers (e.g., 'wait', 'alternatively', 'maybe') that are used to define the distinctive tokens, and the token-restriction ablation blocks exactly those markers; the drop in GPT-4o counts in Table 4 is therefore in part definitional. Moreover, no evidence distinguishes 'behaviors cause performance' from 'longer, more exploratory responses contain more such phrases'. A falsifiable operationalization that is not tied to the banned-token list, or a mediation analysis controlling for length and base rates, would be needed to support the causal language in Section 5.","section":"Section 4.2, Tables 3-4, Appendix D.4"}],"minor_comments":[{"comment":"'GQPA Diamond' should be 'GPQA Diamond'.","section":"Section 3"},{"comment":"'consine' should be 'cosine'; Table 9's '3s hours' should be '3 hours'; and 'the the rest' contains a duplicated article.","section":"Appendix B.2"},{"comment":"'the distill model' should be 'the distilled model'.","section":"Section 4.1"},{"comment":"The in-text references to Table 10 and Table 11 are swapped relative to the table captions: the caption for Table 10 is DAPO-32B, but the text assigns DAPO-32B to Table 11. Please correct the cross-references.","section":"Appendix C.1"},{"comment":"No error bars or inter-annotator statistics are reported for the GPT-4o counts, despite the acknowledged instability of LLM-as-a-judge; please add variance estimates.","section":"Section 4.2, Figure 4"},{"comment":"The sentence noting that some zero-RL training samples include a subset of historical AIME problems should be reconciled with the abstract's 'much more data' claim, since distillation data include teacher solutions and are not directly comparable in sample count.","section":"Section 3"}],"recommendation":"major_revision","confidential_remarks":"The core comparison is the main risk. The authors cite reproducibility pitfalls in [33] but do not yet meet that standard for the third-party checkpoints. I would ask for the extraction audit and confidence intervals before considering publication; the cognitive analysis is interesting but secondary and can be revised. No concerns about citation practice or novelty."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I read this one through twice, including the appendices. The headline result is real enough to take seriously: 920 unfiltered AIME problems, distilled once from DeepSeek R1, SFT for 5 epochs on Qwen2.5-32B, and the resulting model beats three public zero-RL checkpoints on AIME, HMMT, and GPQA. The margins are large. If this holds, it is a genuinely useful recipe — a cheap alternative to zero-RL for sub-32B models — and the comparison to s1/LIMO (which used filtered data and instruct models) is a fair novelty claim.\n\nThe paper does some things well. The token-frequency analysis is straightforward, and the token-restriction experiment is a good control: blocking 'wait', 'perhaps', 'alternatively' drops performance, but the model works around the ban, sometimes switching to Chinese, which suggests something deeper than surface imitation. The appendix is transparent about training cost and evaluation settings, and the robustness check at temperature 0.6 is reassuring.\n\nThe soft spots are real, though. The zero-RL arms are third-party checkpoints, not trained by the authors, and the answer extraction is only documented as adapted for DAPO. ORZ-32B's official format puts answers inside <answer> tags; if the Qwen2.5-Math harness (which prioritizes \\boxed) does not handle that, ORZ's numbers could be depressed. On a 30-problem set with Avg@32, the gap is roughly 4–5 problems per run, so it would take many extraction failures to flip the result — not \"a few responses\" — but the paper should have audited this. The absence of confidence intervals does not help.\n\nThe mechanistic story is weaker. Cognitive behavior counts are not normalized by output length, and the distilled model's responses are ~1.5–2x longer; more behavior instances at greater length may just be a length effect. And the two \"advanced cognitive behaviors\" are defined using the same hedges and connectors that distinguish the model, so the measurement is partly circular. The paper acknowledges some limitations in Appendix A, but not these evaluation concerns.\n\nBottom line: this deserves a serious referee. The main comparison needs an extraction audit and, ideally, an internal zero-RL control, but the recipe is simple enough that the claim is likely to survive in some form. I would cite it cautiously and bring it to a reading group to debate the evaluation.","headline":"A cheap 920-example distillation result that likely holds, but the zero-RL comparison needs an extraction audit and the cognitive-behavior story is confounded by response length.","tokens_in":19271,"tokens_out":3476,"would_cite":true,"duration_ms":38416,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Distilling 920 AIME solutions into Qwen2.5-32B beats zero-RL on hard math benchmarks.","keywords":["distillation","zero-RL","large language models","mathematical reasoning","multi-perspective thinking","metacognitive awareness","flexible reasoning","supervised fine-tuning"],"falsifier":"Train a distilled model on the same 920 prompts but with teacher responses stripped of all anthropomorphic tokens and logical connectors; if that model still matches the full distilled model's AIME2025 score and cognitive-behavior counts, the paper's central mechanism is wrong.","tokens_in":18251,"feed_emoji":"🧮","tokens_out":6163,"duration_ms":58843,"temperature":0.7,"pith_summary":"This paper argues that a simple distillation recipe—supervised fine-tuning the Qwen2.5-32B base model on 920 AIME problems solved by a stronger teacher—can beat zero-RL, which trains the same base with tens of thousands of prompts and far more compute. The claim is not just a performance comparison: the paper identifies why distillation wins. Its answer is flexible reasoning, marked by anthropomorphic expressions and logical connectors that accompany two advanced cognitive behaviors, Multi-Perspective Thinking or Attempting and Metacognitive Awareness, which zero-RL fails to amplify. Blocking the distinctive tokens degrades the distilled model but does not erase its advantage, suggesting the behaviors are internalized rather than surface imitation.","feed_headline":"Distillation with 920 problems beats zero-RL on math","feed_subtitle":"A simple supervised fine-tune on 920 AIME solutions outperforms zero-RL models trained on 8,000-57,000 prompts, and the paper explains why.","key_machinery":"The load-bearing object is the 920-example distillation set constructed from all historical AIME problems (1983-2023), each paired with one DeepSeek R1 solution and used for supervised fine-tuning of Qwen2.5-32B. The argument then runs through two analytic instruments: token-frequency categorization (anthropomorphic tokens, logical connectors, and mathematical reasoning tokens) and GPT-4o-based counting of two advanced cognitive behaviors, Multi-Perspective Thinking or Attempting and Metacognitive Awareness. The token-restriction decoding experiment is the mechanism that separates surface imitation from internalized behavior: banning 'wait', 'maybe', 'alternatively', and related tokens drops performance but leaves the model ahead of zero-RL, and the model works around the ban, sometimes switching to Chinese, to express the same reasoning shifts.","core_discovery":"The central discovery is that fewer than a thousand unfiltered teacher examples can transfer a reasoning style that large-scale zero-RL does not produce. Starting from the same Qwen2.5-32B base, the authors distill all 920 AIME problems from 1983 to 2023 using DeepSeek R1 responses, without filtering for correctness, and fine-tune for five epochs. The resulting model scores 50.0 Avg@32 on AIME2025 versus 32.9 for DAPO-32B, and leads across AIME2024, HMMT Feb 2025, GPQA Diamond, and MATH500. Token-frequency analysis shows the distilled model mirrors the teacher's use of words such as 'wait', 'maybe', 'but', and 'alternatively', while zero-RL models keep the base model's rigid step-by-step style. The authors trace these surface differences to deeper habits: the distilled model exhibits many more instances of Multi-Perspective Thinking or Attempting and Metacognitive Awareness, and these behaviors correlate with benchmark scores. Even when the distinctive tokens are blocked during decoding, the model preserves part of its advantage and finds new ways to express the same behaviors, which the authors take as evidence that distillation teaches reasoning processes, not just word patterns.","pith_inferences":["A concrete testable extension is to run zero-RL with an explicit reward term for Multi-Perspective Thinking or Attempting and Metacognitive Awareness; if the gap with distillation narrows, the paper's behavioral explanation is confirmed rather than stylistic.","The token-restriction results imply that answer-extraction and prompt-template choices can change who appears to win, so a fair head-to-head should report each zero-RL checkpoint under its own intended decoding format, which the paper attempts but the community could audit independently.","Because the distillation data are unfiltered historical AIME problems from a single teacher, the result suggests that data quantity is not the bottleneck; a cheap recipe for other domains is to collect a few hundred hard, representative problems solved by a strong teacher and look for the two behaviors before scaling data.","The observed transfer beyond mathematics, to MMLU-Pro and MMLU-STEM, hints that the two cognitive behaviors are domain-general; testing on code and science reasoning benchmarks would show whether the same mechanism explains improvements there."],"forward_implications":["On the same Qwen2.5-32B base, supervised fine-tuning on 920 unfiltered AIME solutions outperforms three zero-RL baselines on AIME2024, AIME2025, HMMT Feb 2025, GPQA Diamond, and MATH500.","The distilled model's advantage holds at temperature 0.6 and extends to MMLU-Pro and MMLU-STEM, despite the training data being mathematics-only.","Blocking distinctive tokens such as 'wait', 'maybe', 'alternatively', and 'but' during decoding lowers the distilled model's scores, with the largest drop on harder problems, but the restricted model still remains comparable to zero-RL.","The token-restricted model retains more instances of the two advanced cognitive behaviors than zero-RL and works around the ban, sometimes switching to Chinese, which the paper reads as evidence that the behaviors are internalized.","Zero-RL checkpoints show signs of reward hacking and overfitting, such as ending reasoning right after 'let's' or emitting integer answers when the training set contained only integer answers."],"supporting_citations":[{"why":"DeepSeek R1 is the teacher model that generated the 920 distillation responses and supplies the flexible-reasoning style.","marker":"[2]"},{"why":"SimpleRL-32B is one of the three zero-RL baselines, trained on 8,000 prompts.","marker":"[7]"},{"why":"DAPO-32B is the main zero-RL baseline, trained on 17,000 prompts and used for the detailed linguistic comparison.","marker":"[9]"},{"why":"ORZ-32B is the third zero-RL baseline, trained on 57,000 prompts.","marker":"[10]"},{"why":"s1 provides prior evidence that small amounts of high-quality distillation data can yield large reasoning gains.","marker":"[14]"},{"why":"LIMO similarly shows that carefully selected distillation data can greatly improve complex reasoning.","marker":"[15]"},{"why":"Qwen2.5-32B is the shared base model for both the distilled and zero-RL approaches.","marker":"[16]"},{"why":"Qwen2.5-Math supplies the training prompt template and the official evaluation framework used for all models.","marker":"[25]"},{"why":"The four cognitive behaviors identified there ground the paper's own two advanced cognitive behaviors.","marker":"[35]"}],"fun_headline_variants":["920 distilled examples beat 8K+ RL prompts on math reasoning","Why distillation beats zero-RL: flexible reasoning habits","Less data, better reasoning: distillation's edge over RL","920 problems: the surprising key to outperforming RL","Distillation's flexible reasoning outshines zero-RL's rigidity"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central comparison assumes that the three released zero-RL checkpoints are fair, representative instances of zero-RL on Qwen2.5-32B and that the evaluation does not systematically disadvantage them.","fun_headline_variants_meta":{"raw":{"variants":["920 distilled examples beat 8K+ RL prompts on math reasoning","Why distillation beats zero-RL: flexible reasoning habits","Less data, better reasoning: distillation's edge over RL","920 problems: the surprising key to outperforming RL","Distillation's flexible reasoning outshines zero-RL's rigidity"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000267,"raw_usage":{"total_tokens":1635,"prompt_tokens":987,"completion_tokens":648,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":603,"completion_tokens_details":{"reasoning_tokens":565}},"tokens_in":603,"tokens_out":648,"duration_ms":5956,"temperature":1.0,"reasoning_tokens":565,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:36:20.979151+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a distilled model on the same 920 prompts but with teacher responses stripped of all anthropomorphic tokens and logical connectors; if that model still matches the full distilled model's AIME2025 score and cognitive-behavior counts, the paper's central mechanism is wrong.","supporting_citations":[],"review_version":1}