Pith. sign in

REVIEW 5 major objections 5 minor 14 references

A three-stage curriculum lets a 3B model distill long teacher rationales into short, explicit reasoning chains, lifting GSM8K accuracy to 76.19% while cutting output length by 27.4%.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 04:15 UTC pith:5WAXIALJ

load-bearing objection A sensible new curriculum for CoT distillation with a strong Qwen result, but the abstract overstates the comparison and missing details prevent a clean acceptance. the 5 major comments →

arxiv 2602.17686 v4 pith:5WAXIALJ submitted 2026-02-05 cs.LG cs.AI

Curriculum Learning for Efficient Chain-of-Thought Distillation via Structure-Aware Masking and GRPO

Bowen Yu , Maolin Wang , Sheng Zhang , Binhao Wang , Yi Wen , Jingtong Gao , Bowen Liu , Zimo Zhao
show 2 more authors
Wanyu Wang Xiangyu Zhao
This is my paper
classification cs.LG cs.AI
keywords chain-of-thought distillationcurriculum learningGRPOmasked reconstructionreasoning compressionGSM8Kreinforcement learningcapacity mismatch
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that small models fail to learn from verbose teacher reasoning because of a capacity mismatch, and that the fix is to teach structure before compression. It introduces a three-stage curriculum: reconstruct shuffled and masked reasoning steps, then use GRPO with a hierarchical reward that only rewards brevity after correctness, then rewrite teacher solutions for persistent failure cases. On GSM8K, a Qwen2.5-3B base model trained this way reaches 76.19% accuracy with 167 average output tokens, beating both the base and instruct models and prior distillation methods while using 27.4% fewer tokens. The same training transfers zero-shot to SVAMP and MATH-500, suggesting the compressed reasoning patterns generalize.

Core claim

The paper claims that a compact student can internalize explicit, verifiable chains of thought if it learns the logical skeleton first. The method, BRIDGE, stages the learning: Stage 1 performs generative reconstruction of step-shuffled and step-masked teacher rationales, forcing the student to model causal dependencies rather than positional copy. Stage 2 runs GRPO on masked completions with a hierarchical reward that gates all length bonuses behind a correctness condition, preventing reward hacking. Stage 3 collects samples the student still gets wrong, scaffolds them with the teacher's full solution, and runs GRPO to compress the scaffold into the student's own concise style. The result i

What carries the argument

The load-bearing representation is the decomposition of a teacher rationale into discrete reasoning steps (s1,...,sn). Stage 1 applies two corruptions to those steps — shuffling the order and masking about 15% of steps in 70% of samples — and trains the student to reconstruct the original ordered chain, eliminating positional shortcuts. Stage 2 optimizes a hierarchical reward R = R_base + I[correct]·R_eff under GRPO (Group Relative Policy Optimization, an RL update using group averages as baselines), where the efficiency term only activates for correct answers. Stage 3 applies GRPO to rewrite teacher scaffolds for failure cases, with a compression reward relative to teacher length and KL reg

Load-bearing premise

The entire framework assumes a teacher's chain of thought can be split into discrete, semantically independent steps that stay meaningful after shuffling and masking; the paper never specifies how those steps are extracted, so the reconstruction task might teach surface patterns if the segmentation is arbitrary.

What would settle it

Run the full BRIDGE pipeline with two different step-segmentation schemes (e.g., sentence splitting vs. newline splitting) on the same teacher rationales; if GSM8K accuracy differs by more than a couple of points, the method's success depends on an unspecified parse. Alternatively, replace step shuffling with random token-level shuffling: if performance holds, the claimed structural understanding is not needed.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Standard supervised fine-tuning on verbose teacher rationales is not just suboptimal but actively harmful for small models; training should begin with structure reconstruction.
  • The hierarchical reward design removes reward hacking in CoT distillation: brevity is only rewarded after correctness, so the model cannot game the loss by emitting empty answers.
  • Teacher-guided rewriting on failure cases recovers accuracy lost to compression pressure, offering a general recipe for RL-based distillation that avoids SFT consolidation.
  • The method produces explicit, readable chains at 167 tokens, preserving interpretability while compressing — unlike implicit reasoning methods that hide reasoning in hidden states.
  • Zero-shot transfer to SVAMP and MATH-500 indicates the learned compression is a generalizable reasoning skill rather than a dataset-specific template.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves step segmentation unspecified; if different natural segmentations (sentence splits vs. newline splits vs. semantic units) change results materially, the method's success may hinge on an arbitrary parse. A controlled comparison of segmentation schemes would settle this.
  • The curriculum's causal role could be tested by swapping Stage 1 and Stage 2: if structure-before-compression is essential, reversing the order should degrade accuracy and length trade-offs.
  • The observation that 96.83% of failure cases can be compressed when given teacher scaffolds suggests the bottleneck is generation, not comprehension; this asymmetry could be exploited in other distillation settings.
  • A natural extension, listed by the authors as future work but not tested here, is iterative self-evolution where the improved model's own failures become the next Stage 3 dataset — a testable path to autonomous reasoning improvement.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes BRIDGE, a three-stage curriculum for chain-of-thought distillation into 3B base models. Stage 1 uses masked and shuffled step reconstruction to teach reasoning structure; Stage 2 applies GRPO with a hierarchical correctness-first reward to compress outputs; Stage 3 identifies failure cases and applies GRPO with teacher scaffolds to internalize reasoning. On GSM8K the authors report 76.19% accuracy and 167 output tokens for Qwen2.5-3B-Base, versus 64.90%/230 tokens for the base model, and claim to surpass instruction-tuned variants and prior distillation baselines, with zero-shot transfer to SVAMP and MATH-500.

Significance. If the results hold, the paper offers a useful recipe for reasoning distillation at the 3B scale, where capacity mismatch is severe. The curriculum is well motivated, the hierarchical reward is a sensible guard against reward hacking, the ablation isolates component contributions, and the zero-shot transfer experiments are a valuable addition. However, the strength of the empirical evidence is undercut by the contradictory comparison to instruction-tuned models, the lack of variance estimates, and the absence of a precise definition of the 'step' representation on which two of the three stages rest. These issues are fixable, and the central idea remains promising.

major comments (5)
  1. [Abstract & §3.2, Table 1, Fig. E.5] The abstract's claim to 'surpass both instruction-tuned variants' is directly contradicted by Table 1: on Llama-3.2-3B, BRIDGE achieves 42.46% GSM8K versus 53.45% for Llama-3.2-3B-Instruct. Moreover, Figure E.5 states that all models are evaluated with the same raw prompt, without applying the chat template for instruction-tuned models. This is not a fair comparison and substantially weakens the headline comparative claim. Please qualify the claim or evaluate Instruct baselines with their native templates.
  2. [§2.2, Eq. (2)-(3), Algorithm 1] The entire Stage 1 and Stage 2 depend on decomposing a teacher rationale into discrete steps s_1,...,s_n that can be shuffled and masked. The paper never specifies how these steps are extracted (sentence splitting? newline splitting? semantic units?). Without this, the reconstruction task is ill-defined, the results are not reproducible, and the claimed 'structural understanding' may be an artifact of the particular parse. This is a load-bearing reproducibility gap.
  3. [§3.2, Table 1] No error bars or repeated-seed experiments are reported anywhere. The margin over the strongest distillation baseline on Qwen (76.19 vs 75.36, SuperRL) is only 0.83 accuracy points, or roughly 11 test examples, and all evaluations use greedy decoding. A single run cannot support 'outperforming' claims at this margin. Please report mean±std over at least 3 seeds, or demonstrate stability otherwise.
  4. [§2.3, §C, §D] Section 2.3 promises that 'specific reward coefficient values and their sensitivity analysis are provided in Appendix D.' Appendix D only lists the reward configuration; the only sensitivity analysis in the appendices is for GRPO group size G (Table C.1). No analysis is given for the masking probability, step mask ratio, reward coefficients γ, penalties, KL coefficient β_KL, or training subset size. Either provide the promised sensitivity analysis or correct the text.
  5. [§3.4, Table 3] Table 3 is arithmetically inconsistent: the four categories (856 + 264 + 119 + 199) sum to 1,438, but the GSM8K test set has 1,319 samples. Additionally, the claimed net improvement of +145 samples (264 - 119) corresponds to 10.99% of the test set, not the 11.29% stated in the abstract and Section 3.2. These discrepancies undermine confidence in the error analysis and should be corrected and reconciled.
minor comments (5)
  1. [Appendix E vs Table A.1] Appendix E says teacher data generation yields 6,128 correct solutions, while Table A.1 reports Stage 1 and Stage 2 use 2,000 training samples. Explain the relationship: is the 2,000 a subset? If so, how was it sampled?
  2. [Eq. (5)] The objective in Eq. (5) uses an unspecified trade-off parameter λ, which never appears in the reward design (Eqs. (6)-(9)) or in the hyperparameter table. Please clarify the connection.
  3. [§2.2, Fig. 2] The text says 'mask approximately 15% of the reasoning steps ... for a subset of samples (psample=0.7)', while the Figure 2 caption says 'randomly mask one step (with p=0.7)'. These are inconsistent. Clarify both the per-step masking probability and the sample-level probability.
  4. [Throughout] 'SV AMP' should be 'SVAMP' (the benchmark name). There are also several typos: the abstract's 'compressing reasoning into single-step, losing...' is missing an object; Table 1 has inconsistent spacing in the SVAMP column.
  5. [§3.2] The statement that the Qwen Instruct model 'shows no improvement (64.52%)' is made only for GSM8K, but the same paragraph implies general superiority. On SVAMP the Instruct model is at 45.33% versus 79.33% for Base, so qualify the claim to the GSM8K setting.

Circularity Check

1 steps flagged

Length reduction is a direct reward objective rather than an emergent prediction; accuracy gains remain out-of-sample and independent.

specific steps
  1. self definitional [Abstract; Section 2.3, Eqs. 6-9; Table 1]
    "Second, we apply Group Relative Policy Optimization (GRPO) on masked completion tasks, enabling the model to discover its own balance between accuracy and brevity. ... Rcomp(ri)=γ·(1− |ri|/|rT|). ... our approach enables Qwen2.5-3B-Base to achieve an 11.29 percent accuracy improvement while reducing output length by 27.4 percent."

    The reported 27.4% output-length reduction is not an independent emergent result: Eqs. 6-9 define the hierarchical reward as R = R_base + I[Correct]·R_eff with R_eff/R_comp monotonically decreasing in |ri|, and Eqs. 7/10 optimize exactly this reward. Any successful optimization therefore favors shorter outputs by construction; Table 1's length statistic is the optimized objective restated as an outcome. The exact token count is empirical, but the direction (and the headline's use of it as evidence) is forced by the hand-designed reward. Accuracy and zero-shot transfer to SVAMP/MATH-500 are not encoded in the reward and use held-out test sets, so the accuracy half of the central claim retains independent content.

full rationale

No load-bearing self-citation chain or equation-level tautology was found. The accuracy gain (64.90 to 76.19 on GSM8K test) and zero-shot transfer (SVAMP, MATH-500) are out-of-sample evaluations of a policy trained on GSM8K training data, so they are genuine empirical evidence. The compression result is the paper's own optimization target — the hierarchical reward explicitly rewards shorter outputs relative to the teacher — so presenting it as a headline 'reduction' is a mild presentation-level circularity rather than a derivation. The comparison against instruct baselines is weakened by the shared raw-prompt evaluation in Figure E.5 and by the Llama result (42.46 vs 53.45), but those are fairness/correctness issues, not circularity. The unspecified step-segmentation procedure (Stage 1, Eqs. 2-3) is an underspecification, not a circular step. Appendix G's limitations concern dataset scope, not circularity.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 0 invented entities

The central claim rests on several hand-set hyperparameters (masking probabilities, reward coefficients, KL coefficient, sample counts) and on domain assumptions about the structure of teacher CoT and the reliability of answer extraction. No new physical or conceptual entities are introduced. The 27.4% compression figure is essentially the reward function's target.

free parameters (6)
  • Masking probability p = 0.7
    Stage 1 masks a random step in 70% of samples; hand-chosen without sensitivity analysis (Section 2.2, Algorithm 1).
  • Step mask ratio = 15% (min 1 step)
    Mask ratio for Stage 1 reconstruction; no sensitivity reported.
  • Reward coefficient γ (Stage 2/3 efficiency) = max step reduction +0.9, token efficiency +0.2, compression ratio +0.8
    Hand-set in Appendix D; claimed sensitivity analysis is not actually provided.
  • Reward penalties = incorrect −2.0, format −1.0, correct base +1.0
    Hand-set; no sensitivity.
  • KL coefficient β_KL = 0.1
    Used in both GRPO stages; no sensitivity.
  • Training subset size = 2,000 (Stages 1-2); 568 (Stage 3)
    Selected from 6,128 filtered teacher solutions; selection criterion not specified.
axioms (4)
  • domain assumption Teacher CoT can be segmented into discrete reasoning steps (s_1,...,s_n)
    Eq. (2)-(3) treat steps as atomic; no extraction algorithm provided.
  • domain assumption Correctness determination via extracted answer is reliable
    Eq. (5)-(6) reward based on Correct(r); extraction method unspecified.
  • domain assumption GRPO with G=2 converges to a good compression policy
    Relies on standard GRPO; sensitivity only for G.
  • domain assumption Greedy decoding at temperature 0 reflects true model capability
    Evaluation protocol; no stochasticity considered.

pith-pipeline@v1.3.0-alltime-deepseek · 13008 in / 19025 out tokens · 180353 ms · 2026-08-03T04:15:54.188593+00:00 · methodology

0 comments
read the original abstract

Distilling Chain-of-Thought (CoT) reasoning from large language models into compact student models presents a fundamental challenge: teacher rationales are often too verbose for smaller models to faithfully reproduce. Existing approaches either compress reasoning into single-step, losing the interpretability that makes CoT valuable. We present a three-stage curriculum learning framework that addresses this capacity mismatch through progressive skill acquisition. First, we establish structural understanding via masked shuffled reconstruction. Second, we apply Group Relative Policy Optimization (GRPO) on masked completion tasks, enabling the model to discover its own balance between accuracy and brevity. Third, we identify persistent failure cases and guide the student to internalize teacher knowledge through targeted rewriting, again optimized with GRPO. Experiments on GSM8K demonstrate that our approach enables Qwen2.5-3B-Base to achieve an 11.29 percent accuracy improvement while reducing output length by 27.4 percent, surpassing both instruction-tuned variants and prior distillation methods.

Figures

Figures reproduced from arXiv: 2602.17686 by Binhao Wang, Bowen Liu, Bowen Yu, Jingtong Gao, Maolin Wang, Sheng Zhang, Wanyu Wang, Xiangyu Zhao, Yi Wen, Zimo Zhao.

Figure 1
Figure 1. Figure 1: Overview of BRIDGE. Stage 1 establishes structural understanding through masked shuffled reconstruction. Stage 2 applies GRPO on masked completion tasks to balance accuracy and compression. Stage 3 identifies failure cases, applies teacher-guided rewriting for internalization, and uses GRPO to maintain compression capabilities. 2.1 Framework Overview Given a question q sampled from dataset D, a teacher mod… view at source ↗
Figure 2
Figure 2. Figure 2: Illustration of the Structure-Aware Warmup data construction. We randomly mask one step (with [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Illustrative prompt template for the internalization step. The student sees the teacher’s complete solution but [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Output token distribution on GSM8K (Qwen 2.5-3B): Base model vs. [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Std-CoT KD falls into repetition loops when overwhelmed, whereas [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Training dynamics comparison. Solid lines: batch accuracy; dashed lines: output length. [PITH_FULL_IMAGE:figures/full_fig_p013_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

14 extracted references · 8 linked inside Pith

  1. [1]

    Mcc-kd: Multi-cot consis- tent knowledge distillation

    Hongzhan Chen, Siyue Wu, Xiaojun Quan, Rui Wang, Ming Yan, and Ji Zhang. Mcc-kd: Multi-cot consis- tent knowledge distillation. InFindings of the Association for Computational Linguistics: EMNLP 2023, pages 6805–6820,

    2023

  2. [4]

    Micota: Bridging the learnability gap with intermediate cot and teacher assistants.arXiv preprint arXiv:2507.01887,

    Dongyi Ding, Tiannan Wang, Chenghao Zhu, Meiling Tao, Yuchen Eleanor Jiang, and Wangchunshu Zhou. Micota: Bridging the learnability gap with intermediate cot and teacher assistants.arXiv preprint arXiv:2507.01887,

    Pith/arXiv arXiv

  3. [6]

    Distilling step-by-step! outperforming larger language mod- els with less training data and smaller model sizes

    Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alex Ratner, Ranjay Krishna, Chen-Yu Lee, and Tomas Pfister. Distilling step-by-step! outperforming larger language mod- els with less training data and smaller model sizes. InFindings of the Association for Computational Linguistics: ACL 2023, pages 8003–8017,

    2023

  4. [8]

    Implicit reasoning in large language models: A comprehensive survey.arXiv preprint arXiv:2509.02350, 2025a

    Jindong Li, Yali Fu, Li Fan, Jiahong Liu, Yao Shu, Chengwei Qin, Menglin Yang, Irwin King, and Rex Ying. Implicit reasoning in large language models: A comprehensive survey.arXiv preprint arXiv:2509.02350, 2025a. Yuetai Li, Xiang Yue, Zhangchen Xu, Fengqing Jiang, Luyao Niu, Bill Yuchen Lin, Bhaskar Ramasubra- manian, and Radha Poovendran. Small models st...

    Pith/arXiv arXiv 2025

  5. [9]

    Superrl: Reinforcement learning with supervision to boost language model reasoning

    Yihao Liu, Shuocheng Li, Lang Cao, Yuhang Xie, Mengyu Zhou, Haoyu Dong, Xiaojun Ma, Shi Han, and Dongmei Zhang. Superrl: Reinforcement learning with supervision to boost language model reasoning. arXiv preprint arXiv:2506.01096,

    Pith/arXiv arXiv

  6. [11]

    21 Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al

    URL https://arxiv.org/abs/2412.15115. 21 Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300,

    Pith/arXiv arXiv

  7. [13]

    Masked-and-reordered self-supervision for reinforcement learn- ing from verifiable rewards.arXiv preprint arXiv:2511.17473,

    Zhen Wang, Zhifeng Gao, and Guolin Ke. Masked-and-reordered self-supervision for reinforcement learn- ing from verifiable rewards.arXiv preprint arXiv:2511.17473,

    arXiv

  8. [14]

    Tokenskip: Controllable chain-of- thought compression in llms

    Heming Xia, Chak Tou Leong, Wenjie Wang, Yongqi Li, and Wenjie Li. Tokenskip: Controllable chain-of- thought compression in llms. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 3351–3363, 2025a. Runze Xia, Yupeng Ji, Yuxi Zhou, Haodong Liu, Teng Zhang, and Piji Li. From reasoning llms to bert: A two-stage d...

    arXiv 2025

  9. [2020]

    Mixed distillation helps smaller language model better reasoning.arXiv preprint arXiv:2312.10730,

    20 Chenglin Li, Qianglong Chen, Liangyue Li, Caiyu Wang, Yicheng Li, Zulong Chen, and Yin Zhang. Mixed distillation helps smaller language model better reasoning.arXiv preprint arXiv:2312.10730,

    Pith/arXiv arXiv

  10. [2021]

    Implicit chain of thought reasoning via knowledge distillation.arXiv preprint arXiv:2311.01460,

    Yuntian Deng, Kiran Prasad, Roland Fernandez, Paul Smolensky, Vishrav Chaudhary, and Stuart Shieber. Implicit chain of thought reasoning via knowledge distillation.arXiv preprint arXiv:2311.01460,

    Pith/arXiv arXiv

  11. [2022]

    Arkil Patel, Satwik Bhattamishra, and Navin Goyal. Are nlp models really able to solve simple math word problems? InProceedings of the 2021 conference of the North American chapter of the association for computational linguistics: human language technologies, pages 2080–2094,

    2021

  12. [2023]

    Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168,

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168,

    Pith/arXiv arXiv

  13. [2024]

    Codi: Compressing chain-of- thought into continuous space via self-distillation

    Zhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, and Yulan He. Codi: Compressing chain-of- thought into continuous space via self-distillation. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 677–693,

    2025

  14. [2025]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783,

    Pith/arXiv arXiv