Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

On-policy distillation works as an exploration catalyst, not a capability expander, and its success is governed by signal fidelity rather than teacher scale.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 05:17 UTC pith:IP34FEZ2

load-bearing objection Useful empirical study of OPD's failure modes, but the formal mechanism for length exploitation assumes a sequence-averaged objective that the paper's own loss does not specify — as written, the derivation doesn't hold. the 4 major comments →

arxiv 2607.13399 v1 pith:IP34FEZ2 submitted 2026-07-15 cs.CL cs.LG

Demystifying On-Policy Distillation: Roles, Pathologies, and Regulations

Rui Wang , Hongru Wang , Yi Chen , Boyang Xue , Tianqing Fang , Wenhao Yu , Kam-Fai Wong This is my paper
classification cs.CL cs.LG
keywords on-policy distillationexploration catalyststudent-teacher mismatchlength exploitationadvantage clippinglog-scale compressionLLM reasoningpost-training
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

On-policy distillation (OPD) trains a smaller student on token-level preferences from a stronger teacher, using the student's own rollouts. This paper argues that OPD does not transfer new capabilities; it re-weights the student's existing sampling distribution, accelerating early learning without raising the ultimate performance ceiling. That framing implies the teacher's value lies entirely in the fidelity of its per-token signal, and the paper identifies two ways that signal gets corrupted: a student–teacher capability mismatch that turns the guidance into noise, and a length-dependence in the sequence-averaged objective that lets the student game its reward by padding or truncating responses. The proposed fixes — hard clipping and log-scale compression of each token advantage — are claimed to stabilize training and allow a 4B teacher to beat methods using a 30B teacher. A careful reader would care because this repositions OPD from a scaling trick to a signal-quality problem with cheap, in-loop solutions.

Core claim

The paper claims that on-policy distillation does not inject new capabilities into the student; it re-weights the student's existing generative distribution so that correct reasoning paths are found faster in early sampling. Evidence: pass@k curves converge to the base model at high k, and prompt diversity beats per-prompt sampling depth under fixed compute. Because OPD is pure guidance, its effectiveness depends entirely on whether the per-token advantage ∆ℓ_t = log(π_T/π_θ) correlates with rollout correctness. Negative correlation (student-teacher mismatch) steers exploration wrong; length-dependence of the sequence-averaged advantage lets the student pad or truncate (length exploitation).

What carries the argument

The central object is the per-token distillation advantage ∆ℓ_t = log π_T(y_t|y_<t,x) − log π_θ(y_t|y_<t,x), evaluated on the student's own rollouts; it is the token-level reward that drives OPD. The key diagnostic is its sequence average ā = (1/T)Σ_t ∆ℓ_t, whose dependence on length T creates the two exploitation modes: padding increases T to dilute a negative total advantage, and truncation captures a favorable prefix before risky tokens. The regulatory mechanisms are hard clipping (clip(∆ℓ_t, cmin, cmax)) and soft log-scale compression (sign(∆ℓ_t)·log(1+|∆ℓ_t|)), which preserve the teacher's ordinal preference while suppressing outlier magnitudes.

Load-bearing premise

The load-bearing premise is that OPD optimizes the sequence-averaged token advantage ā = (1/T)Σ∆ℓ_t; if the objective instead sums per-token advantages, the padding-dilution and truncation incentives do not directly follow, and the identified pathologies could have a different cause.

What would settle it

Train the same student-teacher pair under OPD with a summed (not averaged) per-token advantage, holding all else fixed; if the student still drifts to endless padding or premature truncation, length exploitation is not caused by the averaging. Equivalently, compute ∂ā/∂T under the actual objective: if it cannot be made positive by padding or truncation, the proposed mechanism fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • OPD should be treated as reward shaping, not knowledge transfer: scaling up the teacher will not help if its per-token signal is misaligned with the student's rollout distribution.
  • Under a fixed compute budget, maximize prompt diversity (one rollout per prompt) rather than sampling depth per problem; the paper finds n=1 consistently beats n=8.
  • The length-exploitation failure modes are objective-level, so any OPD implementation with sequence-averaged token advantages is vulnerable; regulating advantages in the loop is a zero-overhead defense.
  • A well-matched, moderately sized teacher plus signal regulation can outperform a much larger teacher, so teacher selection should be guided by informativeness on student rollouts, not by benchmark score.
  • Hard clipping is preferable to log-scale compression when the teacher-student gap is large; log-scale compression is better when the gap is small.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If OPD is purely an exploration catalyst, its natural partner is outcome-based RL: OPD accelerates the search, outcome rewards set the ceiling. Whether the two can be combined without the pathologies resurfacing is a testable next step the paper leaves open.
  • The length-exploitation analysis assumes the objective averages per-token advantages; an implementation that sums them would weaken the padding-dilution incentive, so a controlled average-vs-sum comparison would isolate the mechanism.
  • The informativeness metric I could be used as a cheap, pre-training screen for teacher selection, though it requires rollout accuracy labels, which may limit it to verifiable domains like math.
  • The same regulations could transfer to other dense-reward settings (e.g., process reward models) where token-level signals are aggregated over variable-length sequences and similar length shortcuts may appear.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies on-policy distillation (OPD) for LLM post-training. It argues that OPD acts as an exploration catalyst: it accelerates learning through dense token-level guidance but does not expand the student's capability ceiling. The authors support this with pass@k convergence experiments, a prompt-diversity-vs-rollout-depth comparison, and an analysis of teacher-student capability gaps. They identify two pathologies—Student-Teacher Mismatch and Length Exploitation—and propose two lightweight signal regulations: hard advantage clipping and soft log-scale compression. Experiments on seven math reasoning benchmarks show these regulations stabilize training and enable a 4B teacher to outperform prior methods that rely on a 30B teacher. The paper concludes that signal quality, not teacher scale, governs OPD success.

Significance. If the claims hold, the paper offers a useful reframing of OPD as exploration guidance rather than capability expansion, with practical implications for compute allocation and teacher selection. The proposed log-scale compression is a simple, parameter-free modification that appears to improve stability and accuracy, and the result that a regulated 4B teacher can beat methods using a 30B teacher is practically significant. The paper is clearly structured and the experimental configuration is reported with reasonable detail. However, the central claims currently rest on qualitative or incomplete quantitative evidence, and the formal derivation of Length Exploitation is not fully grounded in the stated objective. The work is potentially valuable but needs strengthening in these areas.

major comments (4)
  1. [§4.2, Eq. (3) and Eq. (5)] Length Exploitation is derived from the sequence-averaged advantage a_bar = (1/T)Σ a_t, stated in Eq. (5), but the policy-gradient objective in Eq. (3) is per-token and does not specify whether token advantages are summed or averaged over a sequence. The KL objective in Eq. (1) is an expectation of a sum of per-token log-ratios, not their mean. If the implementation sums advantages over tokens (or performs a batch-level mean), the padding-dilution mechanism in Eq. (5) does not follow: appending filler tokens would not asymptotically drive the gradient to zero. Similarly, the truncation inequality in Eq. (6) compares per-token means, which is not implied by a sum objective. The paper should either state explicitly that the implementation uses sequence-level averaging (and justify why this is the standard OPD objective), or re-derive the pathologies under the actual loss normalization. As
  2. [§3, Figure 3] The central claim that OPD accelerates exploration but does not raise the capability ceiling is supported by a single qualitative figure (Figure 3) with no benchmark specified and no numerical pass@k values. The text mentions testing pass@1024, but the figure caption and surrounding description do not say which dataset(s) were used, nor do they report the asymptotic accuracies of the base and trained models. Without quantitative evidence that the high-k performance of OPD-trained models converges to that of the base model, the 'exploration catalyst, not ceiling breaker' conclusion is not empirically verified. Please provide the benchmark names, the concrete pass@k values at large k, and ideally confidence intervals.
  3. [Tables 1 and 2; Figures 4, 5, 9] No error bars, standard deviations, or multiple-seed results are reported for the main experimental tables and figures. The paper repeatedly uses terms such as 'stably surpassing' and 'consistently', but with single runs it is impossible to assess stability. The avg@32 metric is averaged over 10 trials, but that only reduces evaluation noise, not training variance. Please report results over at least three seeds (or otherwise justify that training is deterministic enough) and provide mean±std for the key comparisons against baselines.
  4. [§5.1, Eq. (7); Limitations] The hard clipping regulation introduces two hyperparameters, cmin and cmax, whose values are never disclosed. The paper acknowledges in the Limitations that these require empirical tuning per teacher–student pair. This is a significant reproducibility concern: the claimed superiority of +Clip for the 4B-GRPO teacher may depend on carefully selected bounds. Please report the exact cmin and cmax used for each experiment, and ideally a sensitivity analysis over these bounds. The log-scale compression in Eq. (8) is parameter-free and should be highlighted as the more robust regulation if that is the intended message.
minor comments (5)
  1. [§4.1, Eq. (4)] The Informativeness metric I is defined post hoc using the teacher's log-ratio on correct versus incorrect rollouts. This is a reasonable diagnostic, but the paper should be careful not to overstate its predictive power: I is essentially a correlation measure and may not be causal. Consider discussing this limitation explicitly.
  2. [§3, Figure 5] The prompt-diversity experiment compares n=1, n=2, and n=8 under 'identical total compute', but the exact batch size and total number of prompts for each configuration are not stated in the main text. Please clarify how the budget is held fixed, e.g., whether the number of training steps is adjusted.
  3. [Appendix A] The training configuration mentions a 'block' prefix for non-thinking mode, but the actual token or template is not described. Please specify what this block is so that readers can reproduce the setup.
  4. [Throughout] Several equations use variables without explicit definitions: in Eq. (3), r_t is defined but ϵ is not; in Eq. (4), the expectation is over rollouts but the conditioning on y and r is implicit; in Eq. (5), a_filler is not formally defined. Please pass a notation table or clarify in text.
  5. [References] A few references appear to be in a non-standard format (e.g., 'and 1 others', 'and 41 others' in the reference list). These should be completed or condensed consistently with the journal style.

Circularity Check

1 steps flagged

Length Exploitation is derived from an assumed per-sequence-mean objective rather than from the paper's stated loss; the length shortcut is built into the normalization.

specific steps
  1. other [Section 4.2, Eq. (5) (also Eq. (6))]
    "Crucially, because standard OPD objectives optimize the sequence-averaged advantage ¯a= 1 T PT t=1 at, the student discovers a degenerate shortcut to evade this penalty: appending low-information filler tokens to inflate the denominator T . Formally, suppose the core reasoning block terminates at step T0 with a cumulative negative signal Acore = PT0 t=1 at <0 . By continually generating redundant tokens up to a prolonged length T (T≫T 0), the aggregated advantage decomposes as: ¯a= Acore T + T−T 0 T ¯afiller. (5)"

    The mechanism is not derived from the OPD loss the paper defines. Eq. (1) is an expectation of the sum of per-token log-ratios, and Eq. (3) is a PPO loss over tokens, with no per-sequence division by T. The paper instead asserts that 'standard OPD objectives optimize the sequence-averaged advantage'; once that per-sequence mean is assumed, the padding-dilution (Mode A) and prefix-truncation (Mode B, Eq. (6)) incentives follow algebraically. Thus the length-exploitation 'pathology' is contained in the assumed normalization, i.e., it is an artifact of the chosen definition rather than a consequence of the stated objective. The finding reduces to the premise.

full rationale

Most of the paper is a self-contained empirical study: the exploration-catalyst role is supported by pass@k convergence and prompt-diversity experiments against external baselines, and the regulation comparisons are reported against independent methods and teachers. The Informativeness metric is a post-hoc diagnostic rather than a fitted parameter, so I do not count it as circular. The one load-bearing circular/definitional step is the formal derivation of Length Exploitation, which assumes the very per-sequence length normalization that produces the length shortcut. Because one of the two central pathologies is thereby reduced to an unstated premise, the overall circularity score is 6 rather than 0-2. The cmin/cmax tuning limitation is a correctness/reproducibility concern, not circularity.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 1 invented entities

The central claims rest on a few unstated modeling choices: the sequence-averaging of advantages, the interpretation of pass@k as a proxy for capability ceiling, and the undisclosed clipping bounds. The only invented quantity is the informativeness metric, which is used for explanation rather than prediction.

free parameters (2)
  • cmin (lower clipping bound) = not disclosed
    Used in hard clipping; acknowledged to require empirical tuning per teacher-student pair (Section 5.1, Limitations).
  • cmax (upper clipping bound) = not disclosed
    Same as cmin; not reported in the paper.
axioms (4)
  • ad hoc to paper The training objective optimizes the sequence-averaged token advantage ā = (1/T) Σ a_t.
    Essential for the length exploitation derivation in Eq. (5), but the per-token PPO objective in Eq. (3) does not explicitly state whether sum or average is used; this is a load-bearing assumption.
  • domain assumption The teacher is fixed and the student samples its own rollouts (on-policy).
    This is the definition of OPD from prior work, invoked throughout.
  • standard math Reverse KL is the correct objective for distillation.
    Standard OPD objective as given in Eq. (1).
  • domain assumption The capability ceiling can be measured by pass@k convergence at high k (e.g., k=1024).
    The paper's role conclusion relies on the observation that base and OPD models converge at high sampling budgets; this presupposes that pass@1024 reveals the base model's true latent capability.
invented entities (1)
  • Informativeness metric I no independent evidence
    purpose: Quantify whether the teacher's token-level signal discriminates correct vs. incorrect student rollouts (Eq. 4).
    A new diagnostic metric introduced to explain the Student-Teacher Mismatch; it is not validated against external data and is used only post hoc.

pith-pipeline@v1.3.0-alltime-deepseek · 194 in / 8230 out tokens · 145541 ms · 2026-08-02T05:17:22.708230+00:00 · methodology

0 comments
read the original abstract

On-policy distillation (OPD) has become a key paradigm in LLM post-training, yet its training dynamics remain poorly understood. We present a systematic study examining the role, pathologies, and regulations of OPD. We first clarify the role of OPD as an exploration catalyst: it steers the student toward correct reasoning paths via dense token-level guidance, without expanding capability ceiling. We confirm this by showing that prompt diversity matters more than per-problem sampling numbers, and critically, that the effectiveness of OPD hinges entirely on the quality of its guiding signal. This dependency exposes two pathologies that derail exploration. The Student-Teacher Mismatch occurs when a large teacher-student distributional gap causes the guiding signal to misalign with task correctness, steering exploration in counterproductive directions. Length Exploitation arises when the aggregated token-level objective creates length-dependent shortcuts, allowing the student to game the reward landscape through response truncation or redundant padding, exploring degenerate length modes rather than reasoning strategies. To tame these pathologies, we investigate lightweight signal regulations: advantage clipping and log-scale compression, ensuring exploration is guided by faithful signals. Experiments across seven benchmarks demonstrate that these regulations alleviate length exploitation and enable effective distillation, stably surpassing OPD variants and RLVR baselines, thereby confirming that well-regulated signal quality, rather than mere teacher scale, governs successful exploration in OPD.

Figures

Figures reproduced from arXiv: 2607.13399 by Boyang Xue, Hongru Wang, Kam-Fai Wong, Rui Wang, Tianqing Fang, Wenhao Yu, Yi Chen.

Figure 1
Figure 1. Figure 1: Left: OPD accelerates exploration but cannot exceed the model ceiling. Middle: a stronger teacher does not guarantee a better student. Right: advantage rises while length explodes or crashes. ness (Gu et al., 2026; Yang et al., 2025; DeepSeek￾AI, 2026). Despite its growing adoption, the fun￾damental training dynamics of OPD remain poorly understood. Practitioners often observe highly in￾consistent behavior… view at source ↗
Figure 2
Figure 2. Figure 2: Overview. We characterize OPD’s role as an exploration catalyst (§ [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Pass@k performance of Qwen3-1.7B-Base trained via different methods. OPD variants outperform Base and GRPO models in low-k. However, the conver￾gence at higher k suggests OPD accelerates exploration rather than raising the model’s performance ceiling. et al., 2026), and on-policy RL with dense token￾level rewards (Yang et al., 2026a). While standard distillation intuition suggests that the teacher trans￾fe… view at source ↗
Figure 4
Figure 4. Figure 4: Weak teacher outperforms strong teacher in OPD w/ Clip. Benchmark accuracy (first two panels) and informativeness I on the validation set for OPD w/ Clip runs distilling from four teachers with varying capabilities into the same Qwen3-1.7B-Base student. herent generative space, ensuring optimal solutions are surfaced within the first few attempts. This characterization highlights a critical vulner￾ability:… view at source ↗
Figure 5
Figure 5. Figure 5: Avg@32 across varying prompt-to-rollout configurations. Under a fixed batch B, we vary the roll￾out count n per prompt. Maximizing problem diversity consistently outperforms a larger sampling size. implies that initial learning velocity is an unreliable proxy for the final capability ceiling. The Failure of the Strongest Teacher The most striking observation is the failure of the most capa￾ble model in our… view at source ↗
Figure 6
Figure 6. Figure 6: Length exploitation illustration. Advantage increases, while accuracy drops. Teacher [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: (a) A correct rollout receives a mildly negative token-mean. (b) Mode A (Endless Exploration): a long wrong rollout with negative scores diluted across tokens, failing to penalize the incorrect response. (c) Mode B (Abrupt Degeneration): a short, fully teacher-preferred rollout prefix that yields a strongly positive token-mean despite an incorrect outcome. ing out the penalty gradient (∇θL ≈ 0). As a resul… view at source ↗
Figure 8
Figure 8. Figure 8: Hard Clipping collapses all tokens beyond the [PITH_FULL_IMAGE:figures/full_fig_p007_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Training dynamics of OPD variants. Teacher: [PITH_FULL_IMAGE:figures/full_fig_p008_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Outcome-Confounded Local Supervision in On-Policy Distillation

    cs.LG 2026-07 conditional novelty 6.0

    In on-policy distillation, low teacher–student divergence on failed math trajectories dominates token mass, and local divergence plus a trajectory outcome cannot localize where failure became unrecoverable.

Reference graph

Works this paper leans on

18 extracted references · 17 linked inside Pith · cited by 1 Pith paper

  1. [2]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt

    Minillm: On-policy distillation of large language models.Preprint, arXiv:2306.08543. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt

    Pith/arXiv arXiv

  2. [4]

    Tao Huang, Shan You, Fei Wang, Chen Qian, and Chang Xu

    Uni-opd: Unifying on-policy distilla- tion with a dual-perspective recipe.arXiv preprint arXiv:2605.03677. Tao Huang, Shan You, Fei Wang, Chen Qian, and Chang Xu

    Pith/arXiv arXiv

  3. [6]

    Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, and 1 others

    Entropy-aware on-policy distillation of language models.Preprint, arXiv:2603.07079. Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, and 1 others

    Pith/arXiv arXiv

  4. [7]

    Solving quan- titative reasoning problems with language models. NeurIPS. Yaxuan Li, Yuxin Zuo, Bingxiang He, Jinqian Zhang, Chaojun Xiao, Cheng Qian, Tianyu Yu, Huan ang Gao, Wenkai Yang, Zhiyuan Liu, and Ning Ding. 2026a. Rethinking on-policy distillation of large language models: Phenomenology, mechanism, and recipe.Preprint, arXiv:2604.13016. Yuetai Li...

    Pith/arXiv arXiv

  5. [9]

    Demystifying opd: Length in- flation and stabilization strategies for large language models.Preprint, arXiv:2604.08527. MAA. 2026a. American invitational mathematics examination (AIME). https://www.maa.org/ math-competitions. MAA. 2026b. American mathematics com- petitions (AMC). https://www.maa.org/ math-competitions. Seyed-Iman Mirzadeh, Mehrdad Farajta...

    Pith/arXiv arXiv

  6. [13]

    Nemotron- cascade: Scaling cascaded reinforcement learning for general-purpose reasoning models.Preprint, arXiv:2512.13607. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Day- iheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others

    arXiv

  7. [14]

    Wenkai Yang, Weijie Liu, Ruobing Xie, Kai Yang, Saiy- ong Yang, and Yankai Lin

    Qwen3 technical report.Preprint, arXiv:2505.09388. Wenkai Yang, Weijie Liu, Ruobing Xie, Kai Yang, Saiy- ong Yang, and Yankai Lin. 2026a. Learning beyond teacher: Generalized on-policy distillation with re- ward extrapolation.Preprint, arXiv:2602.12125. Zhuolin Yang, Zihan Liu, Yang Chen, Wenliang Dai, Boxin Wang, Sheng-Chieh Lin, Chankyu Lee, Yangyi Chen...

    Pith/arXiv arXiv

  8. [15]

    Dapo: An open-source llm re- inforcement learning system at scale.Preprint, arXiv:2503.14476. Xinlei Yu, Gen Li, Qingyi Si, Guibin Zhang, Yuqi Xu, Congcong Wang, Shuai Dong, Kaiwen Tuo, Xi- angyu Zeng, Kaituo Feng, Qunzhong Wang, Yang Shi, Xiaobin Hu, Xiangyu Yue, Jiaqi Wang, and Shuicheng Yan

    Pith/arXiv arXiv

  9. [16]

    Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang

    Dopd: Dual on-policy dis- tillation.Preprint, arXiv:2606.30626. Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang

    Pith/arXiv arXiv

  10. [17]

    Siqi Zhu, Xuyan Ye, Hongyu Lu, Weiye Shi, and Ge Liu

    Does reinforcement learning really incentivize rea- soning capacity in llms beyond the base model? Preprint, arXiv:2504.13837. Siqi Zhu, Xuyan Ye, Hongyu Lu, Weiye Shi, and Ge Liu

    Pith/arXiv arXiv

  11. [18]

    A Experiment Configurations We implement the training pipeline using verl

    The many faces of on-policy dis- tillation: Pitfalls, mechanisms, and fixes.Preprint, arXiv:2605.11182. A Experiment Configurations We implement the training pipeline using verl. For the OPD pipeline, policy optimization uses a total batch size of 128, 1 rollout for each prompt. For the GRPO baselines, the group advantage is calculated across a size of 12...

    Pith/arXiv arXiv

  12. [2015]

    Preprint, arXiv:1503.02531

    Distilling the knowledge in a neural network. Preprint, arXiv:1503.02531. Wenjin Hou, Shangpin Peng, Weinong Wang, Zheng Ruan, Yue Zhang, Zhenglin Zhou, Mingqi Gao, Yifei Chen, Kaiqi Wang, Hongming Yang, and 1 oth- ers

    Pith/arXiv arXiv

  13. [2017]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y

    Prox- imal policy optimization algorithms.Preprint, arXiv:1707.06347. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo

    Pith/arXiv arXiv

  14. [2019]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov

    Improved knowl- edge distillation via teacher assistant.Preprint, arXiv:1902.03393. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov

    Pith/arXiv arXiv 1902

  15. [2022]

    Woogyeol Jin, Taywon Min, Yongjin Yang, Swanand Ravindra Kadhe, Yi Zhou, Dennis Wei, Nathalie Baracaldo, and Kimin Lee

    Knowledge distillation from a stronger teacher.Preprint, arXiv:2205.10536. Woogyeol Jin, Taywon Min, Yongjin Yang, Swanand Ravindra Kadhe, Yi Zhou, Dennis Wei, Nathalie Baracaldo, and Kimin Lee

    Pith/arXiv arXiv

  16. [2024]

    Boxin Wang, Chankyu Lee, Nayeon Lee, Sheng- Chieh Lin, Wenliang Dai, Yang Chen, Yangyi Chen, Zhuolin Yang, Zihan Liu, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping

    Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models.Preprint, arXiv:2402.03300. Boxin Wang, Chankyu Lee, Nayeon Lee, Sheng- Chieh Lin, Wenliang Dai, Yang Chen, Yangyi Chen, Zhuolin Yang, Zihan Liu, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping

    Pith/arXiv arXiv

  17. [2025]

    InFind- ings of the Association for Computational Linguis- tics: ACL 2025, pages 25366–25394, Vienna, Austria

    Small models struggle to learn from strong reasoners. InFind- ings of the Association for Computational Linguis- tics: ACL 2025, pages 25366–25394, Vienna, Austria. Association for Computational Linguistics. Zhaoyi Li, Xiangyu Xi, Zhengyu Chen, Wei Wang, Gangwei Jiang, Ranran Shen, Linqi Song, Ying Wei, and Defu Lian. 2026b. On the role of reasoning patte...

    Pith/arXiv arXiv 2025

  18. [2026]

    Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang

    Glm-5: from vibe coding to agentic engineering.Preprint, arXiv:2602.15763. Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang

    Pith/arXiv arXiv