REVIEW 3 major objections 5 minor 31 references
Mismatch Matters: On-Policy Distillation Beyond Token Agreement
T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read The paper argues that token-level agreement in on-policy distillation can mask degenerate loops, and that routing supervision to severe teacher–student mismatch with separate excess and deficit corrections turns that failure into a…
desk verdict A real failure mode with a clean two-branch fix; the teacher-reliability caveat is the one thing a referee should push on. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the pairing of two quantile-gated corrections on student-visited states. The excess branch's Hellinger shaping $h(a)=2(e^{a/2}-1)$ is bounded in $(-2,0)$ for $a<0$, satisfies $h(a)=a+O(a^2)$ near agreement, and yields $E_{v\sim q}[-h(a(v))\nabla_\theta\log q(v|s)]=4\nabla_\theta H^2(p,q)$, so it converts the unstable reverse-KL tail into the exact gradient of a proper divergence. The deficit branch's score $d_t=D_{KL}(\bar p_t\Vert q^K_t)+(-\log Q_t)$ decomposes into probability misallocation inside the teacher's top-$K$ plus the student's total missing mass on that set, which is why the correction is a direct cross-entropy to $\bar p_t$ rather than a sampled advantage. Matched positions receive zero weight, and the two gates are independent, so a position can be updated by neither, one, or both branches.
What would settle it
Build a dataset of naturally occurring repetitive student prefixes and check, at loop states, whether the teacher's top-16 support contains any non-repetitive exit token. The paper reports the teacher continues the loop in 93% of greedy continuations; if the same pattern holds for the top-16 candidates — that is, every candidate at loop states is a loop continuation — then TIDE's deficit branch cannot recover exits and its length and accuracy gains should disappear, confirming that the method depends on the teacher remaining locally reliable.
Extended reading notes
Core claim
The paper's central claim is that informative supervision in OPD lives in directional teacher–student mismatch rather than agreement. Standard OPD minimizes a trajectory-level reverse KL through sampled token advantages $a_t=\log[p(o_t|s_t)/q_\theta(o_t|s_t)]$; this objective is zero-forcing, so it penalizes student-excess tokens harshly while offering almost no pressure on teacher-preferred tokens that the student rarely samples. The paper proposes TIDE, which gates positions by mismatch severity in two independent branches: the excess branch applies $h(a)=2(e^{a/2}-1)$ to negative advantages, a bounded monotone transform that is first-order equal to the log-ratio near agreement and, before selection, equals the gradient of the squared Hellinger divergence ($4\nabla_\theta H^2(p,q)$); the deficit branch scores each state by $d_t=\sum_{v\in K_t}\bar p_t(v)\log[\bar p_t(v)/q_\theta(v|s_t)]$ and distills the teacher's renormalized top-$K$ distribution at the worst states, bypassing sampling. The paper reports this combination raises Avg@8 from 6.9% to 20.3% under strong mismatch, reduces the average response length from 22,395 to 7,294 tokens, and cuts responses without a boxed answer from 65.5% to 5.4%.
Load-bearing premise
The teacher's next-token opinions stay trustworthy even when the student's own output has become repetitive or degenerate.
Editorial extensions
If this is right
- OPD pipelines should allocate supervision by the direction and accessibility of disagreement, not by teacher–student overlap.
- Bounded Hellinger shaping can replace clipping or power transforms for negative token advantages while keeping the update grounded in a divergence gradient.
- Teacher-preferred tokens that are rarely sampled can be recovered analytically from a teacher top-$K$ readout, removing the need to wait for a lucky rollout.
- Under strong teacher–student mismatch, concentrating on mismatched tokens with these two corrections yields both higher accuracy and shorter, better-formatted responses than standard OPD.
- Matched tokens can be dropped from the distillation loss without sacrificing accuracy, since supervising only the most mismatched 20% of positions more than doubles Avg@8 relative to matched-only supervision.
Reading between the lines
- Going beyond the paper, the locally-reliable-teacher assumption could be converted into an adaptive gate: monitor teacher entropy or loop-continuation rate on student prefixes and disable or down-weight the deficit branch when the teacher's top-$K$ itself collapses into repetition.
- A testable extension is to use the deficit score as a teacher-health diagnostic: if $d_t$ stays large at loop states but every top-$K$ candidate is a loop token, the bottleneck is teacher-side, and a trustworthy system should stop trusting that teacher's supervision at those states.
- The two-branch design is not tied to mathematics: code, dialogue, and multilingual post-training also exhibit repetitive loops, so TIDE's excess suppression and top-$K$ recovery could plausibly transfer wherever a locally reliable teacher is available.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper identifies a failure mode of on-policy distillation that it calls degenerate agreement: students learn to produce repetitive loops that achieve near-perfect token-level agreement with the teacher while the overall response is globally degenerate. The authors propose a token-level method, TIDE, which separates teacher-student mismatch into student-excess and student-deficit directions. For student-excess tokens, it applies a bounded Hellinger-shaped correction; for student-deficit tokens, it uses an analytic teacher top-K objective that does not require the deficit token to be sampled. The paper proves two propositions: one that the Hellinger transformation is bounded, locally faithful to the log-ratio, and equals the gradient of the squared Hellinger divergence before position selection, and one that the deficit score decomposes into within-top-K KL mismatch plus a coverage gap. Experiments on nine mathematical reasoning benchmarks with two teacher-student pairs report that TIDE consistently outperforms OPD and several baselines, especially under strong mismatch, with large reductions in response length and formatting failures.
Significance. If the empirical claims hold, TIDE is a valuable contribution to on-policy distillation: it provides a clear conceptual decomposition of mismatch into excess and deficit, offers a principled bounded correction for the unstable log-ratio, and addresses the previously neglected coverage problem with analytic top-K guidance. The two propositions are clean, correctly proved, and give the method a sound theoretical core. The paper also ships code and reports detailed ablations that isolate the contributions of selection, excess suppression, and deficit recovery. However, the central empirical claim of consistent gains rests on two load-bearing points that are not fully established: the reliability of the teacher target at the states selected by the deficit branch, and the statistical stability of the reported single-run accuracy numbers.
major comments (3)
- [Section 5 / Sec. C.1 / Eq. (4)] The deficit branch of TIDE uses the teacher's top-K distribution as the training target at states selected by the deficit score d_t (Eq. 4), yet the paper's own continuation experiments (Sec. C.1, Fig. 6) show that after a repetitive student prefix the Qwen3-8B teacher continues the same loop in 93% of cases, and Fig. 4c shows teacher entropy collapses with repetition. The gate d_t only measures how well the student covers the teacher's top-K, not whether the teacher's top-K is a reliable target at that state, so if the student ever visits a corrupted state with large d_t, the deficit branch would explicitly reinforce the degenerate continuation. The paper states in Section 5 that TIDE assumes a locally reliable teacher, but it provides no direct measurement of teacher top-K quality at the states actually selected by the deficit gate; this is a load-bearing empirical precondition for the central claim of consistent gains under strong mismatch. I request a diagnostic (e.g., teacher top-K accuracy or entropy at deficit-selected states, or an analysis of how often deficit-selected states are within repetitive loops) and, if the precondition fails, an explicit safeguard.
- [Tables 1–5, Section 4.2] All accuracy and generation numbers are from single runs with no standard errors, confidence intervals, or multiple seeds, although the abstract and Section 4.2 claim that TIDE "consistently outperforms" baselines. Some differences are small (e.g., Table 1 weak mismatch: TIDE 46.7 vs FiRe-OPD 46.6 and AOPD 46.2), and without variance estimates the cross-pair and per-benchmark comparisons are difficult to evaluate. Please provide at least three seeds for the main comparison, or if compute constraints are binding, bootstrap confidence intervals over the nine benchmarks and a clear statement of which differences are statistically meaningful.
- [Section 3.1, Eq. (2), Algorithm 1] Proposition 1's unbiased gradient identity (Eq. 2) is derived for the expectation over a token sampled from the student before any position selection, but the actual excess loss in Algorithm 1 applies a quantile mask m^-_t and token-mean normalization over the batch. The resulting update is therefore not the exact gradient of the squared Hellinger divergence, and the statement in Section 3.1 that Hellinger shaping "remains the exact gradient of a proper divergence" is only true for the unselected expectation. Please clarify the scope of Proposition 1 in the text, or, if the divergence interpretation is claimed for the full gated objective, provide the corresponding derivation with the selection mask.
minor comments (5)
- [Section 3.1] The sentence "Accordingly, we take the following the policy gradient in OPD" has a grammar error and should read "Accordingly, we use the following policy gradient in OPD."
- [Section 2.2] The phrase "repeatingI hope it is correct149 times" is missing spaces around the quoted phrase and the number; please fix the formatting.
- [Section 4.3] The sentence "Table 4 and Fig. 5 shows that" should be "Table 4 and Fig. 5 show that" for subject-verb agreement.
- [Table 5] The column header "Dist.-4" is abbreviated, while the text and Appendix C use "Distinct-4"; please make the notation consistent throughout the paper.
- [Appendix A.2] In the proof of Proposition 2, the display for the decomposition uses an unusual symbol in the KL term; standard notation such as D_KL(\bar p_t \parallel q^K_t) would improve readability.
Circularity Check
No significant circularity: the theoretical propositions are algebraic consequences of the paper's own definitions, and the empirical claims are evaluated on held-out benchmarks.
full rationale
The paper's derivation chain is self-contained and does not reduce to its own inputs. Proposition 1 defines h(a)=2(e^{a/2}-1), proves -2<h(a)<0 for a<0 from the softmax full-support assumption, and derives E_v~q[-h(a(v)) grad log q(v)] = 4 grad H^2(p,q) by direct calculation from the definition H^2(p,q)=1-sum_v sqrt(p(v)q(v)). This is a mathematical property of the chosen shaping function, not a fitted or predicted result. Proposition 2 decomposes the deficit score d_t = sum_{v in K_t} bar p_t(v) log(bar p_t(v)/q_theta(v|s_t)) into D_KL(bar p_t || q^K_t) + (-log Q_t) by substituting q(v)=Q_t q^K_t(v) on the teacher top-K support; this is an algebraic identity following from the definitions. Neither proposition imports a result from self-citations or from the benchmark data. The central empirical claims—consistent Avg@8 gains over OPD and baselines, the strong-mismatch improvement from 6.9% to 20.3%, the response-length reduction, and the deficit-only and matched-only ablations—are measured with a fixed decoding and grading pipeline on held-out math problems, and hyperparameters were fixed before benchmark evaluation (Appendix D.5). The deficit branch does use the teacher's top-K as a training target, but its success is an externally evaluated outcome rather than an artifact of the loss definition. The stated limitation that TIDE assumes a locally reliable teacher is an honest precondition for the method's empirical success, not a circular step. There are no load-bearing self-citations or imported uniqueness theorems; references to prior work are used for context and baselines. Overall, no derivation in the paper is equivalent to its inputs by construction.
Assumptions & free parameters
free parameters (4)
- Deficit recovery weight λ =
1.0
- Excess keep-rate ρ− =
0.2
- Deficit keep-rate ρ+ =
0.2
- Teacher top-K support size K =
16
assumptions (3)
- domain assumption Teacher distributions p(·|s) are locally reliable at student-visited states
- standard math Softmax full-support and autoregressive factorization of teacher and student
- domain assumption On-policy sampling gives an unbiased estimate of the reverse-KL objective
Cite this review
Pith. "Pith review of Mismatch Matters: On-Policy Distillation Beyond Token Agreement." pith.science (2026). https://pith.science/paper/DRMGGMGC
@misc{pith2026260809836,
author = {Pith},
title = {Pith review of: Mismatch Matters: On-Policy Distillation Beyond Token Agreement},
year = {2026},
howpublished = {\url{https://pith.science/paper/DRMGGMGC}},
note = {Machine review of arXiv:2608.09836}
}
abstract
On-policy distillation (OPD) has emerged as a core component of modern LLM post-training pipelines, yet we reveal a failure mode: degenerate agreement, where students exploit repetitive loops to achieve near-perfect token agreement with the teacher despite globally flawed responses. We therefore shift our focus from agreement to teacher-student mismatch, and find that mismatch tokens can be mainly categorized into two types: student-excess tokens and student-deficit tokens. Student-excess tokens are generated by the student but assigned near-zero probability by the teacher; their log-ratio corrections grow unbounded and destabilize the update. Student-deficit tokens, in contrast, are preferred by the teacher but rarely sampled by the student; their absence blocks the transfer of the teacher's reasoning patterns. To tackle these mismatch directions, we propose TIDE (Token-level Independent Deficit-Excess correction), which applies bounded Hellinger shaping to suppress the most severe sampled excesses and an analytic teacher top-$K$ injection to restore deficient probability mass without requiring deficit tokens to be sampled. Across mathematical reasoning benchmarks with multiple Qwen3 teacher-student pairs, TIDE consistently outperforms standard OPD and recent token-selection and reward-shaping baselines. Moreover, the gains of TIDE are more pronounced under strong teacher-student mismatch, where it improves Avg@8 from 6.9% to 20.3%, reduces average response length by a factor of 3.6, and substantially reduces formatting failures. Code is available at https://github.com/yzc-666/TIDE
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
On-policy distillation of language models: Learning from self- generated mistakes
Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from self- generated mistakes. InInternational Conference on Learning Representations, volume 2024, pp. 21246–21263,
work page 2024
-
[2]
Thus, the value ofωat which resonance occurs is2
Resonance occurs when the frequency of the external force matches the natural frequency. Thus, the value ofωat which resonance occurs is2 . The failure here is structural rather than lexical: the OPD studentdoesderive the correct answer ω= 2early in the rollout, but instead of terminating it imitates the few-shot format of the prompt and generates an endl...
work page 2024
-
[4]
Nan Jia, Haojin Yang, Xing Ma, Jiesong Lian, Shuailiang Zhang, Weipeng Zhang, Ke Zeng, Xun- liang Cai, and Zequn Sun. Asymmetric on-policy distillation: Bridging exploitation and imitation at the token level.arXiv preprint arXiv:2605.06387,
-
[6]
Rethinking on-policy self-distillation for thinking models
Simran Kaur, Narutatsu Ri, Yinghui He, Liam H Fowl, and Sanjeev Arora. Rethinking on-policy self-distillation for thinking models. InWorkshop on Failure Modes of Agentic AI at ICML 2026,
work page 2026
-
[7]
Junlong Ke, Zichen Wen, Weijia Li, Conghui He, and Linfeng Zhang. Respecting self-uncertainty in on-policy self-distillation for efficient llm reasoning.arXiv preprint arXiv:2605.13255,
-
[8]
Scaling reasoning efficiently via relaxed on-policy distillation.arXiv preprint arXiv:2603.11137,
Jongwoo Ko, Sara Abdali, Young Jin Kim, Tianyi Chen, and Pashmina Cameron. Scaling reasoning efficiently via relaxed on-policy distillation.arXiv preprint arXiv:2603.11137,
-
[9]
Yaxuan Li, Yuxin Zuo, Bingxiang He, Jinqian Zhang, Chaojun Xiao, Cheng Qian, Tianyu Yu, Huan- ang Gao, Wenkai Yang, Zhiyuan Liu, et al. Rethinking on-policy distillation of large language models: Phenomenology, mechanism, and recipe.arXiv preprint arXiv:2604.13016, 2026a. Yuying Li, Leqi Zheng, Yongzi Yu, Wenrui Zhou, Xuchang Zhong, Xing Hu, Jing Jin, Han...
-
[10]
Xinyu Liu, Kechen Jiao, Chunyang Xiao, Runsong Zhao, Junhao Ruan, Bei Li, Jiahao Liu, Qifan Wang, Xin Chen, Jingang Wang, et al. Teacher-guided policy optimization for on-policy reasoning distillation under large policy divergence.arXiv preprint arXiv:2605.13230, 2026a. Yanjiang Liu, Jie Lou, Xinyan Guan, Yuqiu Ji, Hongyu Lin, Ben He, Xianpei Han, Le Sun,...
Show all 31 references
-
[11]
https://thinkingmachines.ai/blog/on-policy-distillation
doi: 10.64434/tml.20251026. https://thinkingmachines.ai/blog/on-policy-distillation. 13 Preprint Feng Luo, Yu-Neng Chuang, Guanchu Wang, Zicheng Xu, Xiaotian Han, Tianyi Zhang, and Vladimir Braverman. Demystifying opd: Length inflation and stabilization strategies for large la...
-
[15]
Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256,
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256,
-
[16]
Kimi k3: Open frontier intelligence.arXiv preprint arXiv:2607.24653,
Kimi Team, Tongtong Bai, Yifan Bai, Yiping Bao, Jianfeng Cai, Xinyuan Cai, Peizhou Cao, Yux- uan Cao, Ziwei Chai, Y Charles, et al. Kimi k3: Open frontier intelligence.arXiv preprint arXiv:2607.24653,
-
[17]
Not all disagreement is learnable: Token teachability in on-policy distillation.arXiv preprint arXiv:2605.26844,
Yuanyi Wang, Su Lu, Yanggan Gu, Pengkai Wang, Yifan Yang, Zhaoyi Yan, Congkai Xie, Jian- min Wu, and Hongxia Yang. Not all disagreement is learnable: Token teachability in on-policy distillation.arXiv preprint arXiv:2605.26844,
-
[18]
Mimo-v2-flash technical report.arXiv preprint arXiv:2601.02780,
Bangjun Xiao, Bingquan Xia, Bo Yang, Bofei Gao, Bowen Shen, Chen Zhang, Chenhong He, Chiheng Lou, Fuli Luo, Gang Wang, et al. Mimo-v2-flash technical report.arXiv preprint arXiv:2601.02780,
-
[19]
Escaping the kl agreement trap in on-policy distillation.arXiv preprint arXiv:2606.09471,
Haoran Xin, Anhao Zhao, Ying Sun, Jin Li, Xiaoyu Shen, and Hui Xiong. Escaping the kl agreement trap in on-policy distillation.arXiv preprint arXiv:2606.09471,
-
[20]
Trust region on-policy distillation.arXiv preprint arXiv:2606.01249,
Xingrun Xing, Haoqing Wang, Boyan Gao, Ziheng Li, and Yehui Tang. Trust region on-policy distillation.arXiv preprint arXiv:2606.01249,
-
[21]
Tip: Token importance in on-policy distillation.arXiv preprint arXiv:2604.14084,
Yuanda Xu, Hejian Sang, Zhengze Zhou, Ran He, Zhipeng Wang, and Alborz Geramifard. Tip: Token importance in on-policy distillation.arXiv preprint arXiv:2604.14084,
-
[22]
Qwen3 technical report.arXiv preprint arXiv:2505.09388,
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388,
-
[23]
Oprd: On-policy representation distilla- tion.arXiv preprint arXiv:2606.06021, 2026a
Shenzhi Yang, Guangcheng Zhu, Bowen Song, Haobo Wang, Mingxuan Xia, Xing Zheng, Yingfan Ma, Zhongqi Chen, Weiqiang Wang, Junbo Zhao, et al. Oprd: On-policy representation distilla- tion.arXiv preprint arXiv:2606.06021, 2026a. Zhicheng Yang, Zhijiang Guo, Yifan Song, Minrui Xu,...
-
[24]
Glm-5: from vibe coding to agentic engineering.arXiv preprint arXiv:2602.15763,
Aohan Zeng, Xin Lv, Zhenyu Hou, Zhengxiao Du, Qinkai Zheng, Bin Chen, Da Yin, Chendi Ge, Chenghua Huang, Chengxing Xie, et al. Glm-5: from vibe coding to agentic engineering.arXiv preprint arXiv:2602.15763,
-
[25]
Fast and effective on-policy distillation from reasoning prefixes
14 Preprint Dongxu Zhang, Zhichao Yang, Sepehr Janghorbani, Jun Han, Andrew Ressler II, Qian Qian, Gre- gory D Lyng, Sanjit Singh Batra, and Robert E Tillman. Fast and effective on-policy distillation from reasoning prefixes. InFindings of the Association for Computational Lin...
2026
-
[26]
Poweropd: Stabiliz- ing on-policy distillation with bounded power transformation.arXiv preprint arXiv:2606.17199, 2026a
Anhao Zhao, Junlong Tong, Yingqi Fan, Ping Nie, Wenjie Li, and Xiaoyu Shen. Poweropd: Stabiliz- ing on-policy distillation with bounded power transformation.arXiv preprint arXiv:2606.17199, 2026a. Qingfei Zhao, Huan Song, Shuyu Tian, Jiawei Shao, and Xuelong Li. Prefix-guided ...
-
[28]
Using P v ∇θq(v) = 0, we obtain Ev∼q [−h(a(v))∇θ logq(v)] =−2 X v s p(v) q(v) −1 ! ∇θq(v) =−2 X v s p(v) q(v) ∇θq(v) = 4∇θH 2(p, q), which proves Equation (2)
By the definition ofH 2, ∇θH 2(p, q) =−1 2 X v∈V s p(v) q(v) ∇θq(v). Using P v ∇θq(v) = 0, we obtain Ev∼q [−h(a(v))∇θ logq(v)] =−2 X v s p(v) q(v) −1 ! ∇θq(v) =−2 X v s p(v) q(v) ∇θq(v) = 4∇θH 2(p, q), which proves Equation (2). A.2 Proof of Proposition 2 Proof.By the definiti...
2026
-
[30]
Findxy. 24 Preprint OPD Student0/8 correct·Distinct-4= 0.009 ANGUAGE 2023-03-15T10:30:00Z ENGINEER 2023-03-15T10:30:00Z SECOND ENGINEER 2023-03-15T10:30:00Z ENGINEER 2023-03-15T10:30:00Z ENGINEER 2023-03-15T10:30:00Z ENGINEER 2023-03-15T10:30:00Z [... the same line repeats 1,3...
2023
-
[31]
错” (“wrong
Conclusion.The value ofxyis25 . Unlike Cases 1 and 2, the student never engages with the problem: every rollout locks into a short periodic loop of problem-irrelevant tokens within the first few tokens and terminates only at the decoding cap. E.4 Case 4: Token Soup Collapsing ...
2023
-
[2005]
Kl for a kl: On-policy distil- lation with control variate baseline.arXiv preprint arXiv:2605.07865,
Minjae Oh, Sangjun Song, Gyubin Choi, Yunho Choi, and Yohan Jo. Kl for a kl: On-policy distil- lation with control variate baseline.arXiv preprint arXiv:2605.07865,
-
[2011]
Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347,
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347,
-
[2017]
Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models.arXiv preprint arXiv:2402.03300,
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models.arXiv preprint arXiv:2402.03300,
-
[2023]
The many faces of on-policy distillation: Pitfalls, mechanisms, and fixes.arXiv preprint arXiv:2605.11182,
Siqi Zhu, Xuyan Ye, Hongyu Lu, Weiye Shi, and Ge Liu. The many faces of on-policy distillation: Pitfalls, mechanisms, and fixes.arXiv preprint arXiv:2605.11182,
-
[2024]
Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948,
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948,
-
[2025]
Justrl: Scaling a 1.5 b llm with a simple rl recipe.arXiv preprint arXiv:2512.16649,
Bingxiang He, Zekai Qu, Zeyuan Liu, Yinghao Chen, Yuxin Zuo, Cheng Qian, Kaiyan Zhang, Weize Chen, Chaojun Xiao, Ganqu Cui, et al. Justrl: Scaling a 1.5 b llm with a simple rl recipe.arXiv preprint arXiv:2512.16649,
-
[2026]
Trajectory-refined distillation.arXiv preprint arXiv:2606.08432,
Li Jiang, Haoran Xu, Yichuan Ding, and Amy Zhang. Trajectory-refined distillation.arXiv preprint arXiv:2606.08432,
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.