Pith. sign in

REVIEW 3 major objections 4 minor 67 references

On-Policy Self-Distillation without Any Supervision

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read On-policy self-distillation can work with no ground-truth labels: the model's own majority-vote consensus serves as the teacher, and the label-free variant beats supervised OPSD by up to 3.2% on five math benchmarks.

desk verdict U-OPSD is a genuinely unsupervised distillation recipe with a real idea, but the headline margins over supervised OPSD rest on best-checkpoint selection and no seeds; the claim is plausible, not yet secure. read the letter →

arxiv 2608.06296 v2 pith:ZJYZJGWD submitted 2026-08-06 cs.LG

Yijiang Li , Bingyang Wang , Yijun Liang , Yunjie Tian , Di Fu , Nuno Vasconcelos This is my paper · ORCID
classification cs.LG
keywords on-policyself-distillationunsupervisedpost-trainingself-consistencymajorityvoteknowledgedistillationmathematicalreasoninglargelanguagemodelslabel-freetraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that on-policy self-distillation of a large language model does not need any ground-truth solution, external feedback, or a stronger teacher. The proposed method, U-OPSD, samples several rollouts per problem, takes the majority-voted final answer as a pseudo-solution when a confidence threshold is met, and then distills the model's own solution-conditioned next-token distribution into the student along the rollouts that disagreed with the vote. On five competition-math benchmarks this label-free procedure improved Qwen3-4B and Qwen3-8B by 8.5% and 10.7% over the base model in non-thinking mode, and outperformed the supervised OPSD baseline by 3.2% and 2.3%. The resulting picture is that the binding constraint for this kind of self-improvement is not access to gold solutions but the machinery for surfacing and correcting the model's own inconsistencies.

What carries the argument

The load-bearing mechanism is consensus as an endogenous privileged context. U-OPSD samples $G$ rollouts from a stop-gradient copy of the policy, parses their final answers, and lets a majority vote fix a pseudo-answer; when the winning fraction crosses $\tau = 1/2$ and at least one rollout disagrees, the longest agreeing rollout $y^+$ replaces the ground-truth solution as the teacher reference. The loss is a per-token divergence $D_\beta(\bar{\pi}(\cdot | x, y^+, y^-_{<n}) \| \pi_\theta(\cdot | x, y^-_{<n}))$ summed over the prefixes of the disagreeing rollouts, where $D_\beta$ is the generalized Jensen-Shannon divergence whose $\beta \to 0$ limit is the forward KL divergence. This turns self-consistency into dense corrective supervision at precisely the positions where the model's own samples contradict its consensus.

What would settle it

Measure the gold-agreement of majority-vote pseudo-labels on a labeled holdout from the training distribution; if agreement falls well below the 86.7% reported on the 64-prompt probe, final accuracy should stop improving and eventually fall below the base model, which would falsify the claim that consensus alone can carry the distillation.

Watch

Extended reading notes

Core claim

The central claim is that the ground-truth solution in on-policy self-distillation can be replaced by the model's own consensus, and that this replacement is not a degradation but often an improvement. Concretely, U-OPSD draws $G$ independent rollouts from a stop-gradient copy of the policy, extracts and canonicalizes their final answers, and takes the plurality answer $\tilde{a}(x)$ as the pseudo-answer. If the winning fraction passes threshold $\tau = 1/2$ and at least one rollout disagrees, the longest agreeing rollout $y^+$ is used as the teacher reference in place of $y^\star$, and the model is trained to match the teacher's next-token distribution, conditioned on $(x, y^+)$, along prefixes of the disagreeing rollouts. The same checkpoint plays teacher and student, so no external signal enters; the paper reports that this unsupervised variant matches or surpasses supervised OPSD, SFT, and GRPO across five math benchmarks in both thinking and non-thinking modes.

Load-bearing premise

The method assumes that a majority vote over the model's own rollouts produces a pseudo-solution that is correct often enough (the paper's 64-prompt probe found 86.7% agreement with gold answers) to serve as a teacher reference, and that this accuracy holds on the training distribution.

Editorial extensions

If this is right

  • Post-training no longer requires labeled solutions: any unlabeled problem set with extractable answers can supply both the teacher reference and the student trajectories.
  • Consensus-derived pseudo-solutions are more effective as a conditioning context for token-level distillation than as a scalar reward: label-free RL baselines under the same budget gained at most 1.5%, while U-OPSD gained 8.5-10.7% in non-thinking mode.
  • Improvement concentrates on the competence frontier: prompts with reliable votes but residual disagreement receive the correction, so the curriculum is set by the model's own vote statistics.
  • In strong thinking-mode models the gains shrink to a tie with supervised OPSD, indicating ceiling effects rather than a failure of the unsupervised signal.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • As an editorial extension, the vote margin could be converted from a binary threshold into a per-prompt confidence weight, so that low-margin consensus contributes less gradient; this is directly testable with the existing code.
  • Soft consensus, for example embedding similarity instead of exact-match answers, would extend the same machinery to open-ended generation, where canonicalizable final answers are unavailable.
  • The reported 86.7% pseudo-label gold-agreement suggests a practical diagnostic: check pseudo-label accuracy on a small labeled holdout before training, since the method's ceiling is bounded by that accuracy.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes U-OPSD, an unsupervised variant of on-policy self-distillation that removes all external supervision from the teacher context. For each unlabeled prompt, the method draws G rollouts from a stop-gradient copy of the policy, majority-votes the parsed answers to form a pseudo-answer, selects the longest agreeing rollout as the pseudo-solution, and then minimizes a token-level forward-KL divergence between the teacher distribution conditioned on the pseudo-solution and the student distribution along the disagreeing rollouts. Experiments on five mathematical reasoning benchmarks with Qwen3-4B and Qwen3-8B in both thinking and non-thinking modes, plus two instruct models, report consistent gains over the base models and average gains over supervised OPSD of 3.2% and 2.3% in non-thinking mode, while roughly tying OPSD in thinking mode. The paper also reports ablations over the confidence threshold, rollout count, teacher update strategy, teacher reference choice, distillation target, and divergence computation strategy. Section 5 explicitly acknowledges the regime dependence of the gains, the dependence on base-model competence, and that seed-replicated error bars are pending.

Significance. If the empirical claims hold, U-OPSD is a meaningful conceptual advance: it shows that the privileged context in on-policy self-distillation need not be a ground-truth solution and that the model's own consensus can serve as the teacher reference. The method is simple, the objective in Eq. (5) is clear, and the comparisons against supervised OPSD, GRPO, SFT, and label-free RL baselines address the right counterfactuals. The paper also has real strengths in presentation: it states its limitations candidly, reports the pseudo-label agreement rate, and will release code. The self-referential training signal is not circular, because the central claim is evaluated on external benchmarks with known answers. However, the headline margins over OPSD rest on an evaluation protocol that selects the best checkpoint without seed replication, and the compute comparison between U-OPSD and OPSD is not fully specified; these issues must be resolved before the central claim is secure.

major comments (3)
  1. [Section 4.1 / Tables 1-2] The main comparison uses an upward-biased checkpoint-selection protocol without error bars. Section 4.1 states that checkpoints are evaluated every 25 steps up to 150 and the best score is reported for U-OPSD and OPSD, while Section 5 explicitly states that seed-replicated error bars are pending. Taking the maximum over six correlated checkpoint evaluations inflates expected performance, and the inflation is method-specific when learning curves have different shapes; Figure 3 suggests U-OPSD is still climbing on AIME24 in non-thinking mode while other methods plateau. The headline claims of 3.2% and 2.3% average gains over OPSD in Table 1, as well as the ablation conclusions in Figure 4, cannot currently be distinguished from run-to-run noise. Please provide at least three seeds for the main U-OPSD and OPSD comparisons and report a preset checkpoint or a last-k average as the primary protocol, with best-checkpoint results retained only as a secondary analysis.
  2. [Section 4.3 / Section 5] The mechanism's ceiling is measured on only 64 training prompts. The pseudo-label quality probe reports 86.7% agreement with gold answers and 13.3% wrong pseudo-labels; this is the load-bearing quantity for the claim that consensus can replace gold supervision, yet no error bar or breakdown is given, and the manuscript does not report how pseudo-label accuracy evolves during training or across the five evaluation distributions. Section 5 appropriately acknowledges the dependence on base-model competence, but the main tables would be materially stronger if pseudo-label accuracy were reported per benchmark and per checkpoint. Without this, the reader cannot tell whether the reported gains are driven by the intended self-correction mechanism or by a few unrepresentative high-consistency prompts.
  3. [Section 4.1 / Implementation details] The cost-matched comparison between U-OPSD and OPSD is not fully specified. The text says U-OPSD uses G=8 rollouts per prompt with a maximum completion length of 4,096 tokens, while supervised OPSD draws a single rollout from each of 32 prompts per optimizer step with a maximum length of 1,024 tokens, but it does not state how many prompts U-OPSD uses per optimizer step. If U-OPSD is allowed more prompts, more tokens, or more total sampling per step, part of its advantage could be attributable to compute rather than to consensus-based conditioning. Please report prompts per step, total sampled tokens per step, and wall-clock time for both methods, and verify the claim that Tables 1 and 2 are cost-matched.
minor comments (4)
  1. [Section 4.1 / Thinking mode] The sentence 'we keep the teacher in thinking mode and distill its behavior into a student in non-thinking mode, while evaluating the resulting model in thinking mode' appears contradictory; please clarify whether the student is trained in thinking mode, non-thinking mode, or both, and how the evaluation mode is chosen.
  2. [Algorithm 1 / Eq. (5)] The algorithm and the objective do not specify how the agreeing reference y+ and the subset B^-_x of disagreeing rollouts are selected; the experiments later refer to 'longest-1' and Figure 5 explores longest/random/shortest. Please state the default selection rule directly in Algorithm 1 and in the text around Eq. (5).
  3. [Table 5 / Section 3.1] The divergence family D_beta is introduced as a generalized Jensen-Shannon divergence whose beta -> 0 limit is forward KL, but Table 5 labels beta=0.5 as JSD and beta=1 as reverse KL without explaining the mapping; please add a sentence defining the beta convention used in the table.
  4. [Figure 3] The horizontal axis in Figure 3 runs from 25 to 100, while the text says checkpoints are evaluated every 25 steps up to 150; please make the axis range and the number of plotted checkpoints consistent with the protocol.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: self-referential training signal is evaluated externally against gold-answer benchmarks; no fitted prediction or load-bearing self-citation.

full rationale

U-OPSD's training signal is self-referential by design, since the teacher reference is a majority vote over the model's own rollouts, but the paper's claims are not circular. The central result, that U-OPSD improves over the base model and matches or exceeds supervised OPSD, is measured on five external math benchmarks with gold answers, so a model that merely reaffirmed its own pseudo-labels would not score well on those held-out evaluations. No parameter is fitted to a subset of the reported benchmarks and then presented as a prediction; hyperparameters such as the confidence threshold tau, rollout count G, teacher update rule, and reference/target selection are swept and reported as ablations rather than as the headline claim. The paper itself explicitly flags the two main caveats in Section 5: majority-vote supervision is bounded by the base model's own answer accuracy, and seed-replicated error bars are pending. The best-checkpoint evaluation protocol described in Section 4.1 is a statistical robustness concern about possible inflation of margins, not a construction that makes the reported accuracy equivalent to the training input. No load-bearing step depends on a self-citation by the present authors: the OPSD recipe is external prior work with released code, and the paper states that all OPSD numbers come from its own rerun under identical conditions. I find no circular step.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The method introduces no new physical entities. The central machinery rests on the empirical validity of majority-vote pseudo-labels, reliable answer extraction, and transfer from the OpenThoughts prompts. The main free parameters, G and tau, are shown in ablations to change outcomes by several points, and the default settings are not the best found.

free parameters (3)
  • G, number of rollouts per prompt = 8 (default)
    Sampling budget chosen to match OPSD's generation budget. Ablation shows G=12 gives +4.7% over the default and G=16 gives back half of that; main results use G=8.
  • tau, self-consistency threshold = 0.5 (default)
    Absolute majority chosen by hand. Ablation shows tau=0.3 is best (58.59 vs 57.10 for the default), while tau=0.9 collapses to 44.40, below the base model. The choice materially affects the result.
  • Teacher update strategy = fixed (initial policy)
    Chosen to match OPSD's setting. Ablation shows EMA 0.995 gains 2.4% over the fixed default at the best checkpoint, so the main tables use a suboptimal teacher update by the paper's own evidence.
assumptions (3)
  • domain assumption Majority vote over independent rollouts yields a pseudo-solution accurate enough for distillation
    Core of the method. Empirically on 64 prompts, 86.7% of pseudo-labels match gold answers, but this is a small probe and the paper acknowledges the accuracy bound in Section 5.
  • domain assumption Answer extraction Ans(.) reliably parses a canonical final answer from rollouts
    Needed to partition rollouts into agreeing and disagreeing sets. The paper notes that under reverse KL the parsable fraction falls from 99% to 33%, showing that the partitioning is fragile when generation degrades.
  • domain assumption The OpenThoughts 30k prompt subset is representative of the evaluation benchmark distribution
    Training on these unlabeled prompts transfers to AIME, HMMT, MATH500, and AMC; no analysis of domain shift is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On-Policy Self-Distillation without Any Supervision." pith.science (2026). https://pith.science/paper/ZJYZJGWD

@misc{pith2026260806296,
  author       = {Pith},
  title        = {Pith review of: On-Policy Self-Distillation without Any Supervision},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZJYZJGWD}},
  note         = {Machine review of arXiv:2608.06296}
}
read the original abstract

On-policy (Self-)Distillation (OPD / OPSD) has shown strong potential for post-training large language models (LLMs). However, existing methods still rely heavily on external supervision, including ground-truth signals, environmental feedback, or guidance from larger models, and therefore fall short of genuine "self"-distillation. In this study, we show that on-policy self-distillation can be achieved using only a model's own generations via internal consistency. We propose unsupervised on-policy self-distillation (U-OPSD). U-OPSD first samples multiple rollouts and constructs a pseudo solution by majority vote under a self-consistency threshold. It then conditions the model's distribution on the pseudo-solution and distills itself on the disagreeing completions, allowing the model to correct itself precisely where it is confidently wrong. Across diverse benchmarks, base models, and training settings, U-OPSD consistently improves over the base models and matches or surpasses supervised methods with ground truth (GT) such as OPSD and GRPO. On five mathematical reasoning benchmarks, i.e., AIME24, AIME25, HMMT25, MATH500, and AMC23, U-OPSD improves over the base model by 8.5% and 10.7% on Qwen3 non-thinking mode at 4B and 8B scales, and outperforms OPSD by 3.2% and 2.3% on average, respectively. In thinking mode, U-OPSD stays on par with OPSD, ahead by 0.9% at 4B and level at 8B and surpassing GRPO by 0.7% and 1.1%, respectively. Code is available at [https://github.com/williamium3000/u-opsd](https://github.com/williamium3000/u-opsd).

Figures

Figures reproduced from arXiv: 2608.06296 by the authors.

Figure 1
Figure 1. Comparison between OPSD / SDFT with ground-truth solution or ICLs (left), SDPO with [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 1
Figure 1. Comparison between OPSD / SDFT with ground-truth solution or ICLs (left), SDPO with [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Overview of Unsupervised On-policy Self-Distillation ( [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figures from the paper (5 more)
Figure 2
Figure 2. Figure 2: Overview of Unsupervised On-policy Self-Distillation ( [PITH_FULL_IMAGE:figures/full_fig_p005_2.png]
Figure 3
Figure 3. Figure 3: Training curves of the Qwen3-4B runs quoted in Tables 1 and 2, on AIME24, AIME25 and [PITH_FULL_IMAGE:figures/full_fig_p009_3.png]
Figure 4
Figure 4. Figure 4: Configuration ablations, on Qwen3-8B non-thinking with longest-1. The vertical axis [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 4
Figure 4. Figure 4: Configuration ablations, on Qwen3-8B non-thinking. The vertical axis is accuracy averaged [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Teacher reference against distillation target, on Qwen3-8B non-thinking, G=8, τ=0.5, k=1; each point is the five-benchmark average. (a) What the teacher is conditioned on. (b) Which disagreeing rollout is distilled [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

67 extracted references · 25 canonical work pages

  1. [1]

    Diverse Cotraining Makes Strong Semi-Supervised Segmentor , year=

    Li, Yijiang and Wang, Xinjiang and Yang, Lihe and Feng, Litong and Zhang, Wayne and Gao, Ying , booktitle=. Diverse Cotraining Makes Strong Semi-Supervised Segmentor , year=

    work page

  2. [2]

    2025 IEEE/CVF International Conference on Computer Vision (ICCV) , pages=

    Videoorion: Tokenizing object dynamics in videos , author=. 2025 IEEE/CVF International Conference on Computer Vision (ICCV) , pages=. 2025 , organization=

    work page 2025

  3. [3]

    International Conference on Learning Representations , volume=

    From pixels to tokens: Byte-pair encoding on quantized visual modalities , author=. International Conference on Learning Representations , volume=

    work page

  4. [4]

    2025 IEEE/CVF International Conference on Computer Vision (ICCV) , pages=

    Unified multimodal understanding via byte-pair visual encoding , author=. 2025 IEEE/CVF International Conference on Computer Vision (ICCV) , pages=. 2025 , organization=

    work page 2025

  5. [5]

    arXiv e-prints , pages=

    Less is enough: synthesizing diverse data in feature space of LLMs , author=. arXiv e-prints , pages=

    work page

  6. [6]

    Vision Language Models Cannot Reason About Physical Transformation

    Vision language models cannot reason about physical transformation , author=. arXiv preprint arXiv:2603.07109 , year=

    work page Pith review arXiv

  7. [7]

    Contextualized Evaluation of Vision Language Models through Dynamic, Multi-turn Interactions

    Contextualized Evaluation of Vision Language Models through Dynamic, Multi-turn Interactions , author=. arXiv preprint arXiv:2607.14499 , year=

    work page Pith review arXiv

  8. [8]

    arXiv preprint arXiv:2603.22281 , year=

    Thinkjepa: Empowering latent world models with large vision-language reasoning model , author=. arXiv preprint arXiv:2603.22281 , year=

    arXiv

Show all 67 references
  1. [9]

    arXiv preprint arXiv:2601.18734 , year=

    Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models , author=. arXiv preprint arXiv:2601.18734 , year=

    arXiv

  2. [10]

    Guo, Daya and Yang, Dejian and Zhang, Haowei and Song, Junxiao and Wang, Peiyi and Zhu, Qihao and Xu, Runxin and Zhang, Ruoyu and Ma, Shirong and Bi, Xiao and Zhang, Xiaokang and Yu, Xingkai and Wu, Yu and Wu, Z. F. and Gou, Zhibin and Shao, Zhihong and Li, Zhuoshu and Gao, Zi...

  3. [11]

    The Eleventh International Conference on Learning Representations , year=

    Self-Consistency Improves Chain of Thought Reasoning in Language Models , author=. The Eleventh International Conference on Learning Representations , year=

  4. [12]

    2025 , url=

    Yuxin Zuo and Kaiyan Zhang and Li Sheng and Shang Qu and Ganqu Cui and Xuekai Zhu and Haozhan Li and Yuchen Zhang and Xinwei Long and Ermo Hua and Biqing Qi and Youbang Sun and Zhiyuan Ma and Lifan Yuan and Ning Ding and Bowen Zhou , booktitle=. 2025 , url=

    2025

  5. [13]

    Proceedings of the 41st International Conference on Machine Learning , articleno =

    Yuan, Weizhe and Pang, Richard Yuanzhe and Cho, Kyunghyun and Li, Xian and Sukhbaatar, Sainbayar and Xu, Jing and Weston, Jason , title =. Proceedings of the 41st International Conference on Machine Learning , articleno =. 2024 , publisher =

    2024

  6. [14]

    Proceedings of the 41st International Conference on Machine Learning , articleno =

    Chen, Zixiang and Deng, Yihe and Yuan, Huizhuo and Ji, Kaixuan and Gu, Quanquan , title =. Proceedings of the 41st International Conference on Machine Learning , articleno =. 2024 , publisher =

    2024

  7. [15]

    2025 , eprint=

    Absolute Zero: Reinforced Self-play Reasoning with Zero Data , author=. 2025 , eprint=

    2025

  8. [16]

    2025 , eprint=

    Right Question is Already Half the Answer: Fully Unsupervised LLM Reasoning Incentivization , author=. 2025 , eprint=

    2025

  9. [17]

    2025 , eprint=

    Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Models , author=. 2025 , eprint=

    2025

  10. [18]

    The Fourteenth International Conference on Learning Representations , year=

    Learning to Reason without External Rewards , author=. The Fourteenth International Conference on Learning Representations , year=

  11. [19]

    arXiv preprint arXiv:2505.22660 , year=

    Maximizing Confidence Alone Improves Reasoning , author=. arXiv preprint arXiv:2505.22660 , year=

    arXiv

  12. [20]

    Does Reinforcement Learning Really Incentivize Reasoning Capacity in

    Yang Yue and Zhiqi Chen and Rui Lu and Andrew Zhao and Zhaokai Wang and Yang Yue and Shiji Song and Gao Huang , booktitle=. Does Reinforcement Learning Really Incentivize Reasoning Capacity in. 2025 , url=

    2025

  13. [21]

    2025 , eprint=

    Qwen3 Technical Report , author=. 2025 , eprint=

    2025

  14. [22]

    Measuring Mathematical Problem Solving With the

    Dan Hendrycks and Collin Burns and Saurav Kadavath and Akul Arora and Steven Basart and Eric Tang and Dawn Song and Jacob Steinhardt , booktitle=. Measuring Mathematical Problem Solving With the. 2021 , url=

    2021

  15. [23]

    Let's Verify Step by Step , url =

    Lightman, Hunter and Kosaraju, Vineet and Burda, Yuri and Edwards, Harrison and Baker, Bowen and Lee, Teddy and Leike, Jan and Schulman, John and Sutskever, Ilya and Cobbe, Karl , booktitle =. Let's Verify Step by Step , url =

  16. [24]

    2022 , eprint=

    STaR: Bootstrapping Reasoning With Reasoning , author=. 2022 , eprint=

    2022

  17. [25]

    2023 , eprint=

    Scaling Relationship on Learning Mathematical Reasoning with Large Language Models , author=. 2023 , eprint=

    2023

  18. [26]

    2026 , eprint=

    Self-Distilled RLVR , author=. 2026 , eprint=

    2026

  19. [27]

    2026 , eprint=

    On-Policy Context Distillation for Language Models , author=. 2026 , eprint=

    2026

  20. [28]

    2026 , eprint=

    Privileged Information Distillation for Language Models , author=. 2026 , eprint=

    2026

  21. [29]

    2026 , eprint=

    CRISP: Compressed Reasoning via Iterative Self-Policy Distillation , author=. 2026 , eprint=

    2026

  22. [30]

    2026 , eprint=

    HERO: Hindsight-Enhanced Reflection from Environment Observations for Agentic Self-Distillation , author=. 2026 , eprint=

    2026

  23. [31]

    2023 , eprint=

    Reinforced Self-Training (ReST) for Language Modeling , author=. 2023 , eprint=

    2023

  24. [32]

    Large Language Models Can Self-Improve

    Huang, Jiaxin and Gu, Shixiang and Hou, Le and Wu, Yuexin and Wang, Xuezhi and Yu, Hongkun and Han, Jiawei. Large Language Models Can Self-Improve. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023. doi:10.18653/v1/2023.emnlp-main.67

    2023 doi

  25. [33]

    2025 , eprint=

    Self-rewarding correction for mathematical reasoning , author=. 2025 , eprint=

    2025

  26. [34]

    2026 , eprint=

    Self-Evolving Visual Questioner , author=. 2026 , eprint=

    2026

  27. [35]

    Yuxian Gu and Li Dong and Furu Wei and Minlie Huang , booktitle=. Mini. 2024 , url=

    2024

  28. [36]

    The Twelfth International Conference on Learning Representations , year=

    On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes , author=. The Twelfth International Conference on Learning Representations , year=

  29. [37]

    ArXiv , year=

    DistiLLM: Towards Streamlined Distillation for Large Language Models , author=. ArXiv , year=

  30. [38]

    2025 , editor =

    Ko, Jongwoo and Chen, Tianyi and Kim, Sungnyun and Ding, Tianyu and Liang, Luming and Zharkov, Ilya and Yun, Se-Young , booktitle =. 2025 , editor =

    2025

  31. [39]

    ArXiv , year=

    Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes , author=. ArXiv , year=

  32. [40]

    2026 , eprint=

    Lightning OPD: Efficient Post-Training for Large Reasoning Models with Offline On-Policy Distillation , author=. 2026 , eprint=

    2026

  33. [41]

    2025 , eprint=

    OpenThoughts: Data Recipes for Reasoning Models , author=. 2025 , eprint=

    2025

  34. [42]

    Thinking Machines Lab: Connectionism , year =

    Kevin Lu and Thinking Machines Lab , title =. Thinking Machines Lab: Connectionism , year =

  35. [43]

    2026 , eprint=

    Demystifying On-Policy Distillation: Roles, Pathologies, and Regulations , author=. 2026 , eprint=

    2026

  36. [44]

    Zhihong Shao and Peiyi Wang and Qihao Zhu and Runxin Xu and Junxiao Song and Mingchuan Zhang and Y. K. Li and Y. Wu and Daya Guo , title=. CoRR , volume=. 2024 , cdate=

    2024

  37. [45]

    2026 , url=

    Bo Liu and Simon Yu and Zichen Liu and Leon Guertler and Penghui Qi and Daniel Balcells and Mickel Liu and Cheston Tan and Weiyan Shi and Min Lin and Wee Sun Lee and Natasha Jaques , booktitle=. 2026 , url=

    2026

  38. [46]

    arXiv preprint arXiv:2508.05004 , year=

    R-Zero: Self-Evolving Reasoning LLM from Zero Data , author=. arXiv preprint arXiv:2508.05004 , year=

    arXiv

  39. [47]

    Genius: A Generalizable and Purely Unsupervised Self-Training Framework For Advanced Reasoning

    Xu, Fangzhi and Yan, Hang and Ma, Chang and Zhao, Haiteng and Sun, Qiushi and Cheng, Kanzhi and He, Junxian and Liu, Jun and Wu, Zhiyong. Genius: A Generalizable and Purely Unsupervised Self-Training Framework For Advanced Reasoning. Proceedings of the 63rd Annual Meeting of t...

    2025 doi

  40. [48]

    Wenkai Fang and Shunyu Liu and Yang Zhou and Kongcheng Zhang and Tongya Zheng and Kaixuan Chen and Mingli Song and Dacheng Tao , booktitle=. Se. 2025 , url=

    2025

  41. [49]

    Forty-second International Conference on Machine Learning , year=

    Self-Consistency Preference Optimization , author=. Forty-second International Conference on Machine Learning , year=

  42. [50]

    2025 , eprint=

    Can Large Reasoning Models Self-Train? , author=. 2025 , eprint=

    2025

  43. [51]

    2026 , eprint=

    Self-Supervised On-Policy Distillation for Reasoning Language Models , author=. 2026 , eprint=

    2026

  44. [52]

    2026 , eprint=

    Respecting Self-Uncertainty in On-Policy Self-Distillation for Efficient LLM Reasoning , author=. 2026 , eprint=

    2026

  45. [53]

    2025 , eprint=

    SDRT: Enhance Vision-Language Models by Self-Distillation with Diverse Reasoning Traces , author=. 2025 , eprint=

    2025

  46. [54]

    Aditya Prakash and Josiah Hester and Jindong Wang and Srijan Kumar , booktitle=

    Yiqiao Jin and Yiyang Wang and Lucheng Fu and Yijia Xiao and Yinyi Luo and Haoxin Liu and B. Aditya Prakash and Josiah Hester and Jindong Wang and Srijan Kumar , booktitle=. Uni. 2026 , url=

    2026

  47. [55]

    2026 , eprint=

    When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoning , author=. 2026 , eprint=

    2026

  48. [56]

    Proceedings of the 35th International Conference on Machine Learning , pages =

    Born Again Neural Networks , author =. Proceedings of the 35th International Conference on Machine Learning , pages =. 2018 , editor =

    2018

  49. [57]

    2018 , eprint=

    Snapshot Distillation: Teacher-Student Optimization in One Generation , author=. 2018 , eprint=

    2018

  50. [58]

    Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation , year=

    Zhang, Linfeng and Song, Jiebo and Gao, Anni and Chen, Jingwei and Bao, Chenglong and Ma, Kaisheng , booktitle=. Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation , year=

  51. [59]

    Emerging Properties in Self-Supervised Vision Transformers , year=

    Caron, Mathilde and Touvron, Hugo and Misra, Ishan and Jegou, Hervé and Mairal, Julien and Bojanowski, Piotr and Joulin, Armand , booktitle=. Emerging Properties in Self-Supervised Vision Transformers , year=

  52. [60]

    Forty-third International Conference on Machine Learning , year=

    Self-Distillation Enables Continual Learning , author=. Forty-third International Conference on Machine Learning , year=

  53. [61]

    Forty-third International Conference on Machine Learning , year=

    Reinforcement Learning via Self-Distillation , author=. Forty-third International Conference on Machine Learning , year=

  54. [62]

    2026 , url=

    Idan Shenfeld and Jyothish Pari and Pulkit Agrawal , booktitle=. 2026 , url=

    2026

  55. [63]

    2025 , url=

    Yixin Ye and Zhen Huang and Yang Xiao and Ethan Chern and Shijie Xia and Pengfei Liu , booktitle=. 2025 , url=

    2025

  56. [64]

    Light-R1: Curriculum SFT , DPO and RL for Long COT from Scratch and Beyond

    Wen, Liang and Cai, Yunke and Xiao, Fenrui and He, Xin and An, Qi and Duan, Zhenyu and Du, Yimin and Liu, Junchen and Tang, Lifu and Lv, Xiaowei and Zou, Haosheng and Deng, Yongchao and Jia, Shousheng and Zhang, Xiangzheng. Light-R1: Curriculum SFT , DPO and RL for Long COT fr...

    2025 doi

  57. [65]

    2025 , eprint=

    Phi-4-reasoning Technical Report , author=. 2025 , eprint=

    2025

  58. [66]

    2025 , eprint=

    MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention , author=. 2025 , eprint=

    2025

  59. [67]

    2025 , eprint=

    GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models , author=. 2025 , eprint=

    2025

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.