Pith. sign in

REVIEW 3 major objections 6 minor 48 references

Geometric self-distillation lifts OOD reasoning by 5.7–8.6 points

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

2026-07-10 00:08 UTC pith:ZRZN4VAB

load-bearing objection GeoSD combines Hellinger overlap-weighting, Fisher-Rao proximal regularization, and K-FAC natural gradients for privileged-context self-distillation. The empirical results are strong and the mechanistic analysis is mostly convincing, but the stress-test concern about misattributed mechanisms is partially valid. the 3 major comments →

arxiv 2607.06855 v1 pith:ZRZN4VAB submitted 2026-07-07 cs.LG cs.CL

Geometric Self-Distillation for Reasoning Generalization

Josip Juki\'c , Ivan Titov This is my paper
classification cs.LG cs.CL
keywords studentteachermodelself-distillationdriftgeosdreasoningsame
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that the out-of-distribution degradation in privileged-context self-distillation is a problem of movement magnitude, not signal selection. When a teacher sees the solution and the student does not, standard divergences pull hardest at the mismatched states — exactly where the teacher's confidence is least justified from the student's own context. Over many updates these pulls compound into drift: the student concentrates probability mass onto the teacher's top choice by draining alternatives, producing confident agreement on wrong answers. GeoSD counters this in a single geometry of next-token distributions, which under the square-root embedding live as points on a hypersphere. A Hellinger loss weights each teacher preference by the geometric-mean overlap between teacher and student, so unsupported teacher confidence exerts no pull. A Fisher-Rao proximal term bounds cumulative drift from a recent checkpoint. A natural-gradient update takes steps in this geometry rather than in parameter space. The student then absorbs privileged supervision in proportion to its own readiness, preserving alternatives rather than collapsing onto a single mode.

Core claim

The failure mode of standard self-distillation — confident agreement on wrong answers out of distribution — traces to a single mechanism: standard divergences concentrate probability mass prematurely at high-entropy states by draining alternatives, and this local concentration propagates downstream into false consensus. The fix is not to select better teacher signals but to regulate how far each signal moves the student, measured in the geometry where next-token distributions live as points on a hypersphere. Near agreement all divergences coincide; they differ only in the low-overlap regime, which is precisely where privileged-context self-distillation operates.

What carries the argument

The square-root embedding maps next-token distributions onto a unit hypersphere, where the Hellinger divergence is the chord distance (bounding per-state teacher pull by teacher-student overlap) and the Fisher-Rao distance is the arc length (tracking cumulative drift from a checkpoint). Both are second-order equivalent to KL near agreement but diverge in the low-overlap regime that governs whether privileged supervision helps or harms.

Load-bearing premise

The method assumes the student's recent predictive behavior is worth preserving: the proximal term penalizes deviations from a recent checkpoint, and overlap weighting suppresses teacher signal where the student assigns little probability. This is the right prior when the base model is broadly competent and high-entropy states reflect healthy uncertainty. But if the base model is weak and those high-entropy states reflect ignorance, the same mechanism that suppresses harmful漂

What would settle it

If one could show that, for a weak base model where high-entropy states reflect ignorance rather than healthy uncertainty, GeoSD's overlap weighting suppresses useful learning signal to the point that the student cannot acquire new capabilities it lacks, then the geometric regulation of movement magnitude would be revealed as a prior about model competence rather than a universal principle of distillation.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If movement magnitude rather than signal selection is the right axis, then filtering and reweighting methods that decide which teacher signals to trust are addressing a secondary concern; the primary intervention should be geometric.
  • The overlap-weighting principle should extend to any asymmetric distillation where teacher confidence is context-dependent, including cross-model distillation where a stronger teacher's confidence may also be unjustified from the student's view.
  • The finding that all divergences coincide near agreement but differ in the low-overlap regime means the choice of divergence is a statement about behavior away from agreement, which may have implications for loss function selection in any distillation setup.
  • The false-consensus diagnostic — confident agreement on wrong answers — is a generalizable tool for detecting premature mode collapse in any generative model post-training, not just self-distillation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The principle that supervision should be absorbed in proportion to the student's readiness rather than the teacher's confidence may apply broadly: in curriculum learning, in human feedback, and in any setting where a supervisor's information is richer than the learner's. The geometric formulation gives a precise way to operationalize readiness as distributional overlap.
  • The moving-anchor approach implies an optimal schedule for checkpoint refresh that depends on the rate of useful learning versus harmful drift — too frequent refreshes would freeze the model, too infrequent ones would allow drift to accumulate. The paper fixes this at 64 steps but does not explore the schedule.
  • Hellinger's position at the self-dual midpoint of the alpha-divergence family suggests that treating teacher and student symmetrically may be the deeper reason for its effectiveness, beyond the specific overlap-weighting of the gradient.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces GeoSD, a geometric self-distillation objective for post-training LLMs in the privileged-context on-policy setting. The method has two components: (1) a Hellinger loss whose gradient weights each teacher preference by the geometric-mean overlap between teacher and student, attenuating pull on tokens the student does not yet support; and (2) a Fisher–Rao proximal term penalizing predictive drift from a recent checkpoint. Both are formulated as distances in the square-root embedding geometry of next-token distributions, and the combined objective is optimized via a K-FAC natural-gradient update. Experiments across three model families (Qwen3-8B, Olmo-3-7B-Think, DS-R1-Llama-8B) and five Qwen3 scales (1.7B–32B) show that GeoSD preserves in-distribution gains while improving average OOD accuracy by 5.7–8.6 points over the base model. A mechanistic analysis (§4) attributes the OOD gains to Hellinger overlap-weighting preventing 'rank-1 collapse' at high-entropy states, which standard KL matching suffers from.

Significance. The paper addresses a practically important problem: privileged-context self-distillation degrades OOD reasoning because the teacher's privileged view produces confidence the student cannot justify. The information-geometry framework (Appendix A) is cleanly developed: the square-root embedding, the equivalence of divergences near agreement (§A.3), and the Hellinger gradient form (Eq. 4, §A.5) are all correct and well-presented. The experimental design is thorough—matched compute budgets, ten seeds with significance tests (Table 6), per-benchmark breakdowns (Table 5), and ablations isolating each component (Table 2). The compute overhead analysis (Appendix D) is transparent and practical. The identification of 'false consensus' (§4.2) as a failure mode is a useful diagnostic. The method ships a working, falsifiable recipe with modest overhead (1.10× wall-clock).

major comments (3)
  1. §4, Figures 6–7: The mechanistic analysis in §4 attributes the prevention of rank-1 collapse and false consensus specifically to Hellinger overlap-weighting, but the experiments compare full GeoSD against FwdKL, conflating two interventions. Table 2 ablations show the Fisher–Rao proximal term contributes 6.7 OOD points (60.5→53.8 when removed) while the Hellinger-vs-JSD swap contributes 4.4 points (60.5→56.1). Since the proximal term penalizes drift from a checkpoint, it would mechanically slow concentration regardless of the loss function. The paper never tests whether the gentler rank-1 growth in Figure 6 persists under Hellinger-without-FR, or whether FwdKL-with-FR also prevents rank-1 collapse. If FwdKL+FR+K-FAC shows similar gentler concentration and lower false-consensus rates, the emphasis on overlap-weighting as the key mechanism (contribution iii) is misattributed. This is load-
  2. §4.1, Figure 6: The claim that GeoSD 'keeps alternatives in reach' is supported only visually. No quantitative metric is reported for rank-1 mass concentration or alternative retention across methods and checkpoints. A simple scalar (e.g., mean rank-1 mass at selected positions, or entropy of the student distribution at high-entropy states) tabulated for FwdKL, GeoSD, and the ablation variants would substantially strengthen the mechanistic claim. As presented, the ridge plots are suggestive but not dispositive.
  3. §2.3, Eq. 5 and Appendix E: The proximal term anchors to a recent checkpoint refreshed every K_ckpt=64 steps. The paper acknowledges (Appendix E) that 'the objective does not prevent the anchor and student from drifting together over much longer horizons.' However, the main text does not discuss this limitation, and Figure 2B shows drift plateauing over 625 steps—readers may infer the drift is permanently controlled. A brief note in §2.3 or §3 on the horizon-dependence of the anchor would improve honesty without weakening the contribution.
minor comments (6)
  1. §2.2, Eq. 3: The expectation over c∼C(·|x) is introduced but C is not formally defined until §2.1. Forward-referencing is minor but could confuse on first read.
  2. Table 1: The ΔOOD column for π0 shows '–' for all three families. Consider showing 0.0 for consistency, or adding a footnote explaining the convention.
  3. Figure 1A: The notation Δ_i ∝ √(p_i q_i) ∇_θ log p_i uses p_i for the student and q_i for the teacher, but the caption text refers to 'teacher–student overlap' without specifying order. Minor, but specifying would help.
  4. Appendix B.5: The hyperparameter sweep ranges are reported, but the selection criterion (validation avg@8) is mentioned only briefly. A one-sentence description of the validation protocol would improve reproducibility.
  5. §3.1: The paper states 'At 14B and 32B scale, we use a more memory-efficient K-FAC' but the distinction between g=16 (≤8B) and g=32 (14B/32B) in Table 9 is not explained in the main text. A brief note on why the block size changes would help.
  6. References: Several citations are to 2026 arXiv preprints (Zhao et al., 2026; Ye et al., 2026; Kim et al., 2026; etc.). Ensure these are consistently formatted and that DOIs/URLs are included where available.

Circularity Check

0 steps flagged

No circularity: the derivation chain is self-contained, grounded in standard information geometry, and evaluated against external benchmarks.

full rationale

The paper's derivation chain proceeds through standard, independently verifiable steps. (1) The Hellinger loss gradient (Eq. 4) is a straightforward calculus result from differentiating the squared Hellinger divergence (Eq. 1) — no self-citation or fitted input is involved. (2) The Fisher–Rao distance (Eq. 2) and the sphere embedding (Appendix A.2) are standard results from information geometry, citing Amari (2016) and Amari & Nagaoka (2000), which are external references with no author overlap. (3) The near-agreement equivalence of divergences (Appendix A.3) is a Taylor expansion with no circular dependency. (4) The natural-gradient update (Eq. 7) and K-FAC preconditioning cite Martens & Grosse (2015), an external reference. (5) The checkpoint pullback derivation (Appendix A.6) shows the proximal term reduces to an L2 penalty as a special case — a standard mathematical reduction, not a self-referential one. (6) All empirical claims are evaluated on external benchmarks (AIME, AMC, MATH-500) disjoint from training data, with hyperparameters tuned on a separate validation set. The mechanistic analysis in §4 compares FwdKL against full GeoSD, which the skeptic correctly notes conflates interventions — but this is a concern about experimental design and causal attribution, not about circularity in the derivation chain. No step in the paper's mathematical or empirical argument reduces to its own inputs by construction.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

The method introduces no new entities or postulated objects. All mathematical objects (Hellinger distance, Fisher-Rao distance, Fisher information, K-FAC) are standard. The free parameters are hyperparameters tuned on a separate validation set, not constants of nature. The domain assumptions are clearly stated in Appendix E as limitations.

free parameters (6)
  • lambda (proximal weight) = 1.0
    Swept over {0.1, 1.0, 3.0}, selected on validation set
  • K_ckpt (checkpoint refresh interval) = 64
    Swept over {32, 64}, selected on validation set
  • gamma (K-FAC damping) = 1e-3
    Swept jointly with learning rate
  • eta (learning rate) = 1e-6
    Swept over {1e-7, 1e-6, 1e-5}
  • K (top-K logit truncation) = 1024
    Applied uniformly to all losses
  • g (K-FAC block size) = 16 or 32
    16 for <=8B models, 32 for 14B/32B
axioms (5)
  • standard math Fisher information is the canonical Riemannian metric on a statistical family (Cencov's theorem)
    Invoked in Appendix A.1 to justify the geometric framework
  • standard math Next-token distributions under square-root embedding live on a unit hypersphere
    Appendix A.2, standard information geometry result
  • domain assumption The student's recent predictive behavior is worth preserving
    Appendix E: the proximal term and overlap weighting encode this prior; appropriate when base policy is competent but may fail for weak base models
  • domain assumption High-entropy states in the base model reflect healthy uncertainty rather than ignorance
    Section 4.1 and Appendix E: the retention of alternatives at these states is assumed beneficial, which holds for competent base models on math reasoning
  • domain assumption On-policy trajectories sampled from the student provide a valid training distribution
    Section 2.1: standard OPD assumption, gradients do not propagate through sampling distribution

pith-pipeline@v1.1.0-glm · 27932 in / 2568 out tokens · 287555 ms · 2026-07-10T00:08:38.222007+00:00 · methodology

0 comments
read the original abstract

On-policy distillation is a practical post-training recipe for large language models, supplying dense teacher supervision on the student's own trajectories. In privileged-context self-distillation, teacher and student are the same model conditioned on the same prefix, but the teacher also sees a hint or the full solution trace. This makes supervision abundant but harder to trust: the teacher can be confident about continuations its privileged view makes obvious but the student cannot yet justify. The distillation pull is strongest where teacher and student disagree most, and over many updates it accumulates into drift that degrades out-of-distribution (OOD) reasoning. We introduce GeoSD, a geometric self-distillation objective that treats this drift as movement in the student's predictive behavior and counters it in two complementary ways. A Hellinger loss scales each teacher preference by the overlap the student already shares with it, attenuating the pull on tokens the student cannot yet support. Since these pulls still compound over training, a proximal term penalizes how far the student's predictions drift from a recent checkpoint, measured as a Fisher-Rao distance. Both are distances in the same geometry of next-token distributions, and a natural-gradient update takes its steps in that geometry rather than in parameter space. Across mathematical reasoning benchmarks and three model families, GeoSD preserves the in-distribution gains of self-distillation while improving average OOD accuracy by 5.7-8.6 points over the base model, with gains holding across model scales from 1.7B to 32B. Analyzing why standard matching fails out of distribution, we find it wins agreement with the teacher by draining mass from alternatives at high-entropy states, resulting in confident agreement on wrong answers, whereas GeoSD keeps those alternatives in reach.

Figures

Figures reproduced from arXiv: 2607.06855 by Ivan Titov, Josip Juki\'c.

Figure 1
Figure 1. Figure 1: GEOSD regulates distillation in the geometry of predictive distributions. (A) The teacher’s pull scales with teacher–student overlap, growing with agreement. (B) Distributions lie on a sphere, where the Hellinger chord bounds each pull and the Fisher–Rao arc tracks drift from a checkpoint. the student commits to the teacher’s continuations before its own information can support them, and the mismatch accum… view at source ↗
Figure 2
Figure 2. Figure 2: Local and global effects in empirical privileged-context OPSD (Qwen3-8B with full-solution [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: In-distribution vs. out-of-distribution gains (percentage points relative to π0) for every method across the three model families; dot size encodes OOD pass@16. of which only the K-FAC state (≈8%) resides on the training workers in the default placement. Wall-clock time grows by only 1.10×, since the checkpoint forward overlaps with rollout generation, which dominates the run and is shared across all metho… view at source ↗
Figure 5
Figure 5. Figure 5: Scale. ID and OOD accuracy (avg@16) across the Qwen3 family (1.7B–32B) for the base model and GEOSD [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: The shape of teacher–student alignment. Student next-token mass at high-entropy decision points (top 10% by π0 entropy) for Qwen3-8B. At each position, tokens are sorted by the teacher’s preference rank; each ridge shows the student’s mean mass over these ranks. 1 2 4 8 16 32 samples k 50 60 70 O O D pass@k (%) A. Exploration payoff base FwdKL RevKL GRPO TrOPD GeoSD base FwdKL RevKL JSD SkewKL TrOPD TIP GR… view at source ↗
Figure 7
Figure 7. Figure 7: From local concentration to false consensus. (A) OOD pass@k versus number of samples k: whether sampling still recovers correct solutions. (B) False-consensus rate: how often samples agree strongly (C(x) ≥ 0.75) on a wrong answer. (C) Majority-answer accuracy versus consensus strength C(x): whether stronger agreement signals a more reliable answer. the one most samples agree on, and the consensus strength … view at source ↗
Figure 8
Figure 8. Figure 8: Teacher–student overlap density. We plot the teacher’s top-1 mass against the student’s mass on the same token. Density in the lower-right region identifies states where privileged context makes the teacher confident in a continuation that the student assigns little probability. B.6 KRONECKER FACTORIZATION GEOSD preconditions with K-FAC (Martens & Grosse, 2015), which approximates each layer’s Fisher block… view at source ↗
Figure 9
Figure 9. Figure 9: Step size and learning rate. (Qwen3-8B, avg@16). Each point is one configuration on the ID–OOD plane: solid lines sweep the learning rate (labels at each point), dashed lines trace early stopping, following training checkpoints at η=10−6 [PITH_FULL_IMAGE:figures/full_fig_p022_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

48 extracted references · 48 canonical work pages · 13 internal anchors

  1. [1]

    Scaling Learning Algorithms Towards

    Bengio, Yoshua and LeCun, Yann , booktitle =. Scaling Learning Algorithms Towards

    work page

  2. [2]

    and Osindero, Simon and Teh, Yee Whye , journal =

    Hinton, Geoffrey E. and Osindero, Simon and Teh, Yee Whye , journal =. A Fast Learning Algorithm for Deep Belief Nets , volume =

    work page

  3. [3]

    Distilling the Knowledge in a Neural Network

    Distilling the knowledge in a neural network , author=. arXiv preprint arXiv:1503.02531 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  4. [4]

    2016 , publisher=

    Deep learning , author=. 2016 , publisher=

    work page 2016

  5. [5]

    Guo, Daya and Yang, Dejian and Zhang, Haowei and Song, Junxiao and Wang, Peiyi and Zhu, Qihao and Xu, Runxin and Zhang, Ruoyu and Ma, Shirong and Bi, Xiao and Zhang, Xiaokang and Yu, Xingkai and Wu, Yu and Wu, Z. F. and Gou, Zhibin and Shao, Zhihong and Li, Zhuoshu and Gao, Ziyi and Liu, Aixin and Xue, Bing and Wang, Bingxuan and Wu, Bochao and Feng, Bei ...

    work page doi:10.1038/s41586-025-09422-z

  6. [6]

    Yuxian Gu and Li Dong and Furu Wei and Minlie Huang , booktitle=. Mini. 2024 , url=

    work page 2024

  7. [7]

    Jongwoo Ko and Tianyi Chen and Sungnyun Kim and Tianyu Ding and Luming Liang and Ilya Zharkov and Se-Young Yun , booktitle=. Disti. 2025 , url=

    work page 2025

  8. [8]

    Jongwoo Ko and Sungnyun Kim and Tianyi Chen and Se-Young Yun , booktitle=. Disti. 2024 , url=

    work page 2024

  9. [9]

    Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?

    Jeonghye Kim and Xufang Luo and Minbeom Kim and Sangmook Lee and Dohyung Kim and Jiwon Jeon and Dongsheng Li and Yuqing Yang , year=. Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of. 2603.24472 , archivePrefix=

    work page internal anchor Pith review Pith/arXiv arXiv

  10. [10]

    2016 , publisher=

    Information geometry and its applications , author=. 2016 , publisher=

    work page 2016

  11. [11]

    1982 , publisher=

    Statistical decision rules and optimal inference , author=. 1982 , publisher=

    work page 1982

  12. [12]

    Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe

    Rethinking on-policy distillation of large language models: Phenomenology, mechanism, and recipe , author=. arXiv preprint arXiv:2604.13016 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  13. [13]

    The Twelfth International Conference on Learning Representations , year=

    On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes , author=. The Twelfth International Conference on Learning Representations , year=

    work page

  14. [14]

    A Survey of On-Policy Distillation for Large Language Models

    A survey of on-policy distillation for large language models , author=. arXiv preprint arXiv:2604.00626 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  15. [15]

    Proceedings of the 32nd International Conference on Machine Learning , pages =

    Optimizing Neural Networks with Kronecker-factored Approximate Curvature , author =. Proceedings of the 32nd International Conference on Machine Learning , pages =. 2015 , editor =

    work page 2015

  16. [16]

    Trust-Region Behavior Blending for On-Policy Distillation

    Trust-Region Behavior Blending for On-Policy Distillation , author=. arXiv preprint arXiv:2605.31159 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  17. [17]

    Extreme Region Policy Distillation

    Extreme Region Policy Distillation , author=. arXiv preprint arXiv:2605.25582 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  18. [18]

    2026 , eprint=

    Trust Region On-Policy Distillation , author=. 2026 , eprint=

    work page 2026

  19. [19]

    Learning to Compress Prompts with Gist Tokens , url =

    Mu, Jesse and Li, Xiang and Goodman, Noah , booktitle =. Learning to Compress Prompts with Gist Tokens , url =

    work page

  20. [20]

    CODI : Compressing Chain-of-Thought into Continuous Space via Self-Distillation

    Shen, Zhenyi and Yan, Hanqi and Zhang, Linhai and Hu, Zhanghao and Du, Yali and He, Yulan. CODI : Compressing Chain-of-Thought into Continuous Space via Self-Distillation. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025. doi:10.18653/v1/2025.emnlp-main.36

    work page doi:10.18653/v1/2025.emnlp-main.36 2025

  21. [21]

    Beyond Autoregression: Fast

    Justin Deschenaux and Caglar Gulcehre , booktitle=. Beyond Autoregression: Fast. 2025 , url=

    work page 2025

  22. [22]

    Sang, Hejian and Xu, Yuanda and Zhou, Zhengze and He, Ran and Wang, Zhipeng and Sun, Jiachen , journal=

    work page

  23. [23]

    arXiv preprint arXiv:2602.01395 , year=

    Rethinking Selective Knowledge Distillation , author=. arXiv preprint arXiv:2602.01395 , year=

    work page arXiv

  24. [24]

    2026 , eprint=

    Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models , author=. 2026 , eprint=

    work page 2026

  25. [25]

    On-Policy Context Distillation for Language Models

    On-policy context distillation for language models , author=. arXiv preprint arXiv:2602.12275 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  26. [26]

    Reinforcement Learning via Self-Distillation

    Reinforcement Learning via Self-Distillation , author=. arXiv preprint arXiv:2601.20802 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  27. [27]

    Privileged Information Distillation for Language Models

    Privileged Information Distillation for Language Models , author=. arXiv preprint arXiv:2602.04942 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  28. [28]

    2026 , url=

    Stein, Alex and Huang, Furong and Goldstein, Tom , journal=. 2026 , url=

    work page 2026

  29. [29]

    2026 , url=

    Xu, Yuanda and Sang, Hejian and Zhou, Zhengze and He, Ran and Wang, Zhipeng and Geramifard, Alborz , journal=. 2026 , url=

    work page 2026

  30. [30]

    Self-Distilled Policy Gradient

    Self-Distilled Policy Gradient , author=. arXiv preprint arXiv:2606.04036 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  31. [31]

    Self-distilled

    Yang, Chenxu and Qin, Chuanyu and Si, Qingyi and Chen, Minghui and Gu, Naibin and Yao, Dingyu and Lin, Zheng and Wang, Weiping and Wang, Jiaqi and Duan, Nan , journal=. Self-distilled. 2026 , url=

    work page 2026

  32. [32]

    arXiv , primaryClass=

    Trajectory-Refined Distillation , author=. arXiv , primaryClass=. 2026 , eprint=

    work page 2026

  33. [33]

    arXiv , primaryClass=

    On the Geometry of On-Policy Distillation , author=. arXiv , primaryClass=. 2026 , eprint=

    work page 2026

  34. [34]

    The Twelfth International Conference on Learning Representations , year=

    Let's Verify Step by Step , author=. The Twelfth International Conference on Learning Representations , year=

    work page

  35. [35]

    2026 , url=

    Qiying Yu and Zheng Zhang and Ruofei Zhu and Yufeng Yuan and Xiaochen Zuo and Yu Yue and Weinan Dai and Tiantian Fan and Gaohong Liu and Juncai Liu and Ling Jun Liu and Xin Liu and Haibin Lin and Zhiqi Lin and Bole Ma and Guangming Sheng and Yuxuan Tong and Chi Zhang and Mofan Zhang and Ru Zhang and Wang Zhang and Hang Zhu and Jinhua Zhu and Jiaze Chen an...

    work page 2026

  36. [36]

    Deepseek-

    Guo, Daya and Yang, Dejian and Zhang, Haowei and Song, Junxiao and Wang, Peiyi and Zhu, Qihao and Xu, Runxin and Zhang, Ruoyu and Ma, Shirong and Bi, Xiao and others , journal=. Deepseek-. 2025 , publisher=

    work page 2025

  37. [37]

    Qwen3 Technical Report

    Qwen3 technical report , author=. arXiv preprint arXiv:2505.09388 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  38. [38]

    Olmo 3

    Olmo 3 , author=. arXiv preprint arXiv:2512.13961 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  39. [39]

    Zhihong Shao and Peiyi Wang and Qihao Zhu and Runxin Xu and Junxiao Song and Xiao Bi and Haowei Zhang and Mingchuan Zhang and Y. K. Li and Y. Wu and Daya Guo , year=. Deep. arXiv preprint arXiv:2402.03300 , primaryClass=. 2402.03300 , archivePrefix=

    work page internal anchor Pith review Pith/arXiv arXiv

  40. [40]

    Neural computation , volume=

    Natural gradient works efficiently in learning , author=. Neural computation , volume=. 1998 , publisher=

    work page 1998

  41. [41]

    Proceedings of the 32nd International Conference on Machine Learning , pages =

    Trust Region Policy Optimization , author =. Proceedings of the 32nd International Conference on Machine Learning , pages =. 2015 , editor =

    work page 2015

  42. [42]

    2000 , publisher=

    Methods of information geometry , author=. 2000 , publisher=

    work page 2000

  43. [43]

    Rusu and Kieran Milan and John Quan and Tiago Ramalho and Agnieszka Grabska-Barwinska and Demis Hassabis and Claudia Clopath and Dharshan Kumaran and Raia Hadsell , title =

    James Kirkpatrick and Razvan Pascanu and Neil Rabinowitz and Joel Veness and Guillaume Desjardins and Andrei A. Rusu and Kieran Milan and John Quan and Tiago Ramalho and Agnieszka Grabska-Barwinska and Demis Hassabis and Claudia Clopath and Dharshan Kumaran and Raia Hadsell , title =. Proceedings of the National Academy of Sciences , volume =. 2017 , doi ...

    work page doi:10.1073/pnas.1611835114 2017

  44. [44]

    IEEE transactions on pattern analysis and machine intelligence , volume=

    Learning without forgetting , author=. IEEE transactions on pattern analysis and machine intelligence , volume=. 2017 , publisher=

    work page 2017

  45. [45]

    Training language models to follow instructions with human feedback , url =

    Ouyang, Long and Wu, Jeffrey and Jiang, Xu and Almeida, Diogo and Wainwright, Carroll and Mishkin, Pamela and Zhang, Chong and Agarwal, Sandhini and Slama, Katarina and Ray, Alex and Schulman, John and Hilton, Jacob and Kelton, Fraser and Miller, Luke and Simens, Maddie and Askell, Amanda and Welinder, Peter and Christiano, Paul F and Leike, Jan and Lowe,...

    work page

  46. [46]

    arXiv , primaryClass=

    Fine-Tuning Language Models from Human Preferences , author=. arXiv , primaryClass=. 2020 , eprint=

    work page 2020

  47. [47]

    arXiv , primaryClass=

    Proximal Policy Optimization Algorithms , author=. arXiv , primaryClass=. 2017 , eprint=

    work page 2017

  48. [48]

    Forty-third International Conference on Machine Learning , year=

    Entropy-Aware On-Policy Distillation of Language Models , author=. Forty-third International Conference on Machine Learning , year=

    work page