Pith. sign in

REVIEW 4 major objections 5 minor 45 references

Teacher-supervised distillation on a diffusion model's own denoising trajectory matches reinforcement learning for math reasoning at a fraction of the rollout cost.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 19:40 UTC pith:EKUNAA7X

load-bearing objection A genuine new distillation idea for dLLMs that deserves refereeing, but the headline gains rest on test-set checkpoint selection and a 96x number the paper itself walks back to 10-15x in the appendix. the 4 major comments →

arxiv 2607.16872 v1 pith:EKUNAA7X submitted 2026-07-18 cs.CL cs.AIcs.LG

Trace-Based On-Policy Distillation for Masked Diffusion Language Models

Haolin Ren , Ziyang Huang , Chenhao Yuan , Jun Zhao , Kang Liu This is my paper
classification cs.CL cs.AIcs.LG
keywords diffusion language modelsmasked diffusionon-policy distillationtrace alignmentreverse KL divergencemathematical reasoningpost-trainingreinforcement learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to show that a diffusion language model can be post-trained for reasoning without reward models or value networks: sample the student's own denoising trajectory, ask a frozen stronger teacher for token distributions on those same partially denoised states, and update only the trace-aligned decisions with Reverse-KL. The central claim is that this attains the MATH500 accuracy of an RL-trained counterpart (+5.7 static, +4.5 dynamic) using four times fewer rollout rounds, with an estimated 96x model-compute speedup to accuracy. A sympathetic reader would care because RL for diffusion LLMs is costly and sparse-reward credit assignment is hard, while this offers dense, state-matched supervision that is on-policy and trajectory-aware. The paper also argues from ablations that each design choice matters: on-policy states beat fixed targets, trace alignment beats random masks, and Reverse-KL beats Forward-KL.

Core claim

TOPD's core discovery is that the "trace" — the subset of token decisions in a diffusion rollout that survives into the final response — is the right unit of supervision for post-training a masked diffusion language model. The paper argues that random-mask supervision can expose later answers while hiding earlier reasoning variables, creating backward-reconstruction contexts that are misaligned with inference-time denoising. By recording the student's own reveal order, evaluating teacher and student distributions at the same partially denoised states and positions, and applying a token-level Reverse-KL objective only to trace-aligned decisions, TOPD replaces sparse reward assignment with den

What carries the argument

The central object is the trace: the subset of token decisions from a sampled on-policy denoising trajectory that survive to the final response. For each surviving decision, the frozen teacher's token distribution is evaluated on the same partially denoised state and position as the student's, and the student is updated with a token-level Reverse-KL objective — a mode-seeking distribution match — implemented as a sampled-token score-function estimator with a stopped log-probability-gap coefficient. This couples three design choices (on-policy states, trace alignment, and Reverse-KL mode-seeking) into a single dense supervision signal.

Load-bearing premise

The load-bearing premise is that a strong frozen teacher with reliable token distributions is available for the student's on-policy states; if no such teacher exists, TOPD has no supervision signal, while RL at least has the environment reward.

What would settle it

Train a dLLM student with TOPD but replace the teacher with the student itself or a randomly initialized teacher; if accuracy still rises, the gains come from something other than teacher knowledge. Alternatively, include the teacher's RL training cost in the compute comparison; if the cost-adjusted speedup disappears, the 96x headline overstates end-to-end savings.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If TOPD is right, dense teacher supervision on student-visited states can substitute for reward models and value networks in dLLM reasoning post-training.
  • The 4x rollout-round reduction implies substantial compute savings: an estimated 96x to-accuracy model-compute speedup over the RL baseline under the paper's accounting, with measured GPU-hour speedup of 10-15x.
  • Because random-mask supervision underperforms trace-aligned supervision, the result implies that where teacher feedback is attached matters as much as which states are sampled.
  • The method transfers to full-attention dLLMs as well as block-attention ones, suggesting the mechanism is not tied to one architecture.
  • The observed failure mode — locally coherent solutions with early algebraic slips — suggests TOPD shapes token-level denoising but does not enforce global consistency.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The central comparison assumes a strong RL-trained teacher exists; the teacher's own training compute is not included in the speedup, so an end-to-end accounting that adds teacher post-training cost could shrink, though not necessarily erase, the advantage.
  • A testable extension: run TOPD with a deliberately weak or untrained teacher; if gains vanish, the method is mainly inheriting teacher quality, but if gains persist, trace alignment itself is doing more work.
  • The method could be combined with a verifier or process-level consistency check to catch the early arithmetic slips the paper documents; the paper leaves this combination open.
  • The Reverse-KL choice pushes the student toward teacher-preferred modes rather than full coverage, which is an advantage in noisy states but a plausible diversity risk in open-ended generation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces TOPD (trace-based on-policy distillation), a post-training method for masked diffusion language models. The student samples its own denoising trajectories, traces are filtered to token decisions that survive into the final response, and a frozen teacher provides per-token distributions on those student-visited states. The student is optimized by a Reverse-KL objective, implemented via a sampled-token score-function estimator. On MATH500, the authors report that TOPD matches or slightly exceeds an RL-trained counterpart (TraDo-4B-Instruct / TraceRL) on SDAR-4B-Chat (+5.7 static, +4.5 dynamic), using 4× fewer rollout rounds and an estimated 96.0× to-accuracy model-compute speedup. Additional experiments cover AIME2024, GSM8K, a full-attention LLaDA setting, and ablations isolating on-policy states, trace alignment, and divergence objective.

Significance. The formalization of trace-aligned on-policy distillation is coherent and the method is clearly presented, with a concrete algorithm, detailed hyperparameter tables, and a reproducible experimental setup. The proposed Eq. (9) estimator is sound, and the paper does not force its gains by construction: comparisons use external baselines and the teacher is frozen. If the efficiency and accuracy claims survive closer scrutiny, TOPD would be a practically valuable addition to dLLM post-training. However, the empirical evaluation as reported does not yet support the headline claims, because checkpoint selection is performed on the test set and the measured speedup is substantially smaller than the headline figure.

major comments (4)
  1. [§C.5, Table 7, Figure 4] The central comparison is undermined by test-set checkpoint selection. Appendix C.5 states: 'Main-table accuracies are taken from the best single run for each method, using the checkpoint selected on MATH500 under the corresponding decoding setting.' Figure 4 shows that TOPD's MATH500 accuracy peaks at step 30 and then declines, while TraceRL is reported at step 120. Thus the +5.7/+4.5 gains and the '4× fewer rollout rounds' claim compare the peak of each method selected on the test benchmark, not typical or fixed-budget performance. No held-out validation set is used and only a single best run is reported for the main table. This is load-bearing for the abstract's central claim. The authors should report mean ± std over multiple runs with a fixed evaluation schedule, or use a held-out validation split for checkpoint selection.
  2. [§4.5, Appendix F, Tables 10/12] The headline '96.0× to-accuracy model-compute speedup' is a theoretical FLOPs proxy, not a measured speedup. Appendix F acknowledges that the measured end-to-end speedup is 10–15×. The 96× figure also depends on the test-selected checkpoint contrast (step 30 vs 120). This distinction must be made prominent in the abstract and conclusion; currently the abstract states the 96.0× number without the measured caveat, which is misleading. The paper should lead with the measured 10–15× result and present the FLOPs proxy only as a secondary analysis.
  3. [Eq. (5), Limitations] The method assumes a strong RL-trained teacher is already available. TraDo-8B-Instruct (and ESPO-trained LLaDA in the full-attention experiment) are themselves products of reward-based post-training. When this teacher cost is included, the 'without reward estimation' framing is incomplete: RL is outsourced to the teacher rather than eliminated. The Limitations section partially acknowledges this, but the main text and abstract still present TOPD as avoiding RL machinery. The authors should quantify or at least clearly scope the efficiency claim as applying to the student-side post-training only, and should state that TOPD is a distillation method that requires an existing strong teacher.
  4. [Table 1, §4.2] The claim that improvements 'extend to AIME2024 and GSM8K' is overstated. Under dynamic decoding, TOPD gains +0.5 on AIME2024 and +0.4 on GSM8K, while TraceRL gains +2.1 and +1.4 respectively. Only the static AIME and GSM8K gains are comparable. The text should qualify the generalization claim and report these per-benchmark trade-offs in the main discussion.
minor comments (5)
  1. [Eq. (22) vs Eq. (9)] The sign and notation of the estimator differ between Eq. (9) and Eq. (22). Eq. (9) presents an ascent estimator with a positive coefficient r; Eq. (22) writes a loss gradient with a negative sign. The relationship between the two should be stated explicitly to avoid confusion.
  2. [§4.4, Tables 4 and 6] The JSD ablation uses a single value β=0.5, and the LLaDA experiment uses top-k=10 token KL without sensitivity analysis. Since these are free parameters of the method, the paper should report at least a small sensitivity sweep or justify the choices.
  3. [Appendix F] The text in Appendix F refers to 'the measured GPU-hour results in Section F' while already inside Appendix F. This should be rephrased to avoid self-reference confusion.
  4. [Figure 8] The right panel uses filled and open markers for static/dynamic decoding, but the caption does not fully explain how to distinguish high-accuracy operating points. A legend or annotation would improve readability.
  5. [Abstract] The phrase 'without reward estimation' is accurate only in the student-side sense. Consider adding 'given a frozen RL-trained teacher' to the abstract to match the method's actual requirements.

Circularity Check

0 steps flagged

No material circularity: TOPD is an empirical distillation method whose supervision comes from an external frozen teacher; no fitted parameter or equation makes the result reduce to its inputs.

full rationale

The derivation chain is not circular. TOPD's objective (Eq. 7) minimizes a Reverse-KL between student and frozen-teacher distributions on student-sampled trace-aligned states; the teacher (TraDo-8B-Instruct) is an external model, not a re-parameterization of the student or of the reported target. The trace definition (Eq. 4) is a design choice borrowed from external TraceRL (Wang et al., 2025), and the paper's ablations test its contribution rather than assuming it. The main MATH500 comparison is empirical against independent baselines; the compute accounting in Appendix F is an estimate based on explicit assumptions, not a fitted prediction. Appendix C.5's test-set checkpoint selection is a methodological/statistical concern about peak-of-curve reporting, not a circularity in the derivation, and the Limitations section candidly identifies the teacher-dependence assumption. There is no self-citation chain, no uniqueness theorem imported from the authors, and no parameter fitted to the benchmark that is then renamed a prediction.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The method introduces no free-floating physical or architectural entities. It relies on domain assumptions about teacher validity on on-policy states and trace-aligned decisions as the correct credit assignment, rather than on a formal first-principles derivation. The main free choices are evaluation hyperparameters and the LLaDA-specific KL truncation.

free parameters (3)
  • Dynamic decoding confidence threshold τ = 0.9
    Chosen for dynamic evaluation/remasking (Appendix A.3); affects which positions are revealed and hence which states receive teacher supervision.
  • LLaDA top-k token KL truncation = 10
    Full-attention experiments use Reverse-KL over only top-10 tokens (Table 6, A.5), deviating from the full-vocab Reverse-KL used in the main SDAR setting.
  • JSD interpolation β = 0.5
    Set for the JSD ablation (A.5); not central, but the divergence comparison in Section 4.4 depends on this choice.
axioms (4)
  • domain assumption Teacher distribution πtea(·|q, st, j) is a valid, useful supervision target on the student's partially denoised states
    Eq. (5); if the teacher cannot evaluate meaningful distributions on student on-policy states, TOPD's dense feedback is misdirected.
  • domain assumption Token decisions that survive into the final response are the ones that should be reinforced
    Eq. (4) inherits TraceRL's trace principle; if final tokens are revised through a non-causal process, the retained set may not correspond to causally responsible decisions.
  • standard math Reverse-KL sampled-token score-function estimator provides an unbiased/stable update
    Eq. (9) uses a score-function estimator with stop-gradient coefficient; unbiasedness relies on the log-policy score having zero mean.
  • domain assumption Student on-policy rollout states approximate inference-time states
    Section 3.1; if rollout decoding differs from evaluation decoding, state alignment is weaker.

pith-pipeline@v1.3.0-alltime-deepseek · 23753 in / 14667 out tokens · 144013 ms · 2026-08-01T19:40:46.696667+00:00 · methodology

0 comments
read the original abstract

Diffusion large language models (dLLMs) are a promising alternative to autoregressive generation. However, reasoning-oriented post-training for dLLMs remains challenging. Supervised fine-tuning (SFT) for dLLMs requires dense but often off-policy masked states, while reinforcement learning (RL) relies on sparse rewards or value modeling. This paper proposes \textbf{trace-based on-policy distillation (TOPD)}, a teacher-supervised framework that transfers reasoning ability to a target dLLM without reward estimation. The key idea is to supervise a dLLM on its own denoising trajectory, focusing on the trace-aligned token decisions that form the final response. Specifically, TOPD samples on-policy diffusion trajectories from the target dLLM, obtains teacher token distributions from a teacher model on the corresponding partially denoised states, and updates the target dLLM with a token-level Reverse Kullback-Leibler (Reverse-KL) objective. This design preserves dense teacher supervision while aligning training with the model's own denoising states. On mathematical reasoning benchmarks, TOPD enables SDAR-4B-Chat to match the MATH500 accuracy of its RL-trained counterpart TraDo-4B-Instruct, with gains of +5.7 under static evaluation and +4.5 under dynamic evaluation. Compared with the RL-trained counterpart, TOPD achieves this with 4$\times$ fewer rollout rounds, corresponding to an estimated 96.0$\times$ to-accuracy model-compute speedup.

Figures

Figures reproduced from arXiv: 2607.16872 by Chenhao Yuan, Haolin Ren, Jun Zhao, Kang Liu, Ziyang Huang.

Figure 1
Figure 1. Figure 1: Conceptual comparison between trace-based [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: A synthetic question-answering example il [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of TOPD. The student samples an on-policy denoising trajectory, the teacher provides step [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Learning curves of TOPD vs. TraceRL on MATH500 under static (solid) and dynamic (dashed) masking. TOPD reaches strong performance by rollout step 30, while TraceRL requires 120 rollout steps for comparable performance. test whether TOPD can match RL-based post￾training on mathematical reasoning benchmarks under both static and dynamic decoding settings. Second, we isolate whether on-policy states, trace￾al… view at source ↗
Figure 6
Figure 6. Figure 6: Trace-based supervision vs. random-mask su [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Average response length on MATH500 for SDAR-4B-Chat + TOPD across three runs. Curves re￾port the across-run mean, and shaded regions show ±1 standard deviation [PITH_FULL_IMAGE:figures/full_fig_p016_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Derived diagnostics for the MATH500 state-source ablation. Left: the static–dynamic accuracy gap over [PITH_FULL_IMAGE:figures/full_fig_p017_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

45 extracted references · 3 linked inside Pith

  1. [1]

    1997 , publisher =

    Dan Gusfield , title =. 1997 , publisher =

    1997

  2. [2]

    Proceedings of the 32nd International Conference on Machine Learning , pages =

    Deep Unsupervised Learning using Nonequilibrium Thermodynamics , author =. Proceedings of the 32nd International Conference on Machine Learning , pages =. 2015 , editor =

    2015

  3. [3]

    Denoising Diffusion Probabilistic Models , booktitle =

    Jonathan Ho and Ajay Jain and Pieter Abbeel , editor =. Denoising Diffusion Probabilistic Models , booktitle =. 2020 , url =

    2020

  4. [4]

    International Conference on Learning Representations , year=

    Score-Based Generative Modeling through Stochastic Differential Equations , author=. International Conference on Learning Representations , year=

  5. [5]

    Advances in Neural Information Processing Systems , editor=

    Structured Denoising Diffusion Models in Discrete State-Spaces , author=. Advances in Neural Information Processing Systems , editor=. 2021 , url=

    2021

  6. [6]

    Diffusion-

    Xiang Lisa Li and John Thickstun and Ishaan Gulrajani and Percy Liang and Tatsunori Hashimoto , booktitle=. Diffusion-. 2022 , url=

    2022

  7. [7]

    Advances in Neural Information Processing Systems , editor=

    Training language models to follow instructions with human feedback , author=. Advances in Neural Information Processing Systems , editor=. 2022 , url=

    2022

  8. [8]

    2017 , eprint=

    Proximal Policy Optimization Algorithms , author=. 2017 , eprint=

    2017

  9. [9]

    Thirty-seventh Conference on Neural Information Processing Systems , year=

    Direct Preference Optimization: Your Language Model is Secretly a Reward Model , author=. Thirty-seventh Conference on Neural Information Processing Systems , year=

  10. [10]

    arXiv preprint arXiv:2504.12216 , year =

    d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning , author =. arXiv preprint arXiv:2504.12216 , year =. doi:10.48550/arXiv.2504.12216 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2504.12216

  11. [11]

    arXiv preprint arXiv:2509.06949 , year =

    Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Models , author =. arXiv preprint arXiv:2509.06949 , year =. doi:10.48550/arXiv.2509.06949 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2509.06949

  12. [12]

    Chengyue Wu and Hao Zhang and Shuchen Xue and Zhijian Liu and Shizhe Diao and Ligeng Zhu and Ping Luo and Song Han and Enze Xie , booktitle=. Fast-d. 2026 , url=

    2026

  13. [13]

    Thinking Machines Lab: Connectionism , year =

    Lu, Kevin and Thinking Machines Lab , title =. Thinking Machines Lab: Connectionism , year =. doi:10.64434/tml.20251026 , abstract =

    doi:10.64434/tml.20251026

  14. [14]

    arXiv preprint arXiv:2603.06743 , year =

    Stabilizing Reinforcement Learning for Diffusion Language Models , author =. arXiv preprint arXiv:2603.06743 , year =. doi:10.48550/arXiv.2603.06743 , abstract =

    doi:10.48550/arxiv.2603.06743

  15. [15]

    The Thirty-eighth Annual Conference on Neural Information Processing Systems , year=

    Simple and Effective Masked Diffusion Language Models , author=. The Thirty-eighth Annual Conference on Neural Information Processing Systems , year=

  16. [16]

    2025 , eprint =

    Large Language Diffusion Models , author =. 2025 , eprint =

    2025

  17. [17]

    Guo, Daya and Yang, Dejian and Zhang, Haowei and Song, Junxiao and Wang, Peiyi and Zhu, Qihao and Xu, Runxin and Zhang, Ruoyu and Ma, Shirong and Bi, Xiao and Zhang, Xiaokang and Yu, Xingkai and Wu, Yu and Wu, Z. F. and Gou, Zhibin and Shao, Zhihong and Li, Zhuoshu and Gao, Ziyi and Liu, Aixin and Xue, Bing and Wang, Bingxuan and Wu, Bochao and Feng, Bei ...

  18. [18]

    2024 , eprint=

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models , author=. 2024 , eprint=

    2024

  19. [19]

    The Twelfth International Conference on Learning Representations , year=

    On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes , author=. The Twelfth International Conference on Learning Representations , year=

  20. [20]

    2015 , eprint =

    Distilling the Knowledge in a Neural Network , author =. 2015 , eprint =

    2015

  21. [21]

    2025 , url=

    Tianzhe Chu and Yuexiang Zhai and Jihan Yang and Shengbang Tong and Saining Xie and Dale Schuurmans and Quoc V Le and Sergey Levine and Yi Ma , booktitle=. 2025 , url=

    2025

  22. [22]

    2025 , eprint=

    Mercury: Ultra-Fast Language Models Based on Diffusion , author=. 2025 , eprint=

    2025

  23. [23]

    The Thirteenth International Conference on Learning Representations , year =

    Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models , author =. The Thirteenth International Conference on Learning Representations , year =

  24. [24]

    The Thirty-eighth Annual Conference on Neural Information Processing Systems , year=

    Simplified and Generalized Masked Diffusion for Discrete Data , author=. The Thirty-eighth Annual Conference on Neural Information Processing Systems , year=

  25. [25]

    Principled

    Jingyang Ou and Jiaqi Han and Minkai Xu and Shaoxuan Xu and Jianwen Xie and Stefano Ermon and Yi Wu and Chongxuan Li , booktitle=. Principled. 2026 , url=

    2026

  26. [26]

    2026 , url=

    Chenyu Wang and Paria Rashidinejad and DiJia Su and Song Jiang and Sid Wang and Siyan Zhao and Cai Zhou and Shannon Zejiang Shen and Feiyu Chen and Tommi Jaakkola and Yuandong Tian and Bo Liu , booktitle=. 2026 , url=

    2026

  27. [27]

    arXiv preprint arXiv:2510.02212 , year =

    DiFFPO: Training Diffusion LLMs to Reason Fast and Furious via Reinforcement Learning , author =. arXiv preprint arXiv:2510.02212 , year =. 2510.02212 , archiveprefix =

    arXiv

  28. [28]

    arXiv preprint arXiv:1502.05477 , year =

    Trust Region Policy Optimization , author =. arXiv preprint arXiv:1502.05477 , year =. 1502.05477 , archiveprefix =

    Pith/arXiv arXiv

  29. [29]

    The Thirteenth International Conference on Learning Representations , year=

    Scaling Diffusion Language Models via Adaptation from Autoregressive Models , author=. The Thirteenth International Conference on Learning Representations , year=

  30. [30]

    2025 , eprint=

    SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation , author=. 2025 , eprint=

    2025

  31. [31]

    2011 , eprint=

    A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , author=. 2011 , eprint=

    2011

  32. [32]

    2021 , eprint=

    Training Verifiers to Solve Math Word Problems , author=. 2021 , eprint=

    2021

  33. [33]

    2020 , eprint=

    DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter , author=. 2020 , eprint=

    2020

  34. [34]

    2024 , eprint=

    A Survey on Knowledge Distillation of Large Language Models , author=. 2024 , eprint=

    2024

  35. [35]

    The Thirteenth International Conference on Learning Representations , year=

    Speculative Knowledge Distillation: Bridging the Teacher-Student Gap Through Interleaved Sampling , author=. The Thirteenth International Conference on Learning Representations , year=

  36. [36]

    Yuxian Gu and Li Dong and Furu Wei and Minlie Huang , booktitle=. Mini. 2024 , url=

    2024

  37. [37]

    Measuring Mathematical Problem Solving With the

    Dan Hendrycks and Collin Burns and Saurav Kadavath and Akul Arora and Steven Basart and Eric Tang and Dawn Song and Jacob Steinhardt , booktitle=. Measuring Mathematical Problem Solving With the. 2021 , url=

    2021

  38. [38]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

  39. [39]

    2024 , howpublished =

    American Invitational Mathematics Examination (AIME) 2024: AIME I and AIME II , author =. 2024 , howpublished =

    2024

  40. [40]

    2025 , eprint=

    DAPO: An Open-Source LLM Reinforcement Learning System at Scale , author=. 2025 , eprint=

    2025

  41. [41]

    2025 , eprint =

    Zhu, Fengqi and Wang, Rongzhen and Nie, Shen and Zhang, Xiaolu and Wu, Chunwei and Hu, Jun and Zhou, Jun and Chen, Jianfei and Lin, Yankai and Wen, Ji-Rong and Li, Chongxuan , journal =. 2025 , eprint =

    2025

  42. [42]

    2025 , eprint =

    He, Haoyu and Renz, Katrin and Cao, Yong and Geiger, Andreas , journal =. 2025 , eprint =. doi:10.48550/arXiv.2508.13148 , abstract =

    doi:10.48550/arxiv.2508.13148 2025

  43. [43]

    2025 , eprint =

    LLaDA2.0: Scaling Up Diffusion Language Models to 100B , author =. 2025 , eprint =

    2025

  44. [44]

    2025 , eprint =

    Dream 7B: Diffusion Large Language Models , author =. 2025 , eprint =

    2025

  45. [45]

    2026 , eprint =

    A Survey of On-Policy Distillation for Large Language Models , author =. 2026 , eprint =

    2026