Pith. sign in

REVIEW 4 major objections 4 minor 40 references

A single language model can learn many tasks at once from soft-prompt teachers, beating full fine-tuning without forgetting.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 09:15 UTC pith:U3QUKCY3

load-bearing objection The prompt-tuned teacher is a real idea, but the 'many teachers' headline is a math-elicitation artifact; send to review with a required re-scope. the 4 major comments →

arxiv 2607.18293 v1 pith:U3QUKCY3 submitted 2026-06-30 cs.LG cs.CL

One Student, Many Teachers: Multi-Task On-Policy Distillation via Soft-Prompt Privileged Context

Yingzi Ma , Zichen Zhu , Ming Jiang , Chaowei Xiao This is my paper
classification cs.LG cs.CL
keywords on-policy distillationsoft-prompt tuningmulti-task learningcatastrophic forgettingknowledge distillationparameter-efficient fine-tuningprivileged contextlarge language models
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that a single large language model can learn several new skills at once by distilling from task-specific soft-prompt teachers, rather than by fine-tuning its weights. The teacher is just a handful of learnable embedding vectors prepended to the frozen backbone, trained on gold input-output pairs; at inference the prompts are thrown away. On four tasks spanning science, tool use, biology, and math, the multi-task student reaches the best average accuracy (56.2 on a 1.7B-parameter base model) while holding its own on held-out general benchmarks — where sequential fine-tuning loses both. If correct, the result offers a parameter-cheap, order-independent way to add capabilities to a model without catastrophic forgetting.

Core claim

PROMPTSD's central claim is that prompt-tuned teachers occupy a unique spot among on-policy distillation teachers: they share the student's exact transformer weights, differ only by a learnable soft prefix, and therefore satisfy three conditions the paper argues are jointly necessary — moderate student–teacher overlap (thinking-pattern consistency), a capability gap that materialises as rising overlap during training, and no gold answer in the teacher's conditioning (no post-hoc rationalization). Distilling from such teachers with reverse-KL on student rollouts, the paper reports, matches or exceeds full fine-tuning with orders of magnitude fewer trainable parameters, and the multi-task vers

What carries the argument

The central object is the soft prompt P: L learnable continuous embeddings prepended to the input embedding sequence of a frozen base model, trained by next-token cross-entropy on (x, ygold) pairs. Because P is the only trainable component, the teacher and student share identical transformer weights, which the paper measures with the top-10 next-token overlap ratio ρ and its change over training Δρ. The argument runs on three desiderata (moderate initial overlap, positive overlap gain, no answer conditioning) and the on-policy reverse-KL objective over student rollouts; the multi-task extension adds per-example routing to the task-tagged teacher.

Load-bearing premise

The load-bearing premise is that the soft-prompt teacher's advantage on Biology and Tool-use comes from the prompt mechanism itself, not from the externally rewritten reasoning targets it was trained on — the paper never trains a PT teacher on the original answer-only targets, so the teacher's edge could be annotation quality.

What would settle it

Train a PT teacher for Biology and Tool-use on the original answer-only targets, without the external reasoning annotation, and run the same on-policy distillation; if the student's accuracy on those tasks collapses toward the zero-shot baseline or no longer beats OPD(FFT) and OPD(LoRA), the claimed advantage of the soft-prompt teacher over weight-modifying teachers is not supported.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Single-task PROMPTSD (OPD with a PT teacher) matches or exceeds full fine-tuning and LoRA-tuned teachers on Science, Tool-use, Biology, and Math while training roughly 0.05–0.1% of the backbone parameters (prompt embeddings only).
  • The multi-task variant reaches the highest average task accuracy (56.2 on Qwen3-1.7B-Base, 56.4 on Phi-4-mini-instruct) and, unlike sequential SFT, does not degrade held-out general benchmarks (MMLU-Pro, HellaSwag, TruthfulQA).
  • The counterfactual answer-swap probe shows the soft-prompt teacher cannot be steered by an injected wrong answer, whereas a gold-answer-conditioned teacher follows it roughly 4–5× above chance—so the student does not inherit backward-chained justification.
  • Teacher scale is not the deciding factor: a same-size PT teacher outperforms a 4B or 8B cross-size teacher on three of four tasks, indicating that carrying new knowledge matters more than parameter count.
  • Leave-one-teacher-out experiments indicate Math is largely a latent capability elicited by cross-task co-training (61.0 without any math teacher or data), while Science, Tool-use, and Biology depend on their own soft-prompt teachers.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same routing mechanism should generalize to other plug-and-play privileged-context modules (LoRA adapters, prefix tuners, task vectors), making post-training a composition problem rather than a sequential fine-tuning problem — the paper hints at this but does not test it.
  • If the Math result holds more broadly, many apparent 'skill gains' from multi-task training are really the surfacing of latent pretraining ability; curricula could deliberately under-sample tasks where the base is already strong and use soft-prompt teachers only for capability injection.
  • A decisive follow-up is to train PT teachers for Biology and Tool-use on the original answer-only targets; if the student's gains vanish, the external reasoning annotation — not the soft-prompt mechanism — is carrying the teacher's capability.
  • The method's parameter and storage footprint scales linearly in the number of tasks (~4.9M parameters for four teachers on a 1.7B model), making it practical for many-task continual adaptation, but interference between many concurrent soft prompts remains untested.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces PROMPTSD, an on-policy self-distillation framework whose teacher is a soft-prompt-tuned version of the frozen student backbone. It motivates three desiderata (thinking-pattern consistency, a non-trivial capability gap, and no post-hoc rationalization), evaluates them on four tasks (Science, Tool Use, Biology, Math) with Qwen3-1.7B-Base and Phi-4-mini-instruct, and reports that the single-task variant (OPD with a prompt-tuned teacher) matches or exceeds FFT/LoRA-based distillation while training far fewer parameters. The paper further proposes a multi-task extension that routes each example to one of K task-specific soft-prompt teachers, reporting that the resulting single student achieves the best average accuracy (56.2 on Qwen3-1.7B-Base) while preserving general-capability benchmarks, in contrast to sequential SFT. The central claim is that a single student can absorb K task-specific teachers in parallel without interference.

Significance. If fully supported, the soft-prompt teacher would be a valuable design point in the OPD landscape: it preserves the student's weights, avoids answer-conditioned rationalization, and provides a parameter-efficient way to inject task knowledge. The paper's strengths include a detailed efficiency analysis, a counterfactual answer-swap probe for D3, a leave-one-teacher-out ablation, and bootstrap/CI analysis on the small Biology split. However, the multi-task mechanism claim is currently not established: the headline average rests on a single seed and is partly contradicted by the paper's own Appendix F, which shows that the Math gain is mostly cross-task elicitation and that Tool Use and Biology regress under multi-task routing. The external annotation of Biology/Tool-use targets further confounds the teacher-mechanism attribution. The single-task OPD-PT result is plausible and useful, but the abstract and §5.2 overstate the multi-task conclusions.

major comments (4)
  1. [§5.2 / Appendix F (Table 11)] The central claim that the multi-task student "absorbs knowledge from K teachers in parallel" and that "PT teachers compose cleanly" is not supported by the paper's own leave-one-teacher-out ablation. Removing the Math teacher and all Math data still yields Math=61.0, versus 67.2 for full PROMPTSD and 51.0 for single-task OPD-PT; the +16.2 Math gain in Table 1 is therefore mostly cross-task elicitation, not absorption of the Math teacher. The same ablation shows Tool Use and Biology regress under multi-task routing (48.4 vs. 51.5 and 54.0 vs. 62.0), which the appendix itself calls an "accepted trade-off." The abstract and §5.2 should be revised to attribute the average gain primarily to cross-task transfer on Math and to acknowledge per-task interference.
  2. [Tables 1–2 / Appendix D] The multi-task PROMPTSD results are reported for a single seed. Appendix D provides three-seed means and standard deviations only for single-task OPD-PT, not for the multi-task setup whose headline average (56.2, +2.3 over OPD-PT) drives the central claim. Given the small Biology (n=50) and Tool Use (n=97) test sets, and the per-task regressions visible in Table 11, the +2.3 average advantage may be within run-to-run noise. Please provide multi-seed estimates, standard deviations, or confidence intervals for PROMPTSD and the main baselines.
  3. [Appendix B.1] The Biology and Tool-use training targets were rewritten by Qwen3-235B-A22B, so the PT teacher's apparent advantage on those tasks may come from the externally injected reasoning rather than from the soft-prompt mechanism itself. No ablation trains a PT teacher on the original answer-only targets. Such an ablation is necessary to attribute the observed gains to prompt-tuning as opposed to the annotation procedure, which is a load-bearing part of the comparison with FFT/LoRA teachers.
  4. [§3.1, Figure 2] Desiderata D1–D3 are derived from the same experiments they are used to explain. In particular, Δρ=ρ(S)−ρ(0) is measured on the very training runs whose gains it is said to predict; a student that improves will by construction move toward the teacher, so the reported correlation (r=0.86) is partly definitional. To support the claimed design principle, the paper should validate Δρ on held-out runs or present an independent control (e.g., intervening on initial overlap) rather than only a post-hoc fit.
minor comments (4)
  1. [Figure 2 caption] The caption states "Initial overlap alone is not predictive," but the reported correlations are r=0.87 for Science and r=0.55 for Tool Use. "Not sufficient" or "weakly predictive" would be more accurate than "not predictive."
  2. [§5.1 vs. Appendix B.3] The main text says "All experiments run on 4 NVIDIA H100 80GB GPUs," while Appendix B.3 says each run uses a single A100, L40S, or H100. This hardware inconsistency should be resolved.
  3. [§3.1] The text says results for k∈{1,5,50} appear in the appendix, but the provided appendix (A–G) does not contain such a k-sensitivity analysis. Either add the section or remove the reference.
  4. [Abstract / §5.2] "Training orders of magnitude fewer parameters" should be qualified by the Appendix C observation that training time is comparable to LoRA/FFT; the savings are in parameter storage and memory footprint, not wall-clock training.

Circularity Check

2 steps flagged

Desiderata and Δρ are post-hoc explanatory fits, but the headline benchmark results are self-contained and Appendix F is honest about per-task trade-offs.

specific steps
  1. self definitional [§3.1 (Desiderata), Eq. (4); Fig. 2; §7 Limitations]
    "We measure this through the change in overlap rather than benchmark accuracy: a real capability gap manifests as steadily rising ρ(s), while a teacher that merely amplifies existing behaviors (Chen et al., 2026) leaves overlap stagnant despite higher accuracy. We require ∆ := ρ(S) − ρ(0) > δ. ... Overlap gain ∆ρ is strongly predictive of student gain (right panels): teachers occupying the moderate-overlap regime (D1) and producing substantial ∆ρ (D2) consistently yield the best students."

    D2 defines 'capability gap absorbed' as Δρ, the change in student–teacher top-k overlap across the same distillation run whose gain it is used to explain. Since the student is optimized to minimize reverse KL to the teacher, ρ is literally the quantity training pushes upward; Δρ is a post-hoc outcome, not an independent teacher property. D1's 'moderate-overlap regime' has no pre-specified ρmin/ρmax; it is read off the same Figure 2 scatter used to validate it. Thus 'D1–D3 validated' and 'Δρ predicts gain' summarize the same experiments twice rather than testing a forward prediction.

  2. self definitional [§3.1 (D3); Appendix E, Table 10]
    "The PT teacher has no answer slot in its input and therefore cannot be steered. its follow-rate is expected to coincide with the chance-level baseline (1−a)/(C−1), where a is the teacher's task accuracy ... This is direct evidence that PT satisfies D3 by construction while observation-based teachers violate it."

    D3 is operationalized as the teacher's conditioning context at distillation time not containing ygold, and the PT teacher is constructed with no answer slot. The chance-level follow-rate in the answer-swap probe is therefore entailed by construction—a model cannot be steered by a token it never receives. The probe does not test whether the soft prompt's learned parameters have internalized the gold answer and rationalized backward from it, so the 'no post-hoc rationalization' claim is definitional rather than empirically derived.

full rationale

PROMPTSD's headline results (Tables 1, 2, 4) are direct empirical comparisons on fixed benchmarks, with no load-bearing self-citation chain; Appendix F explicitly isolates teacher contributions and admits per-task regressions (Tool use 48.4 vs 51.5; Biology 54.0 vs 62.0), so the reported 56.2 average is a reproducible empirical outcome rather than a fitted input renamed as prediction. The circularity is confined to the explanatory overlay: the D1–D3 desiderata are post-hoc summaries of the same teacher-comparison experiments they are used to justify, and Δρ is a post-training movement measure that the reverse-KL objective directly drives. The paper itself concedes in §7 that D1–D3 are 'operationally defined and empirically validated rather than formally proven.' No self-citations are load-bearing, so the score is moderate rather than extreme.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The paper's headline numbers depend on per-task prompt length, per-task Phase B hyperparameters, and especially hand-tuned multi-task routing ratios and divergence choice. The most consequential assumption is that the PT teachers for Biology/Tool-use acquire knowledge from the soft prompt itself, when in fact their training targets were rewritten by a 235B annotator. The D1–D3 desiderata are empirically motivated rather than derived, and their supporting statistic Δρ is a post-hoc measure that is partly definitional.

free parameters (4)
  • Soft-prompt length L per task = 400 (Science, Math), 800 (Tool use, Biology)
    Tuned per task on validation (Table 6); L affects teacher quality and final student accuracy, and the ablations show non-monotonic behavior (e.g., Math drops from 51.0 to 29.0 at L=800).
  • Multi-task routing sampling ratios = Qwen3: Biology 2×, Math 0.5×; Phi-4: Math 0.5×
    Selected via a small validation grid (Appendix B.4, Table 7); directly shapes the headline 56.2 average by up-weighting Biology and down-weighting Math. No sensitivity analysis for uniform routing is provided.
  • Divergence choice in the OPD loss = reverse-KL
    Chosen because reverse-KL substantially outperforms other divergences on Math (Table 3: 51.0 vs 13–18). Under forward KL, symmetric KL, or JSD, the Math score would fall by ~35 points, changing the multi-task average.
  • Phase B learning rate / step count = 1e-5/600 for most tasks; Biology 2e-5/1000
    Tuned per task on validation (Table 6); affects distillation strength and the final balance between task accuracy and forgetting.
axioms (4)
  • domain assumption The soft-prompt teacher does not condition on y_gold at generation time, even though it was trained on (x, y_gold)
    D3 treats the prompt as a 'parametric observation' without post-hoc rationalization. The wrong-answer probe (Appendix E) supports this, but it is an assumption that the trained prompt has not memorized instance-level answers, especially for the small Biology split (n_train=405).
  • domain assumption Top-k next-token overlap ρ and its change Δρ measure thinking-pattern consistency and capability transfer
    The paper defines D1/D2 via ρ but never gives concrete thresholds ρ_min, ρ_max, or δ. The correlation between Δρ and student gain is post-hoc and partly definitional (student moving toward teacher increases overlap by construction).
  • ad hoc to paper The Qwen3-235B-A22B annotated reasoning for Biology and Tool-use is accurate and answer-consistent
    Appendix B.1: the external model generates reasoning for two tasks' training targets, after which the gold answer/action is re-attached deterministically. No manual verification is reported; this external knowledge injection confounds the contribution of the soft-prompt mechanism on those tasks.
  • domain assumption Reverse-KL on student-visited states with a frozen teacher is the right objective for capability injection
    The paper relies on MiniLLM/GKD literature for the reverse-KL objective. On Math it is critical (Table 3), and the paper does not offer a mechanism-level justification for why reverse-KL is needed for capability injection but not for the other three tasks.

pith-pipeline@v1.3.0-alltime-deepseek · 19895 in / 22569 out tokens · 200331 ms · 2026-08-02T09:15:08.379794+00:00 · methodology

0 comments
read the original abstract

On-policy self-distillation (OPSD) teaches large language models new skills through a teacher that shares the student's backbone and supervises its own rollouts. Existing teachers either inject privileged context at the input -- inducing post-hoc rationalization -- or fine-tune weights, accumulating drift and forgetting across tasks. We propose \method, whose teacher differs from the student only by a learnable soft prompt: trained on $(x, y_\text{gold})$ pairs with the backbone frozen, the prompt yields a task-specific teacher that preserves the student's exact representational geometry. \method\ extends naturally to multi-task settings by routing each example in a merged corpus to its corresponding soft-prompt teacher, allowing a single student to absorb knowledge from $K$ teachers in parallel; at inference, all prompts are discarded. On Qwen3-1.7B-Base and Phi-4-mini-instruct across four tasks (Science, Tool Use, Biology, Math), the single-task variant (OPD with a PT teacher) matches or exceeds full fine-tuning while training orders of magnitude fewer parameters, and the multi-task variant achieves the best overall average ($56.2$ on Qwen3-1.7B-Base) while preserving general-capability benchmarks -- in contrast to sequential SFT, which degrades both.

Figures

Figures reproduced from arXiv: 2607.18293 by Chaowei Xiao, Ming Jiang, Yingzi Ma, Zichen Zhu.

Figure 1
Figure 1. Figure 1: Three families of privileged-information teachers for on-policy distillation. All three share the same on-policy reverse-KL objective L(θ) = KL[πs ∥ πt] over student rollouts Ys, but differ in how the teacher acquires privileged information. (a) Observation-based (SDFT, OPSD, SDPO): teacher and student share weights πθ; the teacher is conditioned on extra input-side context c. (b) RL-tuned (CoPD): the teac… view at source ↗
Figure 2
Figure 2. Figure 2: Performance gain vs. overlap dynamics across teacher families. For each teacher (markers), we plot the 1.7B-Base student’s task accuracy gain (pp over zero-shot) against (left of each pair) the initial overlap ρ (0) and (right of each pair) the overlap gain ∆ρ = ρ (S) − ρ (0), on Science and Tooluse. Dashed lines indicate the linear fit. Initial overlap alone is not predictive: high-ρ (0) teachers like RLV… view at source ↗
Figure 3
Figure 3. Figure 3: Overview of PROMPTSD. (a) Task-specific soft prompts {P ⋆ 1 , . . . , P⋆ K} are trained in parallel on {D1, . . . , DK} via next-token cross-entropy (Eq. 6); the base model is frozen. (b) Datasets are merged with task tags; each example is routed to its soft-prompt teacher πθ+P ⋆ k , while the student πθ rolls out on the input alone. The student is updated via per-token reverse KL on student-visited states… view at source ↗
Figure 4
Figure 4. Figure 4: Soft-prompt length L (number of learnable virtual tokens in the PT teacher), across all four tasks. 5.2 Main Results Single-Task Distillation [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 14 linked inside Pith

  1. [1]

    Proceedings of the 2022 conference on empirical methods in natural language processing , pages=

    Rethinking the role of demonstrations: What makes in-context learning work? , author=. Proceedings of the 2022 conference on empirical methods in natural language processing , pages=

    2022

  2. [2]

    , author=

    Lora: Low-rank adaptation of large language models. , author=. Iclr , volume=

  3. [3]

    Proceedings of the 2021 conference on empirical methods in natural language processing , pages=

    The power of scale for parameter-efficient prompt tuning , author=. Proceedings of the 2021 conference on empirical methods in natural language processing , pages=

    2021

  4. [4]

    Advances in neural information processing systems , volume=

    Training language models to follow instructions with human feedback , author=. Advances in neural information processing systems , volume=

  5. [5]

    arXiv preprint arXiv:2505.09388 , year=

    Qwen3 Technical Report , author=. arXiv preprint arXiv:2505.09388 , year=

    Pith/arXiv arXiv

  6. [6]

    Zeng, Aohan and Lv, Xin and Hou, Zhenyu and Du, Zhengxiao and Zheng, Qinkai and Chen, Bin and Yin, Da and Ge, Chendi and Huang, Chenghua and Xie, Chengxing and others , journal=

  7. [7]

    arXiv preprint arXiv:2601.02780 , year=

    Mimo-v2-flash Technical Report , author=. arXiv preprint arXiv:2601.02780 , year=

    Pith/arXiv arXiv

  8. [8]

    Findings of the Association for Computational Linguistics: ACL 2022 , pages=

    Why exposure bias matters: An imitation learning perspective of error accumulation in language generation , author=. Findings of the Association for Computational Linguistics: ACL 2022 , pages=

    2022

  9. [9]

    arXiv preprint arXiv:2601.19897 , year=

    Self-Distillation Enables Continual Learning , author=. arXiv preprint arXiv:2601.19897 , year=

    Pith/arXiv arXiv

  10. [10]

    arXiv preprint arXiv:2601.18734 , year=

    Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models , author=. arXiv preprint arXiv:2601.18734 , year=

    Pith/arXiv arXiv

  11. [11]

    arXiv preprint arXiv:2604.27083 , year=

    Co-Evolving Policy Distillation , author=. arXiv preprint arXiv:2604.27083 , year=

    Pith/arXiv arXiv

  12. [12]

    arXiv preprint arXiv:2601.20802 , year=

    Reinforcement Learning via Self-Distillation , author=. arXiv preprint arXiv:2601.20802 , year=

    Pith/arXiv arXiv

  13. [13]

    Proceedings of the national academy of sciences , volume=

    Overcoming catastrophic forgetting in neural networks , author=. Proceedings of the national academy of sciences , volume=. 2017 , publisher=

    2017

  14. [14]

    IEEE Transactions on Audio, Speech and Language Processing , year=

    An empirical study of catastrophic forgetting in large language models during continual fine-tuning , author=. IEEE Transactions on Audio, Speech and Language Processing , year=

  15. [15]

    arXiv preprint arXiv:2405.09673 , year=

    Lora learns less and forgets less , author=. arXiv preprint arXiv:2405.09673 , year=

    Pith/arXiv arXiv

  16. [16]

    arXiv preprint arXiv:2401.05605 , year=

    Scaling laws for forgetting when fine-tuning large language models , author=. arXiv preprint arXiv:2401.05605 , year=

    Pith/arXiv arXiv

  17. [17]

    Advances in Neural Information Processing Systems , volume=

    Lora vs full fine-tuning: An illusion of equivalence , author=. Advances in Neural Information Processing Systems , volume=

  18. [18]

    International conference on machine learning , pages=

    Parameter-efficient transfer learning for NLP , author=. International conference on machine learning , pages=. 2019 , organization=

    2019

  19. [19]

    Prefix-tuning: Optimizing continuous prompts for generation , author=. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , pages=

  20. [20]

    arXiv preprint arXiv:2511.00088 , year=

    Alpamayo-r1: Bridging reasoning and action prediction for generalizable autonomous driving in the long tail , author=. arXiv preprint arXiv:2511.00088 , year=

    Pith/arXiv arXiv

  21. [21]

    arXiv preprint arXiv:2604.13016 , year=

    Rethinking on-policy distillation of large language models: Phenomenology, mechanism, and recipe , author=. arXiv preprint arXiv:2604.13016 , year=

    Pith/arXiv arXiv

  22. [22]

    Advances in Neural Information Processing Systems , volume=

    Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? , author=. Advances in Neural Information Processing Systems , volume=

  23. [23]

    Advances in Neural Information Processing Systems , volume=

    Language models don't always say what they think: Unfaithful explanations in chain-of-thought prompting , author=. Advances in Neural Information Processing Systems , volume=

  24. [24]

    arXiv preprint arXiv:2503.08679 , year=

    Chain-of-thought reasoning in the wild is not always faithful , author=. arXiv preprint arXiv:2503.08679 , year=

    Pith/arXiv arXiv

  25. [25]

    arXiv preprint arXiv:2103.03874 , year=

    Measuring mathematical problem solving with the math dataset , author=. arXiv preprint arXiv:2103.03874 , year=

    Pith/arXiv arXiv

  26. [26]

    Proceedings of the 31st International Conference on Computational Linguistics , pages=

    Rethinking kullback-leibler divergence in knowledge distillation for large language models , author=. Proceedings of the 31st International Conference on Computational Linguistics , pages=

  27. [27]

    arXiv preprint arXiv:2412.08905 , year=

    Phi-4 technical report , author=. arXiv preprint arXiv:2412.08905 , year=

    Pith/arXiv arXiv

  28. [28]

    Proceedings of the 29th symposium on operating systems principles , pages=

    Efficient memory management for large language model serving with pagedattention , author=. Proceedings of the 29th symposium on operating systems principles , pages=

  29. [29]

    arXiv preprint arXiv:2406.09098 , year=

    Sciknoweval: Evaluating multi-level scientific knowledge of large language models , author=. arXiv preprint arXiv:2406.09098 , year=

    arXiv

  30. [30]

    Advances in Neural Information Processing Systems , volume=

    Gorilla: Large language model connected with massive apis , author=. Advances in Neural Information Processing Systems , volume=

  31. [31]

    Advances in Neural Information Processing Systems , volume=

    Mmlu-pro: A more robust and challenging multi-task language understanding benchmark , author=. Advances in Neural Information Processing Systems , volume=

  32. [32]

    Proceedings of the 57th annual meeting of the association for computational linguistics , pages=

    Hellaswag: Can a machine really finish your sentence? , author=. Proceedings of the 57th annual meeting of the association for computational linguistics , pages=

  33. [33]

    Proceedings of the 60th annual meeting of the association for computational linguistics (volume 1: long papers) , pages=

    Truthfulqa: Measuring how models mimic human falsehoods , author=. Proceedings of the 60th annual meeting of the association for computational linguistics (volume 1: long papers) , pages=

  34. [34]

    International Conference on Learning Representations , volume=

    On-policy distillation of language models: Learning from self-generated mistakes , author=. International Conference on Learning Representations , volume=

  35. [35]

    International Conference on Learning Representations , volume=

    Minillm: Knowledge distillation of large language models , author=. International Conference on Learning Representations , volume=

  36. [36]

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Scott: Self-consistent chain-of-thought distillation , author=. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  37. [37]

    2025 , url =

    Thinking Machines Lab , title =. 2025 , url =

    2025

  38. [38]

    Nature , volume=

    DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning , author=. Nature , volume=. 2025 , publisher=

    2025

  39. [39]

    arXiv preprint arXiv:2402.03300 , year=

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year=

    Pith/arXiv arXiv

  40. [40]

    Advances in neural information processing systems , volume=

    Direct preference optimization: Your language model is secretly a reward model , author=. Advances in neural information processing systems , volume=