Artificial Intelligence · arXiv:2610.38681 v1 · submitted 28 Oct 2026 · 144 references
Weaken to Strengthen: Weakening-Induced On-Policy Self-Distillation for Large Reasoning Models
The paper says
“Self-distillation offers a way to improve reasoning models when a suitable external teacher is unavailable. However, common approaches rely on privileged information to construct a stronger self-teacher, limiting their use when only unlabeled prompts are available.”
more of the abstract
In this paper, we introduce Weakening-Induced On-Policy Self-Distillation (WISD), which instead constructs a weaker counterpart of the original model and uses the contrast between the two as a training signal without requiring privileged information. We explore two ways to construct the weaker model: low-bit quantization and magnitude pruning. We evaluate WISD across diverse architectures and scales on mathematical and coding reasoning tasks. Across these benchmarks, quantization and pruning improve average accuracy over the original models by 4.0 and 5.5 percentage points, respectively, achieving state-of-the-art performance even without privileged information. Notably, pruning raises Qwen3.5-4B's accuracy from 57.7% to 70.5% on HMMT 2025 and from 39.1% to 56.7% on LiveCodeBench v6. Quantization raises Qwen3.6-35B-A3B's accuracy from 68.5% to 74.6% and from 61.8% to 69.5% on the same benchmarks, respectively. These results establish WISD as a novel and effective self-distillation framework for large reasoning models.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 31 Oct 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. A practical label-free self-distillation recipe with consistent gains, but the sign-control experiment and AIME24 hygiene need fixing before the headline fully lands. This line is wrong
4 major objections · 4 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 144 references say.
The paper says, at §3.1–3.2, Eq. (3); Table 3
The review did not quote this passage.
Pith says: Objection 1 of 4 · major
The paper's central interpretive claim is that the contrast r_w in Eq. (3) 'specifies which actions the student should favor more or less', and Section 5.2 attributes the gains to structured exploration and verification supervision. The two controls in Table 3 (self-enhancing, temperature weak) rule out pure confidence reinforcement, but neither varies the sign of r_w nor breaks its token-level structure while preserving the marginal distribution of rewards.
more of objection 1
As written, the results are also compatible with a generic perturbation or regularization account in which any structured token-level signal on self-generated trajectories helps. I request two additional control runs matched to the Section 4.1 protocol on at least one math and one code benchmark: (i) a flipped-sign objective favoring tokens preferred by the weakened model, and (ii) a shuffled-contrast objective that assigns the same multiset of r_w values to randomly chosen candidate tokens. If either control matches the standard WISD gains, the Section 3.2 interpretation and the Section 5.2 attributions would need to be revised; if both are clearly worse, the direction assumption is isolated and the mechanism claim becomes much stronger.
The paper says, at Appendix A.2; Table 2; Figure 3
The review did not quote this passage.
Pith says: Objection 2 of 4 · major
Appendix A.2 states that 'AIME 2024 also serves as the development benchmark for exploring weak-model constructions', and AIME 2024 is a main-results benchmark in Table 2; in addition, the b and tau sweeps in Figure 3 are run on HMMT25 with Qwen3.5-4B, and the defaults b=3 and tau=1/6 chosen from that sweep are then reported for that exact model-benchmark cell in Table 2.
more of objection 2
The paper deserves credit for disclosing the AIME24 role, and the breadth of the gains (21-23 of 25 settings positive) partially mitigates the concern, but the selection-on-evaluation issue is not resolved for the individual cells in question. I ask that the authors either verify the selected constructions and hyperparameters on benchmarks that did not participate in their selection (for example, repeating the Table 3 construction comparison and the Figure 3 sweeps on AIME25 or on a second model), or exclude AIME24 and the swept Qwen3.5-4B/HMMT25 cell from the headline five-benchmark averages and report them separately.
The paper says, at §3.2, Eq. (5); Appendix C.1, Eqs. (10)-(11)
The review did not quote this passage.
Pith says: Objection 3 of 4 · major
The main text states the distillation objective as L_TT(theta) = alpha E[D_KL(pi_theta || q_w)] (Eq. 5), but Appendix C.1 describes the offline implementation as a behavior-weighted squared log-probability regression, ell(p) = (alpha^2/(2m)) sum_j b_j (log p_j - log q_j)^2 (Eq. 10), and derives the gradient in Eq.
more of objection 3
(11) from that expression. These are different objectives: the gradient of the KL in Eq. (5) at p = b would be proportional to b_j - q_j, not to the centered contrast b_j(r_j - sum_k b_k r_k) in Eq. (11). Differentiating Eq. (10) also yields an overall factor alpha that does not appear in Eq. (11). Because both the training-signal analysis in Section 5.2 and the reproducibility of the method depend on which loss is actually optimized, please reconcile the two descriptions and state the implemented objective precisely.
The paper says, at Abstract; §4.2; Appendix A.2
The review did not quote this passage.
Pith says: Objection 4 of 4 · major
The abstract's claim of 'state-of-the-art performance even without privileged information' and Section 4.2's statement that the WISD variants 'surpass the reported five-benchmark averages of baselines that use privileged information' rest on comparisons to published numbers obtained under different protocols, as Appendix A.2 discloses: OPSD and PW-OPSD math results use Avg@12 at temperature 1.0 with a 38,912-token limit, SDPO math uses …
more of objection 4
… Avg@12, and the OPSD/SDPO/LOPD code results are pass@1 at temperature 0.7 with a 16,384-token limit, whereas WISD uses Avg@32 (math), Avg@4 (code), temperature 0.6, and a 40,960-token limit. Moreover, these privileged-information baselines appear in Table 2 only for Qwen3-4B and OLMo-3-7B-Think, so the 'surpass' claim has no direct support for the other three models. I recommend rewording these claims to 'competitive with reported privileged-information baselines under different evaluation protocols', or re-evaluating the baselines under the WISD protocol where checkpoints permit; the same-protocol WISD-versus-original comparisons in Table 2 are unaffected and already carry the paper's main message.
4 minor objections
The paper says, at §5.2
Pith says: minor objection 1 of 4
Between Table 4 and Figure 4 there is a stray passage ('Divide total area by 400: 4021.25 / 400...') that appears to be leftover token-level content from a model rollout; it should be deleted.
The paper says, at Figures 4 and 7
Pith says: minor objection 2 of 4
The math expressions in Figures 4(b) and 7 are garbled in several places (for example, '(a + 2 − 5a − 2)' should presumably read '(a + 2 − 5/(a − 2))'); please regenerate the figures with clean typesetting and re-check the underlined tokens against the surrounding text.
show 2 more
The paper says, at §4.2; Table 2
Pith says: minor objection 3 of 4
No confidence intervals or repeated-seed variance are reported anywhere in the main results; with 30 problems per math benchmark, several reported improvements are on the order of one problem (for example, Qwen3-4B AIME25 rises from 65.6 to 65.8 under quantization), and a sentence noting the per-cell granularity or reporting variance across evaluation seeds would help readers calibrate the aggregate claims.
The paper says, at Abstract and §4.2 and §6
Pith says: minor objection 4 of 4
The abstract, Section 4.2, and Section 6 repeat the same headline numbers (4.0, 5.5, 21-of-25, 23-of-25) nearly verbatim; consider keeping the abstract version and referring back to it in the later sections to reduce redundancy.
What would settle it
Run the standard WISD pipeline on Qwen3.5-4B with AIME25 and LCB v6 (avoiding the AIME24 development benchmark), adding three arms with the exact Table 5 configuration: (1) flipped sign, r' = log piw - log pi0 in Eq. (4); (2) shuffled contrast, where the r_w values at each state are randomly permuted across the 16 cached candidate tokens so the reward multiset is unchanged but token identities are destroyed; (3) the published WISD arm as reference, plus the Base model. Compare Avg@32/Avg@4 accuracy. If arms (1) or (2) match or exceed (3), the sign or structure of the weakening contrast is not load-bearing and the mechanism claim should be weakened. If (3) improves over Base while (1) and (2) do not, the direction assumption is supported.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 31 Oct 2026
- Reader holds up, with conditions
- Skeptic agrees
- Editor wrote the verdict above
- Referee major revision: 4 major objections, 4 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
WISD is a practical, label-free self-improvement recipe, and the core internal result looks real. The idea is simple: take a reasoning model, make a degraded copy by 3-bit quantization or magnitude pruning, and train the original on the token-level log-probability contrast between itself and the degraded copy, using its own unlabeled rollouts. The same-setup comparisons in Table 2 are consistently positive, the ablations show an intermediate-damage sweet spot rather than a monotone hack, and the two confidence baselines (self-enhancing and temperature-softened rewards) stay flat. That combination is credible evidence that weight degradation is doing something useful. The paper also ships code and is careful to credit the prior weak-to-strong OPD and contrastive-decoding work; the citation pattern is fine.
What is genuinely new is using weight degradation of the same checkpoint as the negative member in training-time distillation, with no privileged information. That is a useful move for post-training pipelines where unlabeled prompts are cheap. It is not a new theory of reasoning, and the paper does not claim one.
The main soft spot is exactly what the stress test says: the sign of the contrast is the least controlled premise. Equation (3) assumes tokens favored by the original over the weakened copy are the ones to amplify. The current controls rule out simple confidence reinforcement and show that too much damage hurts, but nothing flips the sign or shuffles the rewards while keeping their marginal distribution. A flipped-sign or shuffled-contrast control would settle whether the specific original-over-weak direction matters, or whether any structured perturbation would do. Given the intermediate sweet spot and the role-level analysis, I suspect the direction assumption is right, but it is not isolated.
Two smaller issues. AIME24 is disclosed in Appendix A.2 as the development benchmark and still sits in the main table; that should be fixed by reporting a held-out version. And there are no repeated-seed or error-bar numbers, which matters on 30-problem benchmarks. The SOTA comparison also mixes protocols: Avg@32 and Avg@4 at temperature 0.6 against Avg@12 and pass@1 baseline numbers from other papers. The internal comparisons stand, but "surpassing privileged-information baselines" is weaker than the same-protocol claim would be.
Who this is for: people working on reasoning-model post-training and self-improvement. It deserves a serious referee. I would send it out and ask for the sign/shuffle control, AIME24 removal or reanalysis, and seed variance reporting. With those, the paper would be solid.
The strongest objection
The load-bearing premise is the sign of Eq. (3): r_w(a|s) = log pi0(a|s) - log piw(a|s) is assumed to identify tokens worth amplifying. All reported WISD runs train the student toward this exact signed contrast. The two controls in Table 3 (self-enhancing and temperature weak) test whether confidence alone explains the gain and find that it does not; the b/tau sweeps in Figure 3 show an intermediate-damage sweet spot. Neither control varies the sign of r_w or breaks its token-level structure while keeping its marginal distribution. If a flipped-sign objective (favor tokens preferred by the weak model) or a shuffled-contrast objective (same multiset of rewards, randomly assigned to candidate tokens) matched or exceeded the standard signal, the gains would not support the claim in Section 3.2 that the contrast 'specifies which actions the student should favor more or less.' The improvement could then be a generic perturbation or regularization effect, and the Section 5.2 attribution of gains to structured exploration and verification supervision would be unconvincing. The empirical gains are consistent with the direction assumption but do not isolate it.
The reader's CONDITIONAL verdict is sensible, and my check targets the same weakest premise. The strongest evidence for WISD is the same-protocol comparison against the original checkpoint across 25 settings, with positive gains in 21 and 23 settings, so the core empirical result is not in doubt. The protocol mismatch against published privileged-information baselines is real but bears on the secondary 'state-of-the-art' claim rather than on whether weakening contrast improves the model. The unisolated premise is the sign and token-level structure of r_w. The Table 3 baselines and Figure 3 ablations rule out pure confidence reinforcement and show an intermediate-damage sweet spot, which is good; they do not rule out that any direction of contrast, or a reward with the same marginal distribution but scrambled token identity, would produce similar gains. Adding flipped-sign and shuffled-contrast arms would settle whether the weakening-induced direction is actually responsible. If those controls reproduce the gain, the mechanism story in Sections 3.2 and 5.2 would need revision, though the empirical method might still stand. This keeps the verdict CONDITIONAL rather than moving it.
The referee's report
If the reported results hold, this is a genuinely useful property: a reasoning model can be improved with no labels, no verifier, and no privileged teacher information, using only a checkpoint, unlabeled prompts, and a cheaply constructed degraded copy. The core internal comparison, WISD against the original checkpoint under an identical protocol, is consistently positive across 25 settings, and the ablations exhibit the intermediate-damage sweet spot one would expect from a contrast-strength trade-off rather than pure confidence reinforcement. The paper deserves credit for shipping code, disclosing that AIME 2024 was used as a development benchmark, running two confidence-control baselines, and characterizing the training signal (Table 4) rather than stopping at aggregate accuracy. The main caveats are interpretative: the sign and token-level structure of the contrast are assumed rather than controlled, and the headline comparisons to privileged-information baselines are made across different evaluation protocols.
Pith's reading of the paper (a machine's summary, not the authors')
This paper claims that a large reasoning model becomes better at mathematics and code by learning against a deliberately weakened copy of itself, requiring no privileged information: no reference solutions, no expert demonstrations, no environment feedback, and no reward labels. The weakened copy is built by a training-free operator on the original weights, either symmetric group-wise 3-bit quantization or magnitude pruning, and the training signal is the per-token log-probability gap between the original model and its damaged copy, evaluated on the original model's own generated trajectories. Across five model families and five benchmarks, the paper reports average accuracy gains of 4.0 percentage points with quantization and 5.5 with pruning, with positive gains in 21 and 23 of 25 model-benchmark settings, and improvements that surpass the published averages of baselines that do use privileged information. If this holds, unlabeled prompts plus a single checkpoint are enough to drive meaningful self-improvement in reasoning.
What carries the argument. The load-bearing object is the weakening contrast $r_w(a|s) = \log \pi_0(a|s) - \log \pi_w(a|s)$, the token-level log-probability gap between the original model and its degraded copy on identical prompts and prefixes. It converts weight damage into a dense, token-specific training signal whose sign tells the student which tokens to favor and which to suppress, via the tilted target $q_w \propto \pi_0 \exp(r_w/\alpha)$ and the distillation loss in Eq. (5). Two training-free operators instantiate the weakening: symmetric round-to-nearest group-wise quantization at 3 bits, and group-relative magnitude pruning at threshold $\tau = 1/6$ with $\ell^2$ row-norm restoration. Their role is to create a structured degradation, coarsened weight values or removed low-magnitude connections, that is comparable token-by-token with the original model while requiring no calibration data.
Core claim. The central claim is that weakening is a source of supervision. WISD takes the original model $\pi_0$ as the stronger member of a pair and a weight-degraded copy $\pi_w$ as the weaker member, then defines the contrast $r_w(a|s) = \log \pi_0(a|s) - \log \pi_w(a|s)$ at every token of the original model's own rollouts. Tokens the original prefers over the weak copy are amplified, and tokens the weak copy prefers are suppressed, through a tilted target $q_w(a|s) \propto \pi_0(a|s)\exp(r_w(a|s)/\alpha)$ and a reverse-KL distillation objective with $\alpha=2$. The paper reports that both constructions improve accuracy over the base model on almost every model-benchmark setting, with pruning raising Qwen3.5-4B's HMMT 2025 accuracy from 57.7% to 70.5% and quantization raising Qwen3.6-35B-A3B's from 68.5% to 74.6%. Ablations show that an intermediate weakening strength is optimal, and comparisons against confidence-based signals, such as the model's own raw log-probabilities or a temperature-softened version of them, indicate the gain comes from the structured contrast rather than from merely reinforcing the original model's confidence.
The premise it rests on. The load-bearing premise is that the contrast $r_w(a|s) = \log \pi_0(a|s) - \log \pi_w(a|s)$ points toward better reasoning: tokens the original model prefers over its quantized or pruned copy are assumed to be the more correct ones, and amplifying them is assumed to improve the student. The paper adopts this direction from earlier weak-to-strong work and supports it empirically with ablations and case studies, but it does not prove it for weight-degraded pairs.
If the paper is right (Pith's inferences, not the authors')
Self-improvement in reasoning no longer requires a separate teacher: a checkpoint and unlabeled prompts suffice, lowering the data barrier for post-training already-capable models.
Weight degradation is a general source of supervision: alternative weak constructions such as Smooth W4A4, Wanda, and SparseGPT also yield gains, so the principle extends beyond the two default operators.
Damage strength is a tunable knob with an optimum: too mild a weakening gives too little contrast and too aggressive a weakening lets noise dominate, giving a practical rule of sweeping $b$ and $\tau$ to balance signal quality against contrast strength.
Trained students generate shorter responses than the base model while scoring higher, indicating the contrast suppresses low-value or verbose tokens as well as promoting correct reasoning steps.
The paper never directly tests whether the contrast is aligned with correctness; a token-level study comparing the rate of correct tokens under positive versus negative contrast would either confirm the proposed mechanism or replace it.
WISD moves the expert-amateur contrast used by contrastive decoding at inference time into training time; a natural extension is to ask whether decoding the trained student against a weakened copy adds anything on top of WISD training, or whether the two capture the same signal.
Because the signal concentrates on exploration and verification tokens, about 22% and 11% of the gradient signal versus 15% and 8% of tokens, WISD may function partly as a targeted entropy regularizer; comparing it against an entropy-bonus reinforcement learning baseline at matched compute would separate these explanations.
The method invites an iteration loop the paper does not run: weaken the improved student and distill again. Compounding gains across iterations would make WISD a genuine self-improvement loop, while a plateau would mark it as a one-shot correction of the original policy.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Self-distillation offers a way to improve reasoning models when a suitable external teacher is unavailable. However, common approaches rely on privileged information to construct a stronger self-teacher, limiting their use when only unlabeled prompts are available. In this paper, we introduce Weakening-Induced On-Policy Self-Distillation (WISD), which instead constructs a weaker counterpart of the original model and uses the contrast between the two as a training signal without requiring privileged information. We explore two ways to construct the weaker model: low-bit quantization and magnitude pruning. We evaluate WISD across diverse architectures and scales on mathematical and coding reasoning tasks. Across these benchmarks, quantization and pruning improve average accuracy over the original models by 4.0 and 5.5 percentage points, respectively, achieving state-of-the-art performance even without privileged information. Notably, pruning raises Qwen3.5-4B's accuracy from 57.7% to 70.5% on HMMT 2025 and from 39.1% to 56.7% on LiveCodeBench v6. Quantization raises Qwen3.6-35B-A3B's accuracy from 68.5% to 74.6% and from 61.8% to 69.5% on the same benchmarks, respectively. These results establish WISD as a novel and effective self-distillation framework for large reasoning models.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-30
The authors' main claim, as the classifier picked it out 0.97
“These results establish WISD as a novel and effective self-distillation framework for large reasoning models.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.13 · states limitation 0.18 · human participants 0.03 · controlled comparison 0.91 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.90 · mentions code or data availability 0.03
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Self-distillation offers a way to improve reasoning models when a suitable external teacher is unavailable. | background 1.00 | 0.05 | 0.03 | asserted 0.77 |
| However, common approaches rely on privileged information to construct a stronger self-teacher, limiting their use when only unlabeled prompts are available. | gap 0.74 | 0.05 | 0.02 | asserted 1.00 |
| In this paper, we introduce Weakening-Induced On-Policy Self-Distillation (WISD), which instead constructs a weaker counterpart of the original model and uses the contrast between the two as a training signal without requiring privileged information. | contribution 0.86 | 0.86 | 0.13 | asserted 0.99 |
| We explore two ways to construct the weaker model: low-bit quantization and magnitude pruning. | method 0.97 | 0.90 | 0.02 | asserted 0.99 |
| We evaluate WISD across diverse architectures and scales on mathematical and coding reasoning tasks. | method 0.78 | 0.95 | 0.03 | asserted 0.89 |
| Across these benchmarks, quantization and pruning improve average accuracy over the original models by 4.0 and 5.5 percentage points, respectively, achieving state-of-the-art performance even without privileged information. | result 1.00 | 0.95 | 0.46 | asserted 1.00 |
| Notably, pruning raises Qwen3.5-4B's accuracy from 57.7% to 70.5% on HMMT 2025 and from 39.1% to 56.7% on LiveCodeBench v6. | result 1.00 | 0.95 | 0.07 | asserted 1.00 |
| Quantization raises Qwen3.6-35B-A3B's accuracy from 68.5% to 74.6% and from 61.8% to 69.5% on the same benchmarks, respectively. | result 1.00 | 0.94 | 0.06 | asserted 1.00 |
| These results establish WISD as a novel and effective self-distillation framework for large reasoning models. | interpretation 0.98 | 0.73 | 0.91 | asserted 1.00 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.97
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-30: 1 request, 7,047 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (7)
Figures are the authors', reproduced from the paper.
References (144)
- [1] Agarwal, Rishabh and Vieillard, Nino and Zhou, Yongchao and Stanczyk, Piotr and Ramos Garea, Sabela and Geist, Matthieu and Bachem, Olivier , booktitle =. On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes , url = on Pith
- [2] Gu, Yuxian and Dong, Li and Wei, Furu and Huang, Minlie , booktitle = on Pith
- [3] Yecheng Wu and Song Han and Han Cai , year=. 2604.13010 , archivePrefix= on PithPith's reviewarXiv
- [4] An Yang and Anfeng Li and Baosong Yang and Beichen Zhang and Binyuan Hui and Bo Zheng and Bowen Yu and Chang Gao and Chengen Huang and Chenxu Lv and Chujie Zheng and Dayiheng Liu and Fan Zhou and Fei Huang and Feng Hu and Hao Ge and Haoran Wei and Huan Lin and Jialong Tang and... on PithPith's reviewarXiv
- [5] Olmo 3 , author=. 2026 , eprint= on Pith
- [6] Core Team and Bangjun Xiao and Bingquan Xia and Bo Yang and Bofei Gao and Bowen Shen and Chen Zhang and Chenhong He and Chiheng Lou and Fuli Luo and Gang Wang and Gang Xie and Hailin Zhang and Hanglong Lv and Hanyu Li and Heyu Chen and Hongshen Xu and Houbin Zhang and Huaqiu L... on PithPith's reviewarXiv
- [7] DeepSeek-AI and Anyi Xu and Bangcai Lin and Bing Xue and Bingxuan Wang and Bingzheng Xu and Bochao Wu and Bowei Zhang and Chaofan Lin and Chen Dong and Chenchen Ling and Chengda Lu and Chenggang Zhao and Chengqi Deng and Chengyu Hou and Chenhao Xu and Chenze Shao and Chong Rua... on PitharXiv
- [8] Zhenpeng Su and Leiyu Pan and Xue Bai and Dening Liu and Guanting Dong and Jiaming Huang and Minxuan Lv and Wenping Hu and Fuzheng Zhang and Kun Gai and Guorui Zhou , year=. 2508.07629 , archivePrefix= on PitharXiv
- [9] Jujie He and Jiacai Liu and Chris Yuhao Liu and Rui Yan and Chaojie Wang and Peng Cheng and Xiaoyu Zhang and Fuxiang Zhang and Jiacheng Xu and Wei Shen and Siyuan Li and Liang Zeng and Tianwen Wei and Cheng Cheng and Bo An and Yang Liu and Yahui Zhou , year=. 2505.22312 , arch... on PithPith's reviewarXiv
- [10] Yu, Qiying and Zhang, Zheng and Zhu, Ruofei and Yuan, Yufeng and Zuo, Xiaochen and Yue, Yu and Dai, Weinan and Fan, Tiantian and Liu, Gaohong and liu, juncai and Liu, LingJun and Liu, Xin and Lin, Haibin and Lin, Zhiqi and Ma, Bole and Sheng, Guangming and Tong, Yuxuan and Zha... on Pith
- [11] Jain, Naman and Han, King and Gu, Alex and Li, Wen-Ding and Yan, Fanjia and Zhang, Tianjun and Wang, Sida and Solar-Lezama, Armando and Sen, Koushik and Stoica, Ion , booktitle = on Pith
- [12] Jiang, Shuyang and Liao, Yusheng and Zhang, Ya and Wang, Yanfeng and Wang, Yu , booktitle =. Overthinking Reduction with Decoupled Rewards and Curriculum Data Scheduling , url = on Pith
- [13] Han Wu and Yuxuan Yao and Shuqi Liu and Zehua Liu and Xiaojin Fu and Xiongwei Han and Xing Li and Hui-Ling Zhen and Tao Zhong and Mingxuan Yuan , year=. Unlocking Efficient Long-to-Short. 2503.20641 , archivePrefix= on PithPith's reviewarXiv
- [14] Yao, Yuxuan and LIU, Shuqi and Liu, Zehua and Li, Qintong and LIU, Mingyang and Han, Xiongwei and Guo, Zhijiang and Wu, Han and Song, Linqi , booktitle =. Activation-Guided Consensus Merging for Large Language Models , url =. doi:10.52202/085713-4133 , editor = on Pithdoi:10.52202/085713-4133
- [15] Xiaochong Lan and Yu Zheng and Shiteng Cao and Yong Li , year=. The Thinking Spectrum: An Empirical Study of Tunable Reasoning in. 2509.22034 , archivePrefix= on PitharXiv
- [16] Shih-Yang Liu and Xin Dong and Ximing Lu and Shizhe Diao and Mingjie Liu and Min-Hung Chen and Hongxu Yin and Yu-Chiang Frank Wang and Kwang-Ting Cheng and Yejin Choi and Jan Kautz and Pavlo Molchanov , year=. 2510.15110 , archivePrefix= on PitharXiv
- [17] Yang Li and Youssef Emad and Karthik Padthe and Jack Lanchantin and Weizhe Yuan and Thao Nguyen and Jason Weston and Shang-Wen Li and Dong Wang and Ilia Kulikov and Xian Li , year=. 2507.01921 , archivePrefix= on PithPith's reviewarXiv
- [18] Luo, Feng and Chuang, Yu-Neng and Wang, Guanchu and Le, Hoang Anh Duy and Zhong, Shaochen and Liu, Hongyi and Yuan, Jiayi and Sui, Yang and Braverman, Vladimir and Chaudhary, Vipin and Hu, Xia. A uto L 2 S : Auto Long-Short Reasoning for Efficient Large Language Models. Findin... on Pithdoi:10.18653/v1/2026.findings-acl.831
- [19] Rame, Alexandre and Couairon, Guillaume and Dancette, Corentin and Gaya, Jean-Baptiste and Shukor, Mustafa and Soulier, Laure and Cord, Matthieu , booktitle =. Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards , url =. doi:... on Pithdoi:10.52202/075280-3114
- [20] Wenhan Ma and Jianyu Wei and Liang Zhao and Hailin Zhang and Bangjun Xiao and Lei Li and Qibin Yang and Bofei Gao and Yudong Wang and Rang Li and Jinhao Dong and Zhifang Sui and Fuli Luo , year=. 2606.30406 , archivePrefix= on PithPith's reviewarXiv
show 80 more
- [21] Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models , author =. Proceedings of the 43rd International Conference on Machine Learning , year = on Pith
- [22] Kun Liang and Clive Bai and Xin Xu and Chenming Tang and Sanwoo Lee and Weijie Liu and Saiyong Yang and Yunfang Wu , year=. 2601.08310 , archivePrefix= on PithPith's reviewarXiv
- [23] Contrastive On-Policy Distillation , author=. 2026 , eprint= on Pith
- [24] Weak-to-Strong Generalization via Direct On-Policy Distillation , author=. 2026 , eprint= on Pith
- [25] Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolation , author=. 2026 , eprint= on Pith
- [26] Weak-to-Strong On-Policy Distillation , author=. 2026 , eprint= on Pith
- [27] Counteraction-Aware Multi-Teacher On-Policy Distillation for General Capability Recovery with Domain Preservation , author=. 2026 , eprint= on Pith
- [28] Yuran Wang and Zekun Wang and Bohan Zeng and Ruixu Zhang and Wenxuan Liu and Liu Yang and Yifan Dai and Yang Shi and Bozhou Li and Chengzhuo Tong and Daili Hua and Yuanxing Zhang and Wentao Zhang , year=. 2607.28022 , archivePrefix= on PithPith's reviewarXiv
- [29] Distilling the Knowledge in a Neural Network , author=. 2015 , eprint= on Pith
- [30] Kim, Yoon and Rush, Alexander M. Sequence-Level Knowledge Distillation. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 2016. doi:10.18653/v1/D16-1139 on Pithdoi:10.18653/v1/d16-1139
- [31] Kevin Lu and Thinking Machines Lab , title =. Thinking Machines Lab: Connectionism , year = on Pith
- [32] Gou, Jianping and Yu, Baosheng and Maybank, Stephen J. and Tao, Dacheng , year=. Knowledge Distillation: A Survey , volume=. International Journal of Computer Vision , publisher=. doi:10.1007/s11263-021-01453-z , number= on Pithdoi:10.1007/s11263-021-01453-z
- [33] A Survey of On-Policy Distillation for Large Language Models , author=. 2026 , eprint= on Pith
- [34] Yecheng Wu and Song Han and Han Cai , year=. 2607.28449 , archivePrefix= on PithPith's reviewarXiv
- [35] Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe , author=. ICML 2026 Workshop on Foundations of Deep Generative Models: Understanding Memorization, Generalization, and Reasoning , year= on Pith
- [36] Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes , author=. Third Conference on Language Modeling , year= on Pith
- [37] Entropy-Aware On-Policy Distillation of Language Models , author =. Proceedings of the 43rd International Conference on Machine Learning , pages =. 2026 , editor = on Pith
- [38] Scaling Reasoning Efficiently via Relaxed On-Policy Distillation , author=. 2026 , eprint= on Pith
- [39] Unmasking On-Policy Distillation: Where It Helps, Where It Hurts, and Why , author=. 2026 , eprint= on Pith
- [40] Shenzhi Yang and Guangcheng Zhu and Bowen Song and Haobo Wang and Mingxuan Xia and Xing Zheng and Yingfan Ma and Zhongqi Chen and Weiqiang Wang and Junbo Zhao and Gang Chen , year=. 2606.06021 , archivePrefix= on PithPith's reviewarXiv
- [41] Xu, Wenda and Han, Rujun and Wang, Zifeng and Le, Long and Madeka, Dhruv and Li, Lei and Wang, William and Agarwal, Rishabh and Lee, Chen-Yu and Pfister, Tomas , booktitle =. Speculative Knowledge Distillation: Bridging the Teacher-Student Gap Through Interleaved Sampling , url = on Pith
- [42] Self-Distillation Enables Continual Learning , author =. Proceedings of the 43rd International Conference on Machine Learning , year = on Pith
- [43] Reinforcement Learning via Self-Distillation , author =. Proceedings of the 43rd International Conference on Machine Learning , year = on Pith
- [44] Junlong Ke and Zichen Wen and Weijia Li and Conghui He and Linfeng Zhang , year=. Respecting Self-Uncertainty in On-Policy Self-Distillation for Efficient. 2605.13255 , archivePrefix= on PithPith's reviewarXiv
- [45] Self-Supervised On-Policy Distillation for Reasoning Language Models , author=. 2026 , eprint= on Pith
- [46] Yuxiao Yang and Xiaoyun Wang and Weitong Zhang , year=. 2605.12400 , archivePrefix= on PithPith's reviewarXiv
- [47] Black-Box On-Policy Distillation of Large Language Models , author=. 2026 , eprint= on Pith
- [48] Privileged Information Distillation for Language Models , author =. Proceedings of the 43rd International Conference on Machine Learning , year = on Pith
- [49] Zhaoyang Zhang and Shuli Jiang and Yantao Shen and Yuting Zhang and Dhananjay Ram and Shuo Yang and Zhuowen Tu and Wei Xia and Stefano Soatto , booktitle =. Reinforcement-aware Knowledge Distillation for. 2026 , url = on Pith
- [50] Haoran Xu and Hongyu Wang and Yifei Gao and Jiaze Li and Xiaofeng Zhang and Xiaosong Yuan , year=. 2606.09304 , archivePrefix= on PithPith's reviewarXiv
- [51] Not All Disagreement Is Learnable: Token Teachability in On-Policy Distillation , author=. 2026 , eprint= on Pith
- [52] On the Position Bias of On-Policy Distillation , author=. 2026 , eprint= on Pith
- [53] Zhuolin Yang and Zihan Liu and Yang Chen and Wenliang Dai and Boxin Wang and Sheng-Chieh Lin and Chankyu Lee and Yangyi Chen and Dongfu Jiang and Jiafan He and Renjie Pi and Grace Lam and Nayeon Lee and Alexander Bukharin and Mohammad Shoeybi and Bryan Catanzaro and Wei Ping ,... on PitharXiv
- [54] Haonan He and Haodi Lei and Yun Luo and Haoran Zhang and Shunkai Zhang and Yizhuo Li and Shengji Tang and Zhilin Wang and Runzhe Zhan and Lei Bai and Ganqu Cui and Fangchen Yu and Yafu Li and Peng Ye and Ning Ding and Yu Cheng , year=. 2608.14277 , archivePrefix= on PithPith's reviewarXiv
- [55] Cross-Tokenizer On-Policy Distillation via Byte-Prefix Marginalization , author=. 2026 , eprint= on Pith
- [56] Jie Sun and Mao Zheng and Mingyang Song and Qiyong Zhong and Gengsheng Li and Zhepei Hong and Chang Wu and Pengfei Liu and Junfeng Fang and Xiang Wang , year=. 2607.11012 , archivePrefix= on PithPith's reviewarXiv
- [57] Tao Liu and Taiqiang Wu and Mao Zheng and Xuan Luo and Runming Yang and Xuewei Yang and Junjie Wang and Yujiu Yang , year=. 2608.06802 , archivePrefix= on PithPith's reviewarXiv
- [58] Xinlei Yu and Gen Li and Qingyi Si and Guibin Zhang and Yuqi Xu and Congcong Wang and Shuai Dong and Kaiwen Tuo and Xiangyu Zeng and Kaituo Feng and Qunzhong Wang and Yang Shi and Xiaobin Hu and Xiangyu Yue and Jiaqi Wang and Shuicheng Yan , year=. 2606.30626 , archivePrefix= on PithPith's reviewarXiv
- [59] Gao, Huan-ang and Haohan Chi and Yong Yan and Shiyuan Feng and Hanlin Wu and Zheng Jiang and Bingxiang He and Wei-Ying Ma and Ya-Qin Zhang and Hao Zhou , year=. 2608.19098 , archivePrefix= on PithPith's reviewarXiv
- [60] Policy Distillation , author =. International Conference on Learning Representations , year = on Pith
- [61] Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning , author =. International Conference on Learning Representations , year = on Pith
- [62] Teh, Yee and Bapst, Victor and Czarnecki, Wojciech M. and Quan, John and Kirkpatrick, James and Hadsell, Raia and Heess, Nicolas and Pascanu, Razvan , booktitle =. Distral: Robust multitask reinforcement learning , url = on Pith
- [63] You, Shan and Xu, Chang and Xu, Chao and Tao, Dacheng , year=. Learning from Multiple Teacher Networks , url=. doi:10.1145/3097983.3098135 , booktitle= on Pithdoi:10.1145/3097983.3098135
- [64] Hydra: Preserving Ensemble Diversity for Model Distillation , author=. 2021 , eprint= on Pith
- [65] Shen, Chengchao and Wang, Xinchao and Song, Jie and Sun, Li and Song, Mingli , year=. Amalgamating Knowledge towards Comprehensive Classification , volume=. Proceedings of the AAAI Conference on Artificial Intelligence , publisher=. doi:10.1609/aaai.v33i01.33013068 , number= on Pithdoi:10.1609/aaai.v33i01.33013068
- [66] Shen, Chengchao and Xue, Mengqi and Wang, Xinchao and Song, Jie and Sun, Li and Song, Mingli , title =. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , month = on Pith
- [67] Li, Lei and Lin, Yankai and Ren, Xuancheng and Zhao, Guangxiang and Li, Peng and Zhou, Jie and Sun, Xu. From Mimicking to Integrating: Knowledge Integration for Pre-Trained Language Models. Findings of the Association for Computational Linguistics: EMNLP 2022. 2022. doi:10.186... on Pithdoi:10.18653/v1/2022.findings-emnlp.477
- [68] Wan, Fanqi and Huang, Xinting and Cai, Deng and Quan, Xiaojun and BI, Wei and Shi, Shuming , booktitle =. Knowledge Fusion of Large Language Models , url = on Pith
- [69] Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models , author=. 2022 , eprint= on Pith
- [70] Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time , author =. Proceedings of the 39th International Conference on Machine Learning , pages =. 2022 , editor = on Pith
- [71] Editing Models with Task Arithmetic , author =. The Eleventh International Conference on Learning Representations , year = on Pith
- [72] Yadav, Prateek and Tam, Derek and Choshen, Leshem and Raffel, Colin and Bansal, Mohit , booktitle =. doi:10.52202/075280-0310 , editor = on Pithdoi:10.52202/075280-0310
- [73] Yang, Enneng and Wang, Zhenyi and Shen, Li and Liu, Shiwei and Guo, Guibing and Wang, Xingwei and Tao, Dacheng , booktitle = on Pith
- [74] Yu, Le and Yu, Bowen and Yu, Haiyang and Huang, Fei and Li, Yongbin , booktitle =. Language Models are. 2024 , editor = on Pith
- [75] Davari, MohammadReza and Belilovsky, Eugene , year=. Model Breadcrumbs: Scaling Multi-task Model Merging with Sparse Masks , ISBN=. doi:10.1007/978-3-031-73226-3_16 , booktitle= on Pithdoi:10.1007/978-3-031-73226-3_16
- [76] Lu, Zhenyi and Fan, Chenghao and Wei, Wei and Qu, Xiaoye and Chen, Dangyang and Cheng, Yu , booktitle =. doi:10.52202/079017-2505 , editor = on Pithdoi:10.52202/079017-2505
- [77] Matena, Michael S and Raffel, Colin , booktitle =. Merging Models with Fisher-Weighted Averaging , url =. doi:10.52202/068431-1287 , editor = on Pithdoi:10.52202/068431-1287
- [78] Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustment , author =. Proceedings of the 41st International Conference on Machine Learning , pages =. 2024 , editor = on Pith
- [79] Zhong, Yifan and Ma, Chengdong and Zhang, Xiaoyuan and Yang, Ziran and Chen, Haojun and Zhang, Qingfu and Qi, Siyuan and Yang, Yaodong , booktitle =. Panacea: Pareto Alignment via Preference Adaptation for. doi:10.52202/079017-2405 , editor = on Pithdoi:10.52202/079017-2405
- [80] Guo, Yiju and Cui, Ganqu and Yuan, Lifan and Ding, Ning and Sun, Zexu and Sun, Bowen and Chen, Huimin and Xie, Ruobing and Zhou, Jie and Lin, Yankai and Liu, Zhiyuan and Sun, Maosong. Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment. Proceedi... on Pithdoi:10.18653/v1/2024.emnlp-main.85
- [81] Luo, Haotian and He, Haiying and Wang, Yibo and Liu, Shiwei and Li, Wei and Cao, Xiaochun and Tao, Dacheng and Tan, Naiqiang and Shen, Li. O1-Pruner : Length-Harmonizing Fine-Tuning for O1 -Like Reasoning Pruning. Findings of the A ssociation for C omputational L inguistics: A... on Pithdoi:10.18653/v1/2026.findings-acl.697
- [82] Han, Tingxu and Wang, Zhenting and Fang, Chunrong and Zhao, Shiyu and Ma, Shiqing and Chen, Zhenyu , booktitle =. Token-Budget-Aware. 2025 , pages =. doi:10.18653/v1/2025.findings-acl.1274 , url = on Pithdoi:10.18653/v1/2025.findings-acl.1274
- [83] Deep Think with Confidence , author =. 2025 , eprint = on Pith
- [84] Aggarwal, Pranjal and Welleck, Sean , booktitle =. 2025 , url = on Pith
- [85] Xia, Heming and Leong, Chak Tou and Wang, Wenjie and Li, Yongqi and Li, Wenjie , booktitle =. 2025 , pages =. doi:10.18653/v1/2025.emnlp-main.165 , url = on Pithdoi:10.18653/v1/2025.emnlp-main.165
- [86] Training Large Language Models to Reason in a Continuous Latent Space , author =. Second Conference on Language Modeling , year = on Pith
- [87] Zhang, Zhen and He, Xuehai and Yan, Weixiang and Shen, Ao and Zhao, Chenyang and Wang, Xin Eric , booktitle =. Soft Thinking: Unlocking the Reasoning Potential of. 2025 , url = on Pith
- [88] Liao, Baohao and Xu, Yuhui and Dong, Hanze and Li, Junnan and Monz, Christof and Savarese, Silvio and Sahoo, Doyen and Xiong, Caiming , booktitle =. Reward-Guided Speculative Decoding for Efficient. 2025 , url = on Pith
- [89] Pan, Rui and Dai, Yinwei and Zhang, Zhihao and Oliaro, Gabriele and Jia, Zhihao and Netravali, Ravi , booktitle =. 2025 , url = on Pith
- [90] Scaling Speculative Decoding with Lookahead Reasoning , author =. Advances in Neural Information Processing Systems , volume =. 2025 , url = on Pith
- [91] Fu, Yichao and Chen, Junda and Zhu, Siqi and Fu, Zheyu and Dai, Zhongdongming and Zhuang, Yonghao and Ma, Yian and Qiao, Aurick and Rosing, Tajana and Stoica, Ion and Zhang, Hao , booktitle =. Efficiently Scaling. 2025 , url = on Pith
- [92] Cai, Zefan and Xiao, Wen and Sun, Hanshi and Luo, Cheng and Zhang, Yikai and Wan, Ke and Li, Yucheng and Zhou, Yeyang and Chang, Li-Wen and Gu, Jiuxiang and Dong, Zhen and Anandkumar, Animashree and Asi, Abedelkadir and Hu, Junjie , booktitle =. 2025 , url = on Pith
- [93] Tian, Jiayi and Azizi, Seyedarmin and Zhao, Yequan and Potraghloo, Erfan Baghaei and McPherson, Sean and Sridhar, Sharath Nittur and Wang, Zhengyang and Zhang, Zheng and Pedram, Massoud and Kundu, Souvik , booktitle =. 2026 , url = on Pith
- [94] The Art of Efficient Reasoning: Data, Reward, and Optimization , author=. 2026 , eprint= on Pith
- [95] Revisiting Model Interpolation for Efficient Reasoning , author=. 2026 , eprint= on Pith
- [96] An, Chenxin and Xie, Zhihui and Li, Xiaonan and Li, Lei and Zhang, Jun and Gong, Shansan and Zhong, Ming and Xu, Jingjing and Qiu, Xipeng and Wang, Mingxuan and Kong, Lingpeng , year = on Pith
- [97] Embarrassingly Simple Self-Distillation Improves Code Generation , author=. 2026 , eprint= on Pith
- [98] Xueliang Zhao and Wei Wu and Jian Guan and Zhuocheng Gong and Lingpeng Kong , year=. 2509.19894 , archivePrefix= on PitharXiv
- [99] Zilin Zhu and Chengxing Xie and Xin Lv and slime Contributors , title =. 2025 , howpublished = on Pith
- [100] Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism , author=. 2020 , eprint= on Pith
The first 100 of 144 references are listed here. All of them are in this paper's citation record.
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment