Pith. sign in

REVIEW 3 major objections 5 minor 30 references

Evolving contexts can teach open-ended preferences if you anchor on a stable teacher and downweight conflicts.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 20:10 UTC pith:XIO7PPC4

load-bearing objection Clean reverse-KL lens plus a practical fix for evolving-context OPD; gains are real but partly ride on per-setting λ schedules, so treat the conflict gate as useful engineering more than automatic theory. the 3 major comments →

arxiv 2607.28022 v1 pith:XIO7PPC4 submitted 2026-07-30 cs.LG cs.AI

Flux-OPD: On-Policy Distillation with Evolving Contexts

Yuran Wang , Zekun Wang , Bohan Zeng , Ruixu Zhang , Wenxuan Liu , Liu Yang , Yifan Dai , Yang Shi
show 5 more authors
Bozhou Li Chengzhuo Tong Daili Hua Yuanxing Zhang Wentao Zhang
This is my paper
classification cs.LG cs.AI
keywords on-policy distillationcontext distillationreverse KLgeometric mean teacherevolving contextsopen-ended preferencesprompt optimizationmedical question answering
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Open-ended LLM tasks rarely offer verifiable rewards, so preferences are hard to turn into training signals. Fixed contexts help at first, then go stale once the student absorbs them; raw evolving contexts make the distillation target jump and often conflict with each other. The paper shows that reverse-KL on-policy context distillation actually pulls the student toward the geometric mean of context-conditioned teachers and carries an explicit conflict term that measures how much those teachers disagree. Flux-OPD uses that fact: it keeps the context-free teacher as a stable anchor, injects only the difference between context-conditioned and context-free teachers as a correction, and scales that correction by the conflict term so consistent contexts push harder and conflicting ones push less. On prompt optimization for video generation and medical question answering, this beats plain on-policy distillation and prior context-distillation variants.

Core claim

Under reverse KL, on-policy context distillation distills the student toward the geometric mean of context-conditioned teachers and isolates a nonnegative conflict term that measures disagreement among them. Flux-OPD exploits this by anchoring on the context-free teacher, adding contextual difference signals from evolving contexts, and weighting those corrections with the conflict term, yielding stronger students on open-ended tasks than OPD, OPCD, or OEL.

What carries the argument

Reverse KL decomposition: Ec[DKL(pθ ∥ qc)] = DKL(pθ ∥ qgeo) + (−log Z), where qgeo is the normalized geometric mean of context-conditioned teachers and −log Z is the conflict term. Flux-OPD builds the target as softmax(log q0 + λk Δk) with Δk = log qgeo,k − log q0 and λk set from the conflict via a clipped schedule.

Load-bearing premise

That a conflict score from only a few extracted contexts at each token is a trustworthy enough signal to decide how strongly those contexts should correct the student.

What would settle it

Train the same student–teacher pairs with Flux-OPD versus fixed-λ correction and pure OPD; if conflict-weighted λk does not improve VBench/HealthBench over the best fixed λ, or if larger context pools reverse the ranking, the weighting claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Open-ended preference learning can keep a dense teacher signal while still adapting to student progress via small, refreshed context pools.
  • Geometric-mean targets and conflict terms become practical design knobs for any reverse-KL on-policy context distillation setup.
  • Prompt-optimization and rubric-rich medical QA both benefit more from difference-signal injection than from fully imitating context-conditioned teachers.
  • Training stability improves when the context-free teacher remains the anchor and contexts only supply controlled corrections.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same reverse-KL conflict term could gate other privileged side-channels (tools, retrieval, multi-teacher ensembles) without rewriting the whole objective.
  • If conflict stays high across iterations, the method is effectively falling back to vanilla OPD—suggesting automatic early-stop or context-regeneration triggers.
  • Tasks where optional supplements (videos, rubrics) already align teachers may need less aggressive downweighting than pure free-form preference domains.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies on-policy distillation (OPD) for open-ended tasks that lack verifiable rewards. It decomposes the reverse-KL OPCD objective (Prop. 1, Eq. 8) into a distillation term toward the normalized geometric mean of context-conditioned teachers and a nonnegative conflict term −log Z that is independent of the student for fixed histories (Cor. 1). Motivated by this, Flux-OPD runs iterative context extraction and distillation within a single training run: it anchors the target to the context-free teacher, injects a contextual difference signal Δ_k = log q_geo,k − log q_0 (Eqs. 17–18), and sets the correction strength λ_k from the conflict term via a clipped schedule (Eq. 20). Empirically, Flux-OPD is reported to outperform OPD, OPCD, and a single-run OEL variant on prompt optimization for video generation and medical QA across several student–teacher pairs (Tabs. 1–3), with ablations on HealthBench (Tab. 4) and stability/generalization checks (Fig. 5).

Significance. If the claims hold, the work offers a practical way to combine dense teacher supervision with evolving, preference-carrying contexts in domains where RLVR-style rewards are unavailable. The reverse-KL decomposition is standard but cleanly applied and correctly identifies geometric-mean targets and a p-independent conflict measure; that analysis is a genuine contribution independent of the empirical method. The empirical setting (video prompt optimization with downstream generators; HealthBench medical QA) is realistic and harder than pure text imitation. Strengths include multi student–teacher pairs, an explicit ablation of evolving contexts / correction / weighting (Tab. 4), a three-run check versus OPD (Fig. 4), and training-stability and OOD IF-Eval probes (Fig. 5). The main significance risk is that headline gains may largely come from a tunable interpolation between q_0 and q_geo rather than from an automatic, theoretically grounded use of −log Z.

major comments (3)
  1. [Sec. 4.3, Eq. (20); Tab. 4; Appendix Tab. 7] Sec. 4.3 and Eq. (20): the central methodological claim is that the reverse-KL conflict δ_k = −log Z_k is a reliable indicator for setting correction strength. In practice every student–teacher–downstream triple uses a different (τ, α, [λ_min, λ_max]) schedule (Appendix Tab. 7; also Tabs. 5–6). The HealthBench ablation (Tab. 4) shows that fixed λ already recovers most of the lift (best static λ ≡ 0.7 → 20.03 vs full 20.61; OPD 19.63), so weighting is incremental rather than decisive. Please either (i) report a single frozen default schedule across all settings and re-evaluate Tabs. 1–3, or (ii) substantially strengthen the claim that conflict-gated λ is necessary by showing that the same frozen schedule systematically beats the best fixed-λ baseline on both tasks. Without this, δ_k functions mainly as a hand-tuned interpolation knob, which undercuts the link from Prop. 1 to the method.
  2. [Sec. 4.1; Eq. (19); Fig. 6] Sec. 4.1 and context pool C_k with M = 3: conflict is estimated from only three extracted contexts per iteration (and R sampled contexts at distillation). Prop. 1 treats E_c as expectation under a fixed context distribution; with M = 3 the Monte Carlo estimate of Z (and thus δ_k) is high-variance, especially under the higher-conflict 8B prompt-optimization regime in Fig. 6 / Fig. 14. The paper should quantify sensitivity of λ_k and final scores to M (and to R), and report variance of δ_k across seeds. If performance collapses or λ becomes effectively constant for other M, the conflict-weighting story needs to be qualified.
  3. [Sec. 5.1 Baselines; Fig. 1; Fig. 5a; Tabs. 1–3] Sec. 5.1–5.2, baselines: OEL is described in related work as alternating deployment and OPCD across runs, but is reimplemented here as single-run context updates every 300 steps—the same regime in which the paper itself documents loss surges (Fig. 1, Fig. 5a). That makes OEL look artificially weak relative to Flux-OPD’s stabilized target. Please either (i) include a multi-run OEL baseline closer to the cited protocol, or (ii) clearly frame the baseline as “single-run evolving OPCD” and avoid claiming superiority over OEL as originally proposed. Also, aside from Fig. 4 (OPD vs Flux-OPD only), main tables are single-run; confidence intervals or at least two more seeds on the full baseline suite would make the ranking credible.
minor comments (5)
  1. [Eqs. (15)–(18); Algorithm 1] Eq. (15)–(18): q_geo is defined with normalization Z_k, then Δ_k uses log q_geo − log q_0; the implementation in Algorithm 1 instead uses the unnormalized log ˜q_geo in the difference inside the softmax. These are equivalent up to a v-independent shift absorbed by softmax only if λ is applied uniformly—which it is—but the notation should be made consistent between the main text and the algorithm to avoid confusion.
  2. [Fig. 2] Fig. 2(a) “extent of distillation target shifts” is described via mean output embeddings but the embedding model, pooling, and distance metric are not specified. A short appendix note would make the figure reproducible.
  3. [Appendix C.1] Top-B = 64 logit truncation (Appendix C.1) is a nontrivial approximation to full-vocab reverse KL; a brief note on whether rankings change with larger B would help.
  4. [Abstract; Sec. 1; Sec. 3.2] Typos / wording: Abstract and §1 “analyzethe”, “motivates us toevolve”; §3.2 “Forward KLPrior work”; occasional missing spaces after periods. A pass for copy-editing is needed.
  5. [Tab. 1; Tab. 5] Table 1 vs Table 3: student sometimes already matches or exceeds the teacher on some generators before OPD; discuss briefly whether distillation is then mostly regularization versus preference transfer, especially when τ is raised because the student outperforms the teacher (Tab. 5).

Circularity Check

0 steps flagged

No significant circularity: reverse-KL decomposition is an algebraic identity; Flux-OPD is a designed method evaluated on external benchmarks.

full rationale

Proposition 1 expands Ec[DKL(pθ∥qc)] into DKL(pθ∥qgeo)+(−log Z) by substituting Ec[log qc]=log qgeo+log Z and applying Jensen; this is a standard rearrangement, not a fit or a definition of the target in terms of the claimed result. Corollary 1 only notes that −log Z is independent of pθ under fixed histories/contexts, so it contributes no student gradient—again an identity, not a tautological prediction. Contextual correction (Δk=log qgeo,k−log q0; qflux=softmax(log q0+λk Δk)) and conflict weighting (λk=α clip(1−δk/τ,…)) are explicit design choices motivated by that decomposition, not quantities forced by construction or by a self-cited uniqueness theorem. Empirical claims rest on external suites (VBench, Video-Bench, HealthBench, IF-Eval) that do not enter the loss definition. Hyperparameter schedules (τ, α, clip bounds in Tab. 7) are ordinary tuning, not fitted inputs relabeled as first-principles predictions. Citations to OPD/OPCD/OEL are external prior paradigms, not load-bearing self-citations that close the argument. No self-definitional loop, no uniqueness import, no renamed known law presented as derivation. Circularity score 0.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 2 invented entities

The central empirical claim rests on standard reverse-KL distillation math, the modeling choice that small pools of LLM-extracted experience items are valid preference contexts, and several hand-set scalars that map conflict into correction strength. No new physical entities are postulated; the ‘corrected teacher’ and ‘contextual difference signal’ are definitional constructions inside the method.

free parameters (4)
  • conflict threshold τ = 0.1 (default) or 0.2
    Sets the scale at which δ_k begins to suppress λ_k; chosen as 0.1 or 0.2 depending on whether the student already beats the teacher on a generator (Tab. 5, Tab. 7).
  • scaling factor α and clip bounds [λ_min, λ_max] = task-dependent (e.g. α=0.9 or 1.0; bounds in [0,1])
    Strength-calibration knobs retuned per teacher size and task (scaling α=0.9 for 8B prompt-opt; clipping ranges such as [0.1,0.9] or [0.7,0.9] for stronger/more consistent settings; Tab. 6–7).
  • context pool size M and update interval = M=3, every 300 steps
    M=3 extraction seeds and context refresh every 300 steps are fixed design choices that determine how fast and how diversely contexts evolve.
  • top-B logit truncation B=64 = 64
    Distillation loss keeps only top-64 teacher logits for efficiency; renormalization can change the effective target, especially in the tails the geometric mean cares about.
axioms (5)
  • domain assumption Reverse KL on-policy distillation is an appropriate training objective for transferring teacher behavior to a student on student-generated histories.
    Inherited from OPD literature (Sec. 3.1); not re-justified beyond citation.
  • domain assumption Experience items extracted by the teacher from a small set of student trajectories (plus optional rubrics/videos) faithfully encode task preferences that the context-free teacher lacks.
    Load-bearing for why evolving contexts add supervision (Sec. 1, 4.1); depends on extraction prompt quality.
  • standard math For fixed histories and fixed context-conditioned teachers, the conflict term −log Z contributes no direct student gradient, so optimizing DKL(p∥q_flux) is the right conditional objective.
    Corollary 1; follows from Z depending only on {qc}, not pθ.
  • ad hoc to paper Monotone down-weighting of λ_k with δ_k (via 1−δ_k/τ, scaled/clipped) improves the distillation target relative to any fixed λ.
    Design choice in Sec. 4.3; supported by Tab. 4 ablation but not derived as optimal.
  • standard math qc(v)>0 for all contexts and tokens so that log qc and the geometric mean are well-defined.
    Stated in Prop. 1; standard softmax support assumption.
invented entities (2)
  • contextual difference signal Δ_k no independent evidence
    purpose: Represents task-preference information as log q_geo,k − log q_0 so it can be added to the context-free anchor instead of replacing it.
    Definitional construction in Sec. 4.2 (Eq. 17); not an external object, but the method’s key intermediate.
  • corrected teacher q_flux_k no independent evidence
    purpose: Stable yet context-aware distillation target via softmax(log q_0 + λ_k Δ_k).
    Eq. 18; exists only inside the training loop.

pith-pipeline@v1.2.0-daily-grok45 · 19317 in / 3937 out tokens · 71684 ms · 2026-07-31T20:10:28.778988+00:00 · methodology

0 comments
read the original abstract

Large language model training in open-ended domains lacks verifiable rewards, making task preferences difficult to formalize as effective supervision. Contexts can convey such preferences, yet provide little additional supervision once distilled into the student, motivating contexts that evolve with student performance. However, directly using evolving contexts as in-training supervision results in an unstable distillation target and conflicting distributions, requiring mechanisms to stabilize target and downweight conflicts. In this paper, we analyze the effect of contexts through a decomposition of the reverse KL objective, revealing two findings: the student is distilled toward the geometric mean of context-conditioned teachers, and the objective contains a conflict term that measures conflicts among these teachers. Based on this decomposition, we propose Flux-OPD, an OPD paradigm that uses evolving contexts as in-training supervision to capture task preferences in open-ended domains. Flux-OPD treats the differences between context-conditioned and context-free teachers as contextual difference signals, injects them as contextual corrections into the context-free teacher anchor, and weights their correction strength using the conflict term as an indicator. Experiments on open-ended tasks show that Flux-OPD outperforms existing OPD paradigms, highlighting the potential to combine teacher supervision with evolving contexts.

Figures

Figures reproduced from arXiv: 2607.28022 by Bohan Zeng, Bozhou Li, Chengzhuo Tong, Daili Hua, Liu Yang, Ruixu Zhang, Wentao Zhang, Wenxuan Liu, Yang Shi, Yifan Dai, Yuanxing Zhang, Yuran Wang, Zekun Wang.

Figure 1
Figure 1. Figure 1: Motivations. (a) OPCD underperforms OPD on Health￾Bench, while OEL does not yield consistent gains. (b) Evolving contexts cause loss surges, and conflicts among them produce con￾flicting target distributions. capturing complex preference priors [16, 17, 25]. An alternative is to use contexts as supervision sig￾nals. Context Distillation and On-Policy Context Distilla￾tion (OPCD) [3, 27] distill the student… view at source ↗
Figure 2
Figure 2. Figure 2: Applying the Reverse KL Decomposition to Training [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of Flux-OPD. (a) illustrates the iterative training process, which divides a single training run into K iterations, each consisting of context extraction and context distillation. (b) illustrates the contextual correction strategy, which corrects the distillation target by combining the context-free teacher distribution with contextual difference signals from context-conditioned teachers. (c) illu… view at source ↗
Figure 4
Figure 4. Figure 4: Results across Three Independent Training Runs. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: (a) Training Stability. The training losses of OPD and Flux-OPD decrease steadily, whereas OEL exhibits loss surges. Flux￾OPD also maintains a stable gradient norm. (b) Out-of-Distribution Performance on IF-Eval. Flux-OPD achieves the highest prompt￾level strict accuracy, outperforming OPD and suggesting better cross-domain generalization. 0 0.6k 1.2k Steps 0.00 0.10 0.20 Conflict Qwen3-VL-Instruct-8B Qwen… view at source ↗
Figure 6
Figure 6. Figure 6: Conflict Dynamics. Results with Wan2.1-VACE-1.3B on the prompt optimization task. the conflict among context-conditioned teachers. We use τ = 0.1 by default and increase it slightly to τ = 0.2 when the initial student outperforms the teacher, allowing contexts to exert greater influence. For example, in Tab. 1, the student underperforms the teacher with Wan2.1-VACE￾1.3B but outperforms it with CogVideoX-2B… view at source ↗
Figure 7
Figure 7. Figure 7: Prompt for Context Extraction in the Prompt Optimization Task. [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Prompt for Context Extraction in the Medical Question Answering Task. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Prompt for Context Distillation in the Prompt Optimization Task. [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Prompt for Context Distillation in the Medical Question Answering Task. [PITH_FULL_IMAGE:figures/full_fig_p014_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: System Prompt for the Prompt Optimization Task. [PITH_FULL_IMAGE:figures/full_fig_p014_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Context Example for the Prompt Optimization Task. [PITH_FULL_IMAGE:figures/full_fig_p014_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Context Example for the Medical Question Answering Task. [PITH_FULL_IMAGE:figures/full_fig_p015_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Conflict Dynamics. The prompt optimization settings with the 8B teacher exhibit higher conflict values, whereas those with the stronger 32B teacher show higher overall context consistency. Despite using an 8B teacher, medical question answering also maintains low conflict, potentially because task-specific rubrics serve as optional supplements during context extraction. 17 [PITH_FULL_IMAGE:figures/full_f… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

30 extracted references · 16 linked inside Pith

  1. [1]

    On-policy distillation of language models: Learn- ing from self-generated mistakes

    Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learn- ing from self-generated mistakes. InInternational Con- ference on Learning Representations, pages 21246–21263,

  2. [2]

    Healthbench: Evaluating large language models towards improved human health.arXiv preprint arXiv:2505.08775, 2025

    Rahul K Arora, Jason Wei, Rebecca Soskin Hicks, Pre- ston Bowman, Joaquin Qui ˜nonero-Candela, Foivos Tsim- pourlas, Michael Sharman, Meghan Shah, Andrea Vallone, Alex Beutel, et al. Healthbench: Evaluating large language models towards improved human health.arXiv preprint arXiv:2505.08775, 2025. 7

    Pith/arXiv arXiv 2025

  3. [3]

    A general language assistant as a laboratory for alignment.arXiv preprint arXiv:2112.00861, 2021

    Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, et al. A general language assistant as a laboratory for alignment.arXiv preprint arXiv:2112.00861, 2021. 1, 2

    Pith/arXiv arXiv 2021

  4. [4]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025. 5

    Pith/arXiv arXiv 2025

  5. [5]

    Vpo: Aligning text-to-video generation models with prompt optimization

    Jiale Cheng, Ruiliang Lyu, Xiaotao Gu, Xiao Liu, Jiazheng Xu, Yida Lu, Jiayan Teng, Zhuoyi Yang, Yuxiao Dong, Jie Tang, et al. Vpo: Aligning text-to-video generation models with prompt optimization. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 15636– 15645, 2025. 6, 11

    2025

  6. [6]

    Revisiting on- policy distillation: Empirical failure modes and simple fixes

    Yuqian Fu, Haohuan Huang, Kaiwen Jiang, Jiacai Liu, Zhuo Jiang, Yuanheng Zhu, and Dongbin Zhao. Revisiting on- policy distillation: Empirical failure modes and simple fixes. arXiv preprint arXiv:2603.25562, 2026. 2

    Pith/arXiv arXiv 2026

  7. [7]

    Minillm: Knowledge distillation of large language models

    Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. Minillm: Knowledge distillation of large language models. InThe twelfth international conference on learning representations,

  8. [8]

    The false promise of imitating proprietary llms.arXiv preprint arXiv:2305.15717, 2023

    Arnav Gudibande, Eric Wallace, Charlie Snell, Xinyang Geng, Hao Liu, Pieter Abbeel, Sergey Levine, and Dawn Song. The false promise of imitating proprietary llms.arXiv preprint arXiv:2305.15717, 2023. 2

    Pith/arXiv arXiv 2023

  9. [9]

    Rubrics as re- wards: Reinforcement learning beyond verifiable domains

    Anisha Gunjal, Anthony Wang, Elaine Lau, Vaskar Nath, Yunzhong He, Bing Liu, and Sean Hendryx. Rubrics as re- wards: Reinforcement learning beyond verifiable domains. arXiv preprint arXiv:2507.17746, 2025. 1, 2, 6

    Pith/arXiv arXiv 2025

  10. [10]

    Video-bench: Human-aligned video gener- ation benchmark

    Hui Han, Siyuan Li, Jiaqi Chen, Yiwen Yuan, Yuling Wu, Yufan Deng, Chak Tou Leong, Hanwen Du, Junchen Fu, Youhua Li, et al. Video-bench: Human-aligned video gener- ation benchmark. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 18858–18868, 2025. 6, 11

    2025

  11. [11]

    Vbench: Comprehensive bench- mark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive bench- mark suite for video generative models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21807–21818, 2024. 6, 11

    2024

  12. [12]

    Reinforcement learning via self-distillation

    Jonas H ¨ubotter, Frederike L¨ubeck, Lejs Behric, Anton Bau- mann, Marco Bagatella, Daniel Marta, Ido Hakimi, Idan Shenfeld, Thomas Kleine Buening, Carlos Guestrin, and An- dreas Krause. Reinforcement learning via self-distillation. CoRR, abs/2601.20802, 2026. 2

    Pith/arXiv arXiv 2026

  13. [13]

    Sur les fonctions convexes et les in ´egalit´es entre les valeurs moyennes.Acta mathematica, 30(1):175–193, 1906

    Johan Ludwig William Valdemar Jensen. Sur les fonctions convexes et les in ´egalit´es entre les valeurs moyennes.Acta mathematica, 30(1):175–193, 1906. 3

    1906

  14. [14]

    Prompt-a-video: Prompt your video diffusion model via preference-aligned llm

    Yatai Ji, Jiacheng Zhang, Jie Wu, Shilong Zhang, Shoufa Chen, Chongjian Ge, Peize Sun, Weifeng Chen, Wenqi Shao, Xuefeng Xiao, et al. Prompt-a-video: Prompt your video diffusion model via preference-aligned llm. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 18725–18735, 2025. 6, 11

    2025

  15. [15]

    Vace: All-in-one video creation and editing

    Zeyinzi Jiang, Zhen Han, Chaojie Mao, Jingfeng Zhang, Yulin Pan, and Yu Liu. Vace: All-in-one video creation and editing. InProceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 17191–17202, 2025. 6

    2025

  16. [16]

    On-policy dis- tillation.Thinking Machines Lab: Connectionism, 2025

    Kevin Lu and Thinking Machines Lab. On-policy dis- tillation.Thinking Machines Lab: Connectionism, 2025. https://thinkingmachines.ai/blog/on-policy-distillation. 1

    2025

  17. [17]

    Lora with- out regret.Thinking Machines Lab: Connectionism, 2025

    John Schulman and Thinking Machines Lab. Lora with- out regret.Thinking Machines Lab: Connectionism, 2025. https://thinkingmachines.ai/blog/lora/. 1

    2025

  18. [18]

    Deepseekmath: Pushing the limits of math- ematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of math- ematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024. 1

    Pith/arXiv arXiv 2024

  19. [19]

    Learning by distilling context.CoRR, abs/2209.15189, 2022

    Charlie Snell, Dan Klein, and Ruiqi Zhong. Learning by distilling context.CoRR, abs/2209.15189, 2022. 2

    Pith/arXiv arXiv 2022

  20. [20]

    Structuring human objectives: A survey of rubrics for evaluation, alignment, and agentic ai.Alignment, and Agentic AI (June 20, 2026), 2026

    Hongru Xiao, Jie Li, Zhirui Li, XIang Li, Sunzhu Li, and Jiale Han. Structuring human objectives: A survey of rubrics for evaluation, alignment, and agentic ai.Alignment, and Agentic AI (June 20, 2026), 2026. 1, 2

    2026

  21. [21]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025. 5

    Pith/arXiv arXiv 2025

  22. [22]

    Self-distilled rlvr.arXiv preprint arXiv:2604.03128, 2026

    Chenxu Yang, Chuanyu Qin, Qingyi Si, Minghui Chen, Naibin Gu, Dingyu Yao, Zheng Lin, Weiping Wang, Jiaqi Wang, and Nan Duan. Self-distilled rlvr.arXiv preprint arXiv:2604.03128, 2026. 2, 3

    Pith/arXiv arXiv 2026

  23. [23]

    Learning beyond teacher: General- ized on-policy distillation with reward extrapolation.arXiv preprint arXiv:2602.12125, 2026

    Wenkai Yang, Weijie Liu, Ruobing Xie, Kai Yang, Saiyong Yang, and Yankai Lin. Learning beyond teacher: General- ized on-policy distillation with reward extrapolation.arXiv preprint arXiv:2602.12125, 2026. 2

    Pith/arXiv arXiv 2026

  24. [24]

    Cogvideox: Text-to- video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xi- aohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to- video diffusion models with an expert transformer. InIn- ternational Conference on Learning Representations, pages 83048–83077, 2025. 6

    2025

  25. [25]

    Llm-as-a-coach: Expe- riential learning for non-verifiable tasks.arXiv preprint arXiv:2607.18110, 2026

    Tianzhu Ye, Li Dong, Guanheng Chen, He Zhu, Xun Wu, Shaohan Huang, and Furu Wei. Llm-as-a-coach: Expe- riential learning for non-verifiable tasks.arXiv preprint arXiv:2607.18110, 2026. 1 9

    Pith/arXiv arXiv 2026

  26. [26]

    Online experiential learning for lan- guage models.arXiv preprint arXiv:2603.16856, 2026

    Tianzhu Ye, Li Dong, Qingxiu Dong, Xun Wu, Shaohan Huang, and Furu Wei. Online experiential learning for lan- guage models.arXiv preprint arXiv:2603.16856, 2026. 1, 2, 4, 6

    Pith/arXiv arXiv 2026

  27. [27]

    On-policy context distillation for language models

    Tianzhu Ye, Li Dong, Xun Wu, Shaohan Huang, and Furu Wei. On-policy context distillation for language models. arXiv preprint arXiv:2602.12275, 2026. 1, 2, 4, 6

    Pith/arXiv arXiv 2026

  28. [28]

    Self-distilled rea- soner: On-policy self-distillation for large language models

    Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover. Self-distilled rea- soner: On-policy self-distillation for large language models. arXiv preprint arXiv:2601.18734, 2026. 2

    Pith/arXiv arXiv 2026

  29. [29]

    Instruction-following evaluation for large language models,

    Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-following evaluation for large language models,

  30. [2023]

    Prompt Template All prompts are intentionally concise and requireminimal human engineering

    8 10 Appendix A. Prompt Template All prompts are intentionally concise and requireminimal human engineering. They contain no human-crafted do- main knowledge, allowing the models to rely on their own capabilities. A.1. Context Extraction Prompt Figs. 7 and 8 show the context extraction prompts used by OPCD, OEL, and Flux-OPD for the prompt optimization an...