Pith. sign in

REVIEW 3 major objections 6 minor 24 references

Matching student and reference trajectories from shared on-policy states beats KL distillation for flow models.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 12:27 UTC pith:22VFYA3T

load-bearing objection Real win is unweighted on-policy velocity match vs SDE-KL weighting; the trajectory-bound story is mostly unused at the headline K=1 result. the 3 major comments →

arxiv 2607.24522 v1 pith:22VFYA3T submitted 2026-07-27 cs.LG cs.CV

FlowCTS: On-policy Continuous Trajectory Supervision of Flow Models

Kaiyang Ye , Yuan Ge , Junxiang Zhang , Bei Li , Ziming Zhu , Haishu Zhao , Xiaoqian Liu , Chenglong Wang
show 3 more authors
Jingbo Zhu Zhengtao Yu Tong Xiao
This is my paper
classification cs.LG cs.CV
keywords flow matchingon-policy distillationtrajectory supervisionvelocity matchingtext-to-imagepost-trainingSDE transition kernels
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Flow models generate images by integrating a continuous velocity field from noise to data, so they lack the natural next-token distributions that make on-policy distillation work for language models. This paper argues that the right dense supervision is continuous trajectory matching: from a state the student actually visits, force the student’s subsequent path to stay close to a reference expert’s path. From the integral link between paths and velocities they derive a weighted velocity-matching upper bound, then discretize it into practical losses controlled by how many future steps K are supervised. In a multi-expert on-policy setup the single-state version already beats transition-kernel KL distillation with faster convergence and higher GenEval, OCR, and PickScore, and also beats a mixed-reward RL baseline; the same idea improves off-policy fine-tuning, especially OCR. The authors locate the gain in better early-denoising supervision: KL weights from auxiliary SDEs pile loss near the end of sampling, exactly where student–reference velocity gaps are smallest.

Core claim

On-policy distillation for flow models is more effective when formulated as continuous trajectory supervision from shared student-visited states than as reverse-KL on auxiliary SDE transition kernels. The trajectory discrepancy admits a temporally weighted velocity-matching upper bound whose Euler discretization yields a family of objectives parameterized by supervision horizon K; K=1 already outperforms vanilla KL-OPD with faster convergence on GenEval, OCR, and PickScore, while longer K trades richer path information against harder optimization.

What carries the argument

Flow Continuous Trajectory Supervision (FlowCTS): the integrated squared gap between student and reference trajectories started from the same state, bounded by a trajectory-derived weighted velocity MSE (the velocity upper bound) and discretized into a K-step weighted velocity-matching loss.

Load-bearing premise

Optimizing the Cauchy–Schwarz velocity upper bound and its short Euler discretization is assumed to be a faithful enough stand-in for true path matching, rather than gains coming mainly from plain on-policy velocity regression without that bound.

What would settle it

A controlled ablation that replaces the trajectory-derived weights with uniform or late-heavy weights (or plain unweighted velocity MSE at the same on-policy states) and checks whether the reported gains over KL-OPD and the early-timestep advantage disappear.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Dense OPD for continuous generators should target early denoising stages, where student–reference velocity gaps and structural decisions are largest.
  • Supervision horizon K becomes an explicit scaling knob for flow post-training, balancing path information against optimization difficulty.
  • The same trajectory objective applies off-policy: multi-step FlowCTS-SFT can beat single-state velocity SFT, especially on text rendering.
  • Multi-reference capability consolidation can be done by routing student states to task experts under one trajectory loss instead of mixed sparse rewards.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If early-path supervision is the real driver, any OPD or RL method whose effective weight peaks late (SDE kernels, terminal rewards) is systematically misallocated for flow and diffusion models.
  • Concurrent direct velocity matching at student states may capture much of the K=1 gain; the distinctive claim to pressure-test is whether the derived multi-step weights and path rollouts add value beyond local field regression.
  • Treating short trajectory segments as the supervision atom could transfer to video or other long-horizon continuous generators where exposure bias compounds over many steps.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes FlowCTS, an on-policy distillation objective for flow models that supervises the discrepancy between student and reference ODE trajectories launched from a shared student-visited state. Starting from the integral identity between trajectory and velocity fields, the authors derive a Cauchy–Schwarz velocity upper bound (VUB, Eq. 8) with a trajectory-derived weight w(r), and discretize it over K Euler steps into a weighted velocity-matching objective with coefficients α_i = [K(K+1)−i(i+1)]/2 (Eq. 10, App. B). In a multi-reference setup on SD3.5-Medium with task-specific Flow-GRPO experts as references, K=1 FlowCTS-OPD reportedly improves over a transition-kernel KL baseline (Vanilla OPD, following Flow-OPD) on GenEval (0.90→0.93), OCR (0.90→0.92), and PickScore (22.75→23.06), with faster convergence (Fig. 3) and better OOD compositional and preference metrics (Tables 2–3). The authors diagnose the KL baseline's weakness as a temporal supervision mismatch: the SDE-induced weight w_KL(t) concentrates supervision late in denoising while student–reference velocity gaps peak early (Fig. 4). An off-policy SFT variant and a study of supervision horizon K are also reported.

Significance. If the results hold, the paper makes a useful contribution to an active area: a controlled multi-reference OPD comparison in which prompt mixture, rollouts, reference routing, and optimization are held fixed between objectives (§4.1, App. C.1), a clean derivation of a trajectory-discrepancy upper bound with trajectory-derived weights (Apps. A–B), a training-budget control for the K=2 comparison (App. E), and a temporal-misallocation analysis of KL-based Flow-OPD (§4.4, Fig. 4) that is consistent with, and adds evidence to, the TempFlow-GRPO timing story. The off-policy extension and OOD evaluations on T2I-CompBench and DrawBench broaden the evidence. However, the significance is currently capped by two factors: the objective at the headline operating point (K=1) coincides with the concurrent DanceOPD field-regression objective, so the distinctive "trajectory" machinery is not what delivers the reported gains; and the multi-step variants that do exercise the machinery give non-monotonic or negative effects (PickScore 22.60 at K=3). The honest, still-valuable version of this paper is "unweighted velocity regression at student-visited states beats KL-weighted OPD, and here is a bound/

major comments (3)
  1. [§3.3, Eq. (10); §4.2, Table 1; §4.4, Eq. (11)] The headline comparison is at K=1, where α_0=1 and Eq. (10) is exactly unweighted mean-squared velocity matching between student and reference at a single student-visited state — by the authors' own Eq. (11), the only difference from Vanilla OPD is the removal of the SDE-induced weight w_KL(t). The VUB derivation (Eq. 8, App. A) and the α-weight profile (Eq. 33, App. B) therefore do no work in the headline experiment, and the K=2,3 rows that would exercise the trajectory machinery show inconsistent or negative effects (PickScore 23.06→22.60 from K=1 to K=3; OCR flat). The abstract and §1 attribute the gains to the 'trajectory-derived velocity-matching objective,' but no experiment separates that attribution from the simpler hypothesis 'uniform velocity regression beats KL-weighted velocity regression.' The clean counterfactual — KL-weighted vs. unweighted velocity regression at identical
  2. [§4.2, Table 1; §4.5, Tables 2–3] All headline deltas (GenEval 0.90→0.93, OCR 0.90→0.92, PickScore 22.75→23.06) are reported from what appears to be a single training run per configuration, with no seed variance, confidence intervals, or significance testing. GenEval and OCR are known to have non-trivial evaluation variance, and a 0.02–0.03 gap is within plausible run-to-run fluctuation for LoRA fine-tuning. The same concern applies to Table 3 (e.g., ImageReward 1.21→1.27) and is acute in Table 2, where Vanilla OPD actually wins Color (0.8350 vs. 0.8295), Texture, and Non-Spatial, undercutting the claim of consistent OOD improvement. At minimum, the main Table 1 comparison and Fig. 3 curves should be replicated over ≥3 seeds (or the evaluation variance quantified by repeated decoding of fixed checkpoints), and claims of 'consistent' improvement should be restricted to metrics where the gap exceeds measured noise.
  3. [§3.2, Eq. (8); Appendix A, Eq. (17)] The VUB is obtained by one application of Cauchy–Schwarz followed by an integral reordering (App. A, Eq. 17); its tightness depends on the velocity mismatch being roughly aligned along the segment, which fails as the student and reference paths diverge. The paper never quantifies how loose L_VUB is relative to L_CTS, nor does it establish that minimizing the bound improves the true trajectory objective. This matters most exactly where the bound is stressed — K>1 — and the non-monotonic K results in Table 1 are consistent with the bound becoming uninformative as paths diverge. A diagnostic (e.g., reporting the empirical ratio L_CTS/L_VUB or the trajectory discrepancy at the end of training for K=1,2,3) would either substantiate or honestly bound the mechanism claim. Relatedly, the discrete derivation (App. B) applies Cauchy–Schwarz per step and then discards the h^3 factor; a brief note o
minor comments (6)
  1. [Abstract] Abstract/§1: 'Beyond on-policy setting,FlowCTS' is missing a space; similar spacing artifacts ('from0.90to0.93', 'Vanilla SFT ,') appear throughout, likely from PDF extraction but worth fixing in source.
  2. [§4.2, Table 1 (Part II)] The off-policy gains over Vanilla SFT are marginal and partly negative: at K=2 PickScore drops 21.68→21.66 and GenEval moves only 0.71→0.72; the 'consistently outperforms vanilla SFT' claim in the abstract rests almost entirely on OCR (+0.03–0.05). The abstract should reflect this metric-selective pattern.
  3. [Appendix E, Table 4] Table 4 uses a 'reduced training configuration' and its K=2 numbers (GenEval 0.93, PickScore 22.80) do not match the Table 1 K=2 row (0.94, 22.96). Please state explicitly what differs between the two configurations and caution that Table 4 is not comparable to Table 1.
  4. [§1, contributions list] The phrase 'a new scaling law dimension' (§1, contributions) overstates what is shown; three values of K with non-monotonic outcomes is a hyperparameter study, not a scaling law. Suggest 'a new design dimension' or similar.
  5. [§4.4, Fig. 4] Fig. 4 would be more convincing with the w_KL(t) weight itself plotted alongside panels (a) and (b), so the reader can see directly that the concentration of the KL loss is weight-driven rather than loss-driven. Also specify over how many states/prompts the curves are averaged.
  6. [§4.1; Appendix C.1] §4.1/Appendix C: the timestep fraction 0.99 yielding 'nine denoising transitions' and the 1:3:1 prompt-cycle ratio are load-bearing hyperparameters with no sensitivity analysis; a brief ablation or at least a justification for these choices would help reproducibility. Code/checkpoint release plans are not stated.

Circularity Check

0 steps flagged

No significant circularity: trajectory bound and weights are derived from ODE error accumulation, and empirical gains are measured against external baselines under a controlled shared pipeline.

full rationale

The paper’s load-bearing derivation is self-contained and does not reduce to its inputs by construction. L_CTS is defined as the integrated squared trajectory discrepancy from a shared state (Eq. 5); the velocity upper bound (Eq. 8 / App. A) follows from the integral relation between states and velocities plus Cauchy–Schwarz, yielding trajectory-derived weights w(r) and discrete α_i that are not fitted to GenEval/OCR/PickScore. Discretization (Eq. 10 / App. B) is a standard Euler reduction of that bound. At K=1 the objective collapses to unweighted velocity MSE—which the paper states explicitly—but that is a special case of the derivation, not a circular fit or a self-definitional loop. Empirical claims compare FlowCTS-OPD to Vanilla OPD (and RL/SFT baselines) under identical prompts, rollouts, references, and hyperparameters on held evaluation protocols; references are external Flow-GRPO checkpoints, not author-fitted uniqueness theorems. Concurrent DanceOPD and the incomplete isolation of “bound/weights vs plain MSE” are novelty/ablation concerns, not circularity. No self-citation is load-bearing for the central math or the metric lifts. Score 0.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 2 invented entities

The central claim rests on standard flow-matching ODE structure, a Cauchy–Schwarz relaxation from path L2 to weighted velocity L2, Euler discretization, and the experimental premise that Flow-GRPO specialists are valid multi-reference teachers under shared SDE rollouts. Free knobs are mainly K and training hyperparameters; no new physical entity is postulated—FlowCTS is a loss construction.

free parameters (4)
  • supervision steps K = K=1 primary; K=2 best overall on-policy in Table 1
    Discrete horizon chosen from {1,2,3,4}; non-monotonic metric trade-offs make K a hand-selected design parameter central to the “new scaling dimension” claim.
  • trajectory segment endpoints (t_e, t_s) / step fraction 0.99 = timestep fraction 0.99; 10 sampling steps
    Defines where on-policy states and multi-step rollouts live; fixed by training recipe rather than derived.
  • prompt mixture cycle OCR:GenEval:PickScore = 1:3:1
    1:3:1 routing affects multi-objective balance and is not justified beyond engineering choice.
  • LoRA rank/alpha, LR, EMA decay = rank 32, alpha 64, LR 1e-4, EMA 0.9
    Optimization hyperparameters shared across OPD variants but still free choices that can move absolute scores.
axioms (4)
  • domain assumption Student and reference states obey the same reverse-time ODE/Euler integration from a shared x_ts, so path difference equals integrated velocity mismatch (Eqs. 6–7).
    Standard flow-matching generative modeling; load-bearing for equating trajectory CTS with velocity objectives.
  • ad hoc to paper Cauchy–Schwarz bound on the time integral yields a valid training surrogate (L_CTS ≤ L_VUB) whose minimizers transfer to better generations.
    Bound is mathematically valid but possibly loose; paper optimizes the bound without proving tightness or equivalence of minimizers (§3.2, App. A).
  • domain assumption SDE student rollouts (Flow-GRPO style) define the correct on-policy state distribution for distillation comparisons.
    App. C.3 prefers SDE over ODE empirically; comparison to Vanilla OPD assumes this shared rollout is fair.
  • domain assumption Task-specific Flow-GRPO checkpoints are appropriate fixed references for multi-reference OPD without extra teachers.
    §4.1 experimental setup; success metrics partly reflect teacher quality.
invented entities (2)
  • FlowCTS / L_CTS continuous trajectory supervision objective no independent evidence
    purpose: Replace sequence-level reverse KL with integrated student–reference path discrepancy from a shared state.
    Core proposed loss; defined in Eq. 5 as a design choice, not an observed natural law.
  • Trajectory-derived velocity weights w(r) and discrete α_i independent evidence
    purpose: Turn path L2 into a practical weighted velocity regression with more weight near segment start.
    Derived via integral reordering after Cauchy–Schwarz; form is paper-specific though the inequality is standard.

pith-pipeline@v1.2.0-grok45-kimik3 · 20658 in / 3784 out tokens · 74417 ms · 2026-07-31T12:27:57.347459+00:00 · methodology

0 comments
read the original abstract

While on-policy distillation (OPD) effectively addresses sparse rewards and exposure bias in large language model post-training, its extension to flow models remains underexplored. To this end, we propose Flow Continuous Trajectory Supervision (FlowCTS), which matches subsequent student and reference trajectories initialized from the same student-visited state. Using the integral relation between trajectories and velocity fields, we derive a temporally weighted velocity-matching upper bound and discretize it into practical objectives parameterized by the number of supervision steps. Under a multi-reference setup, single-state FlowCTS-OPD outperforms vanilla KL-based OPD with faster convergence. FlowCTS-OPD improves GenEval from 0.90 to 0.93, OCR from 0.90 to 0.92, and PickScore from 22.75 to 23.06, while outperforming a mixed-reward RL baseline across all target metrics. Further analysis reveals a clear temporal supervision mismatch in vanilla KL-based OPD arising from its auxiliary SDE transition kernels. Beyond on-policy setting,FlowCTS also consistently outperforms vanilla SFT , particularly on OCR, while increasing supervision steps exhibit a trade-off between richer trajectory information and greater optimization difficulty.

Figures

Figures reproduced from arXiv: 2607.24522 by Bei Li, Chenglong Wang, Haishu Zhao, Jingbo Zhu, Junxiang Zhang, Kaiyang Ye, Tong Xiao, Xiaoqian Liu, Yuan Ge, Zhengtao Yu, Ziming Zhu.

Figure 1
Figure 1. Figure 1: Qualitative comparison of generated images. Our method F [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Autoregressive OPD and trajectory-level supervision for flow models. In autoregres￾sive models, sequence-level reverse KL decomposes by the chain rule into next-token supervision at student-generated prefixes. Since flow models do not naturally admit an analogous next-step fac￾torization, FlowCTS instead matches student and reference trajectories initialized from the same student-visited state. Using the i… view at source ↗
Figure 3
Figure 3. Figure 3: Training dynamics of Vanilla OPD and FLOWCTS-OPD across three task-specific met￾rics. The lower dotted horizontal lines denote the base-model performance, while the upper dashed horizontal lines denote the corresponding expert performance. Blue lines with circular markers rep￾resent OPD-KL, and red lines with square markers represent CTS. 4.3 EFFECT OF THE SUPERVISION STEP IN OPD We next study how far refe… view at source ↗
Figure 4
Figure 4. Figure 4: Temporal signal allocation in KL-based OPD. Both horizontal axes denote normalized [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparison of SDE and ODE sampling for constructing student-visited states during [PITH_FULL_IMAGE:figures/full_fig_p016_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 16 linked inside Pith

  1. [1]

    On-policy distillation of language models: Learning from self- generated mistakes

    Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from self- generated mistakes. InInternational Conference on Learning Representations, volume 2024, pp. 21246–21263,

    2024

  2. [2]

    Models marked with‡in Table 3 are evaluated at1024×1024

    Method Training steps GenEval↑OCR↑PickScore↑ FLOWCTS-OPD (K= 1)N0.91 0.89 22.93 FLOWCTS-OPD (K= 1)2N0.92 0.91 23.01 FLOWCTS-OPD (K= 2)N0.93 0.91 22.80 prompts, generation settings, and evaluator implementations. Models marked with‡in Table 3 are evaluated at1024×1024. Following Flow-GRPO (Liu et al., 2026a), we evaluate GenEval, OCR, and PickScore on thei...

    2023

  3. [3]

    Flow map matching with stochastic interpolants: A mathematical framework for consistency models.arXiv preprint arXiv:2406.07507,

    Nicholas M Boffi, Michael S Albergo, and Eric Vanden-Eijnden. Flow map matching with stochastic interpolants: A mathematical framework for consistency models.arXiv preprint arXiv:2406.07507,

    Pith/arXiv arXiv

  4. [4]

    Directly fine-tuning diffusion models on differentiable rewards

    Kevin Clark, Paul Vicol, Kevin Swersky, and David Fleet. Directly fine-tuning diffusion models on differentiable rewards. InInternational Conference on Learning Representations, volume 2024, pp. 4793–4822,

    2024

  5. [7]

    Minillm: Knowledge distillation of large language models

    Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. Minillm: Knowledge distillation of large language models. InInternational Conference on Learning Representations, volume 2024, pp. 32694–32717,

    2024

  6. [8]

    Tempflow-grpo: When timing matters for grpo in flow models.arXiv preprint arXiv:2508.04324,

    Xiaoxuan He, Siming Fu, Yuke Zhao, Wanli Li, Jian Yang, Dacheng Yin, Fengyun Rao, and Bo Zhang. Tempflow-grpo: When timing matters for grpo in flow models.arXiv preprint arXiv:2508.04324,

    Pith/arXiv arXiv

  7. [10]

    Consistency trajectory models: Learning proba- bility flow ode trajectory of diffusion

    10 Preprint Dongjun Kim, Chieh-Hsin Lai, WeiHsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon. Consistency trajectory models: Learning proba- bility flow ode trajectory of diffusion. InInternational Conference on Learning Representations, volume 2024, pp. 44493–44525,

    2024

  8. [11]

    Optimal flow matching: Learning straight trajectories in just one step.arXiv preprint arXiv:2403.13117,

    Nikita Kornilov, Petr Mokrov, Alexander Gasnikov, and Alexander Korotin. Optimal flow matching: Learning straight trajectories in just one step.arXiv preprint arXiv:2403.13117,

    Pith/arXiv arXiv

  9. [12]

    Kat-coder-v2 technical report.arXiv preprint arXiv:2603.27703, 2026a

    Fengxiang Li, Han Zhang, Haoyang Huang, Jinghui Wang, Jinhua Hao, Kun Yuan, Mengtong Li, Minglei Zhang, Pengcheng Xu, Wenhao Zhuang, et al. Kat-coder-v2 technical report.arXiv preprint arXiv:2603.27703, 2026a. Quanhao Li, Junqiu Yu, Kaixun Jiang, Yujie Wei, Zhen Xing, Pandeng Li, Ruihang Chu, Shiwei Zhang, Yu Liu, and Zuxuan Wu. Diffusionopd: A unified pe...

    arXiv

  10. [14]

    https://thinkingmachines.ai/blog/on-policy-distillation

    doi: 10.64434/tml.20251026. https://thinkingmachines.ai/blog/on-policy-distillation. Wenhan Ma, Jianyu Wei, Liang Zhao, Hailin Zhang, Bangjun Xiao, Lei Li, Qibin Yang, Bofei Gao, Yudong Wang, Rang Li, et al. Mopd: Multi-teacher on-policy distillation for capability integration in llm post-training.arXiv preprint arXiv:2606.30406, 2026a. Xu Ma, Yitian Zhan...

    Pith/arXiv arXiv

  11. [15]

    Progressive distillation for fast sampling of diffusion models.arXiv preprint arXiv:2202.00512,

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models.arXiv preprint arXiv:2202.00512,

    Pith/arXiv arXiv

  12. [16]

    Improving and generalizing flow-based generative models with minibatch optimal transport.arXiv preprint arXiv:2302.00482,

    Alexander Tong, Kilian Fatras, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid Rector- Brooks, Guy Wolf, and Yoshua Bengio. Improving and generalizing flow-based generative models with minibatch optimal transport.arXiv preprint arXiv:2302.00482,

    Pith/arXiv arXiv

  13. [17]

    Mimo-v2-flash technical report.arXiv preprint arXiv:2601.02780,

    Bangjun Xiao, Bingquan Xia, Bo Yang, Bofei Gao, Bowen Shen, Chen Zhang, Chenhong He, Chiheng Lou, Fuli Luo, Gang Wang, et al. Mimo-v2-flash technical report.arXiv preprint arXiv:2601.02780,

    Pith/arXiv arXiv

  14. [18]

    Dancegrpo: Unleashing grpo on visual generation.arXiv preprint arXiv:2505.07818,

    Zeyue Xue, Jie Wu, Yu Gao, Fangyuan Kong, Lingting Zhu, Mengzhao Chen, Zhiheng Liu, Wei Liu, Qiushan Guo, Weilin Huang, et al. Dancegrpo: Unleashing grpo on visual generation.arXiv preprint arXiv:2505.07818,

    Pith/arXiv arXiv

  15. [19]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    Pith/arXiv arXiv

  16. [20]

    Learning beyond teacher: Generalized on-policy distillation with reward extrapolation.arXiv preprint arXiv:2602.12125,

    Wenkai Yang, Weijie Liu, Ruobing Xie, Kai Yang, Saiyong Yang, and Yankai Lin. Learning beyond teacher: Generalized on-policy distillation with reward extrapolation.arXiv preprint arXiv:2602.12125,

    Pith/arXiv arXiv

  17. [21]

    Improved distribution matching distillation for fast image synthesis.Ad- vances in neural information processing systems, 37:47455–47487, 2024a

    Tianwei Yin, Micha ¨el Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and William T Freeman. Improved distribution matching distillation for fast image synthesis.Ad- vances in neural information processing systems, 37:47455–47487, 2024a. Tianwei Yin, Micha¨el Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William T Freeman, and Tae...

    Pith/arXiv arXiv

  18. [22]

    Diffusionnft: Online diffusion reinforcement with forward process.arXiv preprint arXiv:2509.16117,

    Kaiwen Zheng, Huayu Chen, Haotian Ye, Haoxiang Wang, Qinsheng Zhang, Kai Jiang, Hang Su, Stefano Ermon, Jun Zhu, and Ming-Yu Liu. Diffusionnft: Online diffusion reinforcement with forward process.arXiv preprint arXiv:2509.16117,

    Pith/arXiv arXiv

  19. [23]

    Danceopd: On-policy generative field distillation.arXiv preprint arXiv:2606.27377,

    Wei Zhou, Xiongwei Zhu, Zelin Xu, Bo Dong, Lixue Gong, Yongyuan Liang, Meng Chu, Leigang Qu, Lingdong Kong, Wei Liu, et al. Danceopd: On-policy generative field distillation.arXiv preprint arXiv:2606.27377,

    Pith/arXiv arXiv

  20. [2022]

    Flow-grpo: Training flow matching models via online rl.Ad- vances in neural information processing systems, 38:40783–40818, 2026a

    Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Wanli Ouyang. Flow-grpo: Training flow matching models via online rl.Ad- vances in neural information processing systems, 38:40783–40818, 2026a. Jie Liu, Gongye Liu, Jiajun Liang, Ziyang Yuan, Xiaokun Liu, Mingwu Zheng, Xiele Wu, Qiulin Wang, Menghan Xia,...

    Pith/arXiv arXiv

  21. [2023]

    Flow-opd: On-policy distillation for flow matching models.arXiv preprint arXiv:2605.08063,

    Zhen Fang, Wenxuan Huang, Yu Zeng, Yiming Zhao, Shuang Chen, Kaituo Feng, Yunlong Lin, Lin Chen, Zehui Chen, Shaosheng Cao, et al. Flow-opd: On-policy distillation for flow matching models.arXiv preprint arXiv:2605.08063,

    Pith/arXiv arXiv

  22. [2024]

    Training diffusion models with reinforcement learning

    Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. InInternational Conference on Learning Representations, volume 2024, pp. 4965–4987,

    2024

  23. [2025]

    Uni-opd: Unifying on-policy distillation with a dual-perspective recipe.arXiv preprint arXiv:2605.03677,

    Wenjin Hou, Shangpin Peng, Weinong Wang, Zheng Ruan, Yue Zhang, Zhenglin Zhou, Mingqi Gao, Yifei Chen, Kaiqi Wang, Hongming Yang, et al. Uni-opd: Unifying on-policy distillation with a dual-perspective recipe.arXiv preprint arXiv:2605.03677,

    Pith/arXiv arXiv

  24. [2026]

    One step diffusion via shortcut models

    Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. One step diffusion via shortcut models. InInternational Conference on Learning Representations, volume 2025, pp. 34668– 34684,

    2025