Pith. sign in

REVIEW 4 major objections 9 minor 16 references

Matching only CFG-composed velocities lets positive and negative branch errors cancel, and under privileged negative conditioning this becomes antagonistic and guidance-scale fragile; supervising the positive prediction and the CFG directio

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 06:31 UTC pith:IOFVLRXG

load-bearing objection Clean algebraic observation plus a practical fix for CFG-aware OPD when teacher negatives carry privileged context; causal story is a bit confounded but the engineering result holds up. the 4 major comments →

arxiv 2607.24731 v2 pith:IOFVLRXG submitted 2026-07-27 cs.CV cs.AIcs.LG

Rethinking Classifier-Free Guidance in On-Policy Diffusion Distillation

Bingnan Li , Haozhe Wang , Haozhong Xiong , Fangtai Wu , Jinpeng Yu , Yang Shi , Jiaming Liu , Ruihua Huang This is my paper
classification cs.CV cs.AIcs.LG
keywords on-policy distillationclassifier-free guidancediffusion modelsNegative Branch AsymmetryPositive-Direction Matchingdense-to-sparse video controlknowledge transfer
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Modern diffusion systems almost always use classifier-free guidance: the network is run twice (positive and negative conditions) and the two velocities are mixed with a scale chosen at inference. On-policy distillation teaches a student by matching the teacher at states the student itself visits. The natural extension is to match only the mixed, guided velocities. This paper shows that loss is under-identified at the branch level: many pairs of positive and negative errors give the same guided prediction, so the errors can cancel. When teacher and student share the same negative condition, both errors still tend to fall together and the naive objective works. When the teacher’s negative branch keeps privileged inputs the student never sees, joint reduction breaks and naive matching shrinks positive error while growing negative error—a failure mode they call Negative Branch Asymmetry. The fix is Positive–Direction Matching: match the positive prediction and the difference between positive and negative (the CFG direction) separately, so both branch errors must go to zero. In dense-to-sparse video control, that branch-aware objective transfers dense teacher knowledge more reliably and stays stable when the inference guidance scale changes.

Core claim

CFG-composed on-policy distillation is under-identified at the branch level. Under shared negative conditioning the ambiguity is benign and both branch errors fall together; under privileged negative conditioning naive guided matching induces Negative Branch Asymmetry—antagonistic dynamics that cut positive-branch error while raising negative-branch error—producing excess sensitivity to inference guidance scale. Positive–Direction Matching removes that compensation freedom by separately constraining the positive prediction and the CFG conditional direction.

What carries the argument

Positive–Direction Matching (PDM): the pointwise loss ||e+||² + λ||e+ − e−||², which matches the positive branch and the CFG conditional direction d = v+ − v− so that zero loss forces both branch errors to zero and blocks cross-branch cancellation in the guided objective.

Load-bearing premise

The claim that privileged information in the teacher’s negative branch is what turns branch ambiguity into antagonistic error dynamics rests on a two-setting contrast that also changes base model, task, and training setup, not on a single controlled toggle of that privilege alone.

What would settle it

Hold architecture, task, and training fixed and only toggle whether the teacher’s negative branch retains privileged inputs the student lacks; if naive CFG matching then shows joint error reduction without privilege and antagonistic positive-down/negative-up dynamics with privilege—while PDM does not—the NBA causal story stands; if the antagonism appears or disappears independently of that toggle, it fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • When a teacher’s native CFG path keeps privileged control or reference signals on the negative branch, matching only guided velocities is an unreliable OPD objective.
  • Branch-aware losses (PDM, and IBM as foil) should keep student behavior closer to the teacher across inference guidance scales, not only at the training scale.
  • Dense-to-sparse video control (pose, depth, scribble) can transfer dense teacher knowledge more reliably with PDM than with naive guided matching, SFT, or off-policy matching.
  • Supervising only an early slice of the student trajectory can cut video OPD cost while preserving the gains of branch-aware matching.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Any distillation setup that retains separate CFG branches but gives the teacher extra negative-branch context—reference images, dense controls, multi-modal negatives—is a candidate NBA regime, not only the paper’s two cases.
  • Methods that bake CFG into a single student head avoid branch ambiguity by construction, but lose the ability to retune guidance at inference; PDM is the complementary path for branch-retaining students.
  • A clean next experiment is a same-backbone ablation that only adds or strips privileged negative inputs, to separate conditioning asymmetry from model-family confounds.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 9 minor

Summary. The paper studies how classifier-free guidance (CFG) should be handled in on-policy diffusion distillation (OPD) when teacher and student each retain separate positive/negative branches. It first observes that matching only the CFG-composed velocity, L_naive ∝ ‖γe+ + (1−γ)e−‖², is under-identified at the branch level: nonzero branch errors can cancel (Eqs. 4–7). It then argues, through two contrasting image-domain case studies (text rendering on SD3.5 with shared null-text negatives; reference-conditioned style distillation on FLUX.2 with privileged reference latents in the teacher's negative branch), that this ambiguity is benign under shared negative conditioning but induces "Negative Branch Asymmetry" (NBA) — antagonistic branch-error dynamics where ‖e+‖ decreases while ‖e−‖ increases — under privileged negative conditioning. The proposed remedy, Positive–Direction Matching (PDM), supervises ‖e+‖² + λ‖e+ − e−‖² before composition, eliminating the compensation freedom. On a dense-to-sparse video control benchmark (Wan2.1-VACE-1.3B, 600 difficulty-stratified test clips, pose/depth/scribble), PDM and the IBM foil outperform naive OPD, SFT, and off-policy matching at the training scale (Table 2) and are far more robust to inference guidance-scale shifts (Table 3).

Significance. If the results hold, the paper makes a useful and timely contribution to a rapidly growing OPD literature that has so far either collapsed CFG into a single student prediction or only studied shared negative conditioning. The branch-ambiguity observation (Eqs. 5–7) is simple but correct and, to my knowledge, not previously isolated in the OPD context. The diagnostic framing — excess guidance-scale degradation relative to the teacher or a branch-aware student, rather than an absolute drop at γ=1, as the signature of NBA — is a falsifiable prediction and is tested as such (Table 1 vs. Table 3). The practical contribution is well supported: a 600-clip, difficulty-stratified video benchmark with control re-extraction metrics (MPJPE/PCK, affine-invariant depth SI-RMSE, tolerant scribble F1, FID/FVD), matched-budget baselines, and a guidance sweep (Table 3) in which naive OPD collapses at γ=1 (FVD 507.7) while PDM stays flat. The supervision-horizon ablation (Table 5) showing K=8-of-50 suffices at ~6× lower cost is independently useful. PDM is essentially parameter-light (λ ablated in Table 4) and trivially implementable.

major comments (4)
  1. [§3.1, §4.1 (Finding); App. 7.1.1–7.1.2] The load-bearing causal claim of the 'Finding' in §4.1.3 — that privileged negative conditioning (Eq. 14) is what turns branch ambiguity into NBA — rests on a single cross-setting contrast that confounds the conditioning variable with nearly everything else: base model (SD3.5-Medium vs FLUX.2-klein-4B), task (OCR rendering vs style transfer), γ_train (4.5 vs 2), LoRA rank (32 vs 64), data scale (19.6K vs 24 demonstrations), and training budget (1000 vs 2000 steps; §4.1.1, App. 7.1.1–7.1.2). If FLUX.2's editing-interface negative branch, its scheduler, or the 24-example LoRA regime independently produce antagonistic branch dynamics, the attribution in §3.1 is misdiagnosed even though PDM remains practically useful. A controlled toggle is feasible within the paper's own setups and would settle this: e.g., in the reference-conditioned setting, run the identical training with the teacher's n
  2. [§4.1.2, Fig. 2] The positive-only ablation (ℓ+ = ‖e+‖², Fig. 2 bottom) already exhibits antagonistic branch-error dynamics under privileged conditioning, even though it contains no composed objective and hence no compensation freedom of the form in Eq. 6–7. This suggests the harm may flow from conditioning asymmetry plus shared-parameter drift (positive updates actively degrading the negative branch) rather than from the under-identification of the composed loss per se — a different mechanism than the one the paper names. The manuscript should address this directly: e.g., report whether naive matching's negative-branch growth exceeds the positive-only trajectory (it appears similar in Fig. 2), and clarify what the composed objective adds beyond the drift already present under positive-only training. This matters because it determines whether PDM works by 'removing compensation freedom' (the paper's stor
  3. [§4.1.3, Fig. 3; App. 7.1.2] The claim in §4.1.3 that naive matching causes 'excess degradation relative to both the teacher and PDM' in the reference-conditioned setting is supported only qualitatively: Fig. 3 shows 2 held-out prompts, App. 7.1.2 describes 12 test prompts evaluated 'qualitatively by inspecting', with no metric at any guidance scale. By contrast, the shared-conditioning side of the contrast is quantitative (Table 1, 1,018 prompts, OCR reward across γ). The asymmetry weakens the central two-regime claim: one side of the contrast has a rigorous guidance-sweep measurement and the other has cherry-pickable images. Please add a quantitative guidance-scale sweep for the reference-conditioned case — e.g., a style-consistency score (CLIP/DINO similarity to the reference style) and a prompt-alignment score across γ ∈ {1, 1.5, 2, 2.5} on more than 12 prompts — or clearly delimit the reference-conditioned stud
  4. [§4.3.1, Table 4; cf. Table 3] Table 4 complicates the PDM narrative: PDM(λ=0) — i.e., positive-only matching, which Fig. 2 identifies as antagonistic under privileged conditioning — achieves the best overall MPJPE (4.24) and competitive PCK in the pose-only video ablation, and λ=1 improves over λ=0 only marginally on most control metrics (4.27 vs 4.24 MPJPE, 82.12 vs 81.80 PCK@0.1). The guidance-robustness benefit of direction matching is clear in Table 3, but the matched-scale benefit over positive-only is within noise-looking margins, and no γ-sweep is reported for λ=0. Since positive-only is also the natural ablation of PDM's first term, the paper should (a) include a guidance-scale sweep for λ=0 in Table 3 or Fig. 4, and (b) report variance or repeated-seed results sufficient to establish that λ=1's matched-scale gains are real. Otherwise the evidence supports 'branch-aware direction matching buys robustness' mor
minor comments (9)
  1. [§2.2, Eq. (3)] Eq. (3) and elsewhere use a rendered-glyph notation (ev, evT, evS) for the CFG-composed velocity that reads as a typo for v; a distinct symbol (e.g., ṽ or v_cfg, as in Fig. 1) would avoid confusion.
  2. [App. 7.1.1–7.1.2] λ differs across experiments without comment: λ=2 for text-rendering (App. 7.1.1), λ=1 for reference-conditioned and video (App. 7.1.2, §4.2.1). Given Table 4 shows sensitivity to λ, please state how each value was selected.
  3. [App. 7.1.2] In the reference-conditioned setup the 'teacher' adapter is itself updated by EMA of the student (decay 0.9999, App. 7.1.2), so the teacher drifts during training; this is atypical for distillation and could interact with the measured branch-error trajectories in Fig. 2. Please justify and, if possible, report a frozen-teacher variant.
  4. [Fig. 2] Fig. 2 branch-error curves appear to be single runs (seed 42, App. 7.1.1). Since NBA is defined by the sign pattern of these trajectories, error bars or multi-seed replication would strengthen the paper's central diagnostic.
  5. [Table 2] Table 2 bolding is inconsistent with the text's claim that PDM gives 'the most consistent' improvements: for pose, IBM has better Key MPJPE (2.88 vs 2.83 — fine) but also better FID (13.26 vs 13.52); for depth, naive OPD's FVD (72.20) is far worse than SFT (58.13), which deserves comment since it is naive OPD's worst matched-scale result.
  6. [§3.3; App. 7.1.1] Typos: 'smaller K xreduces training cost' (§3.3); 'the target strings to be rendered is given' (App. 7.1.1); 'transition-mean space' (App. 7.1.1) is undefined.
  7. [§3.2, Eq. (11)] IBM's weights γ² and (γ−1)² in Eq. (11) are introduced as retaining 'the per-branch weights of the expanded naive objective'; it would help to show the one-line expansion making this explicit, and to note that IBM's weighting is therefore tied to γ_train while PDM's λ is not — a potentially important practical difference.
  8. [§4.1.3] The reference-conditioned evaluation uses only 12 held-out prompts and qualitative inspection (App. 7.1.2); please state this limitation in the main text near Fig. 3 rather than only in the appendix.
  9. [§5.1] Related work would benefit from citing guidance-aware distillation analyses beyond the few-step literature (e.g., CFG-distillation or guidance-interval training works), to position the branch-retaining setting more precisely.

Circularity Check

0 steps flagged

No significant circularity: algebraic under-identification, empirical NBA diagnosis, and designed PDM objective are independent of each other and of external benchmarks.

full rationale

The paper’s load-bearing chain does not reduce outputs to inputs by construction. (1) Under-identification of naive CFG-composed OPD (Eq. 5–7) is elementary linear algebra on branch errors e+, e−; it does not smuggle in the later empirical claim. (2) Negative Branch Asymmetry is defined as an observed optimization trajectory (antagonistic ||e+||↓ / ||e−||↑ under privileged negative conditioning) and diagnosed on held-out prompts and control-reextraction metrics (OCR, MPJPE/PCK, SI-RMSE, scribble F1, FID/FVD)—not a fitted quantity renamed as a prediction. (3) PDM (ℓ_PDM = ||e+||² + λ||e+ − e−||²) is an intentionally identifiable loss whose zero set forces e+=e−=0; that is objective design, not a self-definitional derivation of a claimed natural law. (4) Citations to concurrent OPD work supply setting and baselines; none is a uniqueness theorem or ansatz that forces the central Finding. Confounds in the shared-vs-privileged contrast affect causal attribution (correctness), not circularity. Score 0; steps empty.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 3 invented entities

This is an empirical methods paper. Load-bearing background is standard diffusion/CFG/OPD practice plus the modeling choice that both teacher and student keep native CFG branches. Free parameters are ordinary loss/training knobs (λ, K, γ_train). Invented entities are named failure mode and objectives—diagnostic labels, not physical entities.

free parameters (4)
  • PDM direction weight λ = 1 (default); 2 in text-rendering appendix
    Balances positive matching vs CFG-direction matching in Eq. 10; default λ=1 chosen from pose-only ablation (Table 4).
  • OPD supervision horizon K = 8 of 50 steps (main video)
    Number of early student states supervised in video OPD (Eq. 12); chosen for cost/quality tradeoff (Table 5).
  • Training guidance scale γ_train = 4.5 (text-rendering), 2 (reference), 5 (video)
    Scale at which composed or branch-aware losses are optimized; differs by experiment and is a deployment knob baked into training.
  • IBM branch weights γ² and (γ−1)² = determined by γ_train
    Inherited from expanding the naive CFG loss; removes cross term but retains γ-dependent branch weighting (Eq. 11).
axioms (5)
  • domain assumption CFG denoising velocity is the linear mix ẽ = γv+ + (1−γ)v− with separately evaluated positive/negative branches.
    Standard CFG (Ho & Salimans); Section 2.2. Underpins the entire under-identification argument.
  • domain assumption Diffusion OPD reduces (up to timestep weight w(t)) to matching teacher and student velocity predictions on student-visited states.
    Taken from cited OPD formulations (Jiang et al., Fang et al., Li et al.); Eq. 2.
  • domain assumption Each model’s native CFG schema may retain auxiliary inputs (reference/control) on the negative branch even when text is null.
    Stated in Section 2.2–3.1; creates privileged negative conditioning in reference and dense-control settings (Eq. 14–15).
  • ad hoc to paper Excess guidance-scale degradation relative to the teacher or a branch-aware student is the diagnostic signature of NBA, not any absolute drop at γ=1.
    Definitional choice in Section 3.1 / Finding box; needed to separate intrinsic teacher CFG dependence from distillation-induced failure.
  • ad hoc to paper Early-trajectory supervision (first K states) is sufficient for dense-to-sparse video OPD performance.
    Efficiency design in Section 3.3; supported only by pose ablation Table 5, used for all main video runs.
invented entities (3)
  • Negative Branch Asymmetry (NBA) no independent evidence
    purpose: Name the failure mode where naive CFG-composed matching under privileged negatives reduces ||e+|| while increasing ||e−|| and induces excess guidance-scale sensitivity.
    Defined in Section 3.1 from observed optimization dynamics; diagnostic construct rather than a new physical mechanism. independent_evidence is behavioral (error curves, scale sweeps) inside this paper’s setups.
  • Positive–Direction Matching (PDM) independent evidence
    purpose: Branch-aware OPD loss matching v+ and the CFG direction d=v+−v− separately to eliminate cross-branch error compensation.
    Eq. 10; primary proposed method. Falsifiable via the reported metrics against naive/IBM baselines.
  • Independent Branch Matching (IBM) independent evidence
    purpose: Foil objective that matches branches independently with naive expanded weights, removing only the cross term.
    Eq. 11; used to separate branch-level supervision from PDM’s particular parameterization.

pith-pipeline@v1.2.0-grok45-kimik3 · 21692 in / 4020 out tokens · 86520 ms · 2026-07-31T06:31:02.572851+00:00 · methodology

0 comments
read the original abstract

On-policy distillation (OPD) adapts diffusion models by querying a teacher along trajectories generated by the current student, but how it should behave under classifier-free guidance (CFG), a default component of modern diffusion systems, remains poorly understood. Existing OPD methods naturally extend velocity matching to the CFG-composed prediction, directly matching teacher and student guided velocities. We show that this objective is under-identified at the branch level: positive- and negative-branch errors can compensate in the guided prediction. Through two contrasting cases, we find that naive matching remains effective under shared negative conditioning, where both branch errors decrease jointly. When the model's native CFG schema retains privileged information in the teacher's negative branch that is unavailable to the student, however, this joint reduction breaks down and the composed objective induces antagonistic branch-error dynamics, reducing the positive-branch error while increasing the negative-branch error. We term this failure mode Negative Branch Asymmetry (NBA). To address NBA, we introduce Positive--Direction Matching (PDM), a branch-aware OPD objective that separately constrains the positive prediction and the CFG conditional direction. We apply PDM to dense-to-sparse video control, where naive guided matching is highly sensitive to inference guidance scales, while branch-aware supervision enables more robust and effective knowledge transfer.

Figures

Figures reproduced from arXiv: 2607.24731 by Bingnan Li, Fangtai Wu, Haozhe Wang, Haozhong Xiong, Jiaming Liu, Jinpeng Yu, Ruihua Huang, Yang Shi.

Figure 1
Figure 1. Figure 1: Naive CFG-based OPD suffers from branch ambiguity. Matching only the CFG￾composed prediction allows positive and negative branch errors to cancel, creating infinitely many degenerate solutions. Our branch-aware objective, Positive–Direction Matching (PDM), resolves this ambiguity by separately constraining the positive prediction and the CFG conditional direction, restoring identifiable supervision and rob… view at source ↗
Figure 2
Figure 2. Figure 2: Branch-error dynamics under shared and privileged negative conditioning. We track the positive- and negative-branch ℓ2 error norms during text-rendering (top) and reference￾conditioned (bottom) distillation. Under shared negative conditioning, both errors decrease jointly for all objectives. With privileged reference conditioning, naive and positive-only matching reduce the positive error while increasing … view at source ↗
Figure 3
Figure 3. Figure 3: Distillation-induced CFG sensitivity under privileged reference conditioning. The left column shows the reference-style exemplars. For two held-out prompts, we compare the reference￾conditioned teacher with text-only students trained using naive matching or PDM at γtrain = 2. All outputs use matched prompts and initial noise. The teacher preserves the reference-supplied style across inference guidance scal… view at source ↗
Figure 4
Figure 4. Figure 4: Guidance-scale robustness in dense-to-sparse video control. Models are trained at γ = 5. PDM remains stable across inference scales, whereas naive OPD degrades off-scale. 4.2.2 MAIN RESULTS [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Additional qualitative results for reference-conditioned distillation. The left column shows reference-style exemplars, followed by outputs for two held-out prompts across inference guidance scales. Text-only students are trained at γtrain = 2 using naive matching or PDM, and the reference-conditioned teacher is shown for comparison. PDM more consistently follows the teacher and preserves the reference-sup… view at source ↗
Figure 6
Figure 6. Figure 6: Additional qualitative visualization of NBA in dense-to-sparse video control. Models are trained at γtrain = 5 and evaluated at lower inference guidance scales. Naive CFG-based OPD exhibits severe blurring and structural degradation as γtest moves away from the training configu￾ration, revealing its sensitivity to a specific CFG composition. PDM consistently preserves subject appearance and video structure… view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative comparison on dense-to-sparse pose control distillation under the default guidance scale γ = 5. The teacher receives dense pose guidance, while the student is conditioned only on sparse keyframes. PDM achieves more consistent structure and visual quality compared with other distillation objectives. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Qualitative comparison on dense-to-sparse depth control distillation under the de￾fault guidance scale γ = 5. The teacher receives dense depth guidance, while the student is con￾ditioned only on sparse keyframes. PDM achieves more consistent structure and visual quality compared with other distillation objectives. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative results on dense-to-sparse scribble control distillation with γ = 5. Com￾pared with other distillation objectives, Positive–Direction Matching better preserves the scribble￾guided structure and generates more consistent visual results under sparse control supervision. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

16 extracted references · 11 linked inside Pith

  1. [1]

    On-policy distillation of language models: Learning from self- generated mistakes

    Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from self- generated mistakes. InInternational Conference on Learning Representations, volume 2024, pp. 21246–21263,

    2024

  2. [3]

    Flow-opd: On-policy distillation for flow matching models.arXiv preprint arXiv:2605.08063,

    Zhen Fang, Wenxuan Huang, Yu Zeng, Yiming Zhao, Shuang Chen, Kaituo Feng, Yunlong Lin, Lin Chen, Zehui Chen, Shaosheng Cao, et al. Flow-opd: On-policy distillation for flow matching models.arXiv preprint arXiv:2605.08063,

    Pith/arXiv arXiv

  3. [7]

    Consistency trajectory models: Learning proba- bility flow ode trajectory of diffusion

    Dongjun Kim, Chieh-Hsin Lai, WeiHsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon. Consistency trajectory models: Learning proba- bility flow ode trajectory of diffusion. InInternational Conference on Learning Representations, volume 2024, pp. 44493–44525,

    2024

  4. [8]

    Diffusionopd: A unified perspective of on-policy distillation in diffusion models.arXiv preprint arXiv:2605.15055, 2026b

    Quanhao Li, Junqiu Yu, Kaixun Jiang, Yujie Wei, Zhen Xing, Pandeng Li, Ruihang Chu, Shiwei Zhang, Yu Liu, and Zuxuan Wu. Diffusionopd: A unified perspective of on-policy distillation in diffusion models.arXiv preprint arXiv:2605.15055, 2026b. Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthe- sizing high-resolu...

    Pith/arXiv arXiv

  5. [11]

    Search beyond what can be taught: Evolving the knowl- edge boundary in agentic visual generation.arXiv preprint arXiv:2607.05382, 2026a

    Haozhe Wang, Weijia Feng, Jinpeng Yu, Che Liu, Ping Nie, Fangzhen Lin, Jiaming Liu, Ruihua Huang, Jimmy Lin, Wenhu Chen, et al. Search beyond what can be taught: Evolving the knowl- edge boundary in agentic visual generation.arXiv preprint arXiv:2607.05382, 2026a. Haozhe Wang, Cong Wei, Weiming Ren, Jiaming Liu, Fangzhen Lin, and Wenhu Chen. Rational- Rew...

    Pith/arXiv arXiv

  6. [12]

    Sod: Step-wise on-policy distillation for small language model agents.arXiv preprint arXiv:2605.07725,

    Qiyong Zhong, Mao Zheng, Mingyang Song, Xin Lin, Jie Sun, Houcheng Jiang, Xiang Wang, and Junfeng Fang. Sod: Step-wise on-policy distillation for small language model agents.arXiv preprint arXiv:2605.07725,

    Pith/arXiv arXiv

  7. [13]

    Danceopd: On-policy generative field distillation.arXiv preprint arXiv:2606.27377,

    Wei Zhou, Xiongwei Zhu, Zelin Xu, Bo Dong, Lixue Gong, Yongyuan Liang, Meng Chu, Leigang Qu, Lingdong Kong, Wei Liu, et al. Danceopd: On-policy generative field distillation.arXiv preprint arXiv:2606.27377,

    Pith/arXiv arXiv

  8. [14]

    7 APPENDIX 7.1 ADDITIONALEXPERIMENTALDETAILS 7.1.1 TEXTRENDERINGDISTILLATION Models.The base model is Stable Diffusion 3.5-Medium (Esser et al., 2024)

    13 Preprint. 7 APPENDIX 7.1 ADDITIONALEXPERIMENTALDETAILS 7.1.1 TEXTRENDERINGDISTILLATION Models.The base model is Stable Diffusion 3.5-Medium (Esser et al., 2024). Both the teacher and the student are LoRA adapters (Hu et al.,

    2024

  9. [15]

    A score of1means the rendered text is read back exactly;0means no characters match

    and score ROCR = 1− Lev(ˆs, s) max(|ˆs|,|s|) ∈[0,1],(16) whereLev(ˆs, s)is the Levenshtein (edit) distance between the two strings—the minimum number of single-character insertions, deletions, or substitutions to turnˆsintos—and| · |denotes string length. A score of1means the rendered text is read back exactly;0means no characters match. Evaluation.Unless...

    2026

  10. [16]

    Inference and qualitative evaluation.At inference, we discard the teacher adapter and reference image; the student receives only the held-out text prompt

    We use bfloat16 mixed precision for the trainable model and V AE, gradient checkpointing, and DeepSpeed ZeRO-2. Inference and qualitative evaluation.At inference, we discard the teacher adapter and reference image; the student receives only the held-out text prompt. We generate512×512images with 40 denoising steps and evaluate guidance scalesγ∈ {1,1.5,2,2...

    2025

  11. [2020]

    Progressive distillation for fast sampling of diffusion models.arXiv preprint arXiv:2202.00512,

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models.arXiv preprint arXiv:2202.00512,

    Pith/arXiv arXiv

  12. [2022]

    Live avatar: Streaming real-time audio-driven avatar generation with infinite length.arXiv preprint arXiv:2512.04677,

    Yubo Huang, Hailong Guo, Fangtai Wu, Weiqiang Wang, Shifeng Zhang, Shijie Huang, Qijun Gan, Lin Liu, Sirui Zhao, Enhong Chen, et al. Live avatar: Streaming real-time audio-driven avatar generation with infinite length.arXiv preprint arXiv:2512.04677,

    Pith/arXiv arXiv

  13. [2023]

    Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314,

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314,

    Pith/arXiv arXiv

  14. [2024]

    Paddleocr 3.0 technical report.arXiv preprint arXiv:2507.05595,

    Cheng Cui, Ting Sun, Manhui Lin, Tingquan Gao, Yubo Zhang, Jiaxuan Liu, Xueqing Wang, Zelun Zhang, Changda Zhou, Hongen Liu, et al. Paddleocr 3.0 technical report.arXiv preprint arXiv:2507.05595,

    Pith/arXiv arXiv

  15. [2025]

    D-opsd: On-policy self-distillation for continuously tuning step-distilled diffusion models.arXiv preprint arXiv:2605.05204,

    Dengyang Jiang, Xin Jin, Dongyang Liu, Zanyi Wang, Mingzhe Zheng, Ruoyi Du, Xiangpeng Yang, Qilong Wu, Zhen Li, Peng Gao, et al. D-opsd: On-policy self-distillation for continuously tuning step-distilled diffusion models.arXiv preprint arXiv:2605.05204,

    Pith/arXiv arXiv

  16. [2026]

    Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598,

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598,

    Pith/arXiv arXiv