Pith. sign in

REVIEW 3 major objections 5 minor 64 references

Scoring preferences and distillation on the same latent features lets video models stay aligned while cutting sampling to 1–4 steps.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 22:43 UTC pith:WK2ZSWUG

load-bearing objection Solid engineering fix for joint preference+distillation in video diffusion: shared latent backbone + dual heads + adaptive λ actually moves the numbers, with the usual closed-stack caveats. the 3 major comments →

arxiv 2607.03960 v1 pith:WK2ZSWUG submitted 2026-07-04 cs.CV

Reward Lightning: Fast Video Generation via Homologous Preference Distillation

Jiaxiang Cheng , Bing Ma , Xuhua Ren , Kai Yu , Peng Zhang , Tianxiang Zheng , Qinglin Lu This is my paper
classification cs.CV
keywords video diffusion modelspreference alignmentdistillation accelerationlatent reward modelhomologous preference distillationfew-step generationflow matching
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Video diffusion models are slow because they need many sampling steps, and making them faster usually weakens how well they match human taste. Prior attempts either mix a pixel-space reward with latent-space distillation or train the two goals in separate stages; both create gradient conflicts that trade quality for speed. This paper argues that the conflict is structural: the two objectives live in mismatched representation spaces. Reward Lightning forces both to evaluate identical latent features through a shared backbone (homology). A latent reward model scores videos without decoding them to pixels, and homologous preference distillation reuses that backbone for joint adversarial distillation and preference alignment. The result is a generator that produces high-fidelity, preference-aligned videos in one to four steps and improves average VBench scores while leading text alignment, motion quality, and visual quality.

Core claim

When preference alignment and adversarial distillation are both evaluated on identical latent features through a shared backbone (structural and space homology), their gradients become sufficiently aligned that a single multi-objective update can produce few-step generators that remain both high-fidelity and human-aligned, rather than forcing a trade-off.

What carries the argument

Homologous Preference Distillation (HPD): a frozen latent reward backbone plus parallel reward and discriminator heads that score the same intermediate flow state; an adaptive weight then gates the preference loss so it only activates once the generator is already realistic.

Load-bearing premise

That a frozen latent reward model, trained once on multi-margin pairs that include few-step samples, continues to supply reliable preference gradients throughout distillation without online re-training or pixel-space checks.

What would settle it

Train the same generator with a pixel-space reward on identical data and steps; if the cosine similarity of the two gradients stays near zero or negative and the 1–4-step VBench scores fail to exceed pure-distillation baselines by the claimed margin, the homology claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Few-step (1–4 NFE) video generators can match or exceed multi-step baselines on text alignment, motion quality, and visual quality without a separate preference stage.
  • Latent-space reward models become preferable to pixel-space ones for joint acceleration and alignment because they eliminate VAE decoding cost and keep gradients on one manifold.
  • Adaptive preference weighting that waits for structural realism can replace fixed linear combinations of distillation and reward losses.
  • The same shared-backbone pattern can be dropped onto other distillation recipes (consistency models, distribution matching) to raise their preference scores.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If homology is the real bottleneck, any future multi-objective diffusion post-training (safety, style, multi-modal) should default to shared latent features rather than cross-space linear sums.
  • The need for few-step augmentation in the reward dataset implies that reward models trained only on multi-step samples will systematically mis-score aggressive distillation trajectories.
  • Cross-VAE latent incompatibility noted by the authors suggests a latent-space converter could turn one trained LRM into a universal preference oracle for many generators.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Reward Lightning, a framework for joint preference alignment and few-step distillation of video diffusion (flow-matching) models. Its central idea is homology: both the adversarial distillation objective and the preference (ReFL-style) objective are evaluated on identical latent features extracted by a shared backbone. A Latent Reward Model (LRM) is first trained on a multi-margin preference dataset (intra-model, inter-model, real-synthetic, and few-step-augmented pairs) with a time-modulated attention head and Bradley-Terry-with-Ties loss plus dynamic margin clipping. Homologous Preference Distillation (HPD) then freezes this LRM, initializes a discriminator head from the reward head, and jointly optimizes a few-step generator with an adaptive preference weight that gates ReFL by discriminator loss. Experiments on Wan2.2-14B report LRM preference accuracy gains of ~11–14.7% on OOD benchmarks and 1–4 NFE generators that improve average VBench by ~2.1% while leading text alignment, motion quality, and visual quality, with supporting gradient-cosine, ablation, and human-preference results.

Significance. If the results hold, the work offers a practical and conceptually clean route to simultaneous acceleration and human alignment for large video generators, an open and industrially relevant problem. The explicit framing of structural and space homology as an implicit gradient regularizer, the latent-space reward design that avoids VAE decoding, the multi-margin dataset with few-step augmentation, and the adaptive λ schedule are concrete engineering contributions. Strengths include extensive ablations (dataset composition, head architecture, regularization, head initialization, adaptive weight), gradient-conflict measurements (cosine similarity rising to +0.41 under homology), human preference votes, and a cross-backbone check on HYVideo1.5. These elements make the paper a useful reference for joint RLHF–distillation pipelines even if some mechanistic claims require tighter validation.

major comments (3)
  1. The central homology claim (§3.3, Alg. 1 lines 14–18, Eqs. 8–10, Tab. 4-a) rests on a frozen LRM continuing to supply correctly ordered, non-hackable preference gradients for every intermediate state the generator produces under 1–4 NFE sampling. The paper shows high static OOD accuracy (Tab. 1) and final VBench gains (Tab. 2), but does not report online reward-ranking fidelity, reward-vs-step curves, or correlation between LRM scores and human judgments on the evolving generator distribution. Without such diagnostics, it remains possible that the observed gains are driven mainly by the adversarial term (or by the few-step augmentation acting as a distillation regularizer) rather than by sustained preference gradients, which would weaken the causal attribution to homology.
  2. Quantitative claims lack uncertainty quantification. Tables 1–4 and the human evaluation (Fig. 5) report point estimates only; no standard deviations across seeds, confidence intervals, or multiple independent runs are given for preference accuracy, VBench dimensions, cosine similarity, or win rates. Given the free parameters (λ schedule, BTT k=5, EMA ρ=0.95, learning rates, 2000-step budget) and the known variance of video generation metrics, the reported +2.1% average and leadership on TA/MQ/VQ cannot be assessed for statistical reliability.
  3. The comparison set for joint preference+distillation methods is narrow and partially self-reproduced (DMDR, FlashDMD on Wan2.2; TurboDiffusion is the main external few-step baseline). Sequential ablations (Tab. 7) are useful but still leave open whether a carefully tuned two-stage pipeline with the same LRM would close most of the gap. A stronger external joint baseline or a controlled sequential-vs-joint experiment with matched compute and the identical frozen LRM would better isolate the contribution of simultaneous homologous training.
minor comments (5)
  1. Notation for the adaptive weight is inconsistent: Eq. (10) writes LHPD = LG_ADV + λ · LReFL while Alg. 1 line 18 writes LG_ADV − λ · LReFL; the sign of LReFL (defined as −R) should be clarified once and used uniformly.
  2. Fig. 2 is dense; the multi-margin dataset panel and the dual-head routing would benefit from a clearer legend and larger type so that the homology (shared Ft) is immediately visible.
  3. The BTT formulation and the value k=5.0 are deferred to the appendix; a short statement of the three-outcome probabilities and the chosen k in the main text (near Eq. 7) would help readers without forcing a jump.
  4. Limitations (Appendix E) correctly note VAE-specific latent spaces; a brief forward pointer in the main conclusion would set expectations for cross-architecture use.
  5. Typos and minor wording: “VisionRewrd” in Tab. 1; occasional missing spaces around citations; “w/o reg.” vs “ρ=0.5” formatting in Tab. 3-c.

Circularity Check

0 steps flagged

No significant circularity: LRM is trained once on an independent multi-margin preference set (with few-step domain coverage) then frozen; HPD and all reported metrics are evaluated against external OOD benchmarks and VBench.

full rationale

The paper is a standard empirical methods contribution. LRM is optimized via BTT + margin-clipping on a curated preference collection (intra-/inter-model, real-synthetic, and 5 k few-step pairs) and then frozen (Sec. 3.2, Eq. 7). HPD re-uses the frozen backbone + a weight-initialized discriminator head for joint ADV + ReFL losses under an adaptive schedule (Alg. 1, Eqs. 8–10); the generator is trained on unpaired real videos. Preference accuracy is measured on held-out VideoGen-RewardBench / GenAI-Bench; generation quality on VBench, FVD and human votes. Gradient-cosine measurements (Tab. 4-a) and ablations are post-hoc diagnostics, not inputs re-labeled as outputs. The few-step augmentation pairs are generated from the same model family later distilled, which is ordinary domain adaptation rather than a self-definitional or fitted-input-as-prediction loop. No uniqueness theorem, ansatz, or load-bearing self-citation forces the central claim. Score 1 only for the mild same-family data overlap; the derivation chain itself is non-circular and externally falsifiable.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 3 invented entities

The central claim rests on standard flow-matching and adversarial-distillation machinery plus several paper-specific design choices (shared backbone, dual identical heads, adaptive λ schedule, multi-margin dataset construction). Free parameters are the usual training hyper-parameters plus the hand-designed form of λ. No new physical entities are postulated; the invented entities are architectural modules whose only evidence is the reported ablation gains.

free parameters (4)
  • adaptive preference weight λ (half-Gaussian squash + EMA)
    Functional form exp(−ReLU(sg(L_ADV^D)−0.5)^2) and EMA β=0.99 are chosen by hand to stabilize early training; not derived from first principles.
  • BTT tie parameter k=5.0
    Fixed threshold controlling tie probability in the Bradley-Terry-with-Ties loss; set without cross-validation reported.
  • EMA decay ρ=0.95 for margin clipping
    Used to bound large-margin reward differences and prevent reward hacking; value selected empirically.
  • learning rates, weight decays, batch size 64, 2000 steps, EMA 0.995
    Standard optimizer hyper-parameters required for the reported HPD numbers.
axioms (4)
  • domain assumption Flow-matching / rectified-flow ODE formulation (zt = (1−t)z0 + t z1) is a valid generative process for video latents.
    Taken as given from Lipman et al. / Liu et al.; all sampling and intermediate-state construction rely on it.
  • ad hoc to paper A frozen latent reward model trained on multi-margin human preferences supplies unbiased preference gradients for generator updates.
    Core premise of HPD; no online reward re-training is performed.
  • ad hoc to paper Structural homology (identical architecture + weight init of discriminator from reward head) plus space homology (shared backbone features) is sufficient to act as an implicit gradient regularizer.
    Stated in §3.3; supported by cosine-similarity measurement but not proved.
  • ad hoc to paper Bradley-Terry-with-Ties loss with dynamic margin clipping prevents reward hacking on large-margin pairs.
    Introduced in §3.2; ablation shows accuracy drop when removed.
invented entities (3)
  • Latent Reward Model (LRM) with time-modulated attention head no independent evidence
    purpose: Score videos directly in latent space and serve as shared backbone for both preference and distillation heads.
    Architectural invention; independent evidence limited to preference-accuracy tables on two OOD benchmarks.
  • Homologous Preference Distillation (HPD) dual-head architecture no independent evidence
    purpose: Jointly optimize adversarial distillation and ReFL-style preference on identical latent features.
    Core method; evidence is VBench gains and gradient cosine similarity.
  • Multi-margin preference dataset with few-step augmentation no independent evidence
    purpose: Expose the reward model to the distribution of 1–4-step samples so it can guide distillation.
    Dataset construction choice; no external release.

pith-pipeline@v1.1.0-grok45 · 26823 in / 3130 out tokens · 22586 ms · 2026-07-11T22:43:40.888541+00:00 · methodology

0 comments
read the original abstract

Achieving simultaneous preference alignment and distillation acceleration in video diffusion models remains an open challenge. Existing methods optimize the two objectives over mismatched representation spaces, where improving one objective often compromises the other. To overcome this, we propose Reward Lightning, a unified framework that aligns and accelerates a video diffusion model within a single shared representation. Its central principle is homology: both objectives are evaluated on identical latent features, which mitigates the gradient conflicts that arise when they are optimized over disjoint representations. As a foundational component, we first introduce a latent reward model (LRM) that scores videos directly in the latent space, without decoding back to the pixel space. Building on the LRM, homologous preference distillation (HPD) reuses this shared backbone to perform adversarial distillation and preference alignment jointly, yielding few-step generators that remain faithful and well aligned. Extensive experiments demonstrate that the LRM surpasses pixel-level and latent-level reward baselines by $11.0\%$ and $14.7\%$ in preference accuracy, and that Reward Lightning generates high-fidelity videos in merely $1$ to $4$ steps, improving the average VBench score by $2.1\%$ while leading in text alignment, motion quality, and visual quality. Project page: https://reward-lightning.github.io.

Figures

Figures reproduced from arXiv: 2607.03960 by Bing Ma, Jiaxiang Cheng, Kai Yu, Peng Zhang, Qinglin Lu, Tianxiang Zheng, Xuhua Ren.

Figure 1
Figure 1. Figure 1: Motivation. (a) Heterogeneous Training: Pixel rewards induce shifts in dis￾tillation trajectories through multi-objective gradient conflicts. (b) Disjoint Training: Sequential training discards original preference distributions through catastrophic for￾getting. (c) Homologous Training: Preference distillation based on homologous struc￾tures and data guides the generator toward the jointly optimal distribut… view at source ↗
Figure 2
Figure 2. Figure 2: Overall architecture of Reward Lightning. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative Results. Upper: Visualization of 80-NFEs and 4-NFEs from the Wan2.2-I2V-A14B. Bottom: Built on the baseline model, compared against a distilla￾tion without homologous preference distillation. we extend the offloading strategy to the EMA generator in addition to the frozen VAE and text encoder. Furthermore, we employ a full-sharding strategy via FSDP and SP across the generator, reward model, an… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative Results For fair comparison, all models are built on Wan2.2- I2V-A14B. Except for TurboDiffusion, we reproduce all results ourselves. Row 1: ReFL model based on latent reward model. Row 2 − 3: Heterogeneous model, composed of pixel ReFL and DMD distillation. Row 4: Distillation model, primarily based on rCM method. Row 5: Homologous model, with structural and space homology. ity score among het… view at source ↗
Figure 5
Figure 5. Figure 5: Human Evaluation. We report the Win/Tie/Lose rate for HPD 1, 4−NFEs, compared with the Euler baseline, distillation, and heterogeneous models. ceiling and accelerates convergence compared to random baselines. In part (e), the adaptive weight λ ensures training stability by delaying preference alignment until the generator achieves structural realism. This self-paced mechanism pre￾vents early gradient inter… view at source ↗
Figure 6
Figure 6. Figure 6: Efficiency-quality Pareto front on Wan2.2-I2V-14B. Dotted lines denote 80-NFE baselines. HPD dominates the top-left region across 1 to 8 NFEs. Efficiency & Quality Analysis. As shown in [PITH_FULL_IMAGE:figures/full_fig_p015_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative Comparison. In each triplet, rows 1 to 3 represent Wan2.2-I2V (80-NFEs), Wan2.2-I2V (4-NFEs), and our Reward Lightning (4-NFEs), respectively [PITH_FULL_IMAGE:figures/full_fig_p027_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Qualitative Comparison. In each triplet, rows 1 to 3 represent TurboDiffu￾sion (4-NFEs), DMDR (4-NFEs), and our Reward Lightning (4-NFEs), respectively [PITH_FULL_IMAGE:figures/full_fig_p028_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative Comparison. In each triplet, rows 1 to 3 represent TurboDif￾fusion (1-NFEs), FlashDMD (1-NFEs), and our Reward Lightning (1-NFEs), respec￾tively [PITH_FULL_IMAGE:figures/full_fig_p029_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

64 extracted references · 15 linked inside Pith

  1. [1]

    Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et al.: Training a helpful and harmless assistant withreinforcementlearningfromhumanfeedback.arXivpreprintarXiv:2204.05862 (2022)

    Pith/arXiv arXiv 2022

  2. [2]

    In: The Twelfth International Conference on Learning Representations (2024),https://openreview.net/forum?id=YCWjhGrJFD

    Black, K., Janner, M., Du, Y., Kostrikov, I., Levine, S.: Training diffusion models with reinforcement learning. In: The Twelfth International Conference on Learning Representations (2024),https://openreview.net/forum?id=YCWjhGrJFD

    2024

  3. [3]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Blattmann, A., Rombach, R., Ling, H., Dockhorn, T., Kim, S.W., Fidler, S., Kreis, K.: Align your latents: High-resolution video synthesis with latent diffusion mod- els. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 22563–22575 (2023)

    2023

  4. [4]

    the method of paired comparisons

    Bradley, R.A., Terry, M.E.: Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika39(3/4), 324–345 (1952),http://www. jstor.org/stable/2334029

    arXiv 1952

  5. [5]

    arXiv preprint arXiv:2511.20549 (2025)

    Chen, G., Huang, S., Liu, K., Zhu, J., Qu, X., Chen, P., Cheng, Y., Sun, Y.: Flash- dmd: Towards high-fidelity few-step image generation with efficient distillation and joint reinforcement learning. arXiv preprint arXiv:2511.20549 (2025)

    arXiv 2025

  6. [6]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    Cheng, J., Ma, B., Ren, X., Jin, H.H., Yu, K., Zhang, P., Li, W., Zhou, Y., Zheng, T., Lu, Q.: Phased one-step adversarial equilibrium for video diffusion models. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 40, pp. 3237– 3245 (2026)

    2026

  7. [7]

    Advances in neural information processing systems30(2017)

    Christiano, P.F., Leike, J., Brown, T., Martic, M., Legg, S., Amodei, D.: Deep reinforcement learning from human preferences. Advances in neural information processing systems30(2017)

    2017

  8. [8]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Ding, Z., Jin, C., Liu, D., Zheng, H., Singh, K.K., Zhang, Q., Kang, Y., Lin, Z., Liu, Y.: Dollar: Few-step video generation via distillation and latent reward optimiza- tion. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 17961–17971 (2025)

    2025

  9. [9]

    In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025),https: //openreview.net/forum?id=DbzREoPwmM

    Eyring, L., Karthik, S., Dosovitskiy, A., Ruiz, N., Akata, Z.: Noise hypernet- works: Amortizing test-time compute in diffusion models. In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025),https: //openreview.net/forum?id=DbzREoPwmM

    2025

  10. [10]

    Trends in cogni- tive sciences3(4), 128–135 (1999)

    French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cogni- tive sciences3(4), 128–135 (1999)

    1999

  11. [11]

    In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025),https://openreview.net/forum?id=uWj4s7rMnR

    Geng, Z., Deng, M., Bai, X., Kolter, J.Z., He, K.: Mean flows for one-step gen- erative modeling. In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025),https://openreview.net/forum?id=uWj4s7rMnR

    2025

  12. [12]

    In: The Twelfth International Conference on Learning Representations (2024),https://openreview.net/forum?id=Fx2SbBgcte

    Guo, Y., Yang, C., Rao, A., Liang, Z., Wang, Y., Qiao, Y., Agrawala, M., Lin, D., Dai, B.: Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. In: The Twelfth International Conference on Learning Representations (2024),https://openreview.net/forum?id=Fx2SbBgcte

    2024

  13. [13]

    In: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing

    He, X., Jiang, D., Zhang, G., Ku, M., Soni, A., Siu, S., Chen, H., Chandra, A., Jiang, Z., Arulraj, A., et al.: Videoscore: Building automatic metrics to simulate fine-grained human feedback for video generation. In: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. pp. 2105–2123 (2024)

    2024

  14. [14]

    Advances in neural information processing systems33, 6840–6851 (2020) Reward Lightning 17

    Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. Advances in neural information processing systems33, 6840–6851 (2020) Reward Lightning 17

    2020

  15. [15]

    Advances in neural information processing systems35, 8633– 8646 (2022)

    Ho, J., Salimans, T., Gritsenko, A., Chan, W., Norouzi, M., Fleet, D.J.: Video diffusion models. Advances in neural information processing systems35, 8633– 8646 (2022)

    2022

  16. [16]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Huang, Z., He, Y., Yu, J., Zhang, F., Si, C., Jiang, Y., Zhang, Y., Wu, T., Jin, Q., Chanpaisit, N., et al.: Vbench: Comprehensive benchmark suite for video gener- ative models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 21807–21818 (2024)

    2024

  17. [17]

    Jacobs, S.A., Tanaka, M., Zhang, C., Zhang, M., Song, S.L., Rajbhandari, S., He, Y.: Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models (2023),https://arxiv.org/abs/2309.14509

    Pith/arXiv arXiv 2023

  18. [18]

    arXiv preprint arXiv:2511.13649 (2025)

    Jiang, D., Liu, D., Wang, Z., Wu, Q., Li, L., Li, H., Jin, X., Liu, D., Li, Z., Zhang, B., et al.: Distribution matching distillation meets reinforcement learning. arXiv preprint arXiv:2511.13649 (2025)

    Pith/arXiv arXiv 2025

  19. [19]

    Advances in Neural Information Processing Systems37, 79889–79908 (2024)

    Jiang, D., Ku, M., Li, T., Ni, Y., Sun, S., Fan, R., Chen, W.: Genai arena: An open evaluation platform for generative models. Advances in Neural Information Processing Systems37, 79889–79908 (2024)

    2024

  20. [20]

    In: The Twelfth International Conference on Learning Rep- resentations (2024),https://openreview.net/forum?id=ymjI8feDTD

    Kim, D., Lai, C.H., Liao, W.H., Murata, N., Takida, Y., Uesaka, T., He, Y., Mitsu- fuji, Y., Ermon, S.: Consistency trajectory models: Learning probability flow ODE trajectory of diffusion. In: The Twelfth International Conference on Learning Rep- resentations (2024),https://openreview.net/forum?id=ymjI8feDTD

    2024

  21. [21]

    arXiv.org (2014)

    Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv.org (2014)

    2014

  22. [22]

    Proceedings of the national academy of sciences114(13), 3521–3526 (2017)

    Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences114(13), 3521–3526 (2017)

    2017

  23. [23]

    Advances in neural information processing systems36, 36652–36663 (2023)

    Kirstain,Y.,Polyak,A.,Singer,U.,Matiana,S.,Penna,J.,Levy,O.:Pick-a-pic:An open dataset of user preferences for text-to-image generation. Advances in neural information processing systems36, 36652–36663 (2023)

    2023

  24. [24]

    arXiv preprint arXiv:2412.03603 (2024)

    Kong, W., Tian, Q., Zhang, Z., Min, R., Dai, Z., Zhou, J., Xiong, J., Li, X., Wu, B., Zhang, J., et al.: Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603 (2024)

    Pith/arXiv arXiv 2024

  25. [25]

    arXiv preprint arXiv:2302.12192 (2023)

    Lee, K., Liu, H., Ryu, M., Watkins, O., Du, Y., Boutilier, C., Abbeel, P., Ghavamzadeh, M., Gu, S.S.: Aligning text-to-image models using human feedback. arXiv preprint arXiv:2302.12192 (2023)

    Pith/arXiv arXiv 2023

  26. [26]

    arXiv preprint arXiv:2402.13929 (2024)

    Lin, S., Wang, A., Yang, X.: Sdxl-lightning: Progressive adversarial diffusion dis- tillation. arXiv preprint arXiv:2402.13929 (2024)

    Pith/arXiv arXiv 2024

  27. [27]

    In: International Conference on Machine Learning

    Lin, S., Xia, X., Ren, Y., Yang, C., Xiao, X., Jiang, L.: Diffusion adversarial post- training for one-step video generation. In: International Conference on Machine Learning. pp. 37959–37974. PMLR (2025)

    2025

  28. [28]

    arXiv preprint arXiv:2506.05343 (2025)

    Lin, W., Chen, R., Liu, B., Yan, S., Feng, R., Wei, J., Zhang, Y., Zhou, Y., Feng, C., Ran, J., et al.: Contentv: Efficient training of video generation models with limited compute. arXiv preprint arXiv:2506.05343 (2025)

    Pith/arXiv arXiv 2025

  29. [29]

    arXiv preprint arXiv:2501.16612 (2025)

    Lin, W., Wei, J., Liu, B., Zhang, Y., Yan, S., Guo, M.: Cascadev: An im- plementation of wurstchen architecture for video generation. arXiv preprint arXiv:2501.16612 (2025)

    Pith/arXiv arXiv 2025

  30. [30]

    In: The Eleventh International Conference on Learning Representations (2023),https://openreview.net/forum?id=PqvMRDCJT9t

    Lipman, Y., Chen, R.T.Q., Ben-Hamu, H., Nickel, M., Le, M.: Flow matching for generative modeling. In: The Eleventh International Conference on Learning Representations (2023),https://openreview.net/forum?id=PqvMRDCJT9t

    2023

  31. [31]

    Cheng et al

    Liu, J., Liu, G., Liang, J., Yuan, Z., Liu, X., Zheng, M., Wu, X., Wang, Q., Xia, M., Wang, X., Liu, X., Yang, F., Wan, P., ZHANG, D., Gai, K., Yang, 18 J. Cheng et al. Y., Ouyang, W.: Improving video generation with human feedback. In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025), https://openreview.net/forum?id=nHkg4yc7SP

    2025

  32. [32]

    In: The Eleventh International Conference on Learning Representations (2023),https://openreview.net/forum?id=XVjTT1nw5z

    Liu, X., Gong, C., qiang liu: Flow straight and fast: Learning to generate and trans- fer data with rectified flow. In: The Eleventh International Conference on Learning Representations (2023),https://openreview.net/forum?id=XVjTT1nw5z

    2023

  33. [33]

    In: International Conferenceon LearningRepresentations(2019),https://openreview.net/forum? id=Bkg6RiCqY7

    Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: International Conferenceon LearningRepresentations(2019),https://openreview.net/forum? id=Bkg6RiCqY7

    2019

  34. [34]

    arXiv preprint arXiv:2310.04378 (2023)

    Luo, S., Tan, Y., Huang, L., Li, J., Zhao, H.: Latent consistency mod- els: Synthesizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.04378 (2023)

    Pith/arXiv arXiv 2023

  35. [35]

    In: Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition

    Meng, C., Rombach, R., Gao, R., Kingma, D., Ermon, S., Ho, J., Salimans, T.: On distillation of guided diffusion models. In: Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition. pp. 14297–14306 (2023)

    2023

  36. [36]

    arXiv preprint arXiv:2511.21541 (2025)

    Mi, X., Yu, W., Lian, J., Jie, S., Zhong, R., Liu, Z., Zhang, G., Zhou, Z., Xu, Z., Zhou, Y., Lu, Q., Tang, F.: Video generation models are good latent reward models. arXiv preprint arXiv:2511.21541 (2025)

    arXiv 2025

  37. [37]

    In: International Conference on Learning Repre- sentations (2018),https://openreview.net/forum?id=B1QRgziT-

    Miyato, T., Kataoka, T., Koyama, M., Yoshida, Y.: Spectral normalization for generative adversarial networks. In: International Conference on Learning Repre- sentations (2018),https://openreview.net/forum?id=B1QRgziT-

    2018

  38. [38]

    Advances in neural information processing sys- tems35, 27730–27744 (2022)

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al.: Training language models to follow instructions with human feedback. Advances in neural information processing sys- tems35, 27730–27744 (2022)

    2022

  39. [39]

    Advances in neural information processing systems36, 53728–53741 (2023)

    Rafailov, R., Sharma, A., Mitchell, E., Manning, C.D., Ermon, S., Finn, C.: Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems36, 53728–53741 (2023)

    2023

  40. [40]

    Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., Liu, P.J.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res.21(1) (Jan 2020)

    2020

  41. [41]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 10684–10695 (2022)

    2022

  42. [42]

    In: International Conference on Learning Representations (2022),https: //openreview.net/forum?id=TIdIXIpzhoI

    Salimans, T., Ho, J.: Progressive distillation for fast sampling of diffusion mod- els. In: International Conference on Learning Representations (2022),https: //openreview.net/forum?id=TIdIXIpzhoI

    2022

  43. [43]

    In: SIGGRAPH Asia 2024 Conference Papers

    Sauer, A., Boesel, F., Dockhorn, T., Blattmann, A., Esser, P., Rombach, R.: Fast high-resolution image synthesis with latent adversarial diffusion distillation. In: SIGGRAPH Asia 2024 Conference Papers. pp. 1–11 (2024)

    2024

  44. [44]

    In: European Conference on Computer Vision

    Sauer, A., Lorenz, D., Blattmann, A., Rombach, R.: Adversarial diffusion distilla- tion. In: European Conference on Computer Vision. pp. 87–103. Springer (2024)

    2024

  45. [45]

    In: The Eleventh International Con- ference on Learning Representations (2023),https://openreview.net/forum?id= nJfylDvgzlq

    Singer, U., Polyak, A., Hayes, T., Yin, X., An, J., Zhang, S., Hu, Q., Yang, H., Ashual, O., Gafni, O., Parikh, D., Gupta, S., Taigman, Y.: Make-a-video: Text- to-video generation without text-video data. In: The Eleventh International Con- ference on Learning Representations (2023),https://openreview.net/forum?id= nJfylDvgzlq

    2023

  46. [46]

    In: Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., Scarlett, J

    Song, Y., Dhariwal, P., Chen, M., Sutskever, I.: Consistency models. In: Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., Scarlett, J. (eds.) Proceedings of the 40th International Conference on Machine Learning. Proceedings of Machine Reward Lightning 19 Learning Research, vol. 202, pp. 32211–32252. PMLR (23–29 Jul 2023),https: //proceeding...

    2023

  47. [47]

    In: Interna- tional Conference on Learning Representations (2021),https://openreview.net/ forum?id=PxTIG12RRHS

    Song, Y., Sohl-Dickstein, J., Kingma, D.P., Kumar, A., Ermon, S., Poole, B.: Score- based generative modeling through stochastic differential equations. In: Interna- tional Conference on Learning Representations (2021),https://openreview.net/ forum?id=PxTIG12RRHS

    2021

  48. [48]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Wallace, B., Dang, M., Rafailov, R., Zhou, L., Lou, A., Purushwalkam, S., Ermon, S., Xiong, C., Joty, S., Naik, N.: Diffusion model alignment using direct preference optimization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 8228–8238 (2024)

    2024

  49. [49]

    arXiv preprint arXiv:2503.20314 (2025)

    Wan, T., Wang, A., Ai, B., Wen, B., Mao, C., Xie, C.W., Chen, D., Yu, F., Zhao, H., Yang, J., et al.: Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314 (2025)

    Pith/arXiv arXiv 2025

  50. [50]

    Advances in neural information processing systems37, 83951–84009 (2024)

    Wang, F.Y., Huang, Z., Bergman, A., Shen, D., Gao, P., Lingelbach, M., Sun, K., Bian, W., Song, G., Liu, Y., et al.: Phased consistency models. Advances in neural information processing systems37, 83951–84009 (2024)

    2024

  51. [51]

    arXiv preprint arXiv:2409.12191 (2024)

    Wang, P., Bai, S., Tan, S., Wang, S., Fan, Z., Bai, J., Chen, K., Liu, X., Wang, J., Ge, W., et al.: Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191 (2024)

    Pith/arXiv arXiv 2024

  52. [52]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Wang, Q., Shi, Y., Ou, J., Chen, R., Lin, K., Wang, J., Jiang, B., Yang, H., Zheng, M., Tao, X., Yang, F., Wan, P., Zhang, D.: Koala-36m: A large-scale video dataset improving consistency between fine-grained conditions and video content. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 8428–8437 (June 2025)

    2025

  53. [53]

    arXiv preprint arXiv:2412.04814 (2024)

    Wang, Y., Tan, Z., Wang, J., Yang, X., Jin, C., Li, H.: Lift: Leveraging human feed- back for text-to-video model alignment. arXiv preprint arXiv:2412.04814 (2024)

    Pith/arXiv arXiv 2024

  54. [54]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Wu, X., Sun, K., Zhu, F., Zhao, R., Li, H.: Human preference score: Better aligning text-to-image models with human preference. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 2096–2105 (2023)

    2096

  55. [55]

    arXiv preprint arXiv:2412.21059 (2024)

    Xu, J., Huang, Y., Cheng, J., Yang, Y., Xu, J., Wang, Y., Duan, W., Yang, S., Jin, Q., Li, S., et al.: Visionreward: Fine-grained multi-dimensional human preference learning for image and video generation. arXiv preprint arXiv:2412.21059 (2024)

    Pith/arXiv arXiv 2024

  56. [56]

    Advances in Neural Information Processing Systems36, 15903–15935 (2023)

    Xu, J., Liu, X., Wu, Y., Tong, Y., Li, Q., Ding, M., Tang, J., Dong, Y.: Imagere- ward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems36, 15903–15935 (2023)

    2023

  57. [57]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Yang, K., Tao, J., Lyu, J., Ge, C., Chen, J., Shen, W., Zhu, X., Li, X.: Using human feedback to fine-tune diffusion models without any reward model. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 8941–8951 (2024)

    2024

  58. [58]

    Advances in neural information processing systems37, 47455–47487 (2024)

    Yin, T., Gharbi, M., Park, T., Zhang, R., Shechtman, E., Durand, F., Freeman, B.: Improved distribution matching distillation for fast image synthesis. Advances in neural information processing systems37, 47455–47487 (2024)

    2024

  59. [59]

    2024 ieee

    Yin, T., Gharbi, M., Zhang, R., Shechtman, E., Durand, F., Freeman, W.T., Park, T.: One-step diffusion with distribution matching distillation. 2024 ieee. In: CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 6613–6623 (2023)

    2024

  60. [60]

    Advances in Neural Information Processing Systems37, 67355–67382 (2024) 20 J

    Zhang, J., Wu, J., Ren, Y., Xia, X., Kuang, H., Xie, P., Li, J., Xiao, X., Huang, W., Wen, S., et al.: Unifl: Improve latent diffusion model via unified feedback learning. Advances in Neural Information Processing Systems37, 67355–67382 (2024) 20 J. Cheng et al

    2024

  61. [61]

    arXiv preprint arXiv:2512.16093 (2025)

    Zhang, J., Zheng, K., Jiang, K., Wang, H., Stoica, I., Gonzalez, J.E., Chen, J., Zhu, J.: Turbodiffusion: Accelerating video diffusion models by 100-200 times. arXiv preprint arXiv:2512.16093 (2025)

    arXiv 2025

  62. [62]

    Zhao,Y.,Gu,A.,Varma,R.,Luo,L.,Huang,C.C.,Xu,M.,Wright,L.,Shojanazeri, H., Ott, M., Shleifer, S., Desmaison, A., Balioglu, C., Damania, P., Nguyen, B., Chauhan, G., Hao, Y., Mathews, A., Li, S.: Pytorch fsdp: Experiences on scaling fully sharded data parallel (2023),https://arxiv.org/abs/2304.11277

    Pith/arXiv arXiv 2023

  63. [63]

    arXiv preprint arXiv:2510.08431 (2025)

    Zheng, K., Wang, Y., Ma, Q., Chen, H., Zhang, J., Balaji, Y., Chen, J., Liu, M.Y., Zhu, J., Zhang, Q.: Large scale diffusion distillation via score-regularized continuous-time consistency. arXiv preprint arXiv:2510.08431 (2025)

    Pith/arXiv arXiv 2025

  64. [64]

    Zheng, K., Wang, Y., Ma, Q., Chen, H., Zhang, J., Balaji, Y., Chen, J., Liu, M.Y., Zhu, J., Zhang, Q.: Large scale diffusion distillation via score- regularized continuous-time consistency. In: The Fourteenth International Con- ference on Learning Representations (2026),https://openreview.net/forum?id= 2uNlM353RI Reward Lightning 21 Reward Lightning: Fast...

    arXiv 2026