Pith. sign in

REVIEW 3 major objections 6 minor 46 references

Pixel-motion rewards and hybrid fine-tuning can restore motion dynamics lost after supervised fine-tuning of video diffusion models.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 23:09 UTC pith:RLFPJZQA

load-bearing objection Practical, well-ablated fix for dynamic-degree collapse after SFT of I2V models; the hybrid objective and motion rewards are solid engineering, not a paradigm shift. the 3 major comments →

arxiv 2603.17426 v2 pith:RLFPJZQA submitted 2026-03-18 cs.CV

SHIFT: Motion Alignment in Video Diffusion Models with Adversarial Hybrid Fine-Tuning

Xi Ye , Wenjia Yang , Yangyang Xu , Xiaoyang Liu , Duo Su , Mengfei Xia , Jun Zhu This is my paper
classification cs.CV
keywords video diffusionmotion alignmentimage-to-videopixel-motion rewardhybrid fine-tuningreward hackingoptical flowadvantage-weighted regression
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Image-conditioned video diffusion models often lose motion strength and long-term temporal coherence after supervised fine-tuning, even as frames look sharper and more realistic. This paper claims the problem can be fixed by defining rewards directly from low-level pixel motion rather than human preferences or language models. Instantaneous rewards come from optical-flow transport residuals between adjacent frames; long-term rewards come from tracked point trajectories over the full clip. These rewards drive SHIFT, a hybrid fine-tuning method that mixes ordinary supervised updates on real videos with advantage-weighted updates on the model's own samples, all inside the forward diffusion process. Adversarial co-training of the reward models and shared noise schedules keep the generator from hacking the reward. On SVD and a larger TI2V model the method raises motion metrics while holding appearance and FVD near the base model, at a fraction of the cost of reverse-process RL baselines.

Core claim

Dynamic-degree collapse after supervised fine-tuning of image-conditioned video diffusion models can be reversed by pixel-motion rewards (instantaneous optical-flow residuals plus long-term point trajectories) together with SHIFT, a forward-process hybrid of supervised anchoring and advantage-weighted updates that uses adversarial reward co-training to avoid reward hacking.

What carries the argument

SHIFT loss: a mixture of an offline supervised denoising term on real data and an online advantage-weighted denoising term on model rollouts, with group-relative advantages from adversarially updated pixel-motion discriminators (IMR from flow residuals, LMR from CoTracker trajectories).

Load-bearing premise

The optical-flow residual and tracked-point features used by the reward models are assumed to be a faithful enough proxy for true motion fidelity that the generator cannot systematically satisfy them with artifacts instead of real dynamics.

What would settle it

If videos that score high on IMR/LMR still show collapsed dynamic degree or physically implausible motion on held-out VBench-style suites, or if ablating the adversarial reward updates causes clear reward hacking that the paper claims is prevented, the central claim fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper studies post-training motion alignment for image-conditioned video diffusion models, which often suffer dynamic-degree collapse after supervised fine-tuning. It introduces two pixel-motion rewards—Instantaneous Motion Reward (IMR) from an optical-flow transport residual (Eq. 6) and Long-term Motion Reward (LMR) from CoTracker trajectories—trained as binary real/fake discriminators. It then proposes SHIFT, a hybrid objective (Eq. 12) that combines an offline SFT data anchor with advantage-weighted updates on model rollouts in the forward diffusion process, plus adversarial reward-model co-training and noise-level alignment. Experiments on SVD (DAVIS2017) and Wan2.2-TI2V (WISA-80K) report restored VBench Motion / Motion Score relative to SFT, appearance near the base model, and substantially lower cost than FlowGRPO, with ablations isolating Noise Alignment, adversarial RM, and LMR.

Significance. Dynamic-degree degradation after SFT of I2V/TI2V models is a practical and widely observed failure mode; a sampler-agnostic, forward-process method that restores motion without appearance collapse is of clear engineering value. Strengths include: (i) dense, annotation-free motion rewards grounded in optical flow and point tracks; (ii) an explicit hybrid Forward-KL + ELBO derivation with stated approximations (Appendix A.3); (iii) systematic ablations (Table 3, Figure 4) showing that IMR alone hacks and that NA + Adv. RM + LMR restore balance; and (iv) a large efficiency gap versus reverse-process RL (2.46× vs 19.35× SFT wall-clock on SVD). If the empirical gains hold under broader evaluation, SHIFT is a useful post-training recipe for motion-aware video diffusion alignment.

major comments (3)
  1. [§5.1 Table 1; §5.2 Table 2] Tables 1–2 report single-run point estimates with no seeds, error bars, or significance tests. Several headline deltas are small (e.g., SHIFT vs FlowGRPO: VBench Motion 86.70 vs 86.67, Overall 84.69 vs 84.63; SHIFT vs Base Motion Score 4.40 vs 4.39). Without variance, it is hard to treat these as reliable improvements rather than run noise. At minimum, multi-seed means±std (or bootstrap CIs) on VBench Motion, Motion Score, and FVD for Base/SFT/SHIFT (and FlowGRPO where feasible) are needed to support the claim that SHIFT is strictly best overall.
  2. [§4.1; §5.3 Table 3 / Figure 4] IMR/LMR are binary real-vs-generated discriminators on motion features (Eq. 7), not calibrated motion-quality scores. Section 5.3 shows IMR alone causes classic reward hacking (Appearance 81.62, FVD 502.60), mitigated by NA and Adv. RM, but the paper never validates that the final reward correlates with human motion preference or independent physical metrics beyond VBench’s dynamic-degree/smoothness suite. A short human preference study (or correlation of r(x) with optical-flow magnitude / trajectory consistency on held-out real/fake pairs) would strengthen the claim that the proxy is faithful rather than merely “real-looking under SEA-RAFT/CoTracker features.”
  3. [§5.2 Table 2] Wan2.2 results (Table 2) compare only Base, SFT, and SHIFT; DenseDPO and FlowGRPO are absent. The SVD comparison is stronger, but the claim that SHIFT “efficiently resolves dynamic-degree collapse in modern video diffusion models” is only partially stress-tested at 5B scale. Either add at least one reverse-process or preference baseline on Wan2.2 (even at reduced budget) or clearly scope the multi-model claim to “SFT collapse is fixed; full RL ranking is shown on SVD.”
minor comments (6)
  1. [Abstract; §4.3 Remark] Abstract and title use “Adversarial Hybrid Fine-Tuning” / “adversarial advantages,” while the method decouples discriminator and generator gradients (Remark, §4.3). A one-sentence clarification early on that this is not min-max GAN training would reduce confusion.
  2. [§4.2 Eq. (12); Appendix A.3.5] Eq. (12) and Algorithm 1 use linearized advantages ˜A_b rather than exp(A/β); the first-order expansion is only in Appendix A.3.5. Cross-reference this in the main text near Eq. (12) so readers do not think the exponential AWR weight was dropped silently.
  3. [Figure 1] Figure 1(b) schematic text is partially garbled/illegible in the manuscript rendering; regenerate with readable labels for “online exploration,” “offline anchor,” and the adversarial loop.
  4. [Table 1; §2] Inconsistent citation of DenseDPO: Table 1 cites [41], related work cites [42, 31]. Unify bibliography keys.
  5. [§5; Appendix B; Table 5] β is described as temperature in Algorithm 1 but realized as β = 1/λ_awr with λ_awr = 0.01 (Table 5). State the mapping once in §5 Implementation to avoid mismatch with the β ∈ {1,10,100} ablation in Appendix B.
  6. [Throughout] Typos / polish: “HybrId” in SHIFT expansion; “Fr´ echet” spacing; “keep optimizing” → “continue optimizing” (§4.3); “difficulty bias” needs a brief definition or citation expansion.

Circularity Check

0 steps flagged

No significant circularity: SHIFT objective and pixel-motion rewards are independently derived approximations and discriminators; VBench gains are external empirical outcomes, not forced by construction.

full rationale

The paper's central derivation (Sec. 4.2 and Appendix A.3) starts from a constrained RL objective, replaces the reverse-KL reference with a stationary data distribution to obtain a Forward-KL target p* ∝ p_data exp(r/β), approximates the expectation by a mixture of offline data and advantage-weighted online rollouts, substitutes the diffusion ELBO for the intractable log-likelihood, and linearizes the exponential advantage weight. Each step is an explicit, standard approximation (ELBO, first-order Taylor, group-relative recentering) whose validity is not assumed by definition of the target metrics. The pixel-motion rewards (IMR via optical-flow residual δ of Eq. 6, LMR via CoTracker trajectories) are ordinary binary ViT discriminators trained on real-vs-generated pairs; their logits become rewards only after training, and the paper itself demonstrates (Sec. 5.3, Fig. 4, Table 3) that IMR alone produces classic reward hacking that is later mitigated by Noise Alignment and adversarial co-training. Reported improvements are measured on external VBench-I2V appearance/motion aggregates, Motion Score, and FVD, none of which appear in the loss or reward definitions. Self-citations are limited to background diffusion/RL literature and do not supply a uniqueness theorem or ansatz that forces the claimed motion recovery. Consequently the derivation chain is self-contained against external benchmarks and exhibits no reduction of a prediction to its own inputs.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 3 invented entities

The central empirical claim rests on standard diffusion ELBO approximations, the brightness-constancy residual as a motion feature, and a handful of free training hyper-parameters (temperature β, buffer reuse K, advantage clip range, LoRA rank). No new physical entities are postulated; the reward models are ordinary ViT discriminators.

free parameters (4)
  • temperature β (realized as 1/λ_awr) = 100
    Controls SFT-vs-RL trade-off; ablated in {1,10,100}, final value 100 chosen for stability.
  • buffer reuse factor K = 10
    Number of gradient steps per on-policy buffer; set to 10 for efficiency.
  • advantage clipping range = [-10,10]
    Hard clip [−10,10] to stabilize LoRA updates.
  • LoRA rank / α = 32 / 32
    Rank-32 adapters on temporal (or all) attention layers.
axioms (3)
  • domain assumption Maximizing log-likelihood of a diffusion model is approximately equivalent to minimizing the simplified noise-prediction ELBO (Eq. 11 / Appendix A.3.4).
    Standard DDPM approximation used to replace the intractable log p_θ term inside the Forward-KL objective.
  • ad hoc to paper Omitting the divergence term from the continuity equation yields a usable transport residual for pixel intensity (Eq. 6).
    Justified by the claim that pixel radiance is not conserved under zoom; empirical observation that full continuity produces blurry static videos.
  • domain assumption Group-relative advantages recentered on the fake-batch mean (without std normalization) provide stable credit assignment.
    Borrowed from GRPO literature; used throughout Algorithm 1.
invented entities (3)
  • Instantaneous Motion Reward (IMR) no independent evidence
    purpose: Dense local reward from optical-flow residual + confidence maps fed to a ViT discriminator.
    New reward architecture; independent evidence is limited to the paper’s own ablation tables.
  • Long-term Motion Reward (LMR) no independent evidence
    purpose: Trajectory-level reward from CoTracker point tracks + correlation maps.
    New reward architecture complementary to IMR; again validated only inside the paper.
  • SHIFT hybrid objective (Eq. 12) no independent evidence
    purpose: Unifies offline SFT anchor with online advantage-weighted denoising loss under Forward KL.
    Core algorithmic contribution; derived from the mixture estimator of the Forward-KL target.

pith-pipeline@v1.1.0-grok45 · 25783 in / 2733 out tokens · 29111 ms · 2026-07-13T23:09:57.095479+00:00 · methodology

0 comments
read the original abstract

Image-conditioned video diffusion models achieve impressive visual realism but often suffer from weakened motion fidelity, e.g., reduced motion dynamics or degraded long-term temporal coherence, especially after fine-tuning. We study motion alignment in video diffusion models post-training. To address this, we introduce pixel-motion rewards based on pixel flux dynamics, capturing both instantaneous and long-term motion consistency. We further propose \underline{S}mooth \underline{H}ybr\underline{i}d \underline{F}ine-\underline{t}uning (SHIFT), a scalable reward-driven framework that unifies supervised fine-tuning and advantage-weighted fine-tuning. Benefiting from novel adversarial advantages, SHIFT improves convergence speed and mitigates reward hacking. Experiments show that our approach efficiently resolves dynamic-degree collapse in modern video diffusion models supervised fine-tuning. Project page: https://xiye20.github.io/projects/SHIFT/.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

46 extracted references · 5 linked inside Pith

  1. [1]

    Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., Man´ e, D.: Concrete problems in ai safety (2016)

    2016

  2. [2]

    In: The Twelfth International Conference on Learning Representations (2024)

    Black, K., Janner, M., Du, Y., Kostrikov, I., Levine, S.: Training diffusion models with reinforcement learning. In: The Twelfth International Conference on Learning Representations (2024)

    2024

  3. [3]

    In: The Fourteenth International Conference on Learning Representations (2026)

    Chen, H., Zheng, K., Zhang, Q., Cui, G., Cui, Y., Ye, H., Lin, T.Y., Liu, M.Y., Zhu, J., Wang, H.: NFT: Bridging supervised learning and reinforcement learning in math reasoning. In: The Fourteenth International Conference on Learning Representations (2026)

    2026

  4. [4]

    Chen, T.S., Lin, C.H., Tseng, H.Y., Lin, T.Y., Yang, M.H.: Motion-conditioned diffusion model for controllable video synthesis (2023)

    2023

  5. [5]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Choi, J., Lee, J., Shin, C., Kim, S., Kim, H., Yoon, S.: Perception prioritized training of diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 11472–11481 (2022)

    2022

  6. [6]

    In: The Thirteenth Interna- tional Conference on Learning Representations (2025)

    Domingo-Enrich, C., Drozdzal, M., Karrer, B., Chen, R.T.Q.: Adjoint matching: Fine-tuning flow and diffusion generative models with memoryless stochastic optimal control. In: The Thirteenth Interna- tional Conference on Learning Representations (2025)

    2025

  7. [7]

    In: Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., Levine, S

    Fan, Y., Watkins, O., Du, Y., Liu, H., Ryu, M., Boutilier, C., Abbeel, P., Ghavamzadeh, M., Lee, K., Lee, K.: Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models. In: Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., Levine, S. (eds.) Advances in Neural Information Processing Systems. vol. 36, pp. 79858–79885. Curran Assoc...

    2023

  8. [8]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Ge, S., Mahapatra, A., Parmar, G., Zhu, J.Y., Huang, J.B.: On the content bias in frechet video distance. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 7277–7288 (June 2024)

    2024

  9. [9]

    In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, H

    Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, H. (eds.) Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual (2020)

    2020

  10. [10]

    Artificial intelligence17(1-3), 185–203 (1981) 11

    Horn, B.K., Schunck, B.G.: Determining optical flow. Artificial intelligence17(1-3), 185–203 (1981) 11

    1981

  11. [11]

    International Journal of Computer Vision133(7), 4384–4400 (2025)

    Hu, Y., Chen, Z., Luo, C.: Lamd: Latent motion diffusion for image-conditional video generation. International Journal of Computer Vision133(7), 4384–4400 (2025)

    2025

  12. [12]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Huang, Z., He, Y., Yu, J., Zhang, F., Si, C., Jiang, Y., Zhang, Y., Wu, T., Jin, Q., Chanpaisit, N., et al.: Vbench: Comprehensive benchmark suite for video generative models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 21807–21818 (2024)

    2024

  13. [13]

    Jaques, N., Ghandeharioun, A., Shen, J.H., Ferguson, C., Lapedriza, A., Jones, N., Gu, S., Picard, R.: Way off-policy batch deep reinforcement learning of implicit human preferences in dialog (2019)

    2019

  14. [14]

    In: 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Jeong, H., Huang, C.H.P., Chul Ye, J., Mitra, N.J., Ceylan, D.: Track4gen: Teaching video diffusion models to track points improves video generation. In: 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 7276–7287 (2025)

    2025

  15. [15]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    Karaev, N., Makarov, Y., Wang, J., Neverova, N., Vedaldi, A., Rupprecht, C.: CoTracker3: Simpler and better point tracking by pseudo-labelling real videos. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 6013–6022 (2025)

    2025

  16. [16]

    In: European conference on computer vision

    Karaev, N., Rocco, I., Graham, B., Neverova, N., Vedaldi, A., Rupprecht, C.: Cotracker: It is better to track together. In: European conference on computer vision. pp. 18–35. Springer (2024)

    2024

  17. [17]

    Lee, K., Liu, H., Ryu, M., Watkins, O., Du, Y., Boutilier, C., Abbeel, P., Ghavamzadeh, M., Gu, S.S.: Aligning text-to-image models using human feedback (2023)

    2023

  18. [18]

    arXiv preprint arXiv:2503.21765 (2025)

    Lin, M., Wang, X., Wang, Y., Wang, S., Dai, F., Ding, P., Wang, C., Zuo, Z., Sang, N., Huang, S., et al.: Exploring the evolution of physics cognition in video generation: A survey. arXiv preprint arXiv:2503.21765 (2025)

    Pith/arXiv arXiv 2025

  19. [19]

    In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025)

    Lin, S., Yang, C., He, H., Jiang, J., Ren, Y., Xia, X., Zhao, Y., Xiao, X., Jiang, L.: Autoregressive adver- sarial post-training for real-time interactive video generation. In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025)

    2025

  20. [20]

    In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025)

    Liu, J., Liu, G., Liang, J., Li, Y., Liu, J., Wang, X., Wan, P., ZHANG, D., Ouyang, W.: Flow-GRPO: Training flow matching models via online RL. In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025)

    2025

  21. [21]

    Liu, Y., Wen, C., Hu, Y., Jayaraman, D., Gao, Y.: Timerewarder: Learning dense reward from passive videos via frame-wise temporal distance (2025)

    2025

  22. [22]

    In: Second Conference on Language Modeling (2025)

    Liu, Z., Chen, C., Li, W., Qi, P., Pang, T., Du, C., Lee, W.S., Lin, M.: Understanding r1-zero-like training: A critical perspective. In: Second Conference on Language Modeling (2025)

    2025

  23. [23]

    IEEE Transactions on Neural Networks and Learning Systems pp

    Ma, Y., Klabjan, D., Utke, J.: Video to video generative adversarial network for few-shot learning based on policy gradient. IEEE Transactions on Neural Networks and Learning Systems pp. 1–15 (2025)

    2025

  24. [24]

    Motamed, S., Culp, L., Swersky, K., Jaini, P., Geirhos, R.: Do generative video models understand physical principles? arXiv preprint arXiv:2501.09038 (2025)

    Pith/arXiv arXiv 2025

  25. [25]

    Nair, A., Gupta, A., Dalal, M., Levine, S.: Awac: Accelerating online reinforcement learning with offline datasets (2021)

    2021

  26. [26]

    Nichol, A., Dhariwal, P.: Improved denoising diffusion probabilistic models (2021)

    2021

  27. [27]

    In: Proceedings of the 24th international conference on Machine learning

    Peters, J., Schaal, S.: Reinforcement learning by reward-weighted regression for operational space con- trol. In: Proceedings of the 24th international conference on Machine learning. pp. 745–750 (2007)

    2007

  28. [28]

    Pfau, D., Vinyals, O.: Connecting generative adversarial networks and actor-critic methods (2016)

    2016

  29. [29]

    arXiv preprint arXiv:1704.00675 (2017) 12

    Pont-Tuset, J., Perazzi, F., Caelles, S., Arbel´ aez, P., Sorkine-Hornung, A., Van Gool, L.: The 2017 davis challenge on video object segmentation. arXiv preprint arXiv:1704.00675 (2017) 12

    Pith/arXiv arXiv 2017

  30. [30]

    arXiv preprint arXiv:2407.08737 (2024)

    Prabhudesai, M., Mendonca, R., Qin, Z., Fragkiadaki, K., Pathak, D.: Video diffusion alignment via reward gradients. arXiv preprint arXiv:2407.08737 (2024)

    Pith/arXiv arXiv 2024

  31. [31]

    In: Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., Levine, S

    Rafailov, R., Sharma, A., Mitchell, E., Manning, C.D., Ermon, S., Finn, C.: Direct preference opti- mization: Your language model is secretly a reward model. In: Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., Levine, S. (eds.) Advances in Neural Information Processing Systems. vol. 36, pp. 53728–53741. Curran Associates, Inc. (2023)

    2023

  32. [32]

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algo- rithms (2017)

    2017

  33. [33]

    Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y.K., Wu, Y., Guo, D.: Deepseekmath: Pushing the limits of mathematical reasoning in open language models (2024)

    2024

  34. [34]

    In: 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021

    Song, J., Meng, C., Ermon, S.: Denoising diffusion implicit models. In: 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net (2021)

    2021

  35. [35]

    In: Wallach, H., Larochelle, H., Beygelzimer, A., d'Alch´ e-Buc, F., Fox, E., Garnett, R

    Song, Y., Ermon, S.: Generative modeling by estimating gradients of the data distribution. In: Wallach, H., Larochelle, H., Beygelzimer, A., d'Alch´ e-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems. vol. 32. Curran Associates, Inc. (2019)

    2019

  36. [36]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Tian, J., Qu, X., Lu, Z., Wei, W., Liu, S., Cheng, Y.: Extrapolating and decoupling image-to-video generation models: Motion modeling is easier than you think. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 12512–12521 (June 2025)

    2025

  37. [37]

    arXiv preprint arXiv:2503.08153 (2025)

    Wang, J., Ma, A., Cao, K., Zheng, J., Zhang, Z., Feng, J., Liu, S., Ma, Y., Cheng, B., Leng, D., et al.: Wisa: World simulator assistant for physics-aware text-to-video generation. arXiv preprint arXiv:2503.08153 (2025)

    Pith/arXiv arXiv 2025

  38. [38]

    In: European Conference on Computer Vision

    Wang, Y., Lipson, L., Deng, J.: Sea-raft: Simple, efficient, accurate raft for optical flow. In: European Conference on Computer Vision. pp. 36–54. Springer (2024)

    2024

  39. [39]

    Machine Learning8, 229–256 (2004)

    Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning8, 229–256 (2004)

    2004

  40. [40]

    Wu, J., Gao, Y., Ye, Z., Li, M., Li, L., Guo, H., Liu, J., Xue, Z., Hou, X., Liu, W., Zeng, Y., Huang, W.: Rewarddance: Reward scaling in visual generation (2025)

    2025

  41. [41]

    In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025)

    Wu, Z., Kag, A., Skorokhodov, I., Menapace, W., Mirzaei, A., Gilitschenski, I., Tulyakov, S., Siarohin, A.: DenseDPO: Fine-grained temporal preference optimization for video diffusion models. In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025)

    2025

  42. [42]

    Xu, J., Huang, Y., Cheng, J., Yang, Y., Xu, J., Wang, Y., Duan, W., Yang, S., Jin, Q., Li, S., Teng, J., Yang, Z., Zheng, W., Liu, X., Zhang, D., Ding, M., Zhang, X., Gu, X., Huang, S., Huang, M., Tang, J., Dong, Y.: Visionreward: Fine-grained multi-dimensional human preference learning for image and video generation (2026)

    2026

  43. [43]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Yuan, H., Zhang, S., Wang, X., Wei, Y., Feng, T., Pan, Y., Zhang, Y., Liu, Z., Albanie, S., Ni, D.: Instructvideo: Instructing video diffusion models with human feedback. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 6463–6474 (June 2024)

    2024

  44. [44]

    In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025)

    Zha, K., Gao, Z., Shen, M., Hong, Z.W., Boning, D.S., Katabi, D.: RL tango: Reinforcing generator and verifier together for language reasoning. In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025)

    2025

  45. [45]

    In: NeurIPS (2024)

    Zhao, M., Zhu, H., Xiang, C., Zheng, K., Li, C., Zhu, J.: Identifying and solving conditional image leakage in image-to-video diffusion model. In: NeurIPS (2024)

    2024

  46. [46]

    a bird with a fish in its beak flying over a field

    Zheng, K., Chen, H., Ye, H., Wang, H., Zhang, Q., Jiang, K., Su, H., Ermon, S., Zhu, J., Liu, M.Y.: DiffusionNFT: Online diffusion reinforcement with forward process. In: The Fourteenth International Conference on Learning Representations (2026) 13 A Supplementary Material A.1 Preliminaries: Diffusion Models The Denoising Diffusion Probabilistic Model (DD...

    arXiv 2026