Pith. sign in

REVIEW 4 major objections 5 minor 39 references

FlowLoss: Dynamic Flow-Conditioned Loss Strategy for Video Diffusion Models

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Direct optical-flow comparison, gated by noise, steadies early video-diffusion training.

desk verdict A genuine but modest idea whose main claimed benefit is confounded by the static-bias problem and extra compute, but the paper is honest and deserves a critical referee. read the letter →

arxiv 2504.14535 v1 pith:UEKN4LBY submitted 2025-04-20 cs.CV

Kuanting Wu , Kei Ota , Asako Kanezaki This is my paper · ORCID
classification cs.CV
keywords FlowLossvideodiffusionmodelsopticalsupervisionnoise-awareweightingmotionstabilitytrainingconvergenceroboticmanipulationvideosgating
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Video diffusion models generate realistic frames but often produce jittery or physically inconsistent motion. FlowLoss attacks this by adding a training term that directly compares optical flow extracted from generated videos with flow from ground-truth videos, instead of using warping-based objectives. Because flow estimates are unreliable on heavily noised inputs, the loss is gated by the diffusion noise level so it acts mainly in low-noise regimes. On robotic manipulation videos, this scheme makes training stabilize earlier and reach a reasonable motion prior faster, although final-generation metrics do not consistently beat the reconstruction-only baseline.

What carries the argument

The method couples the EDM-style diffusion objective $L_{\mathrm{recon}} = \mathbb{E}[\lambda(\sigma)\lVert D_\theta(y+n;\sigma)-y\rVert^2]$ with a flow loss $L_{\mathrm{flow}} = s\cdot\mathbb{E}[\lambda(\sigma)\sum_t o(\alpha_y^t)\lVert f_y^t - f_{\hat{y}}^t\rVert_2^2]$, where $f$ is a pretrained dense flow extractor (DOT, Dense Optical Tracking), $o$ is an occlusion mask, $s=10^{-6}$, and $\lambda(\sigma)=(\sigma^2+1)/\sigma^2$. A hard gating function $w_\psi(\sigma)=1/(\sigma^2+1)$ for $\sigma<\psi$ and $0$ otherwise decides when the flow term participates, so supervision concentrates on clean inputs where flow is reliable.

What would settle it

Train the same model on a robotic dataset with large textureless or heavily occluded regions while keeping FlowLoss; if the early-training flattening of FVD and flow-error curves disappears or flips, the claim that flow supervision itself drives early stabilization fails. Alternatively, replace DOT flow fields with randomized flow fields of the same magnitude; if early training still stabilizes, the effect comes from the loss shape, not from real motion information.

Watch

Extended reading notes

Core claim

The paper's central claim is that direct, noise-conditioned optical-flow comparison can serve as an effective auxiliary objective for video diffusion training. Concretely, with the flow loss active, validation curves for FVD (a video-quality metric) and flow error flatten sooner, and samples at step 100 show less spatial drift and jitter than the reconstruction-only baseline. The authors attribute the early benefit to the flow signal steering the model toward a coherent motion prior in the low-noise stages where denoising is most sensitive, while acknowledging that hard gating and noisy flow estimates in occluded or texture-free regions limit the final gains.

Load-bearing premise

The load-bearing premise is that the pretrained flow estimator, applied to denoised and ground-truth videos at low noise levels, produces gradients that point toward coherent motion rather than artifacts, so that the benefit vanishes if flow estimates are noisy in textureless or occluded scenes.

Editorial extensions

If this is right

  • Early in training, flow supervision flattens FVD and flow-error validation curves sooner, indicating a faster-formed motion prior.
  • Gating flow loss to low-noise steps avoids harmful gradients; applying it at high-noise steps produces flickering, unstable outputs.
  • The benefits are largely confined to early training; final metrics are comparable or mixed, and training time grows substantially with flow loss.
  • A weighted-average variant that emphasizes flow at small $\sigma$ matches the default method's early stability, reinforcing the noise-aware scheduling intuition.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the benefit is concentrated at the start of training, a natural extension is to schedule flow loss with a decaying weight or disable it after a motion prior is established, trading the compute overhead for the early gain.
  • In static-camera robotic scenes, most pixels barely move; direct flow comparison may mostly reinforce static layout, which could explain why final metrics do not improve. Applying FlowLoss on dynamic scenes or with a flow estimator robust to textureless regions is a testable follow-up.
  • The same noise-gated supervision idea could transfer to other motion-related auxiliary signals, such as point trajectories or depth changes, for video diffusion training.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes FlowLoss, an auxiliary training objective for video diffusion models that extracts optical flow from both the denoised prediction and the ground-truth video with a pretrained estimator (DOT), compares the two flow fields directly, and gates the resulting loss by noise level through a hard threshold ψ and a global scale s. The full objective is L = L_recon + w_ψ(σ)·L_flow, where L_flow is only active at low noise levels. Experiments on Bridge v2 with a Stable Video Diffusion 2.1 UNet compare the reconstruction-only baseline with several FlowLoss variants. The authors report early-training stabilization visible in validation curves (Fig. 4) and sample generations at step 100 (Fig. 5), while candidly acknowledging that final quantitative results are mixed and that computation time increases substantially.

Significance. If the central claim held, the paper would make a modest but useful contribution: a direct-flow-matching loss with noise-aware gating is a reasonable alternative to warping-based flow supervision for video diffusion training, and the explicit analysis of low-noise versus high-noise supervision is valuable. The authors are unusually transparent about the limitations of their evidence, including the absence of consistent final-stage gains and the computational overhead. However, the early-training benefit is supported mainly by qualitative samples and by a validation metric that is essentially the training loss itself, on a single dataset and without repeated seeds. The idea is worth pursuing, but the manuscript currently does not establish the central claim with independent, quantitative evidence.

major comments (4)
  1. [§4, Fig. 4(f)] Fig. 4(f) plots Lflow/λ(σ) as a validation metric, but this is essentially the same flow-matching quantity that the proposed training objective minimizes (with gating and scaling). A faster decrease of this quantity under FlowLoss is therefore expected by construction and does not independently confirm that the model has learned a better motion prior. The paper reports 'unweighted Lflow' as its temporal metric in §4 and Fig. 4(f) labels Lflow/λ(σ); either way, this metric is not independent of the training signal. To support the acceleration claim, the authors should report independent temporal consistency metrics (e.g., FVD on dynamic regions, temporal warping error, or human evaluation) with confidence intervals over multiple seeds.
  2. [§3, Eq. (1); §4, static-bias concern] The early stabilization reported in Fig. 5 may be a static-bias artifact rather than evidence of learning a correct motion prior. On Bridge v2 the camera is fixed and, as the authors note in §4, 'most pixels are static.' With ground-truth flow fy near zero, the flow loss in Eq. (1) approximately reduces to s·E||f_yhat||², which directly penalizes any generated motion. A model that outputs a mostly static video will therefore receive a low Lflow and may appear 'stable' at step 100 without having learned the robot-arm motion. The authors should test FlowLoss on a dataset with substantial object or camera motion, or add a control experiment that penalizes temporal differences without using flow, and report metrics restricted to dynamic regions.
  3. [§4, Table 1 and Fig. 4] All convergence comparisons are by training step, but Table 1 shows that the FlowLoss runs add 10–82% wall-clock time (e.g., 14h29m for ψ=0.250 versus 7h56m for the baseline). Comparing by step count therefore conflates 'better sample efficiency' with 'more compute per step.' The paper's practical claim of accelerated convergence should be evaluated by early-stage quality as a function of wall-clock time or total compute, not just step number. The authors should also provide exact early-stage numbers (e.g., FVD, LPIPS at step 100) with multiple seeds; the plotted curves in Fig. 4 appear to be single runs with no error bars.
  4. [§3, Eqs. (1)–(2); §5] The method depends on several hand-chosen constants: the global scaling factor s = 10⁻⁶, the occlusion fallback weight 0.3, and the gating threshold ψ. The reported experiments vary ψ, but provide no sensitivity analysis for s or the occlusion weight, and the ablation in §5 uses a different 'Weighted Average' formulation rather than the default additive objective. Since these constants directly control the magnitude and active region of the flow loss, at least a small sensitivity study (and ideally repeated-seed training) is needed to show that the early-training benefit is robust rather than tuned to the particular choices.
minor comments (5)
  1. [§3, Eq. (1)] The definition of the occlusion mask says 'o(αt(i,j)) = 1 for visible regions and 0.3 otherwise'; the occlusion fallback value 0.3 is not justified and the notation would benefit from an explanation of why a nonzero weight is chosen for occluded pixels.
  2. [Fig. 2] The four panels in Fig. 2 are very small and the curves are hard to read; in particular, panel (c) and (d) would benefit from larger fonts and clearer legends, since the comparison of weighting strategies is central to the method.
  3. [Table 1] The numeric values in Table 1 lack spaces after some commas (e.g., '0.811828.8986' should be parsed as SSIM 0.8118 and PSNR 28.8986), making the table difficult to read. Also, the bold highlighting should be explained in the caption.
  4. [§2, Related Work] There is a grammatical issue in the second paragraph of §2: 'Dense optical flow captures pixel-level motion dynamics, it serves as a valuable signal' should be split into two sentences. Additionally, references [9] and [13] appear to describe the same work (optical-flow guided prompt optimization) with different IDs; the authors should verify and merge duplicates.
  5. [§4, Evaluation Metrics] The description of 'unweighted Lflow' as a metric is ambiguous because Eq. (1) defines Lflow with the λ(σ) weighting and global scale s; the authors should state exactly what is computed for evaluation, including whether the scale and occlusion mask are applied during evaluation.

Circularity Check

1 steps flagged · score 2.0 of 10

One metric (unweighted Lflow) is also the training objective, so the early Lflow curve is partly self-confirming; FVD and qualitative samples remain independent evidence.

  1. other [Section 3 (Flow Loss equation) and Section 4 (Evaluation Metrics, Outcome analysis; Fig. 4f)]
    "For temporal consistency, we report FVD [39] and unweighted Lflow. ... the early flattening of validation curves across FVD and Lflow suggests that flow supervision—when active—helps the model reach a reasonable motion prior more quickly."

    The training objective is L = Lrecon + wψ(σ)·Lflow, where Lflow = s·E[λ(σ)Σ_t o(α)||f_y − f_ŷ||²]. The validation metric called 'unweighted Lflow' is the same loss with the weighting/gating removed. Minimizing a loss will, by construction, reduce that loss on similar data, so the faster drop in Fig. 4(f) is expected and is not an independent confirmation of improved motion learning. This is only partial circularity because FVD, FID, SSIM, PSNR, LPIPS, and visual comparisons provide separate evidence.

full rationale

The paper's central early-training claim is supported by independent metrics (FVD, FID, SSIM, PSNR, LPIPS) and qualitative samples, so it is not forced by construction. The only circular element is the use of an 'unweighted Lflow' validation curve that is essentially the same quantity being minimized, making part of the reported early convergence self-referential. There are no load-bearing self-citations: the cited EDM, DOT, Stable Video Diffusion, and Bridge v2 are all external sources, and the hand-chosen constants s and ψ are not fitted to the evaluation results. Concerns about static-bias artifacts or wall-clock overhead are correctness/experimental-design issues, not circularity. Overall the derivation is self-contained, with a minor metric-related circularity, so the score is 2.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on assumptions about flow-estimator reliability, the transfer from flow loss to temporal quality, and several hand-chosen constants. No new physical entities or latent spaces are introduced.

free parameters (3)
  • global scaling factor s = 1e-6
    Scales Lflow to balance against reconstruction loss; chosen by hand with no sensitivity analysis.
  • gating threshold psi = 0.0625, 0.125, 0.250
    Determines the maximum noise level where flow loss is active; results vary across thresholds and the choice is not derived.
  • occlusion fallback weight = 0.3
    Weight assigned to occluded pixels in the flow loss; fixed by hand.
assumptions (4)
  • domain assumption DOT optical flow extractor yields accurate, differentiable flow on generated and ground-truth video at low noise levels.
    Invoked in Section 3 where Lflow is computed with DOT; Section 4 concedes flow can be noisy or inaccurate in occluded and low-texture regions.
  • domain assumption Reducing L2 distance between generated and ground-truth flow fields improves temporal coherence without harming visual fidelity.
    Underlies the training objective; Table 1 shows final-stage fidelity metrics are mixed, so the assumption is only partially supported.
  • domain assumption The EDM noise weighting lambda(sigma) transfers unchanged from reconstruction loss to flow loss.
    Section 3 reuses lambda(sigma) from EDM without a derivation specific to flow supervision.
  • domain assumption Results on Bridge v2 generalize to other video domains.
    Only one dataset is used; the paper frames conclusions around robotic videos but discusses general video diffusion models.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FlowLoss: Dynamic Flow-Conditioned Loss Strategy for Video Diffusion Models." pith.science (2026). https://pith.science/paper/UEKN4LBY

@misc{pith2026250414535,
  author       = {Pith},
  title        = {Pith review of: FlowLoss: Dynamic Flow-Conditioned Loss Strategy for Video Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UEKN4LBY}},
  note         = {Machine review of arXiv:2504.14535}
}
read the original abstract

Video Diffusion Models (VDMs) can generate high-quality videos, but often struggle with producing temporally coherent motion. Optical flow supervision is a promising approach to address this, with prior works commonly employing warping-based strategies that avoid explicit flow matching. In this work, we explore an alternative formulation, FlowLoss, which directly compares flow fields extracted from generated and ground-truth videos. To account for the unreliability of flow estimation under high-noise conditions in diffusion, we propose a noise-aware weighting scheme that modulates the flow loss across denoising steps. Experiments on robotic video datasets suggest that FlowLoss improves motion stability and accelerates convergence in early training stages. Our findings offer practical insights for incorporating motion-based supervision into noise-conditioned generative models.

Figures

Figures reproduced from arXiv: 2504.14535 by the authors.

Figure 1
Figure 1. Overview of the FlowLoss supervision frame [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Panels (a) and (b) show Lrecon and Lflow computed on a single validation sample, using a VDM built upon the UNet backbone from Stable Video Diffusion 2.1 (Image-to-Video). (a) The original EDM defines the reconstruction loss as Lrecon = λ(σ) · LMSE, where λ(σ) increases as σ decreases, encouraging fine-detail recon￾struction during low-noise steps. (b) Variants of our loss function. (c) Corresponding weighting strat… view at source ↗
Figure 3
Figure 3. Higher noise scales σ lead to corrupted inputs and degraded flow extraction, motivating our noise￾aware flow loss design. 3 Methodology To improve temporal consistency and physical plau￾sibility in video generation, we introduce a training objective that incorporates optical flow supervision di￾rectly. In this section, we first define the flow-based loss term Lflow, then describe how it is integrated with the EDM-st… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Validation performance over training steps for different flow loss strategies. [PITH_FULL_IMAGE:figures/full_fig_p003_4.png]
Figure 5
Figure 5. Figure 5: Comparison of early-stage generation re [PITH_FULL_IMAGE:figures/full_fig_p003_5.png]
Figure 6
Figure 6. Figure 6: Comparison of generated videos at 1,600 training steps using different weighting strategies un￾der the weighted average formulation. These variants serve to validate our hypothesis that flow supervision is more effective when applied in low￾noise stages, where flow ext…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 24 canonical work pages

  1. [1]

    Video diffusion models

    Jonathan Ho, William Chan, Chitwan Saharia, David Fleet, Mohammad Norouzi, and Tim Salimans. Video diffusion models. arXiv preprint arXiv:2204.03458 , 2022

    arXiv 2022

  2. [2]

    Align your latents: High-resolution video syn- thesis with latent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video syn- thesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pat- tern recognition, pages 22563–22575, 2023

    work page 2023

  3. [3]

    Hierarchical patch diffusion models for high-resolution video generation

    Ivan Skorokhodov, Willi Menapace, Aliaksandr Siaro- hin, and Sergey Tulyakov. Hierarchical patch diffusion models for high-resolution video generation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7569–7579, 2024

    work page 2024

  4. [4]

    Struc- ture and content-guided video synthesis with diffusion models

    Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog, and Anastasis Germanidis. Struc- ture and content-guided video synthesis with diffusion models. In Proceedings of the IEEE/CVF interna- tional conference on computer vision, pages 7346–7356, 2023

    work page 2023

  5. [5]

    Magicanimate: Temporally consis- tent human image animation using diffusion model

    Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Han- shu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, and Mike Zheng Shou. Magicanimate: Temporally consis- tent human image animation using diffusion model. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 1481–1490, 2024

    work page 2024

  6. [6]

    Modelscope text-to- video technical report

    Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. Modelscope text-to- video technical report. arXiv preprint arXiv:2308.06571, 2023

    arXiv 2023

  7. [7]

    Onlyflow: Opti- cal flow based motion conditioning for video diffusion models, 2024

    Mathis Koroglu, Hugo Caselles-Dupr´ e, Guillaume Jean- neret Sanmiguel, and Matthieu Cord. Onlyflow: Opti- cal flow based motion conditioning for video diffusion models, 2024

    work page 2024

  8. [8]

    Motion-i2v: Consistent and controllable image-to-video generation with explicit motion modeling

    Xiaoyu Shi, Zhaoyang Huang, Fu-Yun Wang, Weikang Bian, Dasong Li, Yi Zhang, Manyuan Zhang, Ka Chun Cheung, Simon See, Hongwei Qin, et al. Motion-i2v: Consistent and controllable image-to-video generation with explicit motion modeling. In ACM SIGGRAPH 2024 Conference Papers, pages 1–11, 2024

    2024

Show all 39 references
  1. [9]

    Optical-flow guided prompt optimization for co- herent video generation, 2025

    Hyelin Nam, Jaemin Kim, Dohun Lee, and Jong Chul Ye. Optical-flow guided prompt optimization for co- herent video generation, 2025

    2025

  2. [10]

    Flovd: Optical flow meets video diffusion model for enhanced camera-controlled video synthesis

    Wonjoon Jin, Qi Dai, Chong Luo, Seung-Hwan Baek, and Sunghyun Cho. Flovd: Optical flow meets video diffusion model for enhanced camera-controlled video synthesis. arXiv preprint arXiv:2502.08244 , 2025

    2025 arXiv

  3. [11]

    Flowvid: Taming imperfect optical flows for consistent video-to-video synthesis, 2023

    Feng Liang, Bichen Wu, Jialiang Wang, Licheng Yu, Kunpeng Li, Yinan Zhao, Ishan Misra, Jia-Bin Huang, Peizhao Zhang, Peter Vajda, and Diana Marculescu. Flowvid: Taming imperfect optical flows for consistent video-to-video synthesis, 2023

    2023

  4. [12]

    Conditional image-to-video generation with latent flow diffusion models

    Haomiao Ni, Changhao Shi, Kai Li, Sharon X Huang, and Martin Renqiang Min. Conditional image-to-video generation with latent flow diffusion models. In Pro- ceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 18444–18455, 2023

    2023

  5. [13]

    Optical-flow guided prompt optimization for co- herent video generation

    Hyelin Nam, Jaemin Kim, Dohun Lee, and Jong Chul Ye. Optical-flow guided prompt optimization for co- herent video generation. arXiv preprint arXiv:2411.15540, 2024

    2024 arXiv

  6. [14]

    Learning video stabilization using optical flow

    Jiyang Yu and Ravi Ramamoorthi. Learning video stabilization using optical flow. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), June 2020

    2020

  7. [15]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international confer- ence on computer vision , pages 3836–3847, 2023

    2023

  8. [16]

    Diffusion as shader: 3d-aware video diffusion for versatile video generation control

    Zekai Gu, Rui Yan, Jiahao Lu, Peng Li, Zhiyang Dou, Chenyang Si, Zhen Dong, Qifeng Liu, Cheng Lin, Zi- wei Liu, et al. Diffusion as shader: 3d-aware video diffusion for versatile video generation control. arXiv preprint arXiv:2501.03847, 2025

    2025 arXiv

  9. [17]

    Draganything: Motion control for anything using entity representation

    Weijia Wu, Zhuang Li, Yuchao Gu, Rui Zhao, Yefei He, David Junhao Zhang, Mike Zheng Shou, Yan Li, Tingting Gao, and Di Zhang. Draganything: Motion control for anything using entity representation. In European Conference on Computer Vision, pages 331–

  10. [18]

    Image conductor: Precision control for interactive video synthesis

    Yaowei Li, Xintao Wang, Zhaoyang Zhang, Zhouxia Wang, Ziyang Yuan, Liangbin Xie, Ying Shan, and Yuexian Zou. Image conductor: Precision control for interactive video synthesis. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 5031–5038, 2025

    2025

  11. [19]

    Motion prompting: Controlling video generation with motion trajectories

    Daniel Geng, Charles Herrmann, Junhwa Hur, For- rester Cole, Serena Zhang, Tobias Pfaff, Tatiana Lopez- Guevara, Carl Doersch, Yusuf Aytar, Michael Rubin- stein, et al. Motion prompting: Controlling video generation with motion trajectories. arXiv preprint arXiv:2412.02700, 2024

    2024 arXiv

  12. [20]

    Motionctrl: A unified and flexible motion con- troller for video generation

    Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion con- troller for video generation. In ACM SIGGRAPH 2024 Conference Papers, pages 1–11, 2024

    2024

  13. [21]

    Perception-as-control: Fine-grained controllable image animation with 3d-aware motion representation

    Yingjie Chen, Yifang Men, Yuan Yao, Miaomiao Cui, and Liefeng Bo. Perception-as-control: Fine-grained controllable image animation with 3d-aware motion representation. arXiv preprint arXiv:2501.05020, 2025

    2025 arXiv

  14. [22]

    Sparsectrl: Adding sparse controls to text-to-video diffusion models

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Sparsectrl: Adding sparse controls to text-to-video diffusion models. In Euro- pean Conference on Computer Vision , pages 330–348. Springer, 2024

    2024

  15. [23]

    Learning to act from ac- tionless videos through dense correspondences

    Po-Chen Ko, Jiayuan Mao, Yilun Du, Shao-Hua Sun, and Joshua B Tenenbaum. Learning to act from ac- tionless videos through dense correspondences. arXiv preprint arXiv:2310.08576, 2023

    2023 arXiv

  16. [24]

    This&that: Language-gesture controlled video generation for robot planning

    Boyang Wang, Nikhil Sridhar, Chao Feng, Mark Van der Merwe, Adam Fishman, Nima Fazeli, and Jeong Joon Park. This&that: Language-gesture controlled video generation for robot planning. arXiv preprint arXiv:2407.05530, 2024

    2024 arXiv

  17. [25]

    Learning universal policies via text-guided video generation

    Yilun Du, Sherry Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Josh Tenenbaum, Dale Schuurmans, and Pieter Abbeel. Learning universal policies via text-guided video generation. Advances in neural information pro- cessing systems, 36:9156–9172, 2023

    2023

  18. [26]

    Unisim: A neural closed-loop sensor sim- ulator

    Ze Yang, Yun Chen, Jingkang Wang, Sivabalan Mani- vasagam, Wei-Chiu Ma, Anqi Joyce Yang, and Raquel Urtasun. Unisim: A neural closed-loop sensor sim- ulator. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1389–1399, 2023

    2023

  19. [27]

    Videoagent: Self-improving video gener- ation

    Achint Soni, Sreyas Venkataraman, Abhranil Chan- dra, Sebastian Fischmeister, Percy Liang, Bo Dai, and Sherry Yang. Videoagent: Self-improving video gener- ation. arXiv preprint arXiv:2410.10076 , 2024

    2024 arXiv

  20. [28]

    Flownet: Learning optical flow with convolutional net- works

    Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg, Philip H¨ ausser, Caner Hazırba¸ s, Vladimir Golkov, Patrick Van der Smagt, Daniel Cremers, and Thomas Brox. Flownet: Learning optical flow with convolutional net- works. arXiv preprint arXiv:1504.06852 , 2015

    2015 arXiv

  21. [29]

    Efficient sparse- to-dense optical flow estimation using a learned basis and layers

    Jonas Wulff and Michael J Black. Efficient sparse- to-dense optical flow estimation using a learned basis and layers. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 120– 130, 2015

    2015

  22. [30]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 35:26565–26577, 2022

    2022

  23. [31]

    Dense optical tracking: Connecting the dots

    Guillaume Le Moing, Jean Ponce, and Cordelia Schmid. Dense optical tracking: Connecting the dots. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 19187–19197, 2024

    2024

  24. [32]

    Stable video diffusion: Scaling latent video diffu- sion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffu- sion models to large datasets. arXiv preprint arXiv:2311.15127, 2023

    2023 arXiv

  25. [33]

    Bridgedata v2: A dataset for robot learning at scale

    Homer Rich Walke, Kevin Black, Tony Z Zhao, Quan Vuong, Chongyi Zheng, Philippe Hansen-Estruch, An- dre Wang He, Vivek Myers, Moo Jin Kim, Max Du, et al. Bridgedata v2: A dataset for robot learning at scale. In Conference on Robot Learning , pages 1723–

  26. [34]

    Film: Visual reason- ing with a general conditioning layer

    Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reason- ing with a general conditioning layer. In Proceedings of the AAAI conference on artificial intelligence , vol- ume 32, 2018

    2018

  27. [35]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information pro- cessing systems, 30, 2017

    2017

  28. [36]

    Bovik, H.R

    Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simon- celli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004

    2004

  29. [37]

    Image quality metrics: Psnr vs

    Alain Hor´ e and Djemel Ziou. Image quality metrics: Psnr vs. ssim. In 2010 20th International Conference on Pattern Recognition, pages 2366–2369, 2010

    2010

  30. [38]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586–595, 2018

    2018

  31. [39]

    Towards accurate generative models of video: A new metric & challenges

    Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Ku- rach, Raphael Marinier, Marcin Michalski, and Syl- vain Gelly. Towards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717, 2018

    2018 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.