Pith. sign in

REVIEW 4 major objections 5 minor 52 references

A Physical Coherence Benchmark for Evaluating Video Generation Models via Optical Flow-guided Frame Prediction

T0 review · 4 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Optical-flow-guided frame prediction scores video physics closest to human rankings.

desk verdict Useful new benchmark for physical coherence in T2V evaluation, but the automated metric's evidence is thinner than the abstract suggests. read the letter →

arxiv 2502.05503 v3 pith:ZQRKDHMP submitted 2025-02-08 cs.CV cs.AI

Yongfan Chen , Xiuwen Zhu , Tianyu Li This is my paper · ORCID
classification cs.CVcs.AI
keywords physicalcoherencevideogenerationbenchmarktext-to-videoevaluationopticalflowguidanceframepredictionlatentdiffusionmodelanomalydetectionhumanalignment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces PhyCoBench, a benchmark of 120 prompts spanning seven physical principles, and argues that physical coherence of text-to-video output should be judged automatically by comparing a generated video against frames and optical flow predicted from its first frame and prompt. The proposed evaluator, PhyCoPredictor, is a cascade of two latent diffusion models: the first forecasts future optical flow, and the second uses that flow to guide future-frame prediction. On the benchmark, rankings produced by PhyCoPredictor agree with human rankings better than an image-to-video predictor fine-tuned on the same data and better than a video-language-model baseline. The paper's central claim is that predictability under its model is a usable proxy for human judgment of physical coherence, giving video generation a benchmark dimension that most current metrics ignore.

What carries the argument

The central object is PhyCoPredictor, a two-stage latent diffusion frame-prediction model. Its Latent Flow Diffusion Module (a 3D U-Net trained from scratch) takes the first frame and text prompt and predicts future optical flow in latent space; its Latent Video Diffusion Module (initialized from DynamiCrafter) takes the predicted flow, first frame, and prompt and generates future frames. The automatic evaluation score is $\mathrm{score} = 1/\mathrm{MSE}(f,\hat{f}) + 2\,\mathrm{MSE}(v,\hat{v})$, where higher score means judged more physically coherent. The machinery translates physical coherence into frame-predictability: a video that cannot be anticipated from its opening frame and prompt is scored as physically incoherent.

What would settle it

Take a set of videos that are all physically correct but differ in rendering style, texture, or object category, run PhyCoPredictor on them, and check whether scores stay flat. If scores vary substantially with appearance, or if human rankings on a fresh prompt set disagree with the model at near-zero Kendall correlation, the claim that the model evaluates physical coherence rather than distribution fit is falsified.

Watch

Extended reading notes

Core claim

The central claim is that physical coherence can be scored by comparing a generated video with the frames and optical flow that PhyCoPredictor predicts from the first frame and prompt. PhyCoPredictor is a cascade of two latent diffusion modules trained on dynamic-scene video data; at evaluation, its predicted flow and video are compared with the generated video's computed flow and frames through the score $1/\mathrm{MSE}(f,\hat{f}) + 2\,\mathrm{MSE}(v,\hat{v})$, where $f,\hat{f}$ are original and predicted optical flow and $v,\hat{v}$ are original and predicted video frames. On the 120-prompt PhyCoBench, model rankings from this score correlate with manual rankings at Kendall's Tau-b 0.3367 and Spearman's 0.3751, while the flow-free DynamiCrafter baseline and the VideoPhy baseline produce negative or near-zero values. The paper concludes that PhyCoPredictor currently aligns most closely with human evaluation among the compared approaches.

Load-bearing premise

The load-bearing premise is that a generated video is physically coherent exactly when its future frames and optical flow are predictable by PhyCoPredictor; if a physically correct video looks or moves differently than the predictor's training distribution, its MSE will be high and it will be wrongly scored as incoherent.

Editorial extensions

If this is right

  • A practical automatic metric now exists for ranking text-to-video models on physical coherence without per-video human labeling.
  • Researchers can use PhyCoBench's 120 prompts and seven physical categories to compare future text-to-video models against a fixed reference set.
  • Optical flow guidance, rather than raw appearance, is what lets the predictor track motion trajectories, suggesting motion-conditioned training improves downstream physical plausibility.
  • If the correlation holds on other model sets, PhyCoPredictor can serve as a cheap signal for iterating on video generators, replacing expensive manual evaluation.
  • The negative Kendall value for the flow-free DynamiCrafter baseline indicates that without such guidance, frame-prediction error alone misranks physical coherence.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to use the PhyCoPredictor score as a reward or ranking loss when training text-to-video models, treating low predictability as a differentiable penalty.
  • The premise that predictability equals physical coherence could be tested directly by holding physical correctness fixed while varying appearance: if scores move with style, the metric is partly measuring distribution fit rather than physics.
  • Because the predictor is trained on human-action and physics-experiment datasets, it may rank videos with unfamiliar objects or stylistic renderings as less coherent even when their trajectories obey physics; evaluating on out-of-distribution prompts would measure this bias.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces PhyCoBench, a benchmark of 120 text prompts spanning seven categories of physically observable motion (gravity, collision, vibration, friction, fluid dynamics, projectile motion, rotation), together with human rankings of videos generated by four text-to-video models. It also proposes PhyCoPredictor, a two-stage latent diffusion evaluator that predicts future optical flow from the first frame and prompt, then uses that flow to predict future video frames. The automatic score in Eq. (4) combines optical-flow MSE and video MSE, and the authors report that the resulting model ranking correlates with human ranking with Kendall's tau-b of 0.3367, outperforming DynamiCrafter and VideoPhy. The paper claims that PhyCoPredictor 'currently aligns most closely with human evaluation' and can effectively evaluate physical coherence.

Significance. If the central claim is established, the paper would make a useful contribution: it provides a compact, category-structured prompt set for physical coherence, releases generated videos and human rankings, and proposes a reference-based automated evaluator that is more transparent than VLM-based scoring. The authors also release the benchmark, predictor, and dataset, which supports reproducibility. However, the current evidence for the central claim is modest: the headline Kendall's tau-b of 0.3367 is a single aggregate number with no uncertainty quantification, and the automated metric's construct validity -- that prediction error in this specific model equals physical incoherence -- is not directly tested. The overlap between the prompt sources and the predictor's training data (UCF101, PennAction, HAA500, Physics101, filtered OpenVid) further raises the possibility that the metric measures distributional typicality rather than physical correctness. The core idea is defensible, but the validation is not yet load-bearing.

major comments (4)
  1. [§3.2 and §B.1 (Manual Evaluation)] The human evaluation procedure is under-specified: the paper does not state how many evaluators participated, whether all 120 prompts were ranked by the same set of evaluators, how disagreements were resolved, or what inter-annotator agreement was. Since every correlation coefficient in Table 2 is computed against this manual ranking, the target variable itself needs reliability evidence. Please report the number of annotators, the aggregation rule for ties, and an agreement measure such as pairwise agreement or Krippendorff's alpha; without this, the ranking standard is unvalidated.
  2. [§4.3, Eq. (4)] The scoring metric contains two ad-hoc free choices: the relative weight of video MSE versus flow MSE (the factor of 2) and the number of sampled frames N. No sensitivity analysis or ablation is provided. Because the flow and video MSEs are in different units (2-channel displacement versus 3-channel pixel values), the summation is not scale-invariant, and the aggregate tau of 0.3367 could be dominated by either term or by the arbitrary weight. Please report per-category Kendall/Spearman correlations, flow-only and video-only scores, and a sweep over the weight (e.g., 0, 0.5, 1, 2, 5) and over N to show that the conclusion is not an artifact of these choices.
  3. [§4.3 and §A.2 (Construct validity of prediction error as physical incoherence)] The paper's premise is that a video is physically coherent exactly when its future frames and optical flow are predictable by PhyCoPredictor. This premise is not tested. The predictor is trained on UCF101, PennAction, HAA500, Physics101, and filtered OpenVid, and the benchmark prompts were themselves created from the same sources, so the metric may reward videos that resemble the training distribution rather than videos that obey physics. A physically correct but out-of-distribution video (for example, fluid dynamics, which is absent from the action datasets) would receive a large MSE and a low score. Please add control experiments: (i) physically plausible videos from categories or visual styles outside the training distribution, (ii) physically implausible but appearance-predictable videos, and (iii) comparison with human labels at the video level rather than only the model-ranking level. These tests would directly address whether the score measures physical coherence rather than predictability.
  4. [Table 2 / §4.4.1] The central quantitative claim rests on a single Kendall's tau-b of 0.3367 with no confidence interval, no significance test, and no comparison of whether the difference from the baseline correlations (-0.2438 and 0.0147) is statistically reliable. With 120 prompts and tied rankings, a permutation or bootstrap test is straightforward. Please report p-values or bootstrap confidence intervals for each correlation and for the difference between methods; otherwise the claim that PhyCoPredictor 'aligns most closely' is not statistically supported.
minor comments (5)
  1. [§4.2.1] The sentence 'the Latent Adapter... downsamples the feature dimension of zf from 2 to 4' appears to be a typo: zf is described as having 4 channels and the optical flow has 2 channels, so the direction of the dimension change should be clarified.
  2. [§4.2.2, Eq. (3)] Please clarify whether the video loss Lvideo is computed in pixel space or latent space, and specify how the predicted video \(v\) is decoded and compared with the original video.
  3. [§4.3, Eq. (4)] The notation MSE(f, \(f\)) and MSE(v, \(v\)) is introduced without defining the sampling over N frames or the dimensionality of the inputs. Please define all symbols and state how N is chosen.
  4. [Figure 2] The proportion of prompts across the seven categories is shown only graphically; please report the exact counts in the caption or text, especially because the categories are not equally sized.
  5. [Throughout] There are several typographical issues (e.g., 'to to train' in §2.1, 'V AE' in §4.2.1, inconsistent spacing in references). A careful proofread would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the automated score is validated against independent human rankings and is not fitted to them.

full rationale

The claimed derivation chain is: (i) physical coherence is defined as motion following real-world physical laws (Sec. 1); (ii) PhyCoPredictor is trained on normal dynamic videos to predict optical flow and future frames (Secs. 4.2 and 4.2.3); (iii) a generated video is scored by the reciprocal of flow/video prediction MSE (Eq. 4); and (iv) the ranking from this score is compared with manual rankings, yielding Kendall's tau-b 0.3367 (Table 2). No step in this chain is circular by construction: the score is not defined in terms of the human labels, the human labels are not used to fit the model or the 2x weight in Eq. 4, and the comparison in Table 2 is an external validation. The overlap between the prompt sources (UCF101, PennAction, HAA500 in Sec. 3.1) and the Motion Data training set (Sec. 4.2.3) is a distributional confound that could inflate agreement and is a validity threat to the physical-coherence interpretation, but it is not a logical reduction of the output to the input. The only self-citation (Open-Sora author Tianyu Li is also a co-author of this paper) is used to identify an evaluated model, not to support a premise. Accordingly, no circular step is exhibited and the circularity score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on two empirical premises: human rankings are a reliable ground truth, and PhyCoPredictor's prediction error is a valid proxy for physical incoherence. The predictor is a learned model, not a physics simulator, so its validity depends on training data coverage. The score weight of 2 is a hand-chosen free parameter, and the unspecified frame count N is another. No new physical entities are postulated.

free parameters (2)
  • Video MSE weight in score = 2
    Eq. 4 weights video MSE by 2 relative to optical flow MSE. The choice is made by hand with no sensitivity analysis; different weights would change model rankings and the reported correlations.
  • Number of sampled frames N
    Section 4.3 samples videos and flows to N frames but never specifies N; the value affects the MSE computation and comparability across models.
assumptions (4)
  • domain assumption Prediction error is a valid proxy for physical incoherence
    The entire evaluation rests on the premise that a video's deviation from the predictor's output indicates physical anomalies, not just appearance or style differences. Invoked in Section 4.3: 'anomalies can be detected by comparing the original optical flow and video with the predicted optical flow and video.'
  • domain assumption Manual rankings are reliable ground truth
    Human evaluators rank physical coherence consistently; no inter-annotator agreement is reported. Invoked in Section 3.2 and Supplement B.1.
  • domain assumption Optical flow computed by FlowFormer++ is accurate enough for comparison
    The reference flow is computed with FlowFormer++; errors in flow estimation propagate into the MSE score. Invoked in Section 4.
  • domain assumption The training datasets cover the physics in the benchmark prompts
    The predictor is trained on UCF101, PennAction, HAA500, Physics101, and filtered OpenVid; if these datasets do not cover fluid dynamics or other prompt categories, the predictor's predictions are unreliable for those prompts. Invoked in Section 4.2.3 and Supplement A.2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Physical Coherence Benchmark for Evaluating Video Generation Models via Optical Flow-guided Frame Prediction." pith.science (2026). https://pith.science/paper/ZQRKDHMP

@misc{pith2026250205503,
  author       = {Pith},
  title        = {Pith review of: A Physical Coherence Benchmark for Evaluating Video Generation Models via Optical Flow-guided Frame Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZQRKDHMP}},
  note         = {Machine review of arXiv:2502.05503}
}
read the original abstract

Recent advances in video generation models demonstrate their potential as world simulators, but they often struggle with videos deviating from physical laws, a key concern overlooked by most text-to-video benchmarks. We introduce a benchmark designed specifically to assess the Physical Coherence of generated videos, PhyCoBench. Our benchmark includes 120 prompts covering 7 categories of physical principles, capturing key physical laws observable in video content. We evaluated four state-of-the-art (SoTA) T2V models on PhyCoBench and conducted manual assessments. Additionally, we propose an automated evaluation model: PhyCoPredictor, a diffusion model that generates optical flow and video frames in a cascade manner. Through a consistency evaluation comparing automated and manual sorting, the experimental results show that PhyCoPredictor currently aligns most closely with human evaluation. Therefore, it can effectively evaluate the physical coherence of videos, providing insights for future model optimization. Our benchmark, including physical coherence prompts, the automatic evaluation tool PhyCoPredictor, and the generated video dataset, has been released on GitHub at https://github.com/Jeckinchen/PhyCoBench.

Figures

Figures reproduced from arXiv: 2502.05503 by the authors.

Figure 1
Figure 1. Overview of Our Benchmark. We propose PhyCoBench—a benchmark specifically designed to evaluate text-to-video (T2V) models in generating physically coherent videos. We categorize common physical scenarios into seven types and create a comprehensive set of prompts. With these prompts, we generate test set videos with four T2V models and conduct human rankings. We also introduce PhyCoPredictor, an optical flow-guided f… view at source ↗
Figure 2
Figure 2. The proportion of text prompts. Our prompts are grouped into seven types. anomalies within the video. However, this task presents two main challenges: (1) anomalies are diverse and complex, making them difficult to define and quantify; (2) existing datasets lack negative samples, hindering the model’s abil￾ity to learn prior knowledge of anomalies. To address these issues, we take inspiration from previous video ano… view at source ↗
Figure 3
Figure 3. Generated video examples of T2V models. The videos generated by these four models do not consistently adhere to physical coherence, with varying levels of quality. and explanations of motion from physics textbooks from a professional standpoint, while also considering common motion scenarios in action recognition datasets such as UCF101[37], PennAction[48], and HAA500[9] from an ev￾eryday perspective. Ultimately, ba… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Overall ranking result from manual evaluation. 0 1 2 3 4 Gravity Collision Vibration Fluid dynamics Friction Projectile motion Rotation Keling1.5 Dream Machine Gen-3 Alpha OpenSora-STDiT-v3 [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Category-specific ranking results from manual eval￾uation. For the videos generated by the four T2V models, we ini￾tially conduct a manual ranking of the four models for each [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 6
Figure 6. Figure 6: Inference process of PhyCoPredictor. Once we obtain the generated video from the T2V model, we input the first frame and the prompt into PhyCoPredictor. The Latent Flow Diffusion Module predicts the future optical flow, which then guides the Latent Video Diffusion Modu…
Figure 7
Figure 7. Figure 7: Training pipeline. Our model training is divided into two stages. In the first stage, we train the 3D U-Net from scratch to predict future optical flow. In the second stage, we use the pre-trained weights from DynamiCrafter and train the model to generate future video …
Figure 8
Figure 8. Figure 8: Visual comparisons of frame prediction results from DynamiCrafter and our PhyCoPredictor. Additionally, PhyCoPredictor can capture the complex mo￾tion of a rolling stone, whereas DynamiCrafter performs poorly in this scenario. 5. Conclusion The rapid development of the…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 29 canonical work pages

  1. [1]

    Keling1.5

    Keling AI. Keling1.5. https://klingai.kuaishou. com/, 2024. 4, 2

    work page 2024

  2. [2]

    Dream machine

    Luma AI. Dream machine. https://lumalabs.ai/ dream-machine, 2024. 4, 2

    work page 2024

  3. [3]

    Videophy: Evaluating physical commonsense for video generation

    Hritik Bansal, Zongyu Lin, Tianyi Xie, Zeshun Zong, Michal Yarom, Yonatan Bitton, Chenfanfu Jiang, Yizhou Sun, Kai- Wei Chang, and Aditya Grover. Videophy: Evaluating physical commonsense for video generation. arXiv preprint arXiv:2406.03520, 2024. 2, 3

    arXiv 2024

  4. [4]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 3

    arXiv 2023

  5. [5]

    Align your latents: High-resolution video synthesis with la- tent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dock- horn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with la- tent diffusion models. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 22563–22575, 2023. 3

    2023

  6. [6]

    Videocrafter1: Open diffusion models for high-quality video generation

    Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, et al. Videocrafter1: Open diffusion models for high-quality video generation. arXiv preprint arXiv:2310.19512, 2023. 1, 3

    arXiv

  7. [7]

    Videocrafter2: Overcoming data limitations for high-quality video diffusion models

    Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models. In CVPR, pages 7310–7320, 2024. 1, 3

    work page 2024

  8. [8]

    Motion-conditioned diffu- sion model for controllable video synthesis

    Tsai-Shien Chen, Chieh Hubert Lin, Hung-Yu Tseng, Tsung- Yi Lin, and Ming-Hsuan Yang. Motion-conditioned diffu- sion model for controllable video synthesis. arXiv preprint arXiv:2304.14404, 2023. 3

    arXiv 2023

Show all 52 references
  1. [9]

    Haa500: Human-centric atomic action dataset with curated videos

    Jihoon Chung, Cheng-hsin Wuu, Hsuan-ru Yang, Yu-Wing Tai, and Chi-Keung Tang. Haa500: Human-centric atomic action dataset with curated videos. In ICCV, pages 13465– 13474, 2021. 4, 6, 1

    2021

  2. [10]

    Flownet: Learning optical flow with convolutional networks

    Alexey Dosovitskiy, Philipp Fischer, Eddy Ilg, Philip Hausser, Caner Hazirbas, Vladimir Golkov, Patrick Van Der Smagt, Daniel Cremers, and Thomas Brox. Flownet: Learning optical flow with convolutional networks. InICCV, pages 2758–2766, 2015. 3

    2015

  3. [11]

    Generative adversarial nets

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014. 3

    2014

  4. [12]

    Seer: Language instructed video prediction with latent diffusion models

    Xianfan Gu, Chuan Wen, Weirui Ye, Jiaming Song, and Yang Gao. Seer: Language instructed video prediction with latent diffusion models. In ICLR, 2023. 3

    2023

  5. [13]

    Animatediff: Animate your personalized text-to- image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to- image diffusion models without specific tuning. ICLR, 2024. 1, 3

    2024

  6. [14]

    Flexible diffusion modeling of long videos

    William Harvey, Saeid Naderiparizi, Vaden Masrani, Chris- tian Weilbach, and Frank Wood. Flexible diffusion modeling of long videos. Advances in Neural Information Processing Systems, 35:27953–27965, 2022. 3

    2022

  7. [15]

    Latent video diffusion models for high-fidelity long video generation

    Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for high-fidelity long video generation. arXiv preprint arXiv:2211.13221 ,

    arXiv

  8. [16]

    CLIPScore: A reference-free evaluation metric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. CLIPScore: A reference-free evaluation metric for image captioning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Pro- cessing, pages 7514–7528, Online and Punta Cana, Do...

    2021

  9. [17]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in neural information processing systems , 30, 2017. 3

    2017

  10. [18]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. NeurIPS, 33:6840–6851, 2020. 5

    2020

  11. [19]

    Imagen video: High definition video generation with diffusion mod- els

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion mod- els. arXiv preprint arXiv:2210.02303, 2022. 3

    arXiv

  12. [20]

    Video dif- fusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video dif- fusion models. NeurIPS, 35:8633–8646, 2022. 3

    2022

  13. [21]

    Vbench: Comprehensive bench- mark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive bench- mark suite for video generative models. In CVPR, pages 21807–21818, 2024. 2, 3

    2024

  14. [22]

    T2vbench: Benchmarking temporal dynamics for text-to- video generation

    Pengliang Ji, Chuyang Xiao, Huilin Tai, and Mingxiao Huo. T2vbench: Benchmarking temporal dynamics for text-to- video generation. In CVPR, pages 5325–5335, 2024. 2, 3

    2024

  15. [23]

    Text2performer: Text- driven human video generation

    Yuming Jiang, Shuai Yang, Tong Liang Koh, Wayne Wu, Chen Change Loy, and Ziwei Liu. Text2performer: Text- driven human video generation. In ICCV, pages 22747– 22757, 2023. 3

    2023

  16. [24]

    Text2video-zero: Text- to-image diffusion models are zero-shot video generators

    Levon Khachatryan, Andranik Movsisyan, Vahram Tade- vosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Text2video-zero: Text- to-image diffusion models are zero-shot video generators. In ICCV, pages 15954–15964, 2023. 1, 3

    2023

  17. [25]

    Fu- ture frame prediction for anomaly detection–a new baseline

    Wen Liu, Weixin Luo, Dongze Lian, and Shenghua Gao. Fu- ture frame prediction for anomaly detection–a new baseline. In CVPR, pages 6536–6545, 2018. 2, 3

    2018

  18. [26]

    Evalcrafter: Benchmarking and evalu- ating large video generation models

    Yaofang Liu, Xiaodong Cun, Xuebo Liu, Xintao Wang, Yong Zhang, Haoxin Chen, Yang Liu, Tieyong Zeng, Raymond Chan, and Ying Shan. Evalcrafter: Benchmarking and evalu- ating large video generation models. InCVPR, pages 22139– 22149, 2024. 2, 3

    2024

  19. [27]

    Sora: A review on background, technology, limitations, and opportunities of large vision models

    Yixin Liu, Kai Zhang, Yuan Li, Zhiling Yan, Chujie Gao, Ruoxi Chen, Zhengqing Yuan, Yue Huang, Hanchi Sun, Jian- feng Gao, et al. Sora: A review on background, technology, limitations, and opportunities of large vision models. arXiv preprint arXiv:2402.17177, 2024. 3

    2024 arXiv

  20. [28]

    A hybrid video anomaly detection framework via memory-augmented flow reconstruction and flow-guided frame prediction

    Zhian Liu, Yongwei Nie, Chengjiang Long, Qing Zhang, and Guiqing Li. A hybrid video anomaly detection framework via memory-augmented flow reconstruction and flow-guided frame prediction. In ICCV, pages 13588–13597, 2021. 2, 3

    2021

  21. [29]

    Openvid-1m: A large-scale high-quality dataset for text-to- video generation

    Kepan Nan, Rui Xie, Penghao Zhou, Tiehan Fan, Zhen- heng Yang, Zhijie Chen, Xiang Li, Jian Yang, and Ying Tai. Openvid-1m: A large-scale high-quality dataset for text-to- video generation. arXiv preprint arXiv:2407.02371, 2024. 6, 1

    2024 arXiv

  22. [30]

    Anomaly detec- tion in video sequence with appearance-motion correspon- dence

    Trong-Nguyen Nguyen and Jean Meunier. Anomaly detec- tion in video sequence with appearance-motion correspon- dence. In ICCV, pages 1273–1283, 2019. 2, 3

    2019

  23. [31]

    Conditional image-to-video gener- ation with latent flow diffusion models

    Haomiao Ni, Changhao Shi, Kai Li, Sharon X Huang, and Martin Renqiang Min. Conditional image-to-video gener- ation with latent flow diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18444–18455, 2023. 5

    2023

  24. [32]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, pages 10684– 10695, 2022. 5

    2022

  25. [33]

    Gen-3 alpha

    runway. Gen-3 alpha. https://runwayml.com/ research/introducing- gen- 3- alpha, 2024. 4, 2

    2024

  26. [34]

    Improved techniques for training gans

    Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016. 3

    2016

  27. [35]

    Flowformer++: Masked cost volume autoen- coding for pretraining optical flow estimation

    Xiaoyu Shi, Zhaoyang Huang, Dasong Li, Manyuan Zhang, Ka Chun Cheung, Simon See, Hongwei Qin, Jifeng Dai, and Hongsheng Li. Flowformer++: Masked cost volume autoen- coding for pretraining optical flow estimation. In Proceed- ings of the IEEE/CVF conference on computer vision a...

    2023

  28. [36]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792 ,

    arXiv

  29. [37]

    Ucf101: A dataset of 101 human actions classes from videos in the wild

    K Soomro. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402 ,

    arXiv

  30. [38]

    Fvd: A new metric for video generation

    Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Rapha¨el Marinier, Marcin Michalski, and Sylvain Gelly. Fvd: A new metric for video generation. 2019. 3

    2019

  31. [39]

    Modelscope text-to-video technical report

    Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. Modelscope text-to-video technical report. arXiv preprint arXiv:2308.06571, 2023. 3

    2023 arXiv

  32. [40]

    Image quality assessment: from error visibility to structural similarity

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Si- moncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004. 3

    2004

  33. [41]

    Physics 101: Learning phys- ical object properties from unlabeled videos

    Jiajun Wu, Joseph J Lim, Hongyi Zhang, Joshua B Tenen- baum, and William T Freeman. Physics 101: Learning phys- ical object properties from unlabeled videos. In British Ma- chine Vision Conference, 2016. 6

    2016

  34. [42]

    Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation

    Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In ICCV, pages 7623–7633, 2023. 1, 3

    2023

  35. [43]

    Dynamicrafter: Animating open-domain images with video diffusion priors

    Jinbo Xing, Menghan Xia, Yong Zhang, Haoxin Chen, Wangbo Yu, Hanyuan Liu, Gongye Liu, Xintao Wang, Ying Shan, and Tien-Tsin Wong. Dynamicrafter: Animating open-domain images with video diffusion priors. In ECCV, pages 399–417. Springer, 2025. 1, 3, 5, 6, 2

    2025

  36. [44]

    Aid: Adapting image2video diffusion mod- els for instruction-guided video prediction

    Zhen Xing, Qi Dai, Zejia Weng, Zuxuan Wu, and Yu- Gang Jiang. Aid: Adapting image2video diffusion mod- els for instruction-guided video prediction. arXiv preprint arXiv:2406.06465, 2024. 3

    2024 arXiv

  37. [45]

    Qwen2 technical report.arXiv preprint arXiv:2407.10671, 2024

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. Qwen2 technical report.arXiv preprint arXiv:2407.10671, 2024. 2, 6, 1

    2024 arXiv

  38. [46]

    Video event restoration based on keyframes for video anomaly detection

    Zhiwei Yang, Jing Liu, Zhaoyang Wu, Peng Wu, and Xiaotao Liu. Video event restoration based on keyframes for video anomaly detection. In CVPR, pages 14592–14601, 2023. 2, 3

    2023

  39. [47]

    Old is gold: Redefining the adversari- ally learned one-class classifier training paradigm

    Muhammad Zaigham Zaheer, Jin-ha Lee, Marcella Astrid, and Seung-Ik Lee. Old is gold: Redefining the adversari- ally learned one-class classifier training paradigm. In CVPR, pages 14183–14193, 2020. 2, 3

    2020

  40. [48]

    From actemes to action: A strongly-supervised repre- sentation for detailed action understanding

    Weiyu Zhang, Menglong Zhu, and Konstantinos G Derpa- nis. From actemes to action: A strongly-supervised repre- sentation for detailed action understanding. In ICCV, pages 2248–2255, 2013. 4, 6, 1

    2013

  41. [49]

    3d object manipulation in a single image using generative models

    Ruisi Zhao, Zechuan Zhang, Zongxin Yang, and Yi Yang. 3d object manipulation in a single image using generative models. arXiv preprint arXiv:2501.12935, 2025. 3

    2025 arXiv

  42. [50]

    Open-sora: Democratizing efficient video production for all, 2024

    Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all, 2024. 4, 2

    2024

  43. [51]

    The output of the MLM

    Daquan Zhou, Weimin Wang, Hanshu Yan, Weiwei Lv, Yizhe Zhu, and Jiashi Feng. Magicvideo: Efficient video generation with latent diffusion models. arXiv preprint arXiv:2211.11018, 2022. 3 A Physical Coherence Benchmark for Evaluating Video Generation Models via Optical Flow-gui...

    2022 arXiv

  44. [52]

    2 > 1 > 3 > 4,

    Retain information about the movement of objects and people. 2. Re- move descriptions of people’s clothing. 3. Remove descriptions of the environment and background. Table 4. The prompt for generating video captions. Dataset Filtered Action Categories UCF101 Archery, Baseball ...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.