Pith. sign in

REVIEW 4 major objections 6 minor 67 references

Conditioning an image-to-video diffusion model on explicit per-object physical parameters—force vectors, material properties, and gravity—produces videos with measurably better rigid-body physical consistency while preserving visual fidelit

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 13:56 UTC pith:ZMHLTUTZ

load-bearing objection The paper makes a real contribution to explicit instance-level physical control for video generation, but its headline evidence is a closed loop with its own simulator and needs an external anchor before the numbers mean physical correctness. the 4 major comments →

arxiv 2607.18924 v1 pith:ZMHLTUTZ submitted 2026-07-21 cs.CV

Learning Explicit Physical Parameter Control and Benchmarking for Video Generation

Yanxun Li , Hao Wen , Bingze Song , Jiashu Zhu , Aiming Hao , Chubin Chen , Jintao Chen , Jiahong Wu
show 2 more authors
Xiangxiang Chu Miao Wang
This is my paper
classification cs.CV
keywords video generationphysical parameter controlphysics consistencyimage-to-video diffusionrigid-body dynamicsphysics simulation datasetbenchmarkattention routing
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that the missing ingredient for physically grounded video generation is not more data or bigger models but explicit, fine-grained physical parameterization bound to each object. It builds a 130,000-video simulated dataset annotated with forces, masses, friction, restitution, and gravity; a model that routes these parameters to specific object regions through attention; and a benchmark that separates temporal dynamics, spatial stability, and semantic–physical alignment. On that benchmark, the model achieves the best physical-consistency scores, and human raters prefer its outputs, while general visual quality is maintained. If correct, this means users can specify physical quantities—push an object with a chosen force, set gravity to a chosen value—and the generator will produce corresponding motion, a step toward controllable world simulation.

Core claim

PhyParam establishes that instance-level rigid-body physical parameters—external force vectors, mass, friction, restitution, and scene-level gravity—can be injected into a 5-billion-parameter image-to-video diffusion model as tokens, routed through attention so each object attends only to its own attributes, and that this explicit binding shifts generated motion toward the specified physics. Trained on 130K simulated videos covering free fall, collisions, inclined-plane slides, and externally driven motion, the model also supervises an auxiliary conditioning branch's residual features and their temporal differences in a frozen DINOv3 feature space, aligning object identity, boundaries, and d

What carries the argument

Physical Parameter Attention is a lightweight token-routing mechanism: it serializes all per-object and global physical conditions into a single token sequence in an auxiliary conditioning branch, lets each object's mask tokens attend to that object's own physical-attribute tokens in cross-attention, and lets global physics tokens (e.g., gravity) attend to the noise latents. This prevents cross-instance condition leakage while keeping the overhead small. A complementary supervision mechanism uses a frozen self-supervised vision transformer (DINOv3) to align the branch's multi-scale residual features and their first-order temporal differences with reference simulation in semantic-structural s

Load-bearing premise

The paper's core evidence assumes that matching its own synthetic simulation reference videos is a valid proxy for physical correctness; if that assumption fails, the benchmark numbers do not establish real-world physical consistency.

What would settle it

Evaluate the trained model on a held-out set of real-world videos with measured masses and surface properties, vary the specified force magnitude, and check whether the generated object's acceleration scales proportionally as Newton's laws require; if the induced motion does not track the specified parameters quantitatively, the central claim of explicit physical control is falsified.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Users can explicitly set per-object forces, material properties, and gravity, and the model responds with corresponding rigid-body motion, enabling controllable generation for animation, VFX, and previsualization.
  • The dataset–model–benchmark loop offers a standard protocol for evaluating physical-law consistency in video generation, separating temporal dynamics, spatial stability, and semantic–physical alignment.
  • The model transfers to real images with manually annotated masks, suggesting explicit physical parameter conditioning can work beyond the synthetic training distribution.
  • Ablations show Physical Parameter Attention is critical: removing it degrades FVMD from 355.5 to 473.4 and IoU from 17.8 to 11.9, confirming that binding instance parameters to objects is a key factor rather than a mere addition of conditioning.
  • Feature-space supervision contributes most of the motion-consistency gain (FVMD 801.9 → 355.5), implying that temporal feature alignment is a powerful tool for physics learning in diffusion models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's synthetic-to-real transfer is demonstrated qualitatively on a few examples; a quantitative evaluation on real-world videos with tracked trajectories would be needed to confirm that explicit parameter control generalizes beyond the synthetic simulator.
  • The benchmark's reference videos come from the same simulation pipeline as its training data, so the headline numbers partly measure self-consistency with that simulator; an independent physics benchmark built from real experiments would test whether 'physical consistency' here equals real-world correctness.
  • The same mask-attention routing could extend to other per-object attributes (initial velocity, torque, deformable parameters) and to text-to-video or 3D generation, since it decouples instance binding from the base model's attention.
  • The reported correlation between benchmark metrics and human judgment (ρ=0.80) is computed across only four models; a larger model set would be needed to trust the benchmark as a proxy for human-perceived physical consistency.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a closed pipeline for explicit rigid-body physical-parameter control in image-to-video generation. It introduces PhyParam-Dataset, 130K Blender-rendered videos annotated with object-level forces, material properties, and scene-level gravity; PhyParam, a physics-conditioned image-to-video diffusion model built on Wan2.2-TI2V-5B; and PhyParam-Bench, a 380-video benchmark with temporal, spatial, and semantic-physical metrics. The model encodes physical parameters via harmonic embeddings, uses a physical cross-attention routing mechanism to bind parameters to instance masks, and adds DINO feature-space and temporal-difference losses. Experiments report improvements over general baselines on PhyParam-Bench (e.g., FVMD 355.5 vs. 636.9 for HunyuanI2V, IoU 17.8 vs. 15.2), competitive or better results against Force Prompting, positive human evaluation, and preserved VBench quality. The dataset, benchmark, and code are promised for release.

Significance. If the central claim is accepted, the paper offers a useful large-scale resource and a lightweight architecture for instance-level physical control in video generation. The proposed mask-token routing and DINO-based feature supervision are technically interesting, and the ablations (Tables 5–6) support the contribution of each component. The commitment to release data, benchmark, and code is a strength that will benefit the community. However, the quantitative evidence for physical-law consistency is not yet independently anchored: the benchmark shares its generation pipeline with the training data, no statistical uncertainty is reported, and the external validation (ρ=0.80 with human judgment) rests on only four models. The paper's contribution should therefore be evaluated as a promising system plus dataset, pending stronger external validation.

major comments (4)
  1. [§3.2, §4.4, Table 3] The central evidence for physical consistency is obtained in a closed loop. PhyParam-Bench's 380 reference videos are generated by the same Blender pipeline (Algorithms 1–2 in S1) that produced the 130K training videos, and the training losses in Eqs. (5)–(6) explicitly align ControlNet features and their temporal differences to reference simulations. FVMD and IoU therefore measure agreement with the authors' own simulator distribution, and the paper does not report train/test disjointness in object IDs, scenes, materials, or physical parameter ranges. The reported Spearman ρ=0.80 with human judgment is computed across only 4 models, which is far too few to validate the benchmark. This does not disprove the method, but the headline margins over general baselines could reflect in-distribution replication rather than physical-law consistency. Please report disjointness statistics, add exte
  2. [§5.1, Tables 3–6] All quantitative results come from a single 5,000-step run with no random seeds, confidence intervals, or significance tests. Some margins are large (e.g., FVMD 355.5 vs. 636.9), but others are small (e.g., IoU 16.2 vs. 17.8 in Table 5, OC 57.7 vs. 58.1), and without repeated runs it is impossible to assess stability. This is load-bearing for the central claim because the main contribution is measured through these comparisons. Please provide multiple seeds, error bars, and significance tests for the key comparisons in Tables 3, 5, and 6.
  3. [§5.2, Table 3] The comparison with Force Prompting is not accurately summarized. Ours-Force is worse than Force Prompting on FVMD (586.8 vs. 455.3), OC (54.4 vs. 74.3), and SC (96.3 vs. 97.1), and better only on IoU (8.4 vs. 6.2) and VLM-QA (74.4 vs. 74.0). Calling this 'competitive' is defensible, but the text goes on to imply improved physical consistency from the Physical Parameter Attention design. The claim should be qualified, or an analysis should explain why the model loses on temporal and object-consistency metrics while winning on region-level IoU.
  4. [S1, Algorithm 2] For collision scenes, object positions are keyframed in frames 1–10 to 'emulate an initial external impulse', so the labeled force is an effective quantity inferred from keyframe displacement rather than a force actually applied to the body in the simulation. Since force is a core conditioning variable and the model is trained to map these labels to motion, this caveat should be stated in the main text. Please also provide a sensitivity or validation experiment showing that effective-force labels behave like true impulse forces (e.g., varying the keyframe displacement and verifying the generated motion changes accordingly).
minor comments (6)
  1. [§5.1, Table 3] Define Ours-All and Ours-Force explicitly. It is confusing that Ours-All is compared with general models while only Ours-Force is compared with Force Prompting.
  2. [Table 7] Ours-All shows a large drop in VBench Dynamic Degree (47.37 vs. Wan2.2's 92.11), yet the text states that visual quality is preserved. Please discuss whether lower dynamic degree is a deliberate effect of physical stabilization and how it relates to physical consistency.
  3. [S1, References] References [9] and [10] are the same paper (Visual Vibrometry). Please deduplicate.
  4. [Figure 2] The label 'Human-LLM Physical Parameters' is ambiguous. Clarify how LLM-predicted parameters relate to the simulator's ground-truth physical parameters.
  5. [§3.3, S3] The human evaluation description says 10 annotators participate but later says 'Four annotators with high-school physics backgrounds who pass a qualification test perform the final evaluation.' Clarify the roles of the 10 versus 4 annotators.
  6. [S2, VLM-QA] The VLM-QA rubric includes 15 criteria, many of which target appearance artifacts (ghosting, flicker, reflections). As reported, VLM-QA may not isolate physical-law consistency from general visual quality; consider reporting the physics-related subset separately.

Circularity Check

1 steps flagged

Closed-loop Blender train/benchmark: FVMD/IoU largely measure reproduction of the authors' own simulator; ρ=0.80 over 4 models does not anchor it externally.

specific steps
  1. self definitional [Sections 1, 3.1-3.2, 4.4 (Eqs. 5-6), Table 3]
    "Together, we present a closed-loop pipeline for explicit instance-level physical control in I2V generation: a parameterized dataset, a physics-guided generator, and a dedicated benchmark. ... we construct PhyParam-Dataset through a scalable and reproducible pipeline based on the Blender physics engine. ... PhyParam-Bench contains 380 test videos across diverse physical motions ... For each PhyParam-Bench instance, we provide a fixed condition package (caption, physical parameters, and first-frame instance masks exported from Blender)."

    Physical-law consistency is operationalized by FVMD/IoU against Blender reference videos; those references come from the same Blender pipeline that generated the 130K training videos. The training objective explicitly minimizes DINO cosine distance and temporal-difference cosine distance to those reference simulations (Eqs. 5-6), and at test time the model receives the exact Blender-exported parameters and masks. Thus the headline FVMD/IoU improvements are largely a measure of in-distribution replication of the authors' own simulator, not an external check that the dynamics obey physics. No disjointness of test items from training items in object IDs, scenes, materials, or parameter values is reported.

full rationale

The central quantitative claim—that explicit physical-parameter conditioning improves physical consistency—is supported mainly by PhyParam-Bench, whose 380 references are produced by the same Blender pipeline as the 130K training videos. Training explicitly aligns ControlNet outputs to those simulations via DINO feature and temporal-difference losses (Eqs. 5-6), while evaluation compares generated keypoint motion (FVMD) and masks (IoU) to the same kind of references. This makes the benchmark partially self-referential: the model is trained to imitate the authors' simulator and then scored on how well it imitates that same simulator. The reported Spearman ρ=0.80 is computed over only four models, so it does not provide an external anchor, and the human study also samples from the same benchmark. There is no load-bearing self-citation chain, and the architectural components (Physical Parameter Attention, DINO supervision) have independent content; VBench and qualitative real-image tests provide some external signal. However, the paper's own 'closed-loop' framing plus the shared Blender source for training and evaluation means the headline physical-consistency margins are substantially a measure of in-distribution reproduction rather than independent physical-law verification.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

No new physical entities (particles, forces, fields, dimensions) are postulated; the 'Physical Parameter Attention' is an architectural routing mechanism, not an entity. The free parameters are the unreported loss weights and encoding bandwidth, plus hand-chosen simulation sampling ranges that define the whole operating regime. The axioms are dominated by the assumption that Blender simulation output is a faithful proxy for real rigid-body physics — which is simultaneously the dataset's foundation and the benchmark's ground truth.

free parameters (3)
  • lambda_1, lambda_2 (feature and temporal loss weights) = not reported
    Equation (7) in Section 4.4 balances the DINO feature alignment and temporal consistency losses, but the values of lambda_1 and lambda_2 are never stated, so the ablations in Table 5 cannot be exactly reproduced.
  • Harmonic embedding frequency bands L = not reported
    Equation (4) defines gamma(x) with L frequency bands; L is unspecified, and it controls the precision with which scalar physical values (force magnitude, friction, restitution) are encoded into tokens.
  • Simulation parameter sampling ranges = gravity, restitution, friction, mass, density, force ranges sampled; values not enumerated
    The dataset's parameter distributions (Algorithm 2 in S1) define the training distribution and therefore the benchmark's operating regime; the ranges are hand-chosen and not reported, making the 'long-tail' coverage claim untestable.
axioms (5)
  • domain assumption Blender's rigid-body engine produces physically accurate dynamics and the rendered videos faithfully represent those dynamics
    The entire dataset, all supervision signals, and the benchmark ground truth derive from Blender simulations (Section 3.1); the only validation is a self-declared 98% internal consistency check, with no comparison against real measurements.
  • ad hoc to paper For collision scenes, keyframing object positions in frames 1-10 'emulates an initial external impulse' — the labeled force is an effective quantity, not a force actually applied to the body
    Algorithm 2 (S1) prescribes positions at frames 1 and 10, then runs Blender physics from frame 10; force labels for collision scenarios are thus inferred from prescribed kinematics, weakening force-label fidelity for roughly a third of the data.
  • domain assumption Cosine similarity in DINOv3 feature space (single-frame and first-order temporal differences) is a valid supervision signal that improves physical compliance without distorting appearance
    Section 4.4 justifies DINO via temporal stability but provides no evidence that feature-space alignment correlates with physical correctness; the improvement is purely empirical (Table 5).
  • domain assumption A vision-language model's binary answers to 15 checklist questions are a valid proxy for physical plausibility (VLM-QA)
    Section 3.2 and Figure 11 give the full Qwen3-VL prompt, but the metric is only calibrated against human judgment via the aggregate rho=0.80 computed on 4 models — insufficient to establish validity.
  • domain assumption Flow-matching diffusion trained in Wan2.2's VAE latent space can carry explicit scalar physical conditioning into rendered dynamics
    The base framework is assumed (Sections 4.1-4.3); whether injected tokens actually steer the generated motion is exactly the empirical question the paper addresses, so this is a load-bearing framework assumption.

pith-pipeline@v1.3.0-alltime-deepseek · 17758 in / 24101 out tokens · 197037 ms · 2026-08-01T13:56:41.790933+00:00 · methodology

0 comments
read the original abstract

Recent advances in image-to-video generation have improved visual realism, making physically grounded and controllable dynamics an important step toward future world simulation. Current models often generate plausible motion, but it is not reliably governed by explicit physical causes, and instance-level constraints can leak or become entangled in multi-object interactions. We attribute this gap to two missing pieces: large-scale, fine-grained physical parameterization, and model designs that correctly bind physical attributes to instances and emphasize dynamics over appearance. To bridge this gap, we introduce PhyParam-Dataset, an interaction-centric collection of 130K physically simulated videos with dense physical parameterization, including force vectors, object material properties, and environmental constants across five representative rigid-body motion types. Built on this data, we present PhyParam, a physics-guided image-to-video diffusion model that conditions on object-level forces, masses, friction, restitution, and scene-level gravity via a lightweight physical-attention routing mechanism, and further improves motion learning with semantic-structural feature-space supervision. We also establish PhyParam-Bench, a benchmark for physical-law consistency in image-to-video generation, with a multi-level protocol evaluating temporal dynamics, spatial stability, and semantic--physical alignment. Experiments show that PhyParam improves physical consistency while maintaining high visual fidelity, advancing explicit rigid-body physical-parameter control for image-to-video generation. We will publicly release the dataset, benchmark, and code to support future research.

Figures

Figures reproduced from arXiv: 2607.18924 by Aiming Hao, Bingze Song, Chubin Chen, Hao Wen, Jiahong Wu, Jiashu Zhu, Jintao Chen, Miao Wang, Xiangxiang Chu, Yanxun Li.

Figure 1
Figure 1. Figure 1: PhyParam, trained on the PhyParam-Dataset, enables precise and fine-grained control over [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the PhyParam-Dataset generation pipeline. Our pipeline constructs physics [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of the proposed physics-aware conditioning and attention. Left: [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative Comparison. We compare our results with Wan2.2-TI2V-5B [ [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Compared with VJEPA, DINO features are temporally more stable, making them better [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Additional qualitative validation. Top: real-image inputs with manually annotated masks [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative I2V results on real-world-like videos covering diverse motion patterns. (Arrows [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Distribution of CC-textures and PolyHaven environment maps between indoor and outdoor [PITH_FULL_IMAGE:figures/full_fig_p017_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Failure cases. The interface used for user annotations is shown in [PITH_FULL_IMAGE:figures/full_fig_p021_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: The interface used for user annotations in our study. [PITH_FULL_IMAGE:figures/full_fig_p022_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: An overview of the VLM-QA pipeline. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_11.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

67 extracted references · 27 linked inside Pith

  1. [1]

    Interdyn: Controllable interactive dynamics with video diffusion models

    Rick Akkerman, Haiwen Feng, Michael J Black, Dimitrios Tzionas, and Victoria Fernández Abrevaya. Interdyn: Controllable interactive dynamics with video diffusion models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 12467–12479, 2025

    2025

  2. [2]

    Recammaster: Camera-controlled generative rendering from a single video.arXiv preprint arXiv:2503.11647, 2025

    Jianhong Bai, Menghan Xia, Xiao Fu, Xintao Wang, Lianrui Mu, Jinwen Cao, Zuozhu Liu, Haoji Hu, Xiang Bai, Pengfei Wan, et al. Recammaster: Camera-controlled generative rendering from a single video.arXiv preprint arXiv:2503.11647, 2025

    Pith/arXiv arXiv 2025

  3. [3]

    Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond.arXiv preprint arXiv:2308.12966, 2023

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond.arXiv preprint arXiv:2308.12966, 2023

    Pith/arXiv arXiv 2023

  4. [4]

    Qwen2.5-vl technical report

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. a...

    Pith/arXiv arXiv 2025

  5. [5]

    Videophy: Evaluating physical commonsense for video generation.arXiv preprint arXiv:2406.03520, 2024

    Hritik Bansal, Zongyu Lin, Tianyi Xie, Zeshun Zong, Michal Yarom, Yonatan Bitton, Chenfanfu Jiang, Yizhou Sun, Kai-Wei Chang, and Aditya Grover. Videophy: Evaluating physical commonsense for video generation.arXiv preprint arXiv:2406.03520, 2024

    Pith/arXiv arXiv 2024

  6. [6]

    Videophy-2: A challenging action-centric physical commonsense evaluation in video generation.arXiv preprint arXiv:2503.06800, 2025

    Hritik Bansal, Clark Peng, Yonatan Bitton, Roman Goldenberg, Aditya Grover, and Kai-Wei Chang. Videophy-2: A challenging action-centric physical commonsense evaluation in video generation.arXiv preprint arXiv:2503.06800, 2025

    Pith/arXiv arXiv 2025

  7. [7]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, et al. Video generation models as world simulators. OpenAI Blog, 1(8):1, 2024. 12

    2024

  8. [8]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. InProceedings of the International Conference on Computer Vision (ICCV), 2021

    2021

  9. [10]

    Visual vibrometry: Estimating material properties from small motion in video

    Abe Davis, Katherine L Bouman, Justin G Chen, Michael Rubinstein, Fredo Durand, and William T Freeman. Visual vibrometry: Estimating material properties from small motion in video. InProceedings of the ieee conference on computer vision and pattern recognition, pages 5335–5343, 2015

    2015

  10. [11]

    Objaverse-xl: A universe of 10m+ 3d objects.Advances in Neural Information Processing Systems, 36:35799–35813, 2023

    Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram V oleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects.Advances in Neural Information Processing Systems, 36:35799–35813, 2023

    2023

  11. [12]

    Blenderproc2: A procedural pipeline for photorealistic rendering.Journal of Open Source Software, 8(82): 4901, 2023

    Maximilian Denninger, Dominik Winkelbauer, Martin Sundermeyer, Wout Boerdijk, Markus Wendelin Knauer, Klaus H Strobl, Matthias Humt, and Rudolph Triebel. Blenderproc2: A procedural pipeline for photorealistic rendering.Journal of Open Source Software, 8(82): 4901, 2023

    2023

  12. [13]

    Mirrorverse: Pushing diffusion models to realistically reflect the world

    Ankit Dhiman, Manan Shah, and R Venkatesh Babu. Mirrorverse: Pushing diffusion models to realistically reflect the world. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 11239–11249, 2025

    2025

  13. [14]

    Scaling rectified flow trans- formers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow trans- formers for high-resolution image synthesis. InForty-first international conference on machine learning, 2024

    2024

  14. [15]

    Narrlv: Towards a comprehensive narrative-centric evaluation for long video generation models.arXiv preprint arXiv:2507.11245, 2025

    Xiaokun Feng, Haiming Yu, Meiqi Wu, Shiyu Hu, Jintao Chen, Chen Zhu, Jiahong Wu, Xi- angxiang Chu, and Kaiqi Huang. Narrlv: Towards a comprehensive narrative-centric evaluation for long video generation models.arXiv preprint arXiv:2507.11245, 2025

    arXiv 2025

  15. [16]

    Force prompting: Video generation models can learn and generalize physics-based control signals.arXiv preprint arXiv:2505.19386, 2025

    Nate Gillman, Charles Herrmann, Michael Freeman, Daksh Aggarwal, Evan Luo, Deqing Sun, and Chen Sun. Force prompting: Video generation models can learn and generalize physics-based control signals.arXiv preprint arXiv:2505.19386, 2025

    arXiv 2025

  16. [17]

    the public 3d asset library.Avaliable online: https://polyhaven

    Poly Haven. the public 3d asset library.Avaliable online: https://polyhaven. com/hdris (accessed on 26 May 2025), 2025

    2025

  17. [18]

    Videoscore: Building automatic metrics to simulate fine-grained human feedback for video generation

    Xuan He, Dongfu Jiang, Ge Zhang, Max Ku, Achint Soni, Sherman Siu, Haonan Chen, Abhranil Chandra, Ziyan Jiang, Aaran Arulraj, et al. Videoscore: Building automatic metrics to simulate fine-grained human feedback for video generation. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 2105–2123, 2024

    2024

  18. [19]

    Cogvideo: Large-scale pretraining for text-to-video generation via transformers.arXiv preprint arXiv:2205.15868, 2022

    Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers.arXiv preprint arXiv:2205.15868, 2022

    Pith/arXiv arXiv 2022

  19. [20]

    Taming hallucinations: Boosting mllms’ video understanding via counterfactual video generation

    Zhe Huang, Hao Wen, Aiming Hao, Bingze Song, Meiqi Wu, Jiahong Wu, Xiangxiang Chu, Sheng Lu, and Haoqian Wang. Taming hallucinations: Boosting mllms’ video understanding via counterfactual video generation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8153–8163, 2026

    2026

  20. [21]

    Vbench: Comprehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21807–21818, 2024. 13

    2024

  21. [22]

    Vbench++: Comprehensive and versatile benchmark suite for video generative models.arXiv preprint arXiv:2411.13503, 2024

    Ziqi Huang, Fan Zhang, Xiaojie Xu, Yinan He, Jiashuo Yu, Ziyue Dong, Qianli Ma, Nattapol Chanpaisit, Chenyang Si, Yuming Jiang, et al. Vbench++: Comprehensive and versatile benchmark suite for video generative models.arXiv preprint arXiv:2411.13503, 2024

    Pith/arXiv arXiv 2024

  22. [23]

    3d gaussian splatting for real-time radiance field rendering.ACM Trans

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George Drettakis, et al. 3d gaussian splatting for real-time radiance field rendering.ACM Trans. Graph., 42(4):139–1, 2023

    2023

  23. [24]

    Auto-encoding variational bayes.arXiv preprint arXiv:1312.6114, 2013

    Diederik P Kingma and Max Welling. Auto-encoding variational bayes.arXiv preprint arXiv:1312.6114, 2013

    Pith/arXiv arXiv 2013

  24. [25]

    Segment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4015–4026, 2023

    2023

  25. [26]

    Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024

    Pith/arXiv arXiv 2024

  26. [27]

    Pisa experiments: Exploring physics post-training for video diffusion models by watching stuff drop

    Chenyu Li, Oscar Michel, Xichen Pan, Sainan Liu, Mike Roberts, and Saining Xie. Pisa experiments: Exploring physics post-training for video diffusion models by watching stuff drop. arXiv preprint arXiv:2503.09595, 2025

    Pith/arXiv arXiv 2025

  27. [28]

    C-drag: Chain-of-thought driven motion controller for video generation

    Yuhao Li, Mirana Claire Angel, Salman Khan, Yu Zhu, Jinqiu Sun, Yanning Zhang, and Fahad Shahbaz Khan. C-drag: Chain-of-thought driven motion controller for video generation. arXiv preprint arXiv:2502.19868, 2025

    Pith/arXiv arXiv 2025

  28. [29]

    Wonderplay: Dynamic 3d scene generation from a single image and actions.arXiv preprint arXiv:2505.18151, 2025

    Zizhang Li, Hong-Xing Yu, Wei Liu, Yin Yang, Charles Herrmann, Gordon Wetzstein, and Jiajun Wu. Wonderplay: Dynamic 3d scene generation from a single image and actions.arXiv preprint arXiv:2505.18151, 2025

    arXiv 2025

  29. [30]

    Evaluation of text-to-video generation models: A dynamics perspective.Advances in Neural Information Processing Systems, 37:109790–109816, 2024

    Mingxiang Liao, Qixiang Ye, Wangmeng Zuo, Fang Wan, Tianyu Wang, Yuzhong Zhao, Jingdong Wang, Xinyu Zhang, et al. Evaluation of text-to-video generation models: A dynamics perspective.Advances in Neural Information Processing Systems, 37:109790–109816, 2024

    2024

  30. [31]

    Vmbench: A benchmark for perception-aligned video motion generation.arXiv preprint arXiv:2503.10076, 2025

    Xinran Ling, Chen Zhu, Meiqi Wu, Hangyu Li, Xiaokun Feng, Cundian Yang, Aiming Hao, Jiashu Zhu, Jiahong Wu, and Xiangxiang Chu. Vmbench: A benchmark for perception-aligned video motion generation.arXiv preprint arXiv:2503.10076, 2025

    Pith/arXiv arXiv 2025

  31. [32]

    Flow matching for generative modeling.arXiv preprint arXiv:2210.02747, 2022

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling.arXiv preprint arXiv:2210.02747, 2022

    Pith/arXiv arXiv 2022

  32. [33]

    Fr \’echet video motion distance: A metric for evaluating motion consistency in videos.arXiv preprint arXiv:2407.16124, 2024

    Jiahe Liu, Youran Qu, Qi Yan, Xiaohui Zeng, Lele Wang, and Renjie Liao. Fr \’echet video motion distance: A metric for evaluating motion consistency in videos.arXiv preprint arXiv:2407.16124, 2024

    Pith/arXiv arXiv 2024

  33. [34]

    Towards interactive video world modeling: Frontiers, challenges, benchmarks, and future trends.arXiv preprint arXiv:2606.01164, 2026

    Jiuming Liu, Chaojun Ni, Mengmeng Liu, Chensheng Peng, Fangjinhua Wang, Sitian Shen, Marc Pollefeys, Masayoshi Tomizuka, Ayush Tewari, and Per Ola Kristensson. Towards interactive video world modeling: Frontiers, challenges, benchmarks, and future trends.arXiv preprint arXiv:2606.01164, 2026

    Pith/arXiv arXiv 2026

  34. [35]

    Compositional visual generation with composable diffusion models

    Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B Tenenbaum. Compositional visual generation with composable diffusion models. InEuropean conference on computer vision, pages 423–439. Springer, 2022

    2022

  35. [36]

    Physgen: Rigid-body physics-grounded image-to-video generation

    Shaowei Liu, Zhongzheng Ren, Saurabh Gupta, and Shenlong Wang. Physgen: Rigid-body physics-grounded image-to-video generation. InEuropean Conference on Computer Vision, pages 360–378. Springer, 2024

    2024

  36. [37]

    Evalcrafter: Benchmarking and evaluating large video generation models

    Yaofang Liu, Xiaodong Cun, Xuebo Liu, Xintao Wang, Yong Zhang, Haoxin Chen, Yang Liu, Tieyong Zeng, Raymond Chan, and Ying Shan. Evalcrafter: Benchmarking and evaluating large video generation models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22139–22149, 2024. 14

    2024

  37. [38]

    Omni-effects: Unified and spatially- controllable visual effects generation.arXiv preprint arXiv:2508.07981, 2025

    Fangyuan Mao, Aiming Hao, Jintao Chen, Dongxia Liu, Xiaokun Feng, Jiashu Zhu, Meiqi Wu, Chubin Chen, Jiahong Wu, and Xiangxiang Chu. Omni-effects: Unified and spatially- controllable visual effects generation.arXiv preprint arXiv:2508.07981, 2025

    arXiv 2025

  38. [39]

    Towards world simulator: Crafting physical commonsense- based benchmark for video generation.arXiv preprint arXiv:2410.05363, 2024

    Fanqing Meng, Jiaqi Liao, Xinyu Tan, Wenqi Shao, Quanfeng Lu, Kaipeng Zhang, Yu Cheng, Dianqi Li, Yu Qiao, and Ping Luo. Towards world simulator: Crafting physical commonsense- based benchmark for video generation.arXiv preprint arXiv:2410.05363, 2024

    Pith/arXiv arXiv 2024

  39. [40]

    Nerf: Representing scenes as neural radiance fields for view synthesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoor- thi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99–106, 2021

    2021

  40. [41]

    Do generative video models understand physical principles?arXiv preprint arXiv:2501.09038, 2025

    Saman Motamed, Laura Culp, Kevin Swersky, Priyank Jaini, and Robert Geirhos. Do generative video models understand physical principles?arXiv preprint arXiv:2501.09038, 2025

    Pith/arXiv arXiv 2025

  41. [42]

    OpenAI. Sora. Accessed February 25, 2025 [Online] https://openai.com/index/sora/,

    2025

  42. [43]

    Maxime Oquab, Timothée Darcet, Theo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Russell Howes, Po-Yao Huang, Hu Xu, Vasu Sharma, Shang-Wen Li, Wojciech Galuba, Mike Rabbat, Mido Assran, Nicolas Ballas, Gabriel Synnaeve, Ishan Misra, Herve Jegou, Julien Mairal, Patrick Laba...

    2023

  43. [44]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF international conference on computer vision, pages 4195–4205, 2023

    2023

  44. [45]

    Open-sora 2.0: Training a commercial-level video generation model in $200 k.arXiv preprint arXiv:2503.09642, 2025

    Xiangyu Peng, Zangwei Zheng, Chenhui Shen, Tom Young, Xinying Guo, Binluo Wang, Hang Xu, Hongxin Liu, Mingyan Jiang, Wenjun Li, et al. Open-sora 2.0: Training a commercial-level video generation model in $200 k.arXiv preprint arXiv:2503.09642, 2025

    Pith/arXiv arXiv 2025

  45. [46]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  46. [47]

    Learning to generate object interactions with physics-guided video diffusion.arXiv preprint arXiv:2510.02284, 2025

    David Romero, Ariana Bermudez, Hao Li, Fabio Pizzati, and Ivan Laptev. Learning to generate object interactions with physics-guided video diffusion.arXiv preprint arXiv:2510.02284, 2025

    arXiv 2025

  47. [48]

    Motion-i2v: Consistent and controllable image-to-video generation with explicit motion modeling

    Xiaoyu Shi, Zhaoyang Huang, Fu-Yun Wang, Weikang Bian, Dasong Li, Yi Zhang, Manyuan Zhang, Ka Chun Cheung, Simon See, Hongwei Qin, et al. Motion-i2v: Consistent and controllable image-to-video generation with explicit motion modeling. InACM SIGGRAPH 2024 Conference Papers, pages 1–11, 2024

    2024

  48. [49]

    Oriane Siméoni, Huy V V o, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Michaël Ramamonjisoa, et al. Dinov3. arXiv preprint arXiv:2508.10104, 2025

    Pith/arXiv arXiv 2025

  49. [50]

    T2v- compbench: A comprehensive benchmark for compositional text-to-video generation

    Kaiyue Sun, Kaiyi Huang, Xian Liu, Yue Wu, Zihan Xu, Zhenguo Li, and Xihui Liu. T2v- compbench: A comprehensive benchmark for compositional text-to-video generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 8406–8416, 2025

    2025

  50. [51]

    Physmotion: Physics-grounded dynamics from a single image.arXiv preprint arXiv:2411.17189, 2024

    Xiyang Tan, Ying Jiang, Xuan Li, Zeshun Zong, Tianyi Xie, Yin Yang, and Chenfanfu Jiang. Physmotion: Physics-grounded dynamics from a single image.arXiv preprint arXiv:2411.17189, 2024

    Pith/arXiv arXiv 2024

  51. [52]

    Towards accurate generative models of video: A new metric & challenges

    Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717, 2018

    Pith/arXiv arXiv 2018

  52. [53]

    Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    Wan Team, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025. 15

    Pith/arXiv arXiv 2025

  53. [54]

    Wisa: World simulator assistant for physics-aware text-to- video generation.arXiv preprint arXiv:2503.08153, 2025

    Jing Wang, Ao Ma, Ke Cao, Jun Zheng, Zhanjie Zhang, Jiasong Feng, Shanyuan Liu, Yuhang Ma, Bo Cheng, Dawei Leng, et al. Wisa: World simulator assistant for physics-aware text-to- video generation.arXiv preprint arXiv:2503.08153, 2025

    Pith/arXiv arXiv 2025

  54. [55]

    Qwen2-vl: Enhancing vision- language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision- language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024

    Pith/arXiv arXiv 2024

  55. [56]

    Motionctrl: A unified and flexible motion controller for video generation

    Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion controller for video generation. InACM SIGGRAPH 2024 Conference Papers, pages 1–11, 2024

    2024

  56. [57]

    Physanimator: Physics-guided generative cartoon animation

    Tianyi Xie, Yiwei Zhao, Ying Jiang, and Chenfanfu Jiang. Physanimator: Physics-guided generative cartoon animation. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 10793–10804, 2025

    2025

  57. [58]

    Morpheus: Benchmarking physical reasoning of video generative models with real physical experiments.arXiv preprint arXiv:2504.02918, 2025

    Chenyu Zhang, Daniil Cherniavskii, Andrii Zadaianchuk, Antonios Tragoudaras, Antonios V ozikis, Thijmen Nijdam, Derck WE Prinzhorn, Mark Bodracska, Nicu Sebe, and Efstratios Gavves. Morpheus: Benchmarking physical reasoning of video generative models with real physical experiments.arXiv preprint arXiv:2504.02918, 2025

    Pith/arXiv arXiv 2025

  58. [59]

    Adding conditional control to text-to-image diffusion models, 2023

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models, 2023

    2023

  59. [60]

    Tora: Trajectory-oriented diffusion transformer for video generation

    Zhenghao Zhang, Junchao Liao, Menghao Li, Zuozhuo Dai, Bingxue Qiu, Siyu Zhu, Long Qin, and Weizhi Wang. Tora: Trajectory-oriented diffusion transformer for video generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 2063–2073, 2025

    2063

  60. [61]

    Attention never lie: Visual attention defocus reveals and rectifies hallucinations in mllms

    Chenxi Zhao, Yan Zhou, and Jufeng Yang. Attention never lie: Visual attention defocus reveals and rectifies hallucinations in mllms. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8897–8906, 2026

    2026

  61. [62]

    Extending one-step image generation from class labels to text via discriminative text representation

    Chenxi Zhao, Chen Zhu, Xiaokun Feng, Aiming Hao, Jiashu Zhu, Jiachen Lei, Jiahong Wu, Xiangxiang Chu, and Jufeng Yang. Extending one-step image generation from class labels to text via discriminative text representation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 36649–36659, 2026

    2026

  62. [63]

    Vbench-2.0: Advancing video generation benchmark suite for intrinsic faithfulness.arXiv preprint arXiv:2503.21755, 2025

    Dian Zheng, Ziqi Huang, Hongbo Liu, Kai Zou, Yinan He, Fan Zhang, Lulu Gu, Yuanhan Zhang, Jingwen He, Wei-Shi Zheng, et al. Vbench-2.0: Advancing video generation benchmark suite for intrinsic faithfulness.arXiv preprint arXiv:2503.21755, 2025

    Pith/arXiv arXiv 2025

  63. [64]

    Open-sora: Democratizing efficient video production for all

    Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. arXiv preprint arXiv:2412.20404, 2024

    Pith/arXiv arXiv 2024

  64. [65]

    passes the criterion

    Chen Zhu, Jiashu Zhu, Yanxun Li, Meiqi Wu, Bingze Song, Chubin Chen, Jiahong Wu, Xi- angxiang Chu, and Yangang Wang. Artifact-aware evaluation for high-quality video generation. arXiv preprint arXiv:2601.20297, 2026. 16 S1 Dataset Generation Details Basic Scene Setup.We start by placing a sufficiently large planar surface at the scene center, which serves...

    arXiv 2026

  65. [67]

    Time ScoreDoes the video follow physical laws of temporal coherence? For example, is the movement and state change of objects reasonable and natural over time, without jumps, teleportation, or violations of physical time order? •5 points: Fully consistent with physical temporal coherence •0 points: Not consistent, with obvious violations of temporal order...

  66. [68]

    yes" if the anomaly exists or

    Space ScoreDoes the video follow physical laws of spatial coherence? For example, do gravity, friction, elasticity, deformation, and motion paths appear natural and physically plausible? • 5 points: Fully consistent with physical spatial laws; no violations like gravity defiance or implausible deformations •0 points: Clearly violates spatial physical laws...

  67. [2025]

    URLhttps://openai.com/index/sora/