Pith. sign in

REVIEW 2 major objections 2 minor 1 cited by

PhysRAG retrieves examples from a curated 7K-video physics database to correct diffusion-model violations of physical rules.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-26 05:14 UTC pith:RWZ3LCCT

load-bearing objection PhysRAG shows a workable RAG pipeline for physics-aware video diffusion but the 7K dataset's claimed physical improvement rests on unquantified filtering. the 2 major comments →

arxiv 2606.26916 v1 pith:RWZ3LCCT submitted 2026-06-25 cs.CV

PhysRAG: Enhancing Physics-Awareness in Video Generation via Retrieval-Augmented Generation

Kexu Cheng , Zicheng Liu , Mingju Gao , Chunhe Song , Hao Tang This is my paper
classification cs.CV
keywords video generationretrieval-augmented generationphysics-aware generationvideo diffusion modelsphysical rule compliancedata filtering pipeline
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper aims to show that retrieval-augmented generation can make video diffusion models follow physical laws more reliably by pulling relevant real-world motion examples at inference time. Current models often produce impossible dynamics because training data lacks sufficient physical variety and because the generative process itself does not enforce conservation or interaction rules. By first filtering the WISA-80K collection down to 7K high-fidelity clips and then injecting retrieved knowledge through learnable queries, the method claims measurable gains on both visual quality and explicit physical-compliance benchmarks. If the approach works, generated videos would exhibit fewer floating objects, incorrect collisions, or impossible thermal flows without any change to the underlying diffusion architecture.

Core claim

PhysRAG constructs a physical video database from a two-stage filtered subset of WISA-80K, then uses a retrieval mechanism with learnable queries to inject physical knowledge directly into a video diffusion model, producing state-of-the-art results on PhyGenBench and VBench for both visual fidelity and adherence to mechanics, optics, and thermal dynamics.

What carries the argument

The RAG pipeline that retrieves from the 7K-video physical database and injects the information via learnable queries into the diffusion process.

Load-bearing premise

The 7K filtered videos contain physical phenomena that are both accurate and sufficiently diverse that retrieval will correct rather than reinforce model errors.

What would settle it

A controlled test in which the retrieval step is disabled and physical-compliance scores on PhyGenBench fall to or below the scores of the strongest non-RAG baseline.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Video diffusion models can be made more physically consistent without retraining the entire network from scratch.
  • A modest curated database suffices to supply the missing physical priors that large-scale web video data alone do not provide.
  • Learnable queries offer a modular way to condition generation on retrieved physical examples across different motion categories.
  • Ablation results indicate that both the data-filtering stage and the retrieval injection step are necessary for the reported gains.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same retrieval-plus-query pattern could be tested on domains that also suffer from sparse rule-following data, such as biological motion or fluid simulation.
  • If the filtered set proves biased toward certain object scales or lighting conditions, future work could add explicit diversity constraints during the second filtering stage.
  • The approach leaves open whether the retrieved clips are used only at inference or could also be mixed into continued training of the base diffusion model.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The manuscript introduces PhysRAG, a retrieval-augmented generation pipeline for enhancing physical awareness in video diffusion models. It describes a two-stage filtering process applied to the WISA-80K dataset to produce a curated 7K-video database of purported high physical quality, constructs a physical video database, and injects physical knowledge via learnable queries into the diffusion model. The work claims state-of-the-art performance on both visual quality and physical rule compliance on PhyGenBench and VBench, supported by ablation studies on the filtering pipeline, RAG mechanism, and physical information extraction; code, data, and models are slated for release.

Significance. If the empirical gains are shown to be robust under standard controls, the approach could offer a practical route to mitigating physical hallucinations in video generation by leveraging curated retrieval rather than purely parametric learning. The planned artifact release would aid reproducibility in a field where data curation for physics compliance is often opaque.

major comments (2)
  1. [two-stage data filtering pipeline] The two-stage data filtering pipeline (abstract and methods): the claim that the resulting 7K videos exhibit high physical fidelity sufficient for reliable hallucination correction is unsupported by any reported metrics such as physics-engine error, expert-rated compliance on conservation laws/optics, or inter-annotator agreement. This is load-bearing because the RAG mechanism's value rests on the database improving rather than reinforcing dataset biases.
  2. [benchmarks and results] Results on PhyGenBench and VBench (abstract): the SOTA claim is asserted without any quantitative tables, baseline comparisons, error bars, or effect sizes in the provided text, preventing assessment of whether reported gains survive standard controls for data leakage or evaluation variance.
minor comments (2)
  1. [abstract] The abstract states that ablation studies validate the key components but supplies no high-level quantitative outcomes or effect sizes, which would aid immediate assessment of component importance.
  2. [method for physical information extraction] The number and dimension of learnable queries are described as free parameters without discussion of sensitivity or default choices.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments on our manuscript. We address each major point below and indicate planned revisions to improve the rigor of the claims.

read point-by-point responses

  1. Referee: [two-stage data filtering pipeline] The two-stage data filtering pipeline (abstract and methods): the claim that the resulting 7K videos exhibit high physical fidelity sufficient for reliable hallucination correction is unsupported by any reported metrics such as physics-engine error, expert-rated compliance on conservation laws/optics, or inter-annotator agreement. This is load-bearing because the RAG mechanism's value rests on the database improving rather than reinforcing dataset biases.

    Authors: We acknowledge that the manuscript does not report direct quantitative metrics (e.g., physics-engine error or expert-rated compliance scores with inter-annotator agreement) on the physical fidelity of the curated 7K videos. The two-stage pipeline applies automated heuristics based on WISA-80K annotations for visual quality and basic physical plausibility, and its value is supported indirectly via ablation studies showing performance gains on PhyGenBench when the filtered database is used. To strengthen the claim, we will add a new subsection with expert evaluation results on a random subset of the 7K videos, including compliance ratings for conservation laws and optics along with inter-annotator agreement statistics. revision: yes

  2. Referee: [benchmarks and results] Results on PhyGenBench and VBench (abstract): the SOTA claim is asserted without any quantitative tables, baseline comparisons, error bars, or effect sizes in the provided text, preventing assessment of whether reported gains survive standard controls for data leakage or evaluation variance.

    Authors: The full manuscript (Section 4 and associated tables) contains quantitative comparisons against baselines on both PhyGenBench and VBench, reporting metrics for visual quality and physical compliance. However, the current version lacks error bars from repeated runs and formal analysis of effect sizes or data leakage controls. We will revise the experimental section to include these elements, along with explicit discussion of steps taken to avoid overlap between the physical video database and benchmark evaluation prompts. revision: yes

Circularity Check

0 steps flagged

No circularity; empirical pipeline with external benchmarks only

full rationale

The paper presents an empirical pipeline: two-stage filtering of WISA-80K to 7K videos, construction of a physical video database, and injection of physical knowledge via learnable queries into a diffusion model. No equations, derivations, fitted parameters renamed as predictions, or self-citation chains appear in the provided text. SOTA claims rest on external benchmarks (PhyGenBench, VBench) rather than any quantity defined inside the paper. The absence of mathematical structure means none of the six enumerated circularity patterns can be exhibited by quote and reduction.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

The central claim rests on the untested premise that the curated 7K videos plus retrieval will supply sufficient physical constraints. The diffusion backbone itself is treated as a black-box prior. No new physical laws or particles are postulated.

free parameters (1)
  • number and dimension of learnable queries
    Design choice whose count and size are not derived from first principles and must be selected to make retrieval effective.
axioms (1)
  • domain assumption Existing video diffusion models can be conditioned on external retrieved features without destroying their generative capability.
    Invoked when the paper states that physical knowledge is injected via learnable queries.

pith-pipeline@v0.9.1-grok · 5738 in / 1417 out tokens · 20910 ms · 2026-06-26T05:14:50.972576+00:00 · methodology

0 comments
read the original abstract

Developing physically aware video generation models remains a significant challenge due to the difficulty in capturing diverse physical phenomena, such as thermal dynamics, mechanics, and optics. In this work, we introduce PhysRAG, a novel pipeline that enhances physical awareness in video generation through Retrieval-Augmented Generation (RAG). To address the issue of limited high-quality data, we design a two-stage data filtering pipeline based on the WISA-80K dataset, resulting in a curated set of 7K high-quality videos for training. Furthermore, we construct a physical video database and develop a mechanism to inject physical knowledge into a video diffusion model using learnable queries. Our method achieves state-of-the-art performance in both visual quality and physical rule compliance, surpassing existing models in benchmarks such as PhyGenBench and VBench. We conduct extensive ablation studies to validate the effectiveness of our key components, including the data filtering pipeline, RAG mechanism, and method for physical information extraction. To facilitate future research, our code, data, and models are prepared for release at https://github.com/sediment1024/PhysRAG.

Figures

Figures reproduced from arXiv: 2606.26916 by Chunhe Song, Hao Tang, Kexu Cheng, Mingju Gao, Zicheng Liu.

Figure 1
Figure 1. Figure 1: Overview and Qualitative Results of PhysRAG. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Two-stage preprocessing for WISA-80K. We first use Qwen3-VL (caption-only) to select the top 10% candidates, then use Qwen3-VL (key frames + caption) to verify prompt–frame consistency and retain 90%, yielding a high-quality subset (∼7K videos) [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of our retrieval-augmented video diffusion framework. A Video Diffu￾sion Transformer (DiT) generates videos from text and noise while retrieving relevant physical videos from a database. Retrieved features are encoded by Vide MAE and injected into DiT blocks through the Query Inject module. physical phenomena (e.g., collision, combustion, and explosion). During infer￾ence, given a text prompt, we … view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative comparison of physical consistency across different scenarios from PhyGenBench [47]. PhysRAG outperforms Wan2.2-5B and PhyT2V by accurately modeling physical interactions. In the robotic arm etching scenario, PhysRAG cap￾tures the continuous motion required for the task, while baseline models fail to do so. In the fluid dripping example, PhysRAG correctly preserves fluid dynamics and mate￾rial … view at source ↗
Figure 5
Figure 5. Figure 5: Ablation on PhyGenBench [47]. We compare PhysRAG (ours) with SFT and SFT+RAG on (a) drone-over-pool reflection and (b) oil pouring. In (a), SFT and SFT+RAG miss the water-surface reflection, while PhysRAG preserves it. In (b), SFT and SFT+RAG show non-causal liquid growth without coherent downward flow; PhysRAG produces temporally consistent pouring and accumulation [PITH_FULL_IMAGE:figures/full_fig_p013_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. HyBDM: Multi-Scale Hybrid Experts for Time Series Forecasting with Bidirectional Dependency Modeling

    cs.LG 2026-07 conditional novelty 3.0

    HyBDM combines a Mamba-style global-pattern expert with a local window transformer and a learned router to forecast multivariate time series, reporting state-of-the-art results on six benchmarks.

Reference graph

Works this paper leans on

93 extracted references · 59 canonical work pages · cited by 1 Pith paper · 35 internal anchors

  1. [1]

    In: Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., Oh, A

    Alayrac, J.B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Men- sch, A., Millican, K., Reynolds, M., Ring, R., Rutherford, E., Cabi, S., Han, T., Gong, Z., Samangooei, S., Monteiro, M., Menick, J.L., Borgeaud, S., Brock, A., Nematzadeh, A., Sharifzadeh, S., Bińkowski, M.a., Barreira, R., Vinyals, O., Zisser- man, A., Simonyan, K.: Fla...

    2022

  2. [2]

    Qwen3-VL Technical Report

    Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., Ge, W., Guo, Z., Huang, Q., Huang, J., Huang, F., Hui, B., Jiang, S., Li, Z., Li, M., Li, M., Li, K., Lin, Z., Lin, J., Liu, X., Liu, J., Liu, C., Liu, Y., Liu, D., Liu, S., Lu, D., Luo, R., Lv, C., Men, R., Meng, L., Ren, X., Ren, X., Song, S., Sun, Y., Tang, ...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  3. [3]

    arXiv preprint arXiv:2503.06800 (2025)

    Bansal, H., Peng, C., Bitton, Y., Goldenberg, R., Grover, A., Chang, K.W.: Videophy-2: A challenging action-centric physical commonsense evaluation in video generation. arXiv preprint arXiv:2503.06800 (2025)

    work page arXiv 2025

  4. [4]

    In: SIGGRAPH Asia 2024 Conference Papers

    Bar-Tal, O., Chefer, H., Tov, O., Herrmann, C., Paiss, R., Zada, S., Ephrat, A., Hur, J., Liu, G., Raj, A., et al.: Lumiere: A space-time diffusion model for video generation. In: SIGGRAPH Asia 2024 Conference Papers. pp. 1–11 (2024)

    2024

  5. [5]

    Advances in neural infor- mation processing systems29(2016)

    Battaglia, P., Pascanu, R., Lai, M., Jimenez Rezende, D., et al.: Interaction net- works for learning about objects, relations and physics. Advances in neural infor- mation processing systems29(2016)

    2016

  6. [6]

    arXiv preprint arXiv:2106.08261 (2021)

    Bear, D.M., Wang, E., Mrowca, D., Binder, F.J., Tung, H.Y.F., Pramod, R., Hold- away, C., Tao, S., Smith, K., Sun, F.Y., et al.: Physion: Evaluating physical predic- tion from vision in humans and machines. arXiv preprint arXiv:2106.08261 (2021)

    work page arXiv 2021

  7. [7]

    $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control

    Black, K., Brown, N., Driess, D., Esmail, A., Equi, M., Finn, C., Fusai, N., Groom, L., Hausman, K., Ichter, B., et al.:π0: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  8. [8]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Blattmann, A., Rombach, R., Ling, H., Dockhorn, T., Kim, S.W., Fidler, S., Kreis, K.: Align your latents: High-resolution video synthesis with latent diffusion mod- els. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 22563–22575 (2023)

    2023

  9. [9]

    Advances in Neural Information Processing Systems 35, 15309–15324 (2022)

    Blattmann, A., Rombach, R., Oktay, K., Müller, J., Ommer, B.: Retrieval- augmented diffusion models. Advances in Neural Information Processing Systems 35, 15309–15324 (2022)

    2022

  10. [10]

    OpenAI Blog1(8), 1 (2024)

    Brooks, T., Peebles, B., Holmes, C., DePue, W., Guo, Y., Jing, L., Schnurr, D., Taylor, J., Luhman, T., Luhman, E., et al.: Video generation models as world simulators. OpenAI Blog1(8), 1 (2024)

    2024

  11. [11]

    PhyGDPO: Physics-Aware Groupwise Direct Preference Optimization for Physically Consistent Text-to-Video Generation

    Cai, Y., Li, K., Jia, M., Wang, J., Sun, J., Liang, F., Chen, W., Juefei-Xu, F., Wang, C., Thabet, A., et al.: Phygdpo: Physics-aware groupwise direct prefer- ence optimization for physically consistent text-to-video generation. arXiv preprint arXiv:2512.24551 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  12. [12]

    A Compositional Object-Based Approach to Learning Physical Dynamics

    Chang, M.B., Ullman, T., Torralba, A., Tenenbaum, J.B.: A compositional object- based approach to learning physical dynamics. arXiv preprint arXiv:1612.00341 (2016) 18 K. Cheng et al

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  13. [13]

    VideoCrafter1: Open Diffusion Models for High-Quality Video Generation

    Chen, H., Xia, M., He, Y., Zhang, Y., Cun, X., Yang, S., Xing, J., Liu, Y., Chen, Q., Wang, X., et al.: Videocrafter1: Open diffusion models for high-quality video generation. arXiv preprint arXiv:2310.19512 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  14. [14]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Chen, H., Zhang, Y., Cun, X., Xia, M., Wang, X., Weng, C., Shan, Y.: Videocrafter2: Overcoming data limitations for high-quality video diffusion mod- els. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 7310–7320 (2024)

    2024

  15. [15]

    arXiv preprint arXiv:2508.10858 (2025)

    Chen, H.H., Huang, H., Chen, Q., Yang, H., Lim, S.N.: Hierarchical fine-grained preference optimization for physically plausible video generation. arXiv preprint arXiv:2508.10858 (2025)

    work page arXiv 2025

  16. [16]

    In: Pro- ceedings of the 2022 Conference on Empirical Methods in Natural Language Pro- cessing

    Chen, W., Hu, H., Chen, X., Verga, P., Cohen, W.: Murag: Multimodal retrieval- augmented generator for open question answering over images and text. In: Pro- ceedings of the 2022 Conference on Empirical Methods in Natural Language Pro- cessing. pp. 5558–5570 (2022)

    2022

  17. [17]

    arXiv preprint arXiv:2209.14491 (2022)

    Chen, W., Hu, H., Saharia, C., Cohen, W.W.: Re-imagen: Retrieval-augmented text-to-image generator. arXiv preprint arXiv:2209.14491 (2022)

    work page arXiv 2022

  18. [18]

    Douze, M., Guzhva, A., Deng, C., Johnson, J., Szilvasy, G., Mazaré, P.E., Lomeli, M., Hosseini, L., Jégou, H.: The faiss library (2025),https://arxiv.org/abs/ 2401.08281

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  19. [19]

    arXiv preprint arXiv:2501.08453 (2025)

    Fan, W., Si, C., Song, J., Yang, Z., He, Y., Zhuo, L., Huang, Z., Dong, Z., He, J., Pan, D., et al.: Vchitect-2.0: Parallel transformer for scaling up video diffusion models. arXiv preprint arXiv:2501.08453 (2025)

    work page arXiv 2025

  20. [20]

    Seedance 1.0: Exploring the Boundaries of Video Generation Models

    Gao, Y., Guo, H., Hoang, T., Huang, W., Jiang, L., Kong, F., Li, H., Li, J., Li, L., Li, X., et al.: Seedance 1.0: Exploring the boundaries of video generation models. arXiv preprint arXiv:2506.09113 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  21. [21]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Greff, K., Belletti, F., Beyer, L., Doersch, C., Du, Y., Duckworth, D., Fleet, D.J., Gnanapragasam, D., Golemo, F., Herrmann, C., et al.: Kubric: A scalable dataset generator. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 3749–3761 (2022)

    2022

  22. [22]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Guen, V.L., Thome, N.: Disentangling physical dynamics from unknown factors for unsupervised video prediction. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 11474–11484 (2020)

    2020

  23. [23]

    Hao, Y., Chen, C., Mian, A.S., Xu, C., Liu, D.: Enhancing physical plausibility in video generation by reasoning the implausibility (2025),https://arxiv.org/abs/ 2509.24702

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  24. [24]

    Imagen Video: High Definition Video Generation with Diffusion Models

    Ho, J., Chan, W., Saharia, C., Whang, J., Gao, R., Gritsenko, A., Kingma, D.P., Poole, B., Norouzi, M., Fleet, D.J., et al.: Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303 (2022)

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  25. [25]

    Advances in neural information processing systems35, 8633– 8646 (2022)

    Ho, J., Salimans, T., Gritsenko, A., Chan, W., Norouzi, M., Fleet, D.J.: Video diffusion models. Advances in neural information processing systems35, 8633– 8646 (2022)

    2022

  26. [26]

    CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers

    Hong, W., Ding, M., Zheng, W., Liu, X., Tang, J.: Cogvideo: Large-scale pretrain- ing for text-to-video generation via transformers. arXiv preprint arXiv:2205.15868 (2022)

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  27. [27]

    Huang, Y., Lv, X., Xu, J., Yu, Z., Zhang, J., Hu, R., Feng, W., Zou, S., Xiao, H., Zhou, Z., Huang, K., Peng, Z., Xu, J., Zhao, H., Zhu, C., Yi, R., Huang, Y., Wu, D., Zhang, Y., Cheng, K., Song, C., Xue, Y., Zhang, X., Guo, L., Chen, Y., Wu, B., Yu, H., Xu, K.: Paiworld: A 3d-consistent world foundation model for robotic manipulation (2026),https://arxiv...

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  28. [28]

    Huang, J

    Huang, Y., Zhang, J., Zou, S., Liu, X., Hu, R., Xu, K.: Ladi-wm: A latent diffusion- based world model for predictive manipulation. arXiv preprint arXiv:2505.11528 (2025)

    work page arXiv 2025

  29. [29]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Huang, Z., He, Y., Yu, J., Zhang, F., Si, C., Jiang, Y., Zhang, Y., Wu, T., Jin, Q., Chanpaisit, N., et al.: Vbench: Comprehensive benchmark suite for video gener- ative models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 21807–21818 (2024)

    2024

  30. [30]

    GPT-4o System Card

    Hurst, A., Lerer, A., Goucher, A.P., Perelman, A., Ramesh, A., Clark, A., Os- trow, A., Welihinda, A., Hayes, A., Radford, A., et al.: Gpt-4o system card. arXiv preprint arXiv:2410.21276 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  31. [31]

    Intelligence, P., Amin, A., Aniceto, R., Balakrishna, A., Black, K., Conley, K., Connors, G., Darpinian, J., Dhabalia, K., DiCarlo, J., Driess, D., Equi, M., Es- mail, A., Fang, Y., Finn, C., Glossop, C., Godden, T., Goryachev, I., Groom, L., Hancock, H., Hausman, K., Hussein, G., Ichter, B., Jakubczak, S., Jen, R., Jones, T., Katz, B., Ke, L., Kuchi, C.,...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  32. [32]

    Intelligence, P., Black, K., Brown, N., Darpinian, J., Dhabalia, K., Driess, D., Esmail, A., Equi, M., Finn, C., Fusai, N., Galliker, M.Y., Ghosh, D., Groom, L., Hausman, K., Ichter, B., Jakubczak, S., Jones, T., Ke, L., LeBlanc, D., Levine, S., Li-Bell, A., Mothukuri, M., Nair, S., Pertsch, K., Ren, A.Z., Shi, L.X., Smith, L., Springenberg, J.T., Stachow...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  33. [33]

    arXiv preprint arXiv:2510.13809 (2025)

    Ji, S., Chen, X., Tao, X., Wan, P., Zhao, H.: Physmaster: Mastering physical representation for video generation via reinforcement learning. arXiv preprint arXiv:2510.13809 (2025)

    work page arXiv 2025

  34. [34]

    How Far is Video Generation from World Model: A Physical Law Perspective

    Kang, B.,Yue,Y.,Lu, R.,Lin, Z.,Zhao,Y., Wang, K.,Huang,G., Feng,J.: Howfar is video generation from world model: A physical law perspective. arXiv preprint arXiv:2411.02385 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  35. [35]

    HunyuanVideo: A Systematic Framework For Large Video Generative Models

    Kong, W., Tian, Q., Zhang, Z., Min, R., Dai, Z., Zhou, J., Xiong, J., Li, X., Wu, B., Zhang, J., et al.: Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  36. [36]

    Kuaishou Technology: Kling ai: High-fidelity text-to-video generation.https:// www.kuaishou.com/kling(2024)

    2024

  37. [37]

    Advances in neural information processing systems 33, 9459–9474 (2020)

    Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.t., Rocktäschel, T., et al.: Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems 33, 9459–9474 (2020)

    2020

  38. [38]

    arXiv preprint arXiv:2503.09595 (2025) 4

    Li, C., Michel, O., Pan, X., Liu, S., Roberts, M., Xie, S.: Pisa experiments: Explor- ing physics post-training for video diffusion models by watching stuff drop. arXiv preprint arXiv:2503.09595 (2025)

    work page arXiv 2025

  39. [39]

    In: Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., Scarlett, J

    Li, J., Li, D., Savarese, S., Hoi, S.: BLIP-2: Bootstrapping language-image pre- training with frozen image encoders and large language models. In: Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., Scarlett, J. (eds.) Proceedings of the 40th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 202, pp....

    2023

  40. [40]

    IEEE Robotics and Automation Letters9(6), 6012–6019 (2024)

    Liu, C., Shi, K., Zhou, K., Wang, H., Zhang, J., Dong, H.: Rgbgrasp: Image-based object grasping by capturing multiple views during robot arm movement with neural radiance fields. IEEE Robotics and Automation Letters9(6), 6012–6019 (2024)

    2024

  41. [41]

    arXiv preprint arXiv:2501.10928 (2025)

    Liu, D., Zhang, J., Dinh, A.D., Park, E., Zhang, S., Mian, A., Shah, M., Xu, C.: Generative physical ai in vision: A survey. arXiv preprint arXiv:2501.10928 (2025)

    work page arXiv 2025

  42. [42]

    Improving Video Generation with Human Feedback

    Liu, J., Liu, G., Liang, J., Yuan, Z., Liu, X., Zheng, M., Wu, X., Wang, Q., Xia, M., Wang, X., et al.: Improving video generation with human feedback. arXiv preprint arXiv:2501.13918 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  43. [43]

    In: Proceedings of the Com- puter Vision and Pattern Recognition Conference

    Liu, R., Wu, H., Zheng, Z., Wei, C., He, Y., Pi, R., Chen, Q.: Videodpo: Omni- preference alignment for video diffusion generation. In: Proceedings of the Com- puter Vision and Pattern Recognition Conference. pp. 8009–8019 (2025)

    2025

  44. [44]

    Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models

    Liu, Y., Zhang, K., Li, Y., Yan, Z., Gao, C., Chen, R., Yuan, Z., Huang, Y., Sun, H., Gao, J., et al.: Sora: A review on background, technology, limitations, and opportunities of large vision models. arXiv preprint arXiv:2402.17177 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  45. [45]

    Loshchilov, I., Hutter, F.: Decoupled weight decay regularization (2019),https: //arxiv.org/abs/1711.05101

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  46. [46]

    Ma, G., Huang, H., Yan, K., Chen, L., Duan, N., Yin, S., Wan, C., Ming, R., Song, X., Chen, X., Zhou, Y., Sun, D., Zhou, D., Zhou, J., Tan, K., An, K., Chen, M., Ji, W., Wu, Q., Sun, W., Han, X., Wei, Y., Ge, Z., Li, A., Wang, B., Huang, B., Wang, B., Li, B., Miao, C., Xu, C., Wu, C., Yu, C., Shi, D., Hu, D., Liu, E., Yu, G., Yang, G., Huang, G., Yan, G.,...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  47. [47]

    Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation

    Meng, F., Liao, J., Tan, X., Shao, W., Lu, Q., Zhang, K., Cheng, Y., Li, D., Qiao, Y., Luo, P.: Towards world simulator: Crafting physical commonsense-based benchmark for video generation. arXiv preprint arXiv:2410.05363 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  48. [48]

    arXiv preprint arXiv:2502.07007 (2025)

    Meng, S., Luo, Y., Liu, P.: Grounding creativity in physics: A brief survey of physical priors in aigc. arXiv preprint arXiv:2502.07007 (2025)

    work page arXiv 2025

  49. [49]

    In: Proceedings of the AAAI conference on artificial intelligence

    Mou, C., Wang, X., Xie, L., Wu, Y., Zhang, J., Qi, Z., Shan, Y.: T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In: Proceedings of the AAAI conference on artificial intelligence. vol. 38, pp. 4296–4304 (2024)

    2024

  50. [50]

    In: Proceedings of the 2025 International Conference on Multimedia Retrieval

    Peruzzo, E., Xu, D., Xu, X., Shi, H., Sebe, N.: Ragme: Retrieval augmented video generation for enhanced motion realism. In: Proceedings of the 2025 International Conference on Multimedia Retrieval. p. 1081–1090. ICMR ’25, Association for Computing Machinery, New York, NY, USA (2025).https://doi.org/10.1145/ 3731715.3733417,https://doi.org/10.1145/3731715.3733417

    work page doi:10.1145/3731715.3733417 2025

  51. [51]

    Pika Labs: Pika: Text-to-video generation platform.https://pika.art(2024)

    2024

  52. [52]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 10684–10695 (2022) PhysRAG 21

    2022

  53. [53]

    arXiv preprint arXiv:2510.02284 (2025)

    Romero, D., Bermudez, A., Li, H., Pizzati, F., Laptev, I.: Learning to generate rigid body interactions with video diffusion models. arXiv preprint arXiv:2510.02284 (2025)

    work page arXiv 2025

  54. [54]

    Runway ML: Runway gen-3 alpha.https://research.runwayml.com/gen3(2024)

    2024

  55. [55]

    Make-A-Video: Text-to-Video Generation without Text-Video Data

    Singer, U., Polyak, A., Hayes, T., Yin, X., An, J., Zhang, S., Hu, Q., Yang, H., Ashual, O., Gafni, O., et al.: Make-a-video: Text-to-video generation without text- video data. arXiv preprint arXiv:2209.14792 (2022)

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  56. [56]

    Song, S., Xu, Z., Zhang, Z., Zhou, K., Guo, J., Qin, L., Huang, B.: Learning plug- and-play memory for guiding video diffusion models (2025),https://arxiv.org/ abs/2511.19229

    work page arXiv 2025

  57. [57]

    SCOPE: Simulating Cross-game Operations in Playable Environments for FPS World Models

    Tong, Z., Lai, H., Wang, Z., Xing, Z., Cheng, K., Xu, H., Pu, Z., Zhu, S., Feng, R., Zhao, J., et al.: Scope: Simulating cross-game operations in playable environments for fps world models. arXiv preprint arXiv:2605.23345 (2026)

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  58. [58]

    International Conference on Learning Representations (ICLR) , year =

    Toth, P., Rezende, D.J., Jaegle, A., Racanière, S., Botev, A., Higgins, I.: Hamilto- nian generative networks. arXiv preprint arXiv:1909.13789 (2019)

    work page arXiv 1909

  59. [59]

    Advances in Neural Information Processing Systems36, 67048–67068 (2023)

    Tung,H.Y.,Ding,M.,Chen,Z.,Bear,D.,Gan,C.,Tenenbaum,J.,Yamins,D.,Fan, J., Smith, K.: Physion++: Evaluating physical scene understanding that requires online inference of different physical properties. Advances in Neural Information Processing Systems36, 67048–67068 (2023)

    2023

  60. [60]

    Wan: Open and Advanced Large-Scale Video Generative Models

    Wan, T., Wang, A., Ai, B., Wen, B., Mao, C., Xie, C.W., Chen, D., Yu, F., Zhao, H., Yang, J., et al.: Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  61. [61]

    arXiv preprint arXiv:2509.20358 (2025) 4

    Wang, C., Chen, C., Huang, Y., Dou, Z., Liu, Y., Gu, J., Liu, L.: Physctrl: Genera- tive physics for controllable and physics-grounded video generation. arXiv preprint arXiv:2509.20358 (2025)

    work page arXiv 2025

  62. [62]

    Diffusion Model as a Generalist Segmentation Learner

    Wang, H., Xiang, A., Sun, H., Sun, P., Pan, C., Chen, Y., Hong, M., Wang, W., Chen, S., Chen, Y., et al.: Diffusion model as a generalist segmentation learner. arXiv preprint arXiv:2604.24575 (2026)

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  63. [63]

    In: 2025 IEEE International Conference on Robotics and Automation (ICRA)

    Wang, H., Zhou, K., Gu, B., Feng, Z., Wang, W., Sun, P., Xiao, Y., Zhang, J., Dong, H.: Transdiff: Diffusion-based method for manipulating transparent objects using a single rgb-d image. In: 2025 IEEE International Conference on Robotics and Automation (ICRA). pp. 7277–7283. IEEE (2025)

    2025

  64. [64]

    In: Proceedings of the 2024 Con- ference on Empirical Methods in Natural Language Processing

    Wang, J., Wang, C., Huang, K., Huang, J., Jin, L.: Videoclip-xl: Advancing long description understanding for video clip models. In: Proceedings of the 2024 Con- ference on Empirical Methods in Natural Language Processing. pp. 16061–16075 (2024)

    2024

  65. [65]

    arXiv preprint arXiv:2503.08153 (2025)

    Wang, J., Ma, A., Cao, K., Zheng, J., Zhang, Z., Feng, J., Liu, S., Ma, Y., Cheng, B., Leng, D., et al.: Wisa: World simulator assistant for physics-aware text-to-video generation. arXiv preprint arXiv:2503.08153 (2025)

    work page arXiv 2025

  66. [66]

    ModelScope Text-to-Video Technical Report

    Wang, J., Yuan, H., Chen, D., Zhang, Y., Wang, X., Zhang, S.: Modelscope text- to-video technical report. arXiv preprint arXiv:2308.06571 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  67. [67]

    In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Wang, L., Huang, B., Zhao, Z., Tong, Z., He, Y., Wang, Y., Wang, Y., Qiao, Y.: Videomae v2: Scaling video masked autoencoders with dual masking. In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 14549–14560 (June 2023)

    2023

  68. [68]

    arXiv preprint arXiv:2511.03997 , year=

    Wang, P., Wang, W., Li, Q.: Physcorr: Dual-reward dpo for physics-constrained text-to-video generation with automated preference selection. arXiv preprint arXiv:2511.03997 (2025)

    work page arXiv 2025

  69. [69]

    VolSplat: Rethinking Feed-Forward 3D Gaussian Splatting with Voxel-Aligned Prediction

    Wang, W., Chen, Y., Zhang, Z., Liu, H., Wang, H., Feng, Z., Qin, W., Chen, F., Zhu, Z., Chen, D.Y., et al.: Volsplat: Rethinking feed-forward 3d gaussian splatting with voxel-aligned prediction. arXiv preprint arXiv:2509.19297 (2025) 22 K. Cheng et al

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  70. [70]

    World-R1: Reinforcing 3D Constraints for Text-to-Video Generation

    Wang, W., He, X., Gu, Y., Yang, Y., Zhang, Z., He, Y., Ding, Y., Hu, X., Chen, D.Y., He, Z., et al.: World-r1: Reinforcing 3d constraints for text-to-video genera- tion. arXiv preprint arXiv:2604.24764 (2026)

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  71. [71]

    In: European conference on computer vision

    Wang, X., Zhu, Z., Huang, G., Chen, X., Zhu, J., Lu, J.: Drivedreamer: Towards real-world-drive world models for autonomous driving. In: European conference on computer vision. pp. 55–72. Springer (2024)

    2024

  72. [72]

    International Journal of Computer Vision133(5), 3059–3078 (2025)

    Wang, Y., Chen, X., Ma, X., Zhou, S., Huang, Z., Wang, Y., Yang, C., He, Y., Yu, J., Yang, P., et al.: Lavie: High-quality video generation with cascaded latent diffu- sion models. International Journal of Computer Vision133(5), 3059–3078 (2025)

    2025

  73. [73]

    Advances in neural information processing systems30(2017)

    Watters, N., Zoran, D., Weber, T., Battaglia, P., Pascanu, R., Tacchetti, A.: Visual interaction networks: Learning a physics simulator from video. Advances in neural information processing systems30(2017)

    2017

  74. [74]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Wen, Y., Zhao, Y., Liu, Y., Jia, F., Wang, Y., Luo, C., Zhang, C., Wang, T., Sun, X., Zhang, X.: Panacea: Panoramic and controllable video generation for autonomous driving. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 6902–6912 (2024)

    2024

  75. [75]

    Video models are zero-shot learners and reasoners

    Wiedemer, T., Li, Y., Vicol, P., Gu, S.S., Matarese, N., Swersky, K., Kim, B., Jaini, P., Geirhos, R.: Video models are zero-shot learners and reasoners. arXiv preprint arXiv:2509.20328 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  76. [76]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    Wu, J.Z., Ge, Y., Wang, X., Lei, S.W., Gu, Y., Shi, Y., Hsu, W., Shan, Y., Qie, X., Shou, M.Z.: Tune-a-video: One-shot tuning of image diffusion models for text- to-video generation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 7623–7633 (October 2023)

    2023

  77. [77]

    Divide and Conquer: Decoupled Representation Alignment for Multimodal World Models

    Xiao,J.,Liang,D.,Zhou,X.,Ye,Y.,Su,T.,Yi,G.,Xia,B.,Lyu,Q.,Shi,S.,Huang, J., et al.: Divide and conquer: Decoupled representation alignment for multimodal world models. arXiv preprint arXiv:2605.01896 (2026)

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  78. [78]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Xue, Q., Yin, X., Yang, B., Gao, W.: Phyt2v: Llm-guided iterative self-refinement for physics-grounded text-to-video generation. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 18826–18836 (2025)

    2025

  79. [79]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Yang, X., Li, B., Zhang, Y., Yin, Z., Bai, L., Ma, L., Wang, Z., Cai, J., Wong, T.T., Lu, H., et al.: Vlipp: Towards physically plausible video generation with vision and language informed physical prior. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 12360–12370 (2025)

    2025

  80. [80]

    CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer

    Yang, Z., Teng, J., Zheng, W., Ding, M., Huang, S., Xu, J., Yang, Y., Hong, W., Zhang, X., Feng, G., et al.: Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

Showing first 80 references.