Pith. sign in

REVIEW 2 major objections 2 minor 53 references

Joint audio-video generation models lack robust understanding of physical commonsense, with sharp failures on scene transitions and deliberately inconsistent prompts.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-30 23:37 UTC pith:2JPMMZB4

load-bearing objection The paper gives us a new benchmark that flags real gaps in physical consistency for AV generation models, but the abstract leaves open whether the tests cleanly separate physics from prompt following and scene difficulty. the 2 major comments →

arxiv 2605.07061 v2 pith:2JPMMZB4 submitted 2026-05-08 cs.SD cs.AIcs.CVcs.MM

Do Joint Audio-Video Generation Models Understand Physics?

Zijun Cui , Xiulong Liu , Hao Fang , Mingwei Xu , Jiageng Liu , Zexin Xu , Weiguo Pian , Shijian Deng
show 3 more authors
Feiyu Du Chenming Ge Yapeng Tian
This is my paper
classification cs.SD cs.AIcs.CVcs.MM
keywords joint audio-video generationphysical commonsensebenchmark evaluationscene transitionscross-modal consistencyanti-physics promptsmultimodal generation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces a benchmark to test whether these models generate audio and video that obey real-world physics or just produce plausible but inconsistent outputs. It evaluates models on steady scenes, event-driven changes, environment-driven changes, and prompts that request impossible audio-video pairings. Results show that even the strongest model performs poorly overall, with particular drops when transitions or cross-modal consistency are required. A reader would care because these systems are nearing production quality yet still violate basic physical rules that matter for reliable generation in media or simulation.

Core claim

Across proprietary and open-source joint audio-video models, performance on physical commonsense remains limited: the best model excels on steady-state scenes but all systems degrade sharply on event and environment transitions and collapse when asked to generate physically inconsistent audio-video behavior. The benchmark separates semantic adherence from physical commonsense in both modalities and across modalities, revealing cross-modal consistency and transition dynamics as persistent gaps.

What carries the argument

AV-Phys Bench, which organizes test cases into Steady State, Event Transition, and Environment Transition scenes plus Anti-AV-Physics prompts, scored along five dimensions of semantic and physical adherence.

Load-bearing premise

The chosen scene categories and five evaluation dimensions isolate physical commonsense rather than prompt following or visual quality alone.

What would settle it

A model that maintains high physical commonsense scores on every Anti-AV-Physics prompt and every transition scene while human raters confirm the outputs would contradict the claim of lacking robust understanding.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Event-driven and environment-driven transitions expose the largest gaps in current models.
  • Cross-modal physical consistency must be treated as a distinct training objective.
  • Anti-AV-Physics prompts serve as an effective stress test that even leading systems fail.
  • The ReAct-style agent evaluator produces rankings aligned with human judgment and can scale assessment.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Models may be relying on statistical patterns of co-occurrence rather than causal physical rules.
  • Adding explicit physics constraints or simulation feedback during training could close the transition gap.
  • The benchmark categories could be reused to test whether video-only or audio-only models exhibit similar physical deficits.
  • Persistent failures on impossible prompts suggest current architectures lack mechanisms to reject physically invalid requests.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper introduces AV-Phys Bench, a benchmark evaluating physical commonsense in joint audio-video generation models across three scene categories (Steady State, Event Transition, Environment Transition) plus Anti-AV-Physics prompts. Generations are scored on five dimensions (visual/audio semantic adherence, visual/audio physical commonsense, cross-modal physical commonsense). Results across seven models show Seedance 2.0 performing best overall, but all models exhibit sharp drops on transitions and collapse on anti-physics prompts; an AV-Phys Agent (ReAct-style multimodal LLM + acoustic tools) is proposed whose rankings align with human ratings. The central claim is that current models lack robust audio-visual physical understanding.

Significance. If the benchmark validly isolates physical commonsense, the work identifies cross-modal consistency and transition dynamics as open challenges and supplies a reproducible automated evaluator. The empirical nature (no free parameters or derivations) makes the contribution rest entirely on the benchmark's construct validity and the reported performance gaps.

major comments (2)
  1. [Benchmark design and evaluation dimensions] Benchmark design (abstract and § on AV-Phys Bench): the claim that performance drops on Event/Environment Transition scenes and Anti-AV-Physics prompts demonstrate lack of physical understanding assumes the five dimensions cleanly separate physical commonsense from semantic adherence and general generation difficulty. No correlation analysis, ablation on prompt complexity, or orthogonality test is described to support this separation; drops could instead reflect greater difficulty generating multi-step or dynamic content.
  2. [AV-Phys Agent and human evaluation] Results and human alignment (AV-Phys Agent section): the assertion that the agent 'closely align[s] with human ratings' is load-bearing for trusting automated scores, yet the manuscript provides no quantitative agreement metric (e.g., Cohen's kappa, Pearson r), number of rated samples, or inter-rater reliability among humans. Without these, the reported model rankings cannot be confidently attributed to the intended physical dimensions.
minor comments (2)
  1. [AV-Phys Bench description] The abstract states 'physics-grounded subcategories drawn from real-world scenes' but does not list the subcategories or their selection criteria; adding an explicit table or appendix would improve reproducibility.
  2. [Evaluation protocol] Dataset statistics (number of prompts per category, total generations evaluated, model versions and sampling parameters) are referenced only at high level; these details belong in a dedicated table or §3.1 for a benchmark paper.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments on benchmark construct validity and the need for quantitative human agreement metrics. We address each major comment below.

read point-by-point responses

  1. Referee: [Benchmark design and evaluation dimensions] Benchmark design (abstract and § on AV-Phys Bench): the claim that performance drops on Event/Environment Transition scenes and Anti-AV-Physics prompts demonstrate lack of physical understanding assumes the five dimensions cleanly separate physical commonsense from semantic adherence and general generation difficulty. No correlation analysis, ablation on prompt complexity, or orthogonality test is described to support this separation; drops could instead reflect greater difficulty generating multi-step or dynamic content.

    Authors: We agree that the manuscript does not include explicit correlation analysis or ablations to demonstrate orthogonality of the five dimensions. While the Anti-AV-Physics prompts are constructed to isolate physical violations, additional evidence would strengthen the interpretation. In revision we will add a correlation matrix across dimensions and a prompt-complexity ablation to better isolate physical commonsense from general generation difficulty. revision: yes

  2. Referee: [AV-Phys Agent and human evaluation] Results and human alignment (AV-Phys Agent section): the assertion that the agent 'closely align[s] with human ratings' is load-bearing for trusting automated scores, yet the manuscript provides no quantitative agreement metric (e.g., Cohen's kappa, Pearson r), number of rated samples, or inter-rater reliability among humans. Without these, the reported model rankings cannot be confidently attributed to the intended physical dimensions.

    Authors: We acknowledge that the current manuscript reports only qualitative alignment and omits quantitative metrics. In the revised version we will report the number of rated samples, Cohen's kappa, Pearson correlation, and inter-rater reliability to quantify agreement between the AV-Phys Agent and human ratings. revision: yes

Circularity Check

0 steps flagged

No circularity: purely empirical benchmark with no derivations or self-referential reductions

full rationale

The paper introduces AV-Phys Bench with scene categories (Steady State, Event Transition, Environment Transition), Anti-AV-Physics prompts, five evaluation dimensions, and AV-Phys Agent as a ReAct-style evaluator. All central claims rest on observed performance numbers across models, with no equations, fitted parameters, predictions, or derivations that reduce to inputs by construction. No self-citations are load-bearing for any mathematical result, and the work contains no ansatzes, uniqueness theorems, or renamings of known results. This is a standard empirical benchmark study whose validity concerns (e.g., orthogonality of dimensions) fall under correctness rather than circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 2 invented entities

The central claim rests on the validity of the newly introduced benchmark categories and evaluation dimensions as measures of physical commonsense; these are domain assumptions without independent prior evidence.

axioms (1)
  • domain assumption The three scene categories and physics-grounded subcategories drawn from real-world scenes accurately represent physical commonsense requirements for audio-video generation.
    This premise defines the test cases used to evaluate all models.
invented entities (2)
  • AV-Phys Bench no independent evidence
    purpose: Benchmark for evaluating physical commonsense in joint audio-video generation across steady, event, and environment transitions.
    Newly constructed test set introduced in the paper.
  • AV-Phys Agent no independent evidence
    purpose: ReAct-style evaluator that combines multimodal language model with deterministic acoustic measurement tools.
    New automated evaluation system introduced to align with human ratings.

pith-pipeline@v0.9.1-grok · 5793 in / 1433 out tokens · 30369 ms · 2026-06-30T23:37:46.757378+00:00 · methodology

0 comments
read the original abstract

Joint audio-video generation models are rapidly approaching professional production quality, raising a central question: do they understand audio-visual physics, or merely generate plausible sounds and frames that violate real-world consistency? We introduce AV-Phys Bench, a benchmark for evaluating physical commonsense in joint audio-video generation. AV-Phys Bench tests models across three scene categories: Steady State, Event Transition, and Environment Transition. It covers physics-grounded subcategories drawn from real-world scenes, plus Anti-AV-Physics prompts that deliberately request physically inconsistent audio-video behavior. Each generation is evaluated along five dimensions: visual semantic adherence, audio semantic adherence, visual physical commonsense, audio physical commonsense, and cross-modal physical commonsense. Across three proprietary and four open-source models, we find that Seedance 2.0 performs best overall, but all models remain far from robust physical understanding. Performance drops sharply on event-driven and environment-driven transitions, and even strong proprietary systems collapse on Anti-AV-Physics prompts. We further introduce AV-Phys Agent, a ReAct-style evaluator that combines a multimodal language model with deterministic acoustic measurement tools, producing rankings that closely align with human ratings. Our results identify cross-modal physical consistency and transition-driven scene dynamics as key open challenges for joint audio-video generation.

Figures

Figures reproduced from arXiv: 2605.07061 by Chenming Ge, Feiyu Du, Hao Fang, Jiageng Liu, Mingwei Xu, Shijian Deng, Weiguo Pian, Xiulong Liu, Yapeng Tian, Zexin Xu, Zijun Cui.

Figure 1
Figure 1. Figure 1: In the physical world, vision and sound are two observations of the same physical event. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: AV-Phys Bench construction and evaluation pipeline. (a) A physics-grounded taxonomy organizes prompts by how the underlying physics evolve within a clip. Human-in-the-loop curation produces prompts that encode specific, verifiable acoustic outcomes, and each prompt is paired with a five-dimension evaluation rubric covering semantic adherence (SA) and physical commonsense (PC) across video (V), audio (A), a… view at source ↗
Figure 3
Figure 3. Figure 3: AV-Phys Bench’s three scene categories of physics-following prompts, with a per-category [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Worked rubric example for a Category-3 Environment Transition prompt. Top: four frames from the Seedance 2.0 generation showing the outdoor-to-indoor transition through a heavy door. Bottom: the prompt-specific rubric used for evaluation. V-SA and A-SA check whether the described visual entities and sounds are present. V-PC, A-PC, and AV-PC check whether the generated video, audio, and their alignment obey… view at source ↗
Figure 5
Figure 5. Figure 5: Seedance 2.0, Event Transition. The clip strikes the xylophone bars from the longest to the [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Kling 3.0 Omni, Environment Transition. The clip captures the speaker submerging mid [PITH_FULL_IMAGE:figures/full_fig_p015_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Veo 3.1, Event Transition. The clip places a large dog on the left and a small dog on the [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: LTX-2.3, Environment Transition. The clip cuts from a packed stadium interior to an [PITH_FULL_IMAGE:figures/full_fig_p016_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Ovi, Environment Transition. The fire truck is visible only in the first frame and then [PITH_FULL_IMAGE:figures/full_fig_p016_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: JavisDiT++, Event Transition. The clip shows a person knocking gently three times and [PITH_FULL_IMAGE:figures/full_fig_p017_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: MagiHuman, Event Transition. The clip shows a guitarist plucking a string but never [PITH_FULL_IMAGE:figures/full_fig_p017_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Seedance 2.0, Steady State Anti-Physics. The model paints the gym floor with a thin layer [PITH_FULL_IMAGE:figures/full_fig_p018_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Seedance 2.0, Event Transition Anti-Physics. With the handheld microphone held close [PITH_FULL_IMAGE:figures/full_fig_p019_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Seedance 2.0, Environment Transition Anti-Physics. Before the helmet, the voice is clear [PITH_FULL_IMAGE:figures/full_fig_p019_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: The human evaluation interface used to collect the labels in Section 3.3. The header [PITH_FULL_IMAGE:figures/full_fig_p022_15.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

53 extracted references · 53 canonical work pages · 18 internal anchors

  1. [1]

    Cosmos World Foundation Model Platform for Physical AI

    Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, Yifan Ding, et al. Cosmos world foundation model platform for physical ai.arXiv preprint arXiv:2501.03575, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  2. [2]

    VideoPhy: Evaluating Physical Commonsense for Video Generation

    Hritik Bansal, Zongyu Lin, Tianyi Xie, Zeshun Zong, Michal Yarom, Yonatan Bitton, Chenfanfu Jiang, Yizhou Sun, Kai-Wei Chang, and Aditya Grover. Videophy: Evaluating physical commonsense for video generation.arXiv preprint arXiv:2406.03520, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  3. [3]

    arXiv preprint arXiv:2503.06800 (2025)

    Hritik Bansal, Clark Peng, Yonatan Bitton, Roman Goldenberg, Aditya Grover, and Kai-Wei Chang. Videophy-2: A challenging action-centric physical commonsense evaluation in video generation.arXiv preprint arXiv:2503.06800, 2025

    work page arXiv 2025

  4. [4]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Leo Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, et al. Video generation models as world simulators. OpenAI Blog, 1(8):1, 2024

    work page 2024

  5. [5]

    Genie: Generative interactive environments

    Jake Bruce, Michael D Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, et al. Genie: Generative interactive environments. InForty-first International Conference on Machine Learning, 2024

    work page 2024

  6. [6]

    T2AV-Compass: Towards Unified Evaluation for Text-to-Audio-Video Generation

    Zhe Cao, Tao Wang, Jiaming Wang, Yanghai Wang, Yuanxing Zhang, Jialu Chen, Miao Deng, Jiahao Wang, Yubin Guo, Chenxi Liao, et al. T2av-compass: Towards unified evaluation for text-to-audio-video generation.arXiv preprint arXiv:2512.21094, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  7. [7]

    Soundspaces 2.0: A simulation platform for visual-acoustic learning.Advances in Neural Information Processing Systems, 35:8896– 8911, 2022

    Changan Chen, Carl Schissler, Sanchit Garg, Philip Kobernik, Alexander Clegg, Paul Calamia, Dhruv Batra, Philip Robinson, and Kristen Grauman. Soundspaces 2.0: A simulation platform for visual-acoustic learning.Advances in Neural Information Processing Systems, 35:8896– 8911, 2022

    work page 2022

  8. [8]

    Savvy: Spatial awareness via audio-visual llms through seeing and hearing.arXiv preprint arXiv:2506.05414, 2025

    Mingfei Chen, Zijun Cui, Xiulong Liu, Jinlin Xiang, Caleb Zheng, Jingyuan Li, and Eli Shlizerman. Savvy: Spatial awareness via audio-visual llms through seeing and hearing.arXiv preprint arXiv:2506.05414, 2025

    work page arXiv 2025

  9. [9]

    arXiv preprint arXiv:2603.21986 , year=

    Ethan Chern, Hansi Teng, Hanwen Sun, Hao Wang, Hong Pan, Hongyu Jia, Jiadi Su, Jin Li, Junjie Yu, Lijie Liu, et al. Speed by simplicity: A single-stream architecture for fast audio-video generative foundation model.arXiv preprint arXiv:2603.21986, 2026

    work page arXiv 2026

  10. [10]

    Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities

    Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  11. [11]

    Introducing Veo 3.1 and advanced capa- bilities in Flow

    Jess Gallegos, Thomas Iljic, and Google DeepMind. Introducing Veo 3.1 and advanced capa- bilities in Flow. https://blog.google/technology/ai/veo-updates-flow/, October

    work page

  12. [12]

    Technical details inherited from the Veo 3 Tech Re- port,https://storage.googleapis.com/deepmind-media/veo/Veo-3-Tech-Report

    Google Blog, October 15, 2025. Technical details inherited from the Veo 3 Tech Re- port,https://storage.googleapis.com/deepmind-media/veo/Veo-3-Tech-Report. pdf

    work page 2025

  13. [13]

    Look, listen, and act: Towards audio-visual embodied navigation

    Chuang Gan, Yiwei Zhang, Jiajun Wu, Boqing Gong, and Joshua B Tenenbaum. Look, listen, and act: Towards audio-visual embodied navigation. In2020 IEEE International Conference on Robotics and Automation (ICRA), pages 9701–9707. IEEE, 2020

    work page 2020

  14. [14]

    "PhyWorldBench": A Comprehensive Evaluation of Physical Realism in Text-to-Video Models

    Jing Gu, Xian Liu, Yu Zeng, Ashwin Nagarajan, Fangrui Zhu, Daniel Hong, Yue Fan, Qianqi Yan, Kaiwen Zhou, Ming-Yu Liu, et al. " phyworldbench": A comprehensive evaluation of physical realism in text-to-video models.arXiv preprint arXiv:2507.13428, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  15. [15]

    2505.00337 , archivePrefix =

    Xuyang Guo, Jiayan Huo, Zhenmei Shi, Zhao Song, Jiahao Zhang, and Jiale Zhao. T2vphysbench: A first-principles benchmark for physical consistency in text-to-video gen- eration.arXiv preprint arXiv:2505.00337, 2025. 10

    work page arXiv 2025

  16. [16]

    LTX-2: Efficient Joint Audio-Visual Foundation Model

    Yoav HaCohen, Benny Brazowski, Nisan Chiprut, Yaki Bitterman, Andrew Kvochko, Avishai Berkowitz, Daniel Shalem, Daphna Lifschitz, Dudu Moshe, Eitan Porat, et al. Ltx-2: Efficient joint audio-visual foundation model.arXiv preprint arXiv:2601.03233, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  17. [17]

    Videoscore: Building automatic metrics to simulate fine-grained human feedback for video generation

    Xuan He, Dongfu Jiang, Ge Zhang, Max Ku, Achint Soni, Sherman Siu, Haonan Chen, Abhranil Chandra, Ziyan Jiang, Aaran Arulraj, et al. Videoscore: Building automatic metrics to simulate fine-grained human feedback for video generation. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 2105–2123, 2024

    work page 2024

  18. [18]

    Clipscore: A reference-free evaluation metric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning. InProceedings of the 2021 conference on empirical methods in natural language processing, pages 7514–7528, 2021

    work page 2021

  19. [19]

    VABench: A Comprehensive Benchmark for Audio-Video Generation

    Daili Hua, Xizhi Wang, Bohan Zeng, Xinyi Huang, Hao Liang, Junbo Niu, Xinlong Chen, Quanqing Xu, and Wentao Zhang. Vabench: A comprehensive benchmark for audio-video generation.arXiv preprint arXiv:2512.09299, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  20. [20]

    Vbench: Comprehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21807–21818, 2024

    work page 2024

  21. [21]

    A reference-free metric for evaluating music enhancement algorithms

    K Kilgour, M Zuluaga, D Roblek, and M Sharifi. A reference-free metric for evaluating music enhancement algorithms. Interspeech, 2019

    work page 2019

  22. [22]

    The measurement of observer agreement for categorical data.biometrics, pages 159–174, 1977

    J Richard Landis and Gary G Koch. The measurement of observer agreement for categorical data.biometrics, pages 159–174, 1977

    work page 1977

  23. [23]

    Video generation models: A survey of post-training and alignment

    Chaoyu Li, Xiaoyi Gu, Yogesh Kulkarni, Eun Woo Im, Mohammadmahdi Honarmand, Zeyu Wang, Juntong Song, Fei Du, Xilin Jiang, Kexin Zheng, et al. Video generation models: A survey of post-training and alignment. 2026

    work page 2026

  24. [24]

    arXiv preprint arXiv:2503.23377 (2025)

    Kai Liu, Wei Li, Lai Chen, Shengqiong Wu, Yanhao Zheng, Jiayi Ji, Fan Zhou, Jiebo Luo, Ziwei Liu, Hao Fei, et al. Javisdit: Joint audio-video diffusion transformer with hierarchical spatio-temporal prior synchronization.arXiv preprint arXiv:2503.23377, 2025

    work page arXiv 2025

  25. [25]

    Javisdit++: Unified modeling and optimization for joint audio-video generation

    Kai Liu, Yanhao Zheng, Kai Wang, Shengqiong Wu, Rongjunchen Zhang, Jiebo Luo, Dimitrios Hatzinakos, Ziwei Liu, Hao Fei, and Tat-Seng Chua. Javisdit++: Unified modeling and optimization for joint audio-video generation.arXiv preprint arXiv:2602.19163, 2026

    work page arXiv 2026

  26. [26]

    Caven: An embodied conversational agent for efficient audio-visual navigation in noisy environments

    Xiulong Liu, Sudipta Paul, Moitreya Chatterjee, and Anoop Cherian. Caven: An embodied conversational agent for efficient audio-visual navigation in noisy environments. InProceedings of the AAAI conference on artificial intelligence, volume 38, pages 3765–3773, 2024

    work page 2024

  27. [27]

    Tell what you hear from what you see-video to audio generation through text.Advances in Neural Information Processing Systems, 37:101337– 101366, 2024

    Xiulong Liu, Kun Su, and Eli Shlizerman. Tell what you hear from what you see-video to audio generation through text.Advances in Neural Information Processing Systems, 37:101337– 101366, 2024

    work page 2024

  28. [28]

    Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generation

    Chetwin Low, Weimin Wang, and Calder Katyal. Ovi: Twin backbone cross-modal fusion for audio-video generation.arXiv preprint arXiv:2510.01284, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  29. [29]

    Tavgbench: Benchmarking text to audible-video generation

    Yuxin Mao, Xuyang Shen, Jing Zhang, Zhen Qin, Jinxing Zhou, Mochu Xiang, Yiran Zhong, and Yuchao Dai. Tavgbench: Benchmarking text to audible-video generation. InProceedings of the 32nd ACM International Conference on Multimedia, pages 6607–6616, 2024

    work page 2024

  30. [30]

    Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation

    Fanqing Meng, Jiaqi Liao, Xinyu Tan, Wenqi Shao, Quanfeng Lu, Kaipeng Zhang, Yu Cheng, Dianqi Li, Yu Qiao, and Ping Luo. Towards world simulator: Crafting physical commonsense- based benchmark for video generation.arXiv preprint arXiv:2410.05363, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  31. [31]

    Do gener- ative video models understand physical principles? InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 948–958, 2026

    Saman Motamed, Laura Culp, Kevin Swersky, Priyank Jaini, and Robert Geirhos. Do gener- ative video models understand physical principles? InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 948–958, 2026

    work page 2026

  32. [32]

    Sora 2, 2025

    OpenAI. Sora 2, 2025. Accessed: 2026-05-04. 11

    work page 2025

  33. [33]

    OmniSonic: Towards Universal and Holistic Audio Generation from Video and Text

    Weiguo Pian, Saksham Singh Kushwaha, Zhimin Chen, Shijian Deng, Kai Wang, Yunhui Guo, and Yapeng Tian. Omnisonic: Towards universal and holistic audio generation from video and text.arXiv preprint arXiv:2604.04348, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  34. [34]

    Seedance 2.0: Advancing Video Generation for World Complexity

    Team Seedance, De Chen, Liyang Chen, Xin Chen, Ying Chen, Zhuo Chen, Zhuowei Chen, Feng Cheng, Tianheng Cheng, Yufeng Cheng, et al. Seedance 2.0: Advancing video generation for world complexity.arXiv preprint arXiv:2604.14148, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  35. [35]

    Savgbench: Benchmarking spatially aligned audio-video generation

    Kazuki Shimada, Christian Simon, Takashi Shibuya, Shusuke Takahashi, and Yuki Mitsufuji. Savgbench: Benchmarking spatially aligned audio-video generation. InICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 11977–11981. IEEE, 2026

    work page 2026

  36. [36]

    OpenAI GPT-5 System Card

    Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, et al. Openai gpt-5 system card.arXiv preprint arXiv:2601.03267, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  37. [37]

    From vision to audio and beyond: A unified model for audio-visual represen- tation and generation,

    Kun Su, Xiulong Liu, and Eli Shlizerman. From vision to audio and beyond: A unified model for audio-visual representation and generation.arXiv preprint arXiv:2409.19132, 2024

    work page arXiv 2024

  38. [38]

    T2v- compbench: A comprehensive benchmark for compositional text-to-video generation

    Kaiyue Sun, Kaiyi Huang, Xian Liu, Yue Wu, Zihan Xu, Zhenguo Li, and Xihui Liu. T2v- compbench: A comprehensive benchmark for compositional text-to-video generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 8406–8416, 2025

    work page 2025

  39. [39]

    Sonicbench: Dissecting the physical perception bottleneck in large audio language models.arXiv preprint arXiv:2601.11039, 2026

    Yirong Sun, Yanjun Chen, Xin Qiu, Gang Zhang, Hongyu Chen, Daokuan Wu, Chengming Li, Min Yang, Dawei Zhu, Wei Zhang, et al. Sonicbench: Dissecting the physical perception bottleneck in large audio language models.arXiv preprint arXiv:2601.11039, 2026

    work page arXiv 2026

  40. [40]

    Kling-Omni Technical Report

    Kling Team, Jialu Chen, Yuanzheng Ci, Xiangyu Du, Zipeng Feng, Kun Gai, Sainan Guo, Feng Han, Jingbin He, Kang He, et al. Kling-omni technical report.arXiv preprint arXiv:2512.16776, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  41. [41]

    Qwen Team. Qwen3. 5-omni technical report.arXiv preprint arXiv:2604.15804, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  42. [42]

    Towards Accurate Generative Models of Video: A New Metric & Challenges

    Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717, 2018

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  43. [43]

    UniVerse-1: Unified Audio-Video Generation via Stitching of Experts

    Duomin Wang, Wei Zuo, Aojie Li, Ling-Hao Chen, Xinyao Liao, Deyu Zhou, Zixin Yin, Xili Dai, Daxin Jiang, and Gang Yu. Universe-1: Unified audio-video generation via stitching of experts.arXiv preprint arXiv:2509.06155, 2025

    work page Pith review arXiv 2025

  44. [44]

    PhyAVBench: A Challenging Audio Physics-Sensitivity Benchmark for Physically Grounded Text-to-Audio-Video Generation

    Tianxin Xie, Wentao Lei, Kai Jiang, Guanjie Huang, Pengfei Zhang, Chunhui Zhang, Fengji Ma, Haoyu He, Han Zhang, Jiangshan He, et al. Phyavbench: A challenging audio physics- sensitivity benchmark for physically grounded text-to-audio-video generation.arXiv preprint arXiv:2512.23994, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  45. [45]

    A survey on video diffusion models.ACM Computing Surveys, 57(2):1–42, 2024

    Zhen Xing, Qijun Feng, Haoran Chen, Qi Dai, Han Hu, Hang Xu, Zuxuan Wu, and Yu-Gang Jiang. A survey on video diffusion models.ACM Computing Surveys, 57(2):1–42, 2024

    work page 2024

  46. [46]

    A Systematic Post-Train Framework for Video Generation

    Zeyue Xue, Siming Fu, Jie Huang, Shuai Lu, Haoran Li, Yijun Liu, Yuming Li, Xiaoxuan He, Mengzhao Chen, Haoyang Huang, et al. A systematic post-train framework for video generation.arXiv preprint arXiv:2604.25427, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  47. [47]

    ReAct: Synergizing Reasoning and Acting in Language Models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models.arXiv preprint arXiv:2210.03629, 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  48. [48]

    Diverse and aligned audio-to-video generation via text-to-video model adaptation

    Guy Yariv, Itai Gat, Sagie Benaim, Lior Wolf, Idan Schwartz, and Yossi Adi. Diverse and aligned audio-to-video generation via text-to-video model adaptation. InProceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 6639–6647, 2024

    work page 2024

  49. [49]

    A mallet strikes a xylophone from the longest bar to the shortest. Each bar rings higher than the one before, from deep warm tones to bright high notes

    Juan Zhang, Jiahao Chen, Cheng Wang, Zhiwang Yu, Tangquan Qi, Can Liu, and Di Wu. Virbo: Multimodal multilingual avatar video generation in digital marketing.arXiv preprint arXiv:2403.11700, 2024. 12 A Broader Impact and Limitations Broader impact.A V-Phys Bench provides the first systematic diagnostic for where joint audio-video models fail on physics, o...

    work page arXiv 2024

  50. [50]

    video_sa.objects— Are all of the following visually present in the clip: {video.objects}? Answer Yes or No

    work page

  51. [51]

    {video.event}

    video_sa.event— Is the event “{video.event}” visually depicted in the clip? Answer Yes or No

    work page

  52. [52]

    would normally be audible if real-world physics held; answer Yes if they are appropriately represented as such (typically silent here)

    audio_sa.objects— Are the sound source(s) {audio.objects} audible in the clip?(when silence_expected: “would normally be audible if real-world physics held; answer Yes if they are appropriately represented as such (typically silent here)”)

    work page

  53. [53]

    the clip is expected to be silent during the depicted event; answer Yes if it is appropriately silent throughout with no audible leak-through

    audio_sa.sound— Is the sound {audio.sound} clearly audible in the clip?(when silence_expected: “the clip is expected to be silent during the depicted event; answer Yes if it is appropriately silent throughout with no audible leak-through”) Key Standards (Physical Commonsense) Check whether each of the following physics statements is true of the clip. Answ...

    work page arXiv