pith. machine review for the scientific record. sign in

arxiv: 2605.04515 · v1 · submitted 2026-05-06 · 💻 cs.CV

Recognition: unknown

From Priors to Perception: Grounding Video-LLMs in Physical Reality

Zicheng Zhao , Chaofan Gan , Shijie Li , Weiyao Lin
Authors on Pith no claims yet

Pith reviewed 2026-05-08 17:37 UTC · model grok-4.3

classification 💻 cs.CV
keywords video large language modelsphysical reasoningsemantic prior dominanceadversarial curriculumvisual groundingLoRA fine-tuningmultimodal models
0
0 comments X

The pith

Video-LLMs fail physical reasoning because internal narrative priors override visual facts, but fine-tuning on a new adversarial video curriculum corrects the interference without architectural changes.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that Video Large Language Models exhibit systematic deficits in fine-grained physical reasoning because their reasoning processes are hijacked by Semantic Prior Dominance, where pre-trained narrative scripts and statistical expectations take precedence over actual visual evidence in the video. This explains failures not only on physically impossible scenes but also on counter-intuitive cases where the visuals contradict common priors. To isolate the issue, the authors introduce the Programmatic Adversarial Curriculum, a dataset of videos generated directly from physical laws that separates visual perception errors from logical reasoning errors, along with a Visual-Anchored Reasoning Chain that requires models to reference low-level visual details before making judgments. Experiments show that applying standard LoRA fine-tuning to this curriculum on existing state-of-the-art models produces large gains in physical reasoning benchmarks. A sympathetic reader would care because the work reframes the problem as addressable through targeted data and training rather than fundamental model redesign.

Core claim

According to the Unified Attribution Theory, the dual failure modes in anti-physics anomalies and counter-intuitive scenarios where visual facts contradict expectations both arise from Semantic Prior Dominance, in which the model's reasoning mechanism is deeply hijacked by internal narrative scripts rather than from any deficiency in perceiving the video content itself. The Programmatic Adversarial Curriculum supplies high-fidelity adversarial videos synthesized from physical laws to thoroughly decouple visual artifacts from logical errors, while the Visual-Anchored Reasoning Chain forces explicit grounding in low-level visual facts prior to any logical adjudication. Standard LoRA fine-tuned

What carries the argument

Semantic Prior Dominance, defined as the reasoning mechanism being hijacked by internal narrative scripts that override visual evidence from the input video.

If this is right

  • Standard parameter-efficient fine-tuning on physically grounded adversarial data can neutralize prior interference across multiple state-of-the-art Video-LLMs.
  • Forcing explicit visual anchoring before logical steps improves grounding without requiring new model architectures.
  • Adversarial curricula synthesized from physical laws can systematically expose and correct narrative biases that standard training data leaves untouched.
  • Improvements in physical reasoning generalize to both impossible and counter-intuitive scenarios once priors are decoupled from visuals.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar curriculum-based fine-tuning could be tested on other multimodal models to check whether prior dominance affects non-video reasoning domains.
  • The decoupling method in PACC offers a way to create separate benchmarks that measure pure visual perception accuracy versus higher-level physical inference.
  • If the approach scales, it implies that many apparent capability gaps in large models may be mitigated by targeted data rather than scale alone.

Load-bearing premise

The assumption that the models' failures stem specifically from semantic priors overriding perception rather than from an inability to accurately perceive or describe the visual content in the first place, and that the PACC dataset successfully isolates those two sources of error.

What would settle it

If models trained on the PACC curriculum still show no improvement on physical reasoning benchmarks, or if they continue to make logical errors even after correctly describing the low-level visual facts in the same videos, the attribution to prior dominance would be falsified.

Figures

Figures reproduced from arXiv: 2605.04515 by Chaofan Gan, Shijie Li, Weiyao Lin, Zicheng Zhao.

Figure 1
Figure 1. Figure 1: Illustration of Semantic Prior Dominance. (a) SOTA models (e.g., GPT-4o [26]) exhibit dual failures: logical priors drive forced rationalizations in anti-physics scenarios, while statistical priors trigger stereotypical outputs in counter-intuitive cases. Pilot studies (VideoLLaMA3 [43]) rule out knowledge or perception deficits: (b) injecting physical knowledge exacerbates erroneous rationalizations; (c) … view at source ↗
Figure 2
Figure 2. Figure 2: Taxonomy of the PACC dataset. The taxonomy encompasses two adversarial streams across eight fine-grained dimensions. We show only the negative counterpart of each adversarial pair, with red markers highlighting both physical violations and counter-intuitive scenarios. rationalization fallacies: by injecting high-quality physical disruptions into the positive anchor, it forces reasoning based on objective v… view at source ↗
Figure 3
Figure 3. Figure 3: The unified PACC dataset construction pipeline. The HITL paradigm comprises four stages: (1) Selection & Clipping to curate positive samples with explicit physical interactions; (2) Visual Fact Anchoring via MLLMs and expert purification; (3) Adversarial Generation & Label Synthesis, which synthesizes negative counterparts via deterministic manipulation (manual CV editing) or generative synthesis (AI gener… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative comparison on the PACC dataset. The baseline fails to detect the physical violation (top) and hallucinates an expected outcome (bottom). PhyAR consistently anchors its reasoning on objective visual facts view at source ↗
Figure 5
Figure 5. Figure 5: A real qualitative example from the PACC dataset illustrating the Constraint Violation sub-category. The figure displays a paired positive and negative video sample alongside their corresponding VARC labels. In the negative sample, the absence of a shadow despite a strong directional light violates fundamental optical laws and material properties (Property Mismatch). This structured format forces models to… view at source ↗
read the original abstract

While Video Large Language Models (Video-LLMs) excel in general understanding, they exhibit systematic deficits in fine-grained physical reasoning. Existing interventions not only suffer from limited generalization but fundamentally conflate generative artifacts with genuine physical fallacies. Furthermore, we find that models fail systematically not only in anti-physics anomalies but also in counter-intuitive scenarios where visual facts contradict statistical expectations. Accordingly, we propose the Unified Attribution Theory: this dual failure stems not from perception deficiency, but from Semantic Prior Dominance -- the reasoning mechanism is deeply hijacked by internal narrative scripts. To address this, we construct the Programmatic Adversarial Curriculum (PACC), the first high-fidelity adversarial video dataset synthesized based on physical laws, thoroughly decoupling visual artifacts from logical errors. Concurrently, we design the Visual-Anchored Reasoning Chain (VARC) to force models to explicitly ground their judgments in low-level visual facts prior to logical adjudication. Experiments demonstrate that without invasive architectural modifications, standard LoRA fine-tuning with the PACC curriculum effectively neutralizes prior interference in state-of-the-art (SOTA) models, yielding a substantial leap in physical reasoning capabilities.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 1 minor

Summary. The paper claims that Video-LLMs exhibit systematic deficits in fine-grained physical reasoning due to Semantic Prior Dominance rather than perception deficiencies, as formalized in the Unified Attribution Theory. It introduces the Programmatic Adversarial Curriculum (PACC), a high-fidelity adversarial video dataset synthesized from physical laws to decouple visual artifacts from logical errors, and the Visual-Anchored Reasoning Chain (VARC) to force grounding in low-level visual facts. Standard LoRA fine-tuning with PACC is reported to neutralize prior interference in SOTA models and yield substantial gains in physical reasoning without architectural modifications.

Significance. If the empirical results hold with proper controls and the theory receives independent validation, the work could meaningfully advance methods for mitigating statistical prior interference in multimodal models, offering a non-invasive curriculum-based approach grounded in physical laws. The programmatic synthesis of adversarial data represents a potential strength if shown to enforce genuine visual grounding rather than synthetic regularities.

major comments (3)
  1. [Abstract] Abstract: The abstract asserts 'substantial leap in physical reasoning capabilities' and 'effectively neutralizes prior interference' from LoRA fine-tuning with PACC but supplies no quantitative results, baselines, error bars, or experimental details. This is load-bearing for the central claim, as the gains cannot be assessed or attributed to the proposed mechanism.
  2. [Abstract] Abstract: The Unified Attribution Theory is introduced specifically to explain failures observed in the authors' own tests on anti-physics anomalies and counter-intuitive scenarios, creating a circularity risk without mention of independent external validation, pre-existing literature, or falsifiable predictions outside the PACC/VARC setup.
  3. [Abstract] Abstract: The claim that PACC 'thoroughly decoupl[es] visual artifacts from logical errors' and that VARC 'force[s] models to explicitly ground their judgments in low-level visual facts' lacks any description of controls such as testing on real videos with matched physics, ablating visual fidelity while holding logic constant, or verifying unchanged perception-only probes post-training. This is load-bearing for distinguishing prior neutralization from distribution shift or synthetic-data adaptation.
minor comments (1)
  1. [Abstract] Abstract: New terms including 'Semantic Prior Dominance', 'Programmatic Adversarial Curriculum (PACC)', 'Visual-Anchored Reasoning Chain (VARC)', and 'Unified Attribution Theory' are introduced without immediate formal definitions, mathematical formalization, or citations to related prior work on prior interference in LLMs.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the detailed and constructive feedback. We address each major comment point by point below, providing clarifications from the full manuscript and committing to revisions where appropriate to strengthen the presentation of our claims.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The abstract asserts 'substantial leap in physical reasoning capabilities' and 'effectively neutralizes prior interference' from LoRA fine-tuning with PACC but supplies no quantitative results, baselines, error bars, or experimental details. This is load-bearing for the central claim, as the gains cannot be assessed or attributed to the proposed mechanism.

    Authors: We agree that the abstract would benefit from quantitative anchors for the central claims. The full manuscript (Section 4) reports specific results including average accuracy improvements of 18-27% on physical reasoning benchmarks across SOTA models, comparisons against standard LoRA fine-tuning and non-adversarial curricula as baselines, and standard error bars computed over 5 random seeds with statistical significance tests. In the revision, we will condense these key metrics (e.g., 'yielding 22% average gain with p<0.01') into the abstract while retaining the high-level tone. revision: yes

  2. Referee: [Abstract] Abstract: The Unified Attribution Theory is introduced specifically to explain failures observed in the authors' own tests on anti-physics anomalies and counter-intuitive scenarios, creating a circularity risk without mention of independent external validation, pre-existing literature, or falsifiable predictions outside the PACC/VARC setup.

    Authors: The theory is motivated by patterns in our PACC evaluations but is also supported by pre-existing literature on semantic prior dominance and shortcut learning in multimodal models; we will add explicit citations to relevant prior works in the related work and discussion sections. To reduce circularity, the revised manuscript will articulate falsifiable predictions (e.g., models trained on PACC should show reduced bias on held-out counter-intuitive real-world scenarios) and note that independent validation remains an open direction. We acknowledge this as a partial limitation of the current framing. revision: partial

  3. Referee: [Abstract] Abstract: The claim that PACC 'thoroughly decoupl[es] visual artifacts from logical errors' and that VARC 'force[s] models to explicitly ground their judgments in low-level visual facts' lacks any description of controls such as testing on real videos with matched physics, ablating visual fidelity while holding logic constant, or verifying unchanged perception-only probes post-training. This is load-bearing for distinguishing prior neutralization from distribution shift or synthetic-data adaptation.

    Authors: The full experiments section (5.2-5.4) already includes these controls: evaluation on a matched set of real videos, ablation of visual fidelity (e.g., Gaussian noise and downsampling while preserving logical structure), and post-training assessment on perception-only probes showing no degradation. However, these are not explicitly linked back to the abstract claims. We will add a concise summary of the control experiments to the abstract and introduction, plus a dedicated paragraph detailing the results, to make the distinction from distribution shift clearer. revision: yes

Circularity Check

0 steps flagged

No circularity: hypothesis tested via independent empirical evaluation

full rationale

The paper observes systematic failures in Video-LLMs, proposes Unified Attribution Theory as an explanatory hypothesis attributing them to Semantic Prior Dominance rather than perception gaps, constructs PACC (synthesized from physical laws) and VARC accordingly, then reports experimental results from LoRA fine-tuning showing capability gains. No equations, fitted parameters renamed as predictions, or self-citations appear in the provided text. The central claim rests on empirical demonstration rather than definitional reduction or load-bearing self-reference. The theory functions as a testable attribution, not a self-referential loop where success is guaranteed by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 3 invented entities

The central claims rest on the untested premise that prior dominance is the root cause of physical reasoning failures and on the unverified effectiveness of the newly introduced dataset and reasoning chain.

axioms (1)
  • domain assumption Video-LLMs exhibit systematic deficits in fine-grained physical reasoning that stem from Semantic Prior Dominance rather than perception deficiency.
    Directly stated in the abstract as the foundation for both the diagnosis and the proposed interventions.
invented entities (3)
  • Semantic Prior Dominance no independent evidence
    purpose: Explains why models fail in both anti-physics and counter-intuitive physical scenarios.
    New concept introduced to unify the observed failure modes.
  • Programmatic Adversarial Curriculum (PACC) no independent evidence
    purpose: High-fidelity adversarial video dataset synthesized from physical laws to decouple artifacts from logical errors.
    New dataset constructed specifically for this work.
  • Visual-Anchored Reasoning Chain (VARC) no independent evidence
    purpose: Forces models to explicitly ground judgments in low-level visual facts before logical adjudication.
    New reasoning procedure proposed to counteract prior interference.

pith-pipeline@v0.9.0 · 5509 in / 1434 out tokens · 47342 ms · 2026-05-08T17:37:07.443221+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

56 extracted references · 24 canonical work pages · 9 internal anchors

  1. [1]

    Impossible videos

    Zechen Bai, Hai Ci, and Mike Zheng Shou. Impossible videos. InInternational Conference on Machine Learning, 2025

    2025

  2. [2]

    The acquisition of physical knowledge in infancy: A summary in eight lessons

    Renée Baillargeon. The acquisition of physical knowledge in infancy: A summary in eight lessons. InBlackwell handbook of childhood cognitive development, pages 47–83. Wiley Online Library, 2002

    2002

  3. [3]

    Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Do- minik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023

    work page internal anchor Pith review arXiv 2023

  4. [4]

    Video generation models as world simulators.https: //openai.com/research/video-generation-models-as-world-simulators, 2024

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Leo Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, et al. Video generation models as world simulators.https: //openai.com/research/video-generation-models-as-world-simulators, 2024

    2024

  5. [5]

    Activitynet: A large-scale video benchmark for human activity understanding

    Fabian Caba Heilbron, Victor Escorcia, Bernard Ghanem, and Juan Carlos Niebles. Activitynet: A large-scale video benchmark for human activity understanding. InProceedings of the ieee conference on computer vision and pattern recognition, pages 961–970, 2015

    2015

  6. [6]

    VideoJAM: Joint appearance-motion representations for enhanced motion generation in video models.arXiv preprint arXiv:2502.02492,

    Hila Chefer, Uriel Singer, Amit Zohar, Yuval Kirstain, Adam Polyak, Yaniv Taigman, Lior Wolf, and Shelly Sheynin. Videojam: Joint appearance-motion representations for enhanced motion generation in video models.arXiv preprint arXiv:2502.02492, 2025

    work page arXiv 2025

  7. [7]

    Panda-70m: Captioning 70m videos with multiple cross-modality teachers

    Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Ekaterina Deyneka, Hsiang-wei Chao, Byung Eun Jeon, Yuwei Fang, Hsin-Ying Lee, Jian Ren, Ming-Hsuan Yang, et al. Panda-70m: Captioning 70m videos with multiple cross-modality teachers. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13320–13331, 2024

    2024

  8. [8]

    Longvila: Scaling long-context visual language models for long videos

    Yukang Chen, Fuzhao Xue, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Haotian Tang, Shang Yang, Zhijian Liu, et al. Longvila: Scaling long-context visual language models for long videos. InInternational Conference on Learning Representations, 2025

    2025

  9. [9]

    Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271, 2024

    work page internal anchor Pith review arXiv 2024

  10. [10]

    VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs

    Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, et al. Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms.arXiv preprint arXiv:2406.07476, 2024

    work page internal anchor Pith review arXiv 2024

  11. [11]

    Physbench: Benchmarking and enhancing vision-language models for physical world understanding

    Wei Chow, Jiageng Mao, Boyi Li, Daniel Seita, Vitor Guizilini, and Yue Wang. Physbench: Benchmarking and enhancing vision-language models for physical world understanding.arXiv preprint arXiv:2501.16411, 2025. 10

    work page arXiv 2025

  12. [12]

    Video-R1: Reinforcing Video Reasoning in MLLMs

    Kaituo Feng, Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang, Tianshuo Peng, Junfei Wu, Xiaoying Zhang, Benyou Wang, and Xiangyu Yue. Video-r1: Reinforcing video reasoning in mllms.arXiv preprint arXiv:2503.21776, 2025

    work page internal anchor Pith review arXiv 2025

  13. [13]

    Veo: A text-to-video generation system

    Google DeepMind. Veo: A text-to-video generation system. Technical re- port, Google, 2025. URL https://storage.googleapis.com/deepmind-media/veo/ Veo-3-Tech-Report.pdf

    2025

  14. [14]

    something something

    Raghav Goyal, Samira Ebrahimi Kahou, Vincent Michalski, Joanna Materzynska, Susanne Westphal, Heuna Kim, Valentin Haenel, Ingo Fruend, Peter Yianilos, Moritz Mueller-Freitag, et al. The" something something" video database for learning and evaluating visual common sense. InProceedings of the IEEE international conference on computer vision, pages 5842– 5850, 2017

    2017

  15. [15]

    Motionbench: Benchmarking and improving fine-grained video motion understanding for vision language models

    Wenyi Hong, Yean Cheng, Zhuoyi Yang, Weihan Wang, Lefan Wang, Xiaotao Gu, Shiyu Huang, Yuxiao Dong, and Jie Tang. Motionbench: Benchmarking and improving fine-grained video motion understanding for vision language models. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 8450–8460, 2025

    2025

  16. [16]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phil Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. InInternational Conference on Learning Representations, 2022

    2022

  17. [17]

    Noah: Benchmarking narrative prior driven hallucination and omission in video large language models.arXiv preprint arXiv:2511.06475, 2025

    Kyuho Lee, Euntae Kim, Jinwoo Choi, and Buru Chang. Noah: Benchmarking narrative prior driven hallucination and omission in video large language models.arXiv preprint arXiv:2511.06475, 2025

    work page arXiv 2025

  18. [18]

    arXiv preprint arXiv:2502.20694 (2025)

    Dacheng Li, Yunhao Fang, Yukang Chen, Shuo Yang, Shiyi Cao, Justin Wong, Michael Luo, Xiaolong Wang, Hongxu Yin, Joseph E Gonzalez, et al. Worldmodelbench: Judging video generation models as world models.arXiv preprint arXiv:2502.20694, 2025

    work page arXiv 2025

  19. [19]

    arXiv preprint arXiv:2504.06958 (2025)

    Xinhao Li, Ziang Yan, Desen Meng, Lu Dong, Xiangyu Zeng, Yinan He, Yali Wang, Yu Qiao, Yi Wang, and Limin Wang. Videochat-r1: Enhancing spatio-temporal perception via reinforce- ment fine-tuning.arXiv preprint arXiv:2504.06958, 2025

    work page arXiv 2025

  20. [20]

    Videohallu: Evaluating and mitigating multi-modal hallucinations on synthetic video understanding

    Zongxia Li, Xiyang Wu, Guangyao Shi, Yubin Qin, Hongyang Du, Fuxiao Liu, Tianyi Zhou, Dinesh Manocha, and Jordan Lee Boyd-Graber. Videohallu: Evaluating and mitigating multi- modal hallucinations on synthetic video understanding.arXiv preprint arXiv:2505.01481, 2025

    work page arXiv 2025

  21. [21]

    Video-llava: Learning united visual representation by alignment before projection

    Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection. InProceedings of the 2024 conference on empirical methods in natural language processing, pages 5971–5984, 2024

    2024

  22. [22]

    Mitigating hallucination in large multi-modal models via robust instruction tuning

    Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. Mitigating hallucination in large multi-modal models via robust instruction tuning. In B. Kim, Y . Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y . Sun, editors, International Conference on Learning Representations, volume 2024, pages 57689– 57733, 2024. URL https://proceed...

    2024

  23. [23]

    Video-chatgpt: Towards detailed video understanding via large vision and language models

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12585–12602, 2024

    2024

  24. [24]

    Travl: A recipe for making video-language models better judges of physics implausibility.arXiv preprint arXiv:2510.07550, 2025

    Saman Motamed, Minghao Chen, Luc Van Gool, and Iro Laina. Travl: A recipe for making video-language models better judges of physics implausibility.arXiv preprint arXiv:2510.07550, 2025

    work page arXiv 2025

  25. [25]

    Do gener- ative video models understand physical principles? InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 948–958, 2026

    Saman Motamed, Laura Culp, Kevin Swersky, Priyank Jaini, and Robert Geirhos. Do gener- ative video models understand physical principles? InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 948–958, 2026. 11

    2026

  26. [26]

    GPT-4o technical report

    OpenAI. GPT-4o technical report. Technical report, OpenAI, 2024. URL https://openai. com/index/hello-gpt-4o/

    2024

  27. [27]

    Keeping your eye on the ball: Trajectory attention in video transformers

    Mandela Patrick, Dylan Campbell, Yuki Asano, Ishan Misra, Florian Metze, Christoph Feichten- hofer, Andrea Vedaldi, and Joao F Henriques. Keeping your eye on the ball: Trajectory attention in video transformers. InAdvances in Neural Information Processing Systems, volume 34, pages 12493–12506, 2021

    2021

  28. [28]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. InAdvances in Neural Information Processing Systems, volume 36, pages 53728–53741, 2023

    2023

  29. [29]

    SAM 2: Segment Anything in Images and Videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024

    work page internal anchor Pith review arXiv 2024

  30. [30]

    Argus: Hallucination and omission evaluation in video-llms

    Ruchit Rawal, Reza Shirkavand, Heng Huang, Gowthami Somepalli, and Tom Goldstein. Argus: Hallucination and omission evaluation in video-llms. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 20280–20290, 2025

    2025

  31. [31]

    IntPhys: A framework and benchmark for visual intuitive physics reasoning.arXiv preprint arXiv:1803.07616, 2018

    Ronan Riochet, Mario Ynocente Castro, Mathieu Bernard, Adam Lerer, Rob Fergus, Véronique Izard, and Emmanuel Dupoux. Intphys: A framework and benchmark for visual intuitive physics reasoning.arXiv preprint arXiv:1803.07616, 2018

    work page arXiv 2018

  32. [32]

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    work page internal anchor Pith review arXiv 2024

  33. [33]

    Principles of object perception.Cognitive science, 14(1):29–56, 1990

    Elizabeth S Spelke. Principles of object perception.Cognitive science, 14(1):29–56, 1990

    1990

  34. [34]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context.arXiv preprint arXiv:2403.05530, 2024

    work page internal anchor Pith review arXiv 2024

  35. [35]

    CoRR , volume =

    Kling Team, Jialu Chen, Yuanzheng Ci, Xiangyu Du, Zipeng Feng, Kun Gai, Sainan Guo, Feng Han, Jingbin He, Kang He, et al. Kling-omni technical report.arXiv preprint arXiv:2512.16776, 2025

    work page arXiv 2025

  36. [36]

    Qwen 3.5 technical report.https://qwen.ai/blog?id=qwen3.5, 2026

    Qwen Team. Qwen 3.5 technical report.https://qwen.ai/blog?id=qwen3.5, 2026

    2026

  37. [37]

    InternVideo2.5: Empowering video MLLMs with long and rich context modeling.arXiv:2501.12386, 2025

    Yi Wang, Xinhao Li, Ziang Yan, Yinan He, Jiashuo Yu, Xiangyu Zeng, Chenting Wang, Changlian Ma, Haian Huang, Jianfei Gao, et al. Internvideo2. 5: Empowering video mllms with long and rich context modeling.arXiv preprint arXiv:2501.12386, 2025

    work page arXiv 2025

  38. [38]

    Videohallucer: Evaluating intrinsic and extrinsic hallucinations in large video-language models.arXiv preprint arXiv:2406.16338, 2024

    Yuxuan Wang, Yueqian Wang, Dongyan Zhao, Cihang Xie, and Zilong Zheng. Videohallucer: Evaluating intrinsic and extrinsic hallucinations in large video-language models.arXiv preprint arXiv:2406.16338, 2024

    work page arXiv 2024

  39. [39]

    Season: Mitigating temporal hallucination in video large language models via self-diagnostic contrastive decoding.arXiv preprint arXiv:2512.04643, 2025

    Chang-Hsun Wu, Kai-Po Chang, Yu-Yang Sheng, Hung-Kai Chung, Kuei-Chun Wang, and Yu-Chiang Frank Wang. Season: Mitigating temporal hallucination in video large language models via self-diagnostic contrastive decoding.arXiv preprint arXiv:2512.04643, 2025

    work page arXiv 2025

  40. [40]

    MiniCPM-V: A GPT-4V Level MLLM on Your Phone

    Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level mllm on your phone.arXiv preprint arXiv:2408.01800, 2024

    work page internal anchor Pith review arXiv 2024

  41. [41]

    Clevrer: Collision events for video representation and reasoning

    Kexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, and Joshua B Tenenbaum. Clevrer: Collision events for video representation and reasoning. InInternational Conference on Learning Representations, 2020

    2020

  42. [42]

    Phyvllm: Physics-guided video language model with motion-appearance disentanglement.arXiv preprint arXiv:2512.04532, 2025

    Yu-Wei Zhan, Xin Wang, Hong Chen, Tongtong Feng, Wei Feng, Ren Wang, Guangyao Li, Qing Li, and Wenwu Zhu. Phyvllm: Physics-guided video language model with motion-appearance disentanglement.arXiv preprint arXiv:2512.04532, 2025. 12

    work page arXiv 2025

  43. [43]

    VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding

    Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, et al. Videollama 3: Frontier multimodal foundation models for image and video understanding.arXiv preprint arXiv:2501.13106, 2025

    work page internal anchor Pith review arXiv 2025

  44. [44]

    Flash-vstream: Memory- based real-time understanding for long video streams.arXiv preprint arXiv:2406.08085, 2024

    Haoji Zhang, Yiqin Wang, Yansong Tang, Yong Liu, Jiashi Feng, Jifeng Dai, and Xiaojie Jin. Flash-vstream: Memory-based real-time understanding for long video streams.arXiv preprint arXiv:2406.08085, 2024

    work page arXiv 2024

  45. [45]

    Eventhallusion: Diagnosing event hallucinations in video llms.arXiv preprint arXiv:2409.16597, 2024

    Jiacheng Zhang, Yang Jiao, Shaoxiang Chen, Na Zhao, Zhiyu Tan, Hao Li, Xingjun Ma, and Jingjing Chen. Eventhallusion: Diagnosing event hallucinations in video llms.arXiv preprint arXiv:2409.16597, 2024

    work page arXiv 2024

  46. [46]

    Cogstream: Context-guided streaming video question answering

    Zicheng Zhao, Kangyu Wang, Shijie Li, Rui Qian, Weiyao Lin, and Huabin Liu. Cogstream: Context-guided streaming video question answering. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 13332–13341, 2026

    2026

  47. [47]

    Propainter: Improving propagation and transformer for video inpainting

    Shangchen Zhou, Chongyi Li, Kelvin CK Chan, and Chen Change Loy. Propainter: Improving propagation and transformer for video inpainting. InProceedings of the IEEE/CVF international conference on computer vision, pages 10477–10486, 2023

    2023

  48. [48]

    empty air

    Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. InConference on Robot Learning, pages 2165–2183. PMLR, 2023. 13 A Detailed Taxonomy and Qualitative Examples of PACC Building upon the taxonomy int...

    2023

  49. [49]

    Select the most appropriate Typical Scenario

  50. [50]

    visual_fact_caption

    Output a Target Fallacy Scenario that clearly describes how the fallacy manifests. Step 3: Generation Method Decision Basis: Input Video + Step 2 Target Scenario + Modification Principles. Principle for [Manual_CV_Edit]: Select this if the Target Fallacy Scenario can be achieved by rearranging existing pixels, removing objects, or layering. Principle for ...

  51. [51]

    Focus ONLY on Subjects, Actions, and Environment

    Observation Rule: Generate a strictly objective visual description. Focus ONLY on Subjects, Actions, and Environment. No causal speculations. Positive Sample Observation: Base this entirely on the Visual Fact Caption. Negative Sample Observation: Integrate the abnormal actions from the Target Fallacy Scenario into the Visual Fact Caption

  52. [52]

    Positive Sample Attribution: Explain which fundamental laws the video adheres to

    Attribution Rule: Use the PACC Category Dictionary below to explain physical adherence. Positive Sample Attribution: Explain which fundamental laws the video adheres to. Negative Sample Attribution: Point out which physical law the Target Fallacy Scenario violates

  53. [53]

    positive_sample

    Verdict Positive Sample Verdict: Summarize that the video is real and conforms to logic. Negative Sample Verdict: Summarize that the video is forged and contains fallacies. PACC Category Dictionary {Dynamically injected definitions based on the selected fallacy category} Output Requirements Strictly return the output in JSON format. { "positive_sample": {...

  54. [54]

    reasoning

    "reasoning": (String) A step-by-step analysis comparing the entities, state transitions, and causal logic

  55. [55]

    accuracy

    "accuracy": (Integer) 0 or 1. STRICTLY evaluate ONLY the final binary verdict. (1 if matches, 0 if contradicts). DO NOT let flawed reasoning lower the accuracy to 0. If the model guessed the right verdict for the wrong reasons, the accuracy MUST be 1

  56. [56]

    reasoning

    "score": (Integer) 1 to 5. The reasoning quality score: 1: (Requires accuracy=0) Wrong verdict, and severe hallucinations. 2: (Requires accuracy=0) Wrong verdict, but correctly observed some entities. 3: (Requires accuracy=1) Correct verdict, BUT reasoning is hallucinated (Right for the wrong reasons). 4: (Requires accuracy=1) Correct verdict, reasoning b...