Pith. sign in

REVIEW 4 major objections 4 minor 65 references

Video generators can produce plausible future videos while failing explicit tests of causal perception, a gap the paper calls the Perception-Prediction Gap.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 17:42 UTC pith:TXHE6NPU

load-bearing objection Clever dual-judge evaluation, but the implicit task leaks the future via the co-injected description, so the Perception-Prediction Gap is not yet well-founded. the 4 major comments →

arxiv 2607.17523 v1 pith:TXHE6NPU submitted 2026-07-20 cs.CV cs.AIcs.CL

Thinking in Video: Can Video Generators Really Reason About the Real World?

Yongheng Zhang , Guang Yang , Ruihan Hou , Qiguang Chen , Ziang Liu , Xiaolong Liu , Manman Zhang , Yanchao Hao
show 7 more authors
Zheng Wei Hao Wu Libo Qin Peishan Dai Yinghui Li Di Yin Xing Sun
This is my paper
classification cs.CV cs.AIcs.CL
keywords world modelsvideo generationcausal reasoningPerception-Prediction Gapvideo generation evaluationcausal perceptiongenerative predictionaudio-visual misalignment
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks whether video generators reason about the real world or merely imitate pixel statistics. To find out, it introduces the Causal-Generative Dual-Judge (CGDJ), which tests each model twice: explicit causal perception (can it answer questions about a scenario's causal structure from a flattened grid of frames?) and implicit generative prediction (can it render the physically consistent future video given the pre-event frames?). Across four representative generators, the paper finds a sharp Perception-Prediction Gap: one open-source model scores 41.5% on generative plausibility but only 0.7% on explicit causal perception, while a frontier closed-source model reaches 51.2% explicit and 61.6% implicit. It also documents audio-visual misalignment, where models verbalize correct causal logic more reliably than they render it, which challenges the 'world simulator' narrative. The upshot: current video generators are partial imitators of world dynamics, and the strongest systems are only beginning to align perception with prediction.

Core claim

On the paper's own terms, current video generators exhibit a 'Perception-Prediction Gap': they can synthesize visually plausible continuations without reliable explicit causal perception. Empirically, an open-weights generator scored 41.5% on implicit generative action yet 0.7% on explicit causal understanding, whereas an advanced closed-source generator scored 61.6% implicit and 51.2% explicit. The paper interprets this as evidence that open models operate via statistical pixel mimicry, while scaled closed models show early but limited alignment between explicit reasoning and generative simulation. A further audio-visual analysis shows models articulate correct causal logic more often than

What carries the argument

The central object is the Causal-Generative Dual-Judge (CGDJ), a benchmark-and-evaluation framework built on the 'Flatten Temporal Video' strategy. That strategy maps a video and a rasterized text query onto a single static image grid (e.g., 7×10 frames), so video generators, which accept only image inputs, can be given temporal context. The framework then runs two complementary protocols: Explicit Causal Perception, a spatio-temporal flattened visual question-answering task, and Implicit Generative Prediction, which splits a video at a manually identified causal inflection point and asks the model to generate the post-inflection future from the pre-event frames. The gap between a model's sc

Load-bearing premise

The paper's conclusion depends on the assumption that the detailed semantic video description co-injected with the input video in the implicit generation task does not reveal the future outcome after the causal inflection point; if it does, the 'prediction' collapses into text-to-video translation and the Perception-Prediction Gap interpretation fails.

What would settle it

Take the implicit generative task, and for each test video, redact or surgically remove from the injected video description any words that describe the post-inflection future. If open-source models' scores remain near 41.5%, the Perception-Prediction Gap is real; if their scores fall sharply or the closed-source scores change dramatically, the original protocol leaked the outcome through the text, and the gap is partly an artifact of description-following rather than genuine causal prediction.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the Perception-Prediction Gap holds for current generators, high visual plausibility scores cannot be taken as evidence of causal understanding; generative evaluation must include a discriminative causal-perception probe.
  • Open-weights models that generate plausible continuations while scoring near zero on explicit perception are best described as statistical mimics of world dynamics, not world simulators.
  • Audio-visual misalignment implies that a model's narrated reasoning in a rollout is not reliable evidence that it has grounded physical knowledge; the visual channel must be inspected separately.
  • The strongest closed-source models show early alignment between explicit reasoning and generative simulation, suggesting that scaling or instruction-following alignment can partially close the gap.
  • Evaluation benchmarks for video reasoning should adopt a dual-judge design that separately audits semantic comprehension and physical rendering, as the paper's CGDJ does.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A direct extension would be to test whether training video generators on explicit causal question-answering improves their implicit generative scores; the paper's gap suggests transfer will be limited, and if so, separate objectives for perception and prediction are needed.
  • The co-injected semantic description in the implicit task is a potential leak path: if it describes the post-inflection future, the task becomes text-to-video re-rendering. Redacting or blinding that description would isolate true future prediction from description-following.
  • The audio-visual misalignment could be turned into a cheap screening metric: transcribe the audio of generated rollouts and compare verbal-versus-visual correctness before running full video evaluation, possibly catching hallucinated reasoning early.
  • The flattening protocol may disadvantage models trained with native temporal encoders or internal prompt-rewriting pipelines; a native video-conditioning variant might reduce the measured gap and reveal whether the gap is architectural or intrinsic.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces the Causal-Generative Dual-Judge (CGDJ), a two-component benchmark for evaluating whether video generative models can reason about causality. Explicit Causal Perception converts video and a rasterized multiple-choice question into a flattened image grid and scores the model's ability to produce a video that contains the correct answer. Implicit Generative Prediction splits videos at a manually identified causal inflection point, feeds the pre-event frames (plus a Gemini-3-Pro-generated semantic description) to the generator, and asks it to produce the post-event future; the output is scored by Gemini-3-Pro against the gold video. Applying CGDJ to Wan-2.2, HunyuanVideo-1.5, Veo-3.1, and Sora-2, the paper reports a Perception-Prediction Gap: open-source models score near zero on explicit causal perception (e.g., Wan-2.2: 0.7%) but moderate on implicit generation (41.5%), while closed-source models show higher and more aligned scores (e.g., Sora-2: 51.2% explicit, 61.6% implicit). The paper also reports audio-visual misalignment and validates its automated judge against human ratings.

Significance. If the implicit task genuinely measures causal prediction, CGDJ would provide a useful diagnostic that separates perceptual plausibility from causal correctness, and the reported gap would be an important empirical observation. The paper has concrete strengths: it benchmarks multiple open and closed generators, ablates the flattening grid and keyframe count, reports human alignment for the automated judge (Pearson r = 0.82), and makes code/data publicly available. However, the central claim rests on two load-bearing assumptions: that the co-injected semantic description does not reveal the future outcome, and that the explicit and implicit scores are commensurable enough to be compared as a gap. Both assumptions are currently insufficiently supported.

major comments (4)
  1. [Section 2.1.2 and Figure 2(b); Section 2.2.2] The co-injected Gemini-3-Pro detailed semantic video description is not specified as covering only the pre-inflection input video. If it describes the full original video, including the post-inflection future, then the implicit generative task is not causal prediction but text-to-video re-rendering of an already-provided outcome. This would collapse the empirical anchor of the Perception-Prediction Gap. The paper must (a) state explicitly whether the description is restricted to the pre-event portion, and (b) provide an ablation that removes the description or uses a redacted pre-event-only description. The evaluation prompt in Figure 4(b) also uses VideoDescription as the governing text, so a leaked description would inflate the implicit scores through the judge as well.
  2. [Section 3.2, Tables 2 and 3] The headline Perception-Prediction Gap compares two non-commensurable measurements: accuracy (%) on a 900-video multiple-choice task (Explicit) versus a 0.0-1.0 quality score on a different 600-video generation task (Implicit). The datasets, protocols, and metrics differ, so the gap could reflect task difficulty or metric calibration rather than a genuine dissociation between perception and prediction. The authors should either design a controlled comparison (e.g., evaluate the same video set with both protocols) or clearly justify why the two numbers can be placed on a common interpretative scale.
  3. [Section 3.3.1, Figure 5] The Audio-Visual Misalignment result is derived from the explicit task's generated videos. For open-source models with near-zero explicit accuracy, the audio channel may be absent, unintelligible, or unrelated to the rasterized question; comparing Audio accuracy with Video accuracy without per-model breakdowns is therefore confounded by instruction-following ability. Please report per-model audio and video accuracies separately, define exactly what the audio channel contains (e.g., transcribed speech, answer tokens), and show the comparison conditioned on cases where the model actually attempts the instructed format.
  4. [Section 3.1, Table 2] The near-zero explicit scores of Wan-2.2 and HunyuanVideo-1.5 are interpreted as a lack of causal perception, but these models may simply be unable to read rasterized text queries from the flattened grid; they are not trained for visual question answering in this format. The MLLM baselines demonstrate that the flattened representation retains information for language models, but they do not validate that video generators can extract and follow the text instruction. A control experiment (e.g., providing the question as a native text prompt or using an image captioning pretask) is needed to separate instruction-following failure from causal-perception failure.
minor comments (4)
  1. [Tables 2 and 3 captions] The phrase 'The Red represents better A VG performance' should be corrected to 'AVG'; the colors are not visible in a text-only rendering and should be described.
  2. [Figure 5] The numerical labels in the figure are unexplained. Define Audio∩Video, Video-Only, Audio-Only and state which number corresponds to which model/condition.
  3. [Section 2.1.2] The Causal Inflection Point annotation is performed by expert annotators but no inter-annotator agreement or quality-control statistics are reported; please add these to justify the reliability of the split.
  4. [Section 3.2] The claim 'performing on par with an 8B-parameter MLLM' is imprecise: Qwen3-VL-8B achieves 55.1% in the flattened setting, while Veo-3.1 and Sora-2 score 49.9% and 51.2%, respectively. Please specify the comparison baseline and note the difference.

Circularity Check

2 steps flagged

Implicit generative 'prediction' is conditioned on a Gemini-3-Pro video description that is not restricted to the pre-event segment, so the Perception–Prediction Gap may reduce to text-guided re-rendering of the disclosed outcome.

specific steps
  1. self definitional [§2.1.2, Video Splitting (Implicit Generative Action Data)]
    "❶ Input Video: The pre-event sequence serves as the visual condition, providing the necessary causal context for the subsequent generation. Furthermore, we use Gemini-3-Pro [44] to produce a detailed semantic video description, which is co-injected with the video to facilitate the generation. ❷ Gold Video: The subsequent temporal segment functions as the immutable ground truth reference, capturing the realized outcome of the initial conditions."

    The implicit task is defined as causal prediction: from the pre-event visual condition, the model must synthesize the post-event Gold Video. But the prompt also co-injects a 'detailed semantic video description' with no stated restriction to the pre-event segment. If that description is generated from the full original video, it already contains the outcome the model is supposed to predict. The generated 'consequent future sequence' then reduces to a text-to-video re-rendering of a described future. The reported gap between Wan-2.2's implicit score (41.5%) and explicit score (0.7%) is therefore not a clean measure of causal prediction versus pixel mimicry; it can be inflated simply by following the leaked semantic description.

  2. self definitional [§2.2.2, Implicit Generative Evaluation]
    "It systematically benchmarks the video generated by the model against the ground truth 'Gold Video', while ensuring strict adherence to the governing textual description."

    The evaluator's prompt includes VideoDescription as an input and explicitly scores the model's output for strict adherence to that textual description. If the same co-injected description contains the future outcome, then the 'Reference Consistency' dimension measures how faithfully the model re-renders the description it was given, not whether it independently predicted the causal consequence. The implicit score is thus internally tied to the very text fed into the generator, making the metric a text-following check rather than an external audit of causal prediction.

full rationale

The paper's central empirical claim—the Perception–Prediction Gap—rests on comparing near-zero explicit scores (Wan-2.2: 0.7%) with substantial implicit generative scores (Wan-2.2: 41.5%). That comparison is only meaningful if the implicit task truly requires predicting a future from a pre-event visual condition. However, §2.1.2 says the model receives a 'detailed semantic video description' produced by Gemini-3-Pro and 'co-injected with the video', without specifying that the description is limited to the pre-inflection segment. As written, the task permits the GPT/LLM-generated description to summarize the full original video, including the post-inflection Gold Video. In that case, the model is not predicting; it is translating a text description that already states the outcome into pixels. The evaluation in §2.2.2 compounds this by scoring outputs for 'strict adherence to the governing textual description', so the implicit score can reflect description-following rather than physical simulation. This is not a case of condemnation of normal self-citation: the paper's self-citations are not the load-bearing element, and the explicit QA benchmark, external data sources, and human-alignment check provide independent content. The circularity is localized to the implicit generative protocol, but that protocol is the empirical anchor for the headline Perception–Prediction Gap. Because the confound is untested (the ablation in §3.3 varies only frame count, never removing the description), the central interpretation is partially forced by construction. I therefore score the circularity as 6 rather than 8 or 10: the paper has independent components, but the key implicit 'prediction' measurement is potentially reducible to its own textual input.

Axiom & Free-Parameter Ledger

2 free parameters · 5 axioms · 0 invented entities

No new physical entities are invented. The free parameters are evaluation hyperparameters selected by running the exact test models, which is a mild form of fitting to the data. The axioms are the experimental design assumptions that the causal interpretation rests on, especially the non-leakage of the future in the implicit task.

free parameters (2)
  • Flatten grid layout (rows × cols) = 7×10
    Selected as the default after ablation (Fig. 7) because it yielded the highest average accuracy across Veo-3.1 and Sora-2; the choice is tuned to the very models being evaluated.
  • Number of input keyframes for implicit task = 7
    Selected after comparing 1/4/7/10 frames (Fig. 8); the authors state 7 gives the best balance, again tuned on the test models.
axioms (5)
  • domain assumption The Flatten Temporal Video composite image preserves enough spatiotemporal information for video generators to perform the explicit task.
    The entire explicit protocol depends on this; MLLMs are used to validate that information is preserved, but video generators may not share that capability.
  • domain assumption The semantic video description provided to the generator in the implicit task describes only the pre-event context, not the future outcome.
    This is the weakest assumption: the paper does not state the description is limited to the pre-event segment, and if it leaks the future, the prediction is circular.
  • domain assumption Gemini-3-Pro is a reliable judge of video semantic alignment and physical validity.
    Used as the automated evaluator for both explicit and implicit tasks; human alignment is shown only for the implicit task.
  • domain assumption Whisper-large-v2 accurately transcribes generated audio.
    The audio-visual misalignment analysis and explicit adjudication rely on Whisper transcripts.
  • domain assumption The videos from Video-MME and the aggregated datasets contain identifiable causal structure relevant to the questions.
    The benchmark assumes that the flattened clips and split points reflect causal reasoning problems, not just appearance-based associations.

pith-pipeline@v1.3.0-alltime-deepseek · 15814 in / 12757 out tokens · 109257 ms · 2026-08-01T17:42:56.941654+00:00 · methodology

0 comments
read the original abstract

Recent advances in world models and video generation have given rise to an emerging reasoning paradigm that leverages video generative models to simulate, predict, and reason about real-world dynamics. We redefine this paradigm as Thinking in Video, where video is not merely an output artifact but a medium for constructing, extending, and verifying causal thought. However, this promise remains unverified: convincing rollouts may reflect memorized appearances rather than causal understanding, while existing metrics separate perceptual fidelity from semantic logic. To evaluate whether video generators support such reasoning, we introduce the Causal-Generative Dual-Judge (CGDJ), auditing World Model Consistency from two perspectives. Explicit Causal Perception tests whether a generator reads a video scenario as a reasoning problem through spatio-temporal flattened visual question answering, while Implicit Generative Perception-Prediction Gap evaluates whether it renders the causal consequence as a consistent future video. Applying CGDJ to representative open- and closed-source generators reveals a clear Perception-Prediction Gap: open-source models produce plausible dynamics despite near-zero explicit causal perception, whereas advanced closed-source systems show stronger but still limited alignment between reasoning and generation. Further analysis exposes audio-visual misalignment, where models verbalize correct causal logic more reliably than they render it, challenging the "world simulator" narrative.

Figures

Figures reproduced from arXiv: 2607.17523 by Di Yin, Guang Yang, Hao Wu, Libo Qin, Manman Zhang, Peishan Dai, Qiguang Chen, Ruihan Hou, Xiaolong Liu, Xing Sun, YanChao Hao, Yinghui Li, Yongheng Zhang, Zheng Wei, Ziang Liu.

Figure 1
Figure 1. Figure 1: The overall workflow of the Causal Generative Dual Judge ( [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The construction process of the CGDJ benchmark. (a) Explicit Causal Perception Data utilizes a Flatten Temporal Video strategy to map temporal dynamics into a spatial grid, combining visual history with rasterized text queries for direct reasoning. (b) Implicit Generative Prediction Data evaluates simulation by splitting videos at a critical inflection point; The pre-event frames serve as a flattened visua… view at source ↗
Figure 3
Figure 3. Figure 3: Statistical distribution of the Causal-Generative [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: The overall evaluation pipeline of the Causal–Generative Dual-Judge consists of two distinct yet complementary [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparison of Audio ∩ Video, Video-Only and Audio-Only. The empirical results show that Audio accu￾racy consistently surpasses Video accuracy by a significant margin. This indicates models can semantically describe out￾comes but fail to physically generate them. 3.3 Causal–Generative Dual-Judge Analysis To gain deeper insights into the underlying mechanisms of the “Perception-Prediction Gap” and address po… view at source ↗
Figure 7
Figure 7. Figure 7: Performance comparison across different spatial grid layouts and video durations. The bar chart and table illustrate [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Performance comparison across varying inputs for [PITH_FULL_IMAGE:figures/full_fig_p008_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

65 extracted references · 20 linked inside Pith

  1. [1]

    Youtu-llm: Unlocking the native agentic potential for lightweight large language models, 2026

    Junru Lu, Jiarui Qin, Lingfeng Qiao, Yinghui Li, Xinyi Dai, Bo Ke, Jianfeng He, Ruizhi Qiao, Di Yin, Xing Sun, Yunsheng Wu, Yinsong Liu, Shuangyin Liu, Mingkong Tang, Haodong Lin, Jiayi Kuang, Fanxu Meng, Xiaojuan Tang, Yunjia Xi, Junjie Huang, Haotong Yang, Zhenyi Shen, Yangning Li, Qianwen Zhang, Yifei Yu, Siyu An, Junnan Dong, Qiufeng Wang, Jie Wang, K...

    arXiv 2026

  2. [2]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023

    Pith/arXiv arXiv 2023

  3. [3]

    Sora 2.https://openai.com/index/sora-2, 2025

    OpenAI. Sora 2.https://openai.com/index/sora-2, 2025

    2025

  4. [4]

    Hunyuanvideo 1.5 technical report, 2025

    Bing Wu, Chang Zou, Changlin Li, Duojun Huang, Fang Yang, Hao Tan, Jack Peng, Jianbing Wu, Jiangfeng Xiong, Jie Jiang, Linus, Patrol, Peizhen Zhang, Peng Chen, Penghao Zhao, Qi Tian, Songtao Liu, Weijie Kong, Weiyan Wang, Xiao He, Xin Li, Xinchi Deng, Xuefei Zhe, Yang Li, Yanxin Long, Yuanbo Peng, Yue Wu, Yuhong Liu, Zhenyu Wang, Zuozhuo Dai, Bo Peng, Coo...

    Pith/arXiv arXiv 2025

  5. [5]

    Veo 3.https://aistudio.google.com/models/veo-3, 2025

    Google. Veo 3.https://aistudio.google.com/models/veo-3, 2025

    2025

  6. [6]

    Wan: Open and advanced large-scale video generative models, 2025

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, Pandeng Li, Pingyu Wu, Ruihang Chu, Ruili Feng, Shiwei Zhang, Siyang Sun, Tao Fang, T...

    Pith/arXiv arXiv 2025

  7. [7]

    From chatbot to digital colleague: The paradigm shift toward persistent autonomous ai.arXiv preprint arXiv:2606.14502, 2026

    Yongheng Zhang, Ziang Liu, Jiaxuan Zhu, Shuai Wang, Xiangqi Chen, Haojing Huang, Jiayi Kuang, Siyu Chen, Ao Shen, Hao Wu, et al. From chatbot to digital colleague: The paradigm shift toward persistent autonomous ai.arXiv preprint arXiv:2606.14502, 2026

    Pith/arXiv arXiv 2026

  8. [8]

    World simulation with video foundation models for physical ai.arXiv preprint arXiv:2511.00062, 2025

    Arslan Ali, Junjie Bai, Maciej Bala, Yogesh Balaji, Aaron Blakeman, Tiffany Cai, Jiaxin Cao, Tianshi Cao, Elizabeth Cha, Yu-Wei Chao, et al. World simulation with video foundation models for physical ai.arXiv preprint arXiv:2511.00062, 2025

    Pith/arXiv arXiv 2025

  9. [9]

    Understanding world or predicting future? a comprehensive survey of world models.ACM Computing Surveys, 58(3):1–38, 2025

    Jingtao Ding, Yunke Zhang, Yu Shang, Yuheng Zhang, Zefang Zong, Jie Feng, Yuan Yuan, Hongyuan Su, Nian Li, Nicholas Sukiennik, et al. Understanding world or predicting future? a comprehensive survey of world models.ACM Computing Surveys, 58(3):1–38, 2025

    2025

  10. [10]

    Videophy-2: A challenging action-centric physical com- monsense evaluation in video generation.arXiv preprint arXiv:2503.06800, 2025

    Hritik Bansal, Clark Peng, Yonatan Bitton, Roman Goldenberg, Aditya Grover, and Kai-Wei Chang. Videophy-2: A challenging action-centric physical com- monsense evaluation in video generation.arXiv preprint arXiv:2503.06800, 2025

    Pith/arXiv arXiv 2025

  11. [11]

    The past mistake is the future wisdom: Error-driven contrastive probability optimization for chinese spell checking

    Yinghui Li, Qingyu Zhou, Yangning Li, Zhongli Li, Ruiyang Liu, Rongyi Sun, Zizhen Wang, Chao Li, Yunbo Cao, and Hai-Tao Zheng. The past mistake is the future wisdom: Error-driven contrastive probability optimization for chinese spell checking. InFindings of the Association for Computational Linguistics: ACL 2022, pages 3202–3213, 2022

    2022

  12. [12]

    Think- ing with video: Video generation as a promising multimodal reasoning paradigm

    Jingqi Tong, Yurong Mou, Hangcheng Li, Mingzhe Li, Yongzhuo Yang, Ming Zhang, Qiguang Chen, Tianyi Liang, Xiaomeng Hu, Yining Zheng, et al. Think- ing with video: Video generation as a promising multimodal reasoning paradigm. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pages 41121–41129, 2026

    2026

  13. [13]

    Reasoning via video: The first evaluation of video models’ reasoning abilities through maze-solving tasks.arXiv preprint arXiv:2511.15065, 2025

    Cheng Yang, Haiyuan Wan, Yiran Peng, Xin Cheng, Zhaoyang Yu, Jiayi Zhang, Junchi Yu, Xinlei Yu, Xiawu Zheng, Dongzhan Zhou, et al. Reasoning via video: The first evaluation of video models’ reasoning abilities through maze-solving tasks.arXiv preprint arXiv:2511.15065, 2025

    arXiv 2025

  14. [14]

    Weave: Unleashing and benchmarking the in-context interleaved comprehension and generation

    Wei Chow, Jiachun Pan, Yongyuan Liang, Mingze Zhou, Xue Song, Liyu Jia, Sain- ing Zhang, Siliang Tang, Juncheng Li, Fengda Zhang, et al. Weave: Unleashing and benchmarking the in-context interleaved comprehension and generation. arXiv preprint arXiv:2511.11434, 2025

    arXiv 2025

  15. [15]

    Tivibench: Benchmarking think-in-video reasoning for video generative models

    Harold Haodong Chen, Disen Lan, Wen-Jie Shu, Qingyang Liu, Zihan Wang, Sirui Chen, Wenkai Cheng, Kanghao Chen, Hongfei Zhang, Zixin Zhang, et al. Tivibench: Benchmarking think-in-video reasoning for video generative models. arXiv preprint arXiv:2511.13704, 2025

    arXiv 2025

  16. [16]

    Evaluating gemini robotics policies in a veo world simulator.arXiv preprint arXiv:2512.10675, 2025

    Gemini Robotics Team, Coline Devin, Yilun Du, Debidatta Dwibedi, Ruiqi Gao, Abhishek Jindal, Thomas Kipf, Sean Kirmani, Fangchen Liu, Anirudha Majumdar, et al. Evaluating gemini robotics policies in a veo world simulator.arXiv preprint arXiv:2512.10675, 2025

    arXiv 2025

  17. [17]

    Ggbench: A geometric generative reasoning benchmark for unified multimodal models

    Jingxuan Wei, Caijun Jia, Xi Bai, Xinglong Xu, Siyuan Li, Linzhuang Sun, Bihui Yu, Conghui He, Lijun Wu, and Cheng Tan. Ggbench: A geometric generative reasoning benchmark for unified multimodal models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5199– 5210, 2026

    2026

  18. [18]

    Let’s think with images efficiently! an interleaved-modal chain-of-thought reasoning framework with dynamic and precise visual thoughts

    Xu Liu, Yongheng Zhang, Qiguang Chen, Yao Li, Sheng Wang, and Libo Qin. Let’s think with images efficiently! an interleaved-modal chain-of-thought reasoning framework with dynamic and precise visual thoughts. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 32213–32221, 2026

    2026

  19. [19]

    Physgen: Rigid-body physics-grounded image-to-video generation

    Shaowei Liu, Zhongzheng Ren, Saurabh Gupta, and Shenlong Wang. Physgen: Rigid-body physics-grounded image-to-video generation. InEuropean Conference on Computer Vision, pages 360–378. Springer, 2024

    2024

  20. [20]

    Exploring the evolution of physics cognition in video generation: A survey.arXiv preprint arXiv:2503.21765, 2025

    Minghui Lin, Xiang Wang, Yishan Wang, Shu Wang, Fengqi Dai, Pengxiang Ding, Cunxiang Wang, Zhengrong Zuo, Nong Sang, Siteng Huang, et al. Exploring the evolution of physics cognition in video generation: A survey.arXiv preprint arXiv:2503.21765, 2025

    Pith/arXiv arXiv 2025

  21. [21]

    Improved techniques for training gans.Advances in neural information processing systems, 29, 2016

    Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Rad- ford, and Xi Chen. Improved techniques for training gans.Advances in neural information processing systems, 29, 2016

    2016

  22. [22]

    FVD: A new metric for video generation,

    Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphaël Marinier, Marcin Michalski, and Sylvain Gelly. FVD: A new metric for video generation,

  23. [23]

    On the content bias in fréchet video distance

    Songwei Ge, Aniruddha Mahapatra, Gaurav Parmar, Jun-Yan Zhu, and Jia-Bin Huang. On the content bias in fréchet video distance. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7277– 7288, 2024

    2024

  24. [24]

    The essential role of causality in foundation world models for embodied ai.arXiv preprint arXiv:2402.06665, 2024

    Tarun Gupta, Wenbo Gong, Chao Ma, Nick Pawlowski, Agrin Hilmkil, Meyer Scetbon, Marc Rigter, Ade Famoti, Ashley Juan Llorens, Jianfeng Gao, et al. The essential role of causality in foundation world models for embodied ai.arXiv preprint arXiv:2402.06665, 2024

    Pith/arXiv arXiv 2024

  25. [25]

    Diffusion art or digital forgery? investigating data replication in diffusion models

    Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Diffusion art or digital forgery? investigating data replication in diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6048–6058, 2023

    2023

  26. [26]

    Karthik Valmeekam, Matthew Marquez, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati. Planbench: An extensible benchmark for evaluating large language models on planning and reasoning about change.Advances in Neural Information Processing Systems, 36:38975–38987, 2023

    2023

  27. [27]

    Do generative video models understand physical principles? InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 948–958, 2026

    Saman Motamed, Laura Culp, Kevin Swersky, Priyank Jaini, and Robert Geirhos. Do generative video models understand physical principles? InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 948–958, 2026

    2026

  28. [28]

    Roboalign-r1: Distilled multimodal reward alignment for robot video world models, 2026

    Hao Wu, Yuqi Li, Yuan Gao, Fan Xu, Fan Zhang, Kun Wang, Penghao Zhao, Qiufeng Wang, Yizhou Zhao, Weiyan Wang, Yingli Tian, Xian Wu, and Xiaomeng Huang. Roboalign-r1: Distilled multimodal reward alignment for robot video world models, 2026. URL https://arxiv.org/abs/2605.03821

    Pith/arXiv arXiv 2026

  29. [29]

    Vbench: Comprehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21807–21818, 2024

    2024

  30. [30]

    Evalcrafter: Benchmarking and evaluating large video generation models

    Yaofang Liu, Xiaodong Cun, Xuebo Liu, Xintao Wang, Yong Zhang, Haoxin Chen, Yang Liu, Tieyong Zeng, Raymond Chan, and Ying Shan. Evalcrafter: Benchmarking and evaluating large video generation models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22139–22149, 2024

    2024

  31. [31]

    Video generation models as world simulators.OpenAI Blog, 1(8):1, 2024

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, et al. Video generation models as world simulators.OpenAI Blog, 1(8):1, 2024

    2024

  32. [32]

    What about gravity in video generation? post-training newton’s laws with verifiable rewards.arXiv preprint arXiv:2512.00425, 2025

    Minh-Quan Le, Yuanzhi Zhu, Vicky Kalogeiton, and Dimitris Samaras. What about gravity in video generation? post-training newton’s laws with verifiable rewards.arXiv preprint arXiv:2512.00425, 2025

    arXiv 2025

  33. [33]

    T2v-compbench: A comprehensive benchmark for compositional text-to- video generation

    Kaiyue Sun, Kaiyi Huang, Xian Liu, Yue Wu, Zihan Xu, Zhenguo Li, and Xihui Liu. T2v-compbench: A comprehensive benchmark for compositional text-to- video generation. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 8406–8416, 2025

    2025

  34. [34]

    Vibe: A text-to-video benchmark for evaluating hal- lucination in large multimodal models

    Vipula Rawte, Sarthak Jain, Aarush Sinha, Garv Kaushik, Aman Bansal, Prathik- sha Rumale Vishwanath, Samyak Rajesh Jain, Aishwarya Naresh Reganti, Vinija Jain, Aman Chadha, et al. Vibe: A text-to-video benchmark for evaluating hal- lucination in large multimodal models. InProceedings of the 5th Workshop on Trustworthy NLP (TrustNLP 2025), pages 232–246, 2025

    2025

  35. [35]

    World models.arXiv preprint arXiv:1803.10122, 2(3), 2018

    David Ha and Jürgen Schmidhuber. World models.arXiv preprint arXiv:1803.10122, 2(3), 2018. Zhang et al

    Pith/arXiv arXiv 2018

  36. [36]

    A path towards autonomous machine intelligence version 0.9

    Yann LeCun. A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27.Open Review, 62(1):1–62, 2022

    2022

  37. [37]

    Have we unified image generation and understanding yet? an empirical study of gpt-4o’s image generation ability.arXiv preprint arXiv:2504.08003, 2025

    Ning Li, Jingran Zhang, and Justin Cui. Have we unified image generation and understanding yet? an empirical study of gpt-4o’s image generation ability.arXiv preprint arXiv:2504.08003, 2025

    Pith/arXiv arXiv 2025

  38. [38]

    Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis

    Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 24108–24118, 2025

    2025

  39. [39]

    The sound of water: Inferring physical properties from pouring liquids

    Piyush Bagad, Makarand Tapaswi, Cees GM Snoek, and Andrew Zisserman. The sound of water: Inferring physical properties from pouring liquids. InICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2025

    2025

  40. [40]

    Physion: Evaluating physical prediction from vision in humans and machines

    Daniel M Bear, Elias Wang, Damian Mrowca, Felix J Binder, Hsiao-Yu Fish Tung, RT Pramod, Cameron Holdaway, Sirui Tao, Kevin Smith, Fan-Yun Sun, et al. Physion: Evaluating physical prediction from vision in humans and machines. arXiv preprint arXiv:2106.08261, 2021

    Pith/arXiv arXiv 2021

  41. [41]

    Tld: A vehicle tail light signal dataset and benchmark.arXiv preprint arXiv:2409.02508, 2024

    Jinhao Chai, Shiyi Mu, and Shugong Xu. Tld: A vehicle tail light signal dataset and benchmark.arXiv preprint arXiv:2409.02508, 2024

    Pith/arXiv arXiv 2024

  42. [42]

    The kinetics human action video dataset.arXiv preprint arXiv:1705.06950, 2017

    Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, et al. The kinetics human action video dataset.arXiv preprint arXiv:1705.06950, 2017

    Pith/arXiv arXiv 2017

  43. [43]

    Ego4d: Around the world in 3,000 hours of egocentric video

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18995–19012, 2022

    2022

  44. [44]

    Gemma 3 technical report, 2025

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieil- lard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penchev, Gaël Liu, Francesco Visin, Kathleen Kenealy, Lucas B...

    Pith/arXiv arXiv 2025

  45. [45]

    Robust speech recognition via large-scale weak supervision

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. InInternational conference on machine learning, pages 28492–28518. PMLR, 2023

    2023

  46. [46]

    Qwen3-vl technical report, 2025

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan ...

    Pith/arXiv arXiv 2025

  47. [47]

    Openai gpt-5 system card, 2025

    Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El- Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, Akshay Nathan, Alan Luo, Alec Helyar, Aleksander Madry, Aleksandr Efremov, Aleksan- dra Spyra, Alex Baker-Whitcomb, Alex Beutel, Alex Karpenko, Alex Makelov, Alex Neitz, Alex Wei, Alexandra Barr, Alexandre Kirchmeyer, ...

    Pith/arXiv arXiv 2025

  48. [48]

    Scaling zero-shot reference-to-video generation.arXiv preprint, 2025

    Zijian Zhou, Shikun Liu, Haozhe Liu, Haonan Qiu, Zhaochong An, Weiming Ren, Zhiheng Liu, Xiaoke Huang, Kam Woh Ng, Tian Xie, Xiao Han, Yuren Cong, Hang Li, Chuyan Zhu, Aditya Patel, Tao Xiang, and Sen He. Scaling zero-shot reference-to-video generation.arXiv preprint, 2025

    2025

  49. [49]

    Reward forcing: Efficient streaming video generation with rewarded distribution matching distillation, 2025

    Yunhong Lu, Yanhong Zeng, Haobo Li, Hao Ouyang, Qiuyu Wang, Ka Leong Cheng, Jiapeng Zhu, Hengyuan Cao, Zhipeng Zhang, Xing Zhu, Yujun Shen, and Min Zhang. Reward forcing: Efficient streaming video generation with rewarded distribution matching distillation, 2025. URL https://arxiv.org/abs/2512.04678

    Pith/arXiv arXiv 2025

  50. [50]

    Stable video infinity: Infinite-length video generation with error recycling.arXiv preprint arXiv:2510.09212, 2025

    Wuyang Li, Wentao Pan, Po-Chien Luan, Yang Gao, and Alexandre Alahi. Stable video infinity: Infinite-length video generation with error recycling.arXiv preprint arXiv:2510.09212, 2025

    arXiv 2025

  51. [51]

    thinking with long videos

    Zuhao Yang, Sudong Wang, Kaichen Zhang, Keming Wu, Sicong Leng, Yifan Zhang, Bo Li, Chengwei Qin, Shijian Lu, Xingxuan Li, et al. Longvt: Incentivizing" thinking with long videos" via native tool calling.arXiv preprint arXiv:2511.20785, 2025

    Pith/arXiv arXiv 2025

  52. [52]

    V-reasonbench: Toward unified reasoning benchmark suite for video generation models, 2025

    Yang Luo, Xuanlei Zhao, Baijiong Lin, Lingting Zhu, Liyao Tang, Yuqi Liu, Ying- Cong Chen, Shengju Qian, Xin Wang, and Yang You. V-reasonbench: Toward unified reasoning benchmark suite for video generation models, 2025. URL https://arxiv.org/abs/2511.16668

    arXiv 2025

  53. [53]

    Vitcot: Video-text interleaved chain-of-thought for boosting video understanding in large language models

    Yongheng Zhang, Xu Liu, Ruihan Tao, Qiguang Chen, Hao Fei, Wanxiang Che, and Libo Qin. Vitcot: Video-text interleaved chain-of-thought for boosting video understanding in large language models. InProceedings of the 33rd ACM International Conference on Multimedia, pages 5267–5276, 2025

    2025

  54. [54]

    Ruler-bench: Probing rule-based reasoning abilities of next-level video generation models for vision foundation intelligence, 2025

    Xuming He, Zehao Fan, Hengjia Li, Fan Zhuo, Hankun Xu, Senlin Cheng, Di Weng, Haifeng Liu, Can Ye, and Boxi Wu. Ruler-bench: Probing rule-based reasoning abilities of next-level video generation models for vision foundation intelligence, 2025. URL https://arxiv.org/abs/2512.02622

    arXiv 2025

  55. [55]

    Can world simulators reason? gen-vire: A generative visual reasoning benchmark,

    Xinxin Liu, Zhaopan Xu, Ming Li, Kai Wang, Yong Jae Lee, and Yuzhang Shang. Can world simulators reason? gen-vire: A generative visual reasoning benchmark,

  56. [56]

    CCHall: A novel benchmark for joint cross-lingual and cross- modal hallucinations detection in large language models

    Yongheng Zhang, Xu Liu, Ruoxi Zhou, Qiguang Chen, Hao Fei, Wenpeng Lu, and Libo Qin. CCHall: A novel benchmark for joint cross-lingual and cross- modal hallucinations detection in large language models. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Proceedings of the 63rd Annual Meeting of the Association for Com...

  57. [57]

    Large language models meet nlp: A survey.Frontiers of Computer Science, 20(11):2011361, 2026

    Libo Qin, Qiguang Chen, Xiachong Feng, Yang Wu, Yongheng Zhang, Yinghui Li, Min Li, Wanxiang Che, and Philip S Yu. Large language models meet nlp: A survey.Frontiers of Computer Science, 20(11):2011361, 2026

    2026

  58. [58]

    Are video models ready as zero-shot reasoners? an empirical study with the mme-cof benchmark.arXiv preprint arXiv:2510.26802, 2025

    Ziyu Guo, Xinyan Chen, Renrui Zhang, Ruichuan An, Yu Qi, Dongzhi Jiang, Xiangtai Li, Manyuan Zhang, Hongsheng Li, and Pheng-Ann Heng. Are video models ready as zero-shot reasoners? an empirical study with the mme-cof benchmark.arXiv preprint arXiv:2510.26802, 2025

    arXiv 2025

  59. [59]

    ISBN 979-8-89176-251-0

    Association for Computational Linguistics. ISBN 979-8-89176-251-0. doi: 10. 18653/v1/2025.acl-long.1485. URL https://aclanthology.org/2025.acl-long.1485/

    2025

  60. [60]

    Latent visual cache for video reasoning.arXiv preprint arXiv:2607.02607, 2026

    Yongheng Zhang, Zhipeng Xu, Hao Wu, Yinghui Li, Di Yin, Xing Sun, and Philip S Yu. Latent visual cache for video reasoning.arXiv preprint arXiv:2607.02607, 2026

    Pith/arXiv arXiv 2026

  61. [61]

    Mmgr: Multi-modal generative reasoning.arXiv preprint arXiv:2512.14691, 2025

    Zefan Cai, Haoyi Qiu, Tianyi Ma, Haozhe Zhao, Gengze Zhou, Kung-Hsiang Huang, Parisa Kordjamshidi, Minjia Zhang, Xiao Wen, Jiuxiang Gu, et al. Mmgr: Multi-modal generative reasoning.arXiv preprint arXiv:2512.14691, 2025

    arXiv 2025

  62. [62]

    Video models start to solve chess, maze, sudoku, mental rotation, and raven’matrices.arXiv preprint arXiv:2512.05969, 2025

    Hokin Deng. Video models start to solve chess, maze, sudoku, mental rotation, and raven’matrices.arXiv preprint arXiv:2512.05969, 2025

    arXiv 2025

  63. [65]

    Beyond the last frame: Process-aware evaluation for generative video reasoning, 2026

    Yifan Li, Yukai Gu, Yingqian Min, Zikang Liu, Yifan Du, Kun Zhou, Min Yang, Wayne Xin Zhao, and Minghui Qiu. Beyond the last frame: Process-aware evaluation for generative video reasoning, 2026. URL https://arxiv.org/abs/2512. 24952

    2026

  64. [2019]

    URL https://openreview.net/forum?id=rylgEULtdN

  65. [2025]

    URL https://arxiv.org/abs/2511.13853

    arXiv