{"id":"5686fc14-b874-4caa-9c90-2f35be975af1","arxiv_id":"2605.30346","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"YoCausal benchmark shows video diffusion models detect the arrow of time but lack genuine causal understanding relative to humans.","lead":"YoCausal introduces a benchmark that reverses real videos to test whether video diffusion models understand causality or only statistical patterns. A smart generalist might read it to gauge how close current AI video tools are to functioning as reliable world simulators.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Time-reversal as counterfactuals and VLM stratification lack validation for isolating causality","rationale":"The reader's weakest_assumption exactly matches the two load-bearing premises above. The abstract-only review already flagged them; the method description in the provided abstract supplies no additional safeguards (e.g., human validation or synthetic controls), so the concern remains load-bearing and the UNVERDICTED status is unchanged.","tokens_in":1676,"tokens_out":390,"duration_ms":25476,"concrete_test":"Sample 100 videos from the YoCausal dataset; obtain independent labels from 3 human raters using the paper's causal/non-causal criteria; compute Cohen's kappa between human majority vote and the VLM stratification used for CCI. If kappa < 0.6, the stratification fails to isolate genuine causal reasoning.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that RSI (arrow-of-time perception via denoising loss on reversed videos) and CCI (causality via VLM stratification) are separable and that VDMs show a gap to human causal cognition—rests on two unverified premises. First, temporally reversing real-world videos is treated as producing 'natural counterfactual samples,' but reversal simultaneously violates multiple irreversible physical processes (entropy, friction, gravity) without corresponding to a targeted do-intervention or single-cause counterfactual in a causal graph; this risks conflating general physics violation detection with causal reasoning. Second, CCI relies on an off-the-shelf VLM to partition videos into causal vs. non-causal subsets, yet no calibration against human judgments or formal causal criteria is described, leaving open that CCI largely reflects VLM priors or annotation artifacts rather than VDM causal understanding. Because the paper reports that 'perceiving the arrow of time does not imply understanding causality' solely from the divergence of these two indices, any shared bias between RSI and CCI undermines the dissociation.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper introduces YoCausal, a two-level benchmark for evaluating causal understanding in video diffusion models (VDMs) inspired by the Violation of Expectation paradigm. It treats temporally reversed real-world videos as natural counterfactual samples, defines the Reverse Surprise Index (RSI) to quantify arrow-of-time perception via denoising loss, and the Causality Cognition Index (CCI) via VLM-based stratification into causal vs. non-causal subsets. Evaluation across 13 state-of-the-art VDMs concludes that arrow-of-time perception does not imply causal understanding and that a significant gap remains relative to human causal cognition.","tokens_in":1890,"tokens_out":499,"duration_ms":25041,"significance":"If the premises hold, this provides a scalable, real-world, zero-cost protocol for disentangling temporal bias from causal reasoning in generative video models, extending cognitive science methods to assess progress toward world models. It offers falsifiable indices and highlights a dissociation that could guide future VDM development.","major_comments":[{"comment":"Abstract: Treating temporally reversed videos as 'natural counterfactual samples' is load-bearing for the central dissociation claim, yet reversal simultaneously violates multiple irreversible processes (entropy, gravity, friction) without corresponding to a targeted do-intervention or single-cause counterfactual in a causal graph; this risks conflating general physics-violation detection with causal reasoning.","section":"Abstract"},{"comment":"Abstract: CCI relies on an off-the-shelf VLM to partition videos into causal vs. non-causal subsets with no reported calibration against human judgments or formal causal criteria; without this, the reported gap between RSI and CCI may reflect VLM annotation artifacts rather than VDM causal understanding.","section":"Abstract"},{"comment":"Abstract: The evaluation on 13 VDMs reports a dissociation and gap to humans but supplies no quantitative details on CCI computation, error bars, dataset sizes, or controls for VLM bias, preventing verification that the data support the central claim.","section":"Abstract"}],"minor_comments":[{"comment":"The abstract would benefit from explicit references to causal inference literature (e.g., Pearl's do-calculus) and prior VoE implementations to situate the protocol.","section":null}],"recommendation":"major_revision","confidential_remarks":"The provided abstract lacks the quantitative and methodological details needed for a full assessment; the full manuscript should be checked for whether the validation experiments requested above are already present."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for their constructive feedback. We address each major comment below with clarifications on our methodology and indicate planned revisions where appropriate.","responses":[{"response":"We agree that time reversal is not a targeted do-intervention on a single causal variable. Our method draws directly from the Violation of Expectation paradigm, using reversal to create scalable, real-world violations of expected physical dynamics rather than precise graph interventions. RSI quantifies detection of such violations as a necessary (but not sufficient) component of causal perception. We will revise the abstract and method sections to describe these as 'approximate natural counterfactuals' to prevent overstatement.","revision_made":"partial","referee_comment":"Treating temporally reversed videos as 'natural counterfactual samples' is load-bearing for the central dissociation claim, yet reversal simultaneously violates multiple irreversible processes (entropy, gravity, friction) without corresponding to a targeted do-intervention or single-cause counterfactual in a causal graph; this risks conflating general physics-violation detection with causal reasoning."},{"response":"The concern is valid. The current manuscript applies an off-the-shelf VLM with prompts targeting agent-driven cause-effect relations but does not report human calibration. We will add a human validation study on a data subset, report agreement metrics, and include the exact stratification prompts and criteria in the revised version.","revision_made":"yes","referee_comment":"CCI relies on an off-the-shelf VLM to partition videos into causal vs. non-causal subsets with no reported calibration against human judgments or formal causal criteria; without this, the reported gap between RSI and CCI may reflect VLM annotation artifacts rather than VDM causal understanding."},{"response":"The full manuscript contains dataset sizes, the CCI formula, and per-model results. We agree that error bars, explicit dataset statistics, and VLM bias controls are insufficiently detailed. We will add a table with video counts, standard errors across VLM runs, and a discussion of bias mitigation in the revision.","revision_made":"yes","referee_comment":"The evaluation on 13 VDMs reports a dissociation and gap to humans but supplies no quantitative details on CCI computation, error bars, dataset sizes, or controls for VLM bias, preventing verification that the data support the central claim."}],"tokens_in":1365,"tokens_out":502,"duration_ms":26349,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main thing to know is that YoCausal applies denoising loss on reversed real videos to measure arrow-of-time sensitivity (RSI) and then uses a VLM to split videos into causal versus non-causal groups (CCI), claiming the two do not align in 13 tested models.\n\nThe new elements are the zero-cost reversal protocol on real footage and the two-level split that tries to move past synthetic data. Applying this to multiple diffusion models and linking it to the Violation of Expectation idea gives a concrete way to probe the issue.\n\nThe soft spots are in the core premises. Reversing videos breaks entropy, gravity, and other processes together, so the RSI may capture general physics violation detection rather than targeted causal reasoning. The CCI depends on an external VLM for stratification with no reported calibration to human judgments or formal causal criteria, which leaves room for the index to track VLM artifacts instead.\n\nThe abstract gives no dataset sizes, CCI computation details, or error bars, making the size of the claimed gap to humans hard to assess. If the full paper adds those controls and shows the dissociation survives them, the result would be sharper.\n\nThis is for groups building video models for robotics or simulation who need causality checks. Readers can take the protocol as a starting point but should add their own validation.\n\nIt deserves peer review so the authors can strengthen the counterfactual and labeling steps.","headline":"The paper's time-reversal benchmark for separating temporal perception from causality in video models has a practical setup but rests on unvalidated assumptions about counterfactuals and VLM labels.","tokens_in":2379,"tokens_out":365,"would_cite":false,"duration_ms":22667,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Video diffusion models notice when time runs backward but do not grasp cause and effect like humans do.","keywords":["video diffusion models","causality","world models","benchmark","counterfactuals","arrow of time","violation of expectation"],"falsifier":"A model that scores equally on causal and non-causal subsets in the Causality Cognition Index or reaches human-level scores on both indices would contradict the reported gap between time perception and causal understanding.","tokens_in":2586,"feed_emoji":"🔄","tokens_out":623,"duration_ms":22800,"temperature":0.7,"pith_summary":"The paper tests whether video diffusion models function as world models by checking if they understand causality or only statistical timing patterns. It introduces YoCausal, which reverses real videos at no cost to create natural counterfactual examples and measures both time-direction awareness and causal reasoning separately. On 13 current models, time-direction detection turns out to be unrelated to actual causal understanding, leaving a clear shortfall compared with human performance. This distinction matters because models positioned as simulators of the physical world need reliable cause-effect reasoning to predict what happens next in new situations.","feed_headline":"Video models detect time reversal but miss causality","feed_subtitle":"Reversing real videos in a new benchmark shows 13 diffusion models still lag human causal reasoning.","key_machinery":"YoCausal benchmark that creates natural counterfactuals by temporally reversing real videos, then computes Reverse Surprise Index for time-direction sensitivity and Causality Cognition Index to isolate genuine causal reasoning from temporal bias.","core_discovery":"YoCausal is a two-level benchmark that first quantifies arrow-of-time perception through a Reverse Surprise Index based on denoising loss when videos are played backward, then applies a Causality Cognition Index that uses a vision-language model to split videos into causal and non-causal groups. Evaluation across 13 state-of-the-art video diffusion models shows that strong performance on the first index does not produce strong performance on the second, revealing that temporal pattern recognition alone does not deliver causal cognition and that current models remain far from human levels on real-world videos.","pith_inferences":["Improving the Causality Cognition Index could lead models to generate more physically consistent future frames.","The same reversal technique might expose causal gaps in other generative domains such as audio or 3D scenes.","Explicit causal objectives beyond standard diffusion training may be needed to close the human gap."],"forward_implications":["Models can detect time reversal without acquiring causal reasoning.","Synthetic-data benchmarks may overlook real-world causal failures.","Current video diffusion models fall short of human causal cognition.","The two-level protocol can be extended to new models at low cost."],"fun_headline_variants":["Reversal seen but causality missed in video models","New test shows video AI lacks causal cognition","VDMs perceive time arrow without understanding cause","Causality gap persists in video diffusion models"],"cache_read_input_tokens":64,"weakest_assumption_plain":"Reversing real-world videos produces valid natural counterfactual samples, and a vision-language model can accurately separate causal from non-causal videos.","fun_headline_variants_meta":{"raw":{"variants":["Reversal seen but causality missed in video models","New test shows video AI lacks causal cognition","VDMs perceive time arrow without understanding cause","Causality gap persists in video diffusion models"]},"model":"grok-4.3","cost_usd":0.007175,"raw_usage":{"total_tokens":3304,"prompt_tokens":653,"num_sources_used":0,"completion_tokens":55,"cost_in_usd_ticks":71749500,"prompt_tokens_details":{"text_tokens":653,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2596,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":653,"tokens_out":55,"duration_ms":19737,"temperature":1.0,"reasoning_tokens":2596,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-29T08:24:31.341787+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A model that scores equally on causal and non-causal subsets in the Causality Cognition Index or reaches human-level scores on both indices would contradict the reported gap between time perception and causal understanding.","supporting_citations":[],"review_version":1}