Pith. sign in

REVIEW 11 cited by

Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2501.03230 v1 pith:CAENBIY4 submitted 2024-05-07 cs.AI cs.CV

Hao Fei , Shengqiong Wu , Wei Ji , Hanwang Zhang , Meishan Zhang , Mong-Li Lee , Wynne Hsu This is my paper
classification cs.AIcs.CV
keywords videoreasoningcomplexspatial-temporalunderstandingcomprehensionexistingfine-grained
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Existing research of video understanding still struggles to achieve in-depth comprehension and reasoning in complex videos, primarily due to the under-exploration of two key bottlenecks: fine-grained spatial-temporal perceptive understanding and cognitive-level video scene comprehension. This paper bridges the gap by presenting a novel solution. We first introduce a novel video Multimodal Large Language Model (MLLM), MotionEpic, which achieves fine-grained pixel-level spatial-temporal video grounding by integrating video spatial-temporal scene graph (STSG) representation. Building upon MotionEpic, we then develop a Video-of-Thought (VoT) reasoning framework. VoT inherits the Chain-of-Thought (CoT) core, breaking down a complex task into simpler and manageable sub-problems, and addressing them step-by-step from a low-level pixel perception to high-level cognitive interpretation. Extensive experiments across various complex video QA benchmarks demonstrate that our overall framework strikingly boosts existing state-of-the-art. To our knowledge, this is the first attempt at successfully implementing the CoT technique for achieving human-level video reasoning, where we show great potential in extending it to a wider range of video understanding scenarios. Project is open at https://haofei.vip/VoT

Discussion (0). Sign in to comment.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. EndoCoT: Scaling Endogenous Chain-of-Thought Reasoning in Diffusion Models

    cs.CV 2026-03 conditional novelty 6.0 of 10

    Iterative latent thought refinement plus terminal text grounding lets diffusion models solve multi-step visual reasoning tasks at 92.1% average accuracy, beating DiffThinker by 8.3 points.

  2. LaRe: Latent Refocusing for Multimodal Reasoning

    cs.CV 2025-11 reject novelty 6.0 of 10

    LaRe performs iterative visual refocusing in latent space and reports accuracy gains with fewer tokens, but its main experiments compare against baselines trained with less data.

  3. AdsQA: Towards Advertisement Video Understanding

    cs.CV 2025-09 conditional novelty 6.0 of 10

    AdsQA adds an ad-video question-answering benchmark and ReAd-R, a GRPO-trained model that beats 7B baselines but not larger closed models.

  4. ProPy: Building Interactive Prompt Pyramids upon CLIP for Partially Relevant Video Retrieval

    cs.CV 2025-08 conditional novelty 6.0 of 10

    A hierarchical prompt pyramid over CLIP with ancestor-descendant attention improves partially relevant video retrieval.

  5. CausalStep: A Benchmark for Explicit Stepwise Causal Reasoning in Videos

    cs.CV 2025-07 conditional novelty 6.0 of 10

    CausalStep introduces a stepwise video QA protocol and reports that top multimodal models (chain success rate 51%) remain far below human performance (79%) on explicit causal chains.

  6. DAVID-XR1: Detecting AI-Generated Videos with Explainable Reasoning

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A video-language model fine-tuned on a new defect-annotated dataset detects AI-generated videos from unseen generators with 76.7% accuracy and gives written explanations, though the test set is small and the dataset i...

  7. VFaith: Do Large Multimodal Models Really Reason on Seen Images Rather than Previous Memories?

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A new benchmark with edited-image question pairs shows that multimodal reasoning models lose accuracy when visual cues change, suggesting their reasoning is often not faithfully tied to the image.

  8. VRBench: A Benchmark for Multi-Step Reasoning in Long Narrative Videos

    cs.CV 2025-06 conditional novelty 6.0 of 10

    VRBench is a benchmark of 960 long narrative videos with 8,243 human-written multi-step questions, plus a two-level evaluation of answer accuracy and reasoning quality for 31 large models.

  9. Uncertainty-o: One Model-agnostic Framework for Unveiling Uncertainty in Large Multimodal Models

    cs.CV 2025-06 conditional novelty 5.0 of 10

    Uncertainty-o estimates uncertainty in large multimodal models by perturbing prompts and computing entropy over semantically clustered answers, improving hallucination detection across five modalities.

  10. Argus Inspection: Do Multimodal Large Language Models Possess the Eye of Panoptes?

    cs.CV 2025-06 conditional novelty 5.0 of 10

    A new 1,430-item multimodal benchmark shows that leading multimodal LLMs rarely notice small visual traps needed for commonsense safety reasoning, with top scores near 0.46 on a scale whose maximum is about 0.97.

  11. Empowering Multimodal LLMs with External Tools: A Comprehensive Survey

    cs.CV 2025-08 unverdicted novelty 2.0 of 10

    A survey paper maps how external tools are used to augment multimodal large language models across data, tasks, evaluation, and future directions.

Pith tools