REVIEW 8 cited by
CATER: A diagnostic dataset for Compositional Actions and TEmporal Reasoning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Computer vision has undergone a dramatic revolution in performance, driven in large part through deep features trained on large-scale supervised datasets. However, much of these improvements have focused on static image analysis; video understanding has seen rather modest improvements. Even though new datasets and spatiotemporal models have been proposed, simple frame-by-frame classification methods often still remain competitive. We posit that current video datasets are plagued with implicit biases over scene and object structure that can dwarf variations in temporal structure. In this work, we build a video dataset with fully observable and controllable object and scene bias, and which truly requires spatiotemporal understanding in order to be solved. Our dataset, named CATER, is rendered synthetically using a library of standard 3D objects, and tests the ability to recognize compositions of object movements that require long-term reasoning. In addition to being a challenging dataset, CATER also provides a plethora of diagnostic tools to analyze modern spatiotemporal video architectures by being completely observable and controllable. Using CATER, we provide insights into some of the most recent state of the art deep video architectures.
Forward citations
Cited by 8 Pith papers
-
Do Vision-Language Models Have Internal World Models? Towards an Atomic Evaluation
A new 23-dimension benchmark finds that even the best VLMs score near random on motion trajectory, temporal extension, and several prediction tasks, far below humans, suggesting weak internal world models.
-
Finding the Trigger: Causal Abductive Reasoning on Video Events
A new video benchmark and graph-based method for identifying the root-cause trigger event behind a target event, using counterfactually generated labels.
-
Video Representation Learning with Joint-Embedding Predictive Architectures
VJ-VCR applies variance-covariance regularization to a video joint-embedding predictive architecture and beats a generative baseline at probing dynamics from frozen representations.
-
IMBench: A Benchmark for Intuitive Robotic Manipulation
IMBench is a 35-task robosuite benchmark with a three-stage evaluation showing current VLMs and robot policies fail to convert physical reasoning into executable manipulation.
-
An Empirical Study of Autoregressive Pre-training from Videos
Autoregressive next-token prediction on video and image tokens yields competitive visual representations across recognition, tracking, and robotics benchmarks, with scaling laws that are slower than those of language models.
-
VideoCogQA: A Controllable Benchmark for Evaluating Cognitive Abilities in Video-Language Models
A new controllable synthetic-video benchmark shows that even state-of-the-art video-language models struggle with abstract and symbolic video cognition, with accuracy falling as task difficulty rises.
-
Video Understanding by Design: How Datasets Shape Video Models
A dataset-centric framework that explains video architectures as responses to structural properties of benchmark datasets.
-
Do Language Models Understand Time?
A survey arguing that video-LLMs rely on pretrained encoders and short-biased datasets, leaving them weak at long-term temporal reasoning such as causality and event progression.
Discussion (0). Continue with ORCID to comment.