REVIEW 23 cited by
WorldSimBench: Towards Video Generation Models as World Simulators
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Recent advancements in predictive models have demonstrated exceptional capabilities in predicting the future state of objects and scenes. However, the lack of categorization based on inherent characteristics continues to hinder the progress of predictive model development. Additionally, existing benchmarks are unable to effectively evaluate higher-capability, highly embodied predictive models from an embodied perspective. In this work, we classify the functionalities of predictive models into a hierarchy and take the first step in evaluating World Simulators by proposing a dual evaluation framework called WorldSimBench. WorldSimBench includes Explicit Perceptual Evaluation and Implicit Manipulative Evaluation, encompassing human preference assessments from the visual perspective and action-level evaluations in embodied tasks, covering three representative embodied scenarios: Open-Ended Embodied Environment, Autonomous, Driving, and Robot Manipulation. In the Explicit Perceptual Evaluation, we introduce the HF-Embodied Dataset, a video assessment dataset based on fine-grained human feedback, which we use to train a Human Preference Evaluator that aligns with human perception and explicitly assesses the visual fidelity of World Simulators. In the Implicit Manipulative Evaluation, we assess the video-action consistency of World Simulators by evaluating whether the generated situation-aware video can be accurately translated into the correct control signals in dynamic environments. Our comprehensive evaluation offers key insights that can drive further innovation in video generation models, positioning World Simulators as a pivotal advancement toward embodied artificial intelligence.
Forward citations
Cited by 23 Pith papers
-
WorldSimProbe: Diagnosing Simulator Faithfulness in Action-Conditioned World Models for Embodied Manipulation
WorldSimProbe is a five-suite benchmark showing that six action-conditioned world models systematically degrade in action-to-motion fidelity and interaction grounding across RoboTwin, ManiSkill, and LIBERO.
-
KineBench: Benchmarking Embodied World Models via IDM-Free Kinematic Grounding
KineBench evaluates embodied world models by extracting 6D robot-pose trajectories from generated videos, executing them in a physics simulator, and scoring them with SPARC and manipulability metrics.
-
MemoBench: Benchmarking World Modeling in Dynamically Changing Environments
None of ten tested video-generation models reliably remembers objects after occlusion in dynamic scenes; static-camera videos inflate consistency scores.
-
What if? Emulative Simulation with World Models for Situated Reasoning
WanderDream supplies 15.8K panoramic mental-exploration videos and 158K QA pairs showing that world-model imagination measurably improves situated spatial reasoning without active exploration.
-
WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivity
Across 1,474 cases and 20 models, WorldExam shows that video world models split along paradigm lines — camera-, action-, and language-driven models each dominate one capability, and none combines strong reactivity wit...
-
DynaVieW: Schema-Guided World Modeling for Understanding Hierarchical Visual Dynamics
Schema-guided interleaved state-transition pretraining with selective attention and reweighted loss improves hierarchical visual dynamics modeling for narrative generation and world simulation.
-
GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation
Long-horizon action-faithful consistency, not short-term visual realism, dominates world-model reliability for robot policy evaluation; GigaWorld-1 implements that roadmap and gains 14.9% on evaluator-alignment metrics.
-
Towards Spatial Trace with Reasoning in Vision-Language Models for Robotics
A 3D-aware VLM, RoboTracer, generates metric-grounded spatial traces for robot manipulation using scale supervision and metric-sensitive reinforcement rewards.
-
BMMR: A Large-Scale Bilingual Multimodal Multi-Discipline Reasoning Dataset
BMMR provides a 110k-question bilingual, multimodal, college-level dataset across 300 subjects where state-of-the-art models score at most about 50%.
-
CDP: Towards Robust Autoregressive Visuomotor Policy Learning via Causal Diffusion
Causal Diffusion Policy adds historical action conditioning and attention cache sharing to diffusion-based robot policies, improving success rates on most tested manipulation tasks under degraded observations.
-
From Generation to Generalization: Emergent Few-Shot Learning in Video Diffusion Models
Frozen CogVideoX1.5, adapted with LoRA on 3 to 30 input-output videos, performs segmentation, pose estimation, and abstract reasoning (ARC-AGI 16.75%) with modest but real generalization.
-
Can VLMs Predict Future States? Bootstrapping World Models from Inverse Dynamics
Using an inverse dynamics model to label and verify frame transitions lets vision-language models learn forward dynamics and outperform specialized image editors on Aurora-Bench.
-
Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval
Context-as-Memory conditions video generation on selected historical frames chosen by camera FOV overlap, improving scene consistency in long generated videos.
-
MOVi: Training-free Text-conditioned Multi-Object Video Generation
MOVi improves multi-object video generation without retraining by using LLM-planned trajectories to reinitialize the diffusion noise and by reweighting attention to stop objects from mixing together.
-
DanceTogether! Identity-Preserving Multi-Person Interactive Video Generation
A diffusion model fuses per-person masks with pose keypoints to generate identity-preserving, two-person interactive videos from a single reference image, outperforming prior single-person-animation pipelines.
-
Generative Physical AI in Vision: A Survey
A structured review that categorizes physics-aware generative models in vision into explicit-simulation and implicit-learning families and proposes six integration paradigms.
-
Is Your World Simulator a Good Story Presenter? A Consecutive Events-Based Benchmark for Future Long Video Generation
A new story-completion benchmark, StoryEval, shows that 11 current text-to-video models complete fewer than half of the consecutive events in short story prompts.
-
Owl-1: Omni World Model for Consistent Long Video Generation
Owl-1 generates long, multi-scene videos by using a language model to maintain a latent state and predict text dynamics, then rendering each clip with a video diffusion model.
-
Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detection
A VLM-generated code monitor tracks geometric elements and evaluates spatio-temporal constraints, enabling real-time reactive and proactive failure detection for robotic manipulation.
-
Quo Vadis, World Modeling?
An agent-centric reframing of world modeling, replacing physical state prediction with 'information transitions' organized into six proxy functions and three empowerment levels.
-
A Definition and Roadmap for World Models
A perspective article defining world models as finite-resource compression of physical state transitions and outlining a roadmap toward physical AGI via unified representations and interactive simulators.
-
Towards Conscious Service Robots
Service robots should be built with a two-system cognitive architecture, including a global workspace and metacognitive monitoring, to generalize to novel situations.
-
A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges
A survey that catalogs large vision-language models, their alignment methods, benchmarks, and challenges, but is compromised by inconsistent counts and misclassified entries.
Discussion (0). Continue with ORCID to comment.