REVIEW 8 cited by
iVideoGPT: Interactive VideoGPTs are Scalable World Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
World models empower model-based agents to interactively explore, reason, and plan within imagined environments for real-world decision-making. However, the high demand for interactivity poses challenges in harnessing recent advancements in video generative models for developing world models at scale. This work introduces Interactive VideoGPT (iVideoGPT), a scalable autoregressive transformer framework that integrates multimodal signals--visual observations, actions, and rewards--into a sequence of tokens, facilitating an interactive experience of agents via next-token prediction. iVideoGPT features a novel compressive tokenization technique that efficiently discretizes high-dimensional visual observations. Leveraging its scalable architecture, we are able to pre-train iVideoGPT on millions of human and robotic manipulation trajectories, establishing a versatile foundation that is adaptable to serve as interactive world models for a wide range of downstream tasks. These include action-conditioned video prediction, visual planning, and model-based reinforcement learning, where iVideoGPT achieves competitive performance compared with state-of-the-art methods. Our work advances the development of interactive general world models, bridging the gap between generative video models and practical model-based reinforcement learning applications. Code and pre-trained models are available at https://thuml.github.io/iVideoGPT.
Forward citations
Cited by 8 Pith papers
-
DreamX-Phi 1.0: Action-Conditioned Video World Model for Robotic Manipulation
A geometry-aware video world model for bimanual manipulation that ranks first on WorldArena 2.0 Track 1 and ties for second on Track 2.
-
Open-AoE: An Open Egocentric Manipulation Dataset and Toolchain for Embodied Learning
Open-AoE releases 2,000 hours of smartphone egocentric manipulation video with MANO hand poses, camera trajectories, atomic action labels, and training tools for VLA and world-model pipelines.
-
Causally Debiased Latent Action Model for Embodied Action Conditioned World Models
Three lightweight LAM fine-tuning objectives (foreground-weighted reconstruction, primitive contrastive learning, zero-transition calibration) debias latent actions and yield stronger, cheaper robot-action following i...
-
Met$^2$Net: A Decoupled Two-Stage Spatio-Temporal Forecasting Model for Complex Meteorological Systems
Met2Net trains multivariable weather forecasters with per-variable encoders and a two-stage latent-space objective, beating TAU on WeatherBench and ERA5.
-
Graph World Model
The Graph World Model uses action nodes and graph message passing to unify multimodal and graph-structured tasks, but its 'outperforms or matches' claim is contradicted by results on Goodreads.
-
FLIP: Flow-Centric Generative Planning as General-Purpose Manipulation World Model
FLIP combines flow generation, flow-conditioned video prediction, and a learned value function to plan long robot manipulations from only image and language inputs.
-
Quo Vadis, World Modeling?
An agent-centric reframing of world modeling, replacing physical state prediction with 'information transitions' organized into six proxy functions and three empowerment levels.
-
On the Benefits of Instance Decomposition in Video Prediction Models
Explicit instance decomposition with per-class shared weights improves latent-transformer video prediction in the paper's experiments, but the claimed advantage is weakened by mismatched parameter counts and test-set ...
Discussion (0). Continue with ORCID to comment.