REVIEW 20 cited by
Transformers are Sample-Efficient World Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Deep reinforcement learning agents are notoriously sample inefficient, which considerably limits their application to real-world problems. Recently, many model-based methods have been designed to address this issue, with learning in the imagination of a world model being one of the most prominent approaches. However, while virtually unlimited interaction with a simulated environment sounds appealing, the world model has to be accurate over extended periods of time. Motivated by the success of Transformers in sequence modeling tasks, we introduce IRIS, a data-efficient agent that learns in a world model composed of a discrete autoencoder and an autoregressive Transformer. With the equivalent of only two hours of gameplay in the Atari 100k benchmark, IRIS achieves a mean human normalized score of 1.046, and outperforms humans on 10 out of 26 games, setting a new state of the art for methods without lookahead search. To foster future research on Transformers and world models for sample-efficient reinforcement learning, we release our code and models at https://github.com/eloialonso/iris.
Forward citations
Cited by 20 Pith papers
-
TaskSense: Focusing on What Matters in World Models
TaskSense filters visual observations with latent-conditioned stochastic spatial attention before encoding, improving world-model control under visual distractions relative to DreamerV3.
-
Weights or Skills? A Survey of Robot-Learning Techniques: from Action-Predicting Weights to Robots that Write their Own Skills
A taxonomy of robot learning on a weights-versus-skills axis, with a five-rung self-improvement ladder whose top cell (feedback plus memory plus search) holds only a few recent systems.
-
Autoregressive Diffusion World Models for Off-Policy Evaluation of LLM Agents
ADWM learns a latent diffusion world model with per-transition independent denoising and policy-conditioned guidance to enable accurate offline evaluation of LLM agent policies.
-
Latent Policy Barrier: Learning Robust Visuomotor Policies by Staying In-Distribution
Latent Policy Barrier improves behavior-cloned visuomotor policies by using a latent dynamics model trained on expert and rollout data to guide actions back toward in-distribution expert states.
-
Dyn-O: Building Structured World Models with Object-Centric Representations
Dyn-O learns object-centric world models directly from pixels in complex Procgen games, using SAM2-guided slot attention and Mamba state-space dynamics, and reports better rollout prediction than DreamerV3.
-
ParticleFormer: A 3D Point Cloud World Model for Multi-Object, Multi-Material Robotic Manipulation
ParticleFormer uses a Transformer over point-cloud particles and a hybrid Chamfer-Hausdorff loss to predict multi-material object dynamics, and it reports lower errors than GNN and image-based baselines in simulation ...
-
EgoM2P: Egocentric Multimodal Multitask Pretraining
A masked pretraining model over RGB, depth, gaze, and camera-pose tokens matches specialist egocentric vision systems on four tasks while running at 300+ frames per second.
-
Celo: Training Versatile Learned Optimizers on a Compute Diet
A compute-cheap learned optimizer, Celo, beats tuned hand-crafted and learned optimizers on a 17-task out-of-distribution suite by combining task augmentation, a simple hierarchical design, and two-stage decoupled met...
-
GLAM: Global-Local Variation Awareness in Mamba-based World Model
GLAM improves world model prediction in model-based RL by feeding state differences into two parallel Mamba modules and training agents on imagined variation-aware trajectories.
-
FLIP: Flow-Centric Generative Planning as General-Purpose Manipulation World Model
FLIP combines flow generation, flow-conditioned video prediction, and a learned value function to plan long robot manipulations from only image and language inputs.
-
Quo Vadis, World Modeling?
An agent-centric reframing of world modeling, replacing physical state prediction with 'information transitions' organized into six proxy functions and three empowerment levels.
-
LEMUR: Learning to Align with Multi-Objective Reinforcement Learning from Preference Feedback
LEMUR jointly learns a separate reward model for each teacher's preferences and uses them to train a population of multi-objective policies, beating baselines that merge feedback into one reward.
-
RynnVLA-002: A Unified Vision-Language-Action and World Model
A single model that jointly predicts robot actions and future images outperforms separate action-only and video-only models on LIBERO and real SO100 manipulation tasks.
-
Higher Embedding Dimension Creates a Stronger World Model for a Simple Sorting Task
In an RL-trained transformer solving adjacent-swap sorting, larger embedding dimensions improve the monotonicity of an internal order-encoding in attention weights and the match to a largest-adjacent-difference swap r...
-
Improving Transformer World Models for Data-Efficient RL
A transformer world model agent using a static patch tokenizer, warmup before imagination training, and block teacher forcing reaches 69.66% reward on Craftax-classic, beating DreamerV3 and the human expert figure.
-
TransZero: Parallel Tree Expansion in MuZero using Transformer Networks
TransZero parallelizes Monte Carlo tree search expansion using a transformer dynamics network and a variance-based evaluator, achieving up to an 11x wall-clock speedup over MuZero without sacrificing final reward.
-
Large Model Empowered Embodied AI: A Survey on Decision-Making and Embodied Learning
A review that categorizes large-model-empowered embodied AI into hierarchical and end-to-end decision-making, imitation and reinforcement learning, and world models.
-
TransDreamerV3: Implanting Transformer In DreamerV3
TransDreamerV3 replaces DreamerV3's GRU memory with a simple transformer encoder and reports mixed gains on four Atari and Crafter tasks.
-
Reinforcement Learning: From Algorithms To Foundation Models
A dissertation uniting the author's published results: non-exploitable Nash-DQN policies and the FightLadder benchmark for games, plus diffusion/consistency-model world models for RL — a compilation rather than new results.
-
Advances in Transformers for Robotic Applications: A Review
A survey of Transformer applications in robotic perception, planning, control, human-robot interaction, and reinforcement learning.
Discussion (0). Continue with ORCID to comment.