DeepMind Control Suite

Abbas Abdolmaleki; Alistair Muldal; Andrew Lefrancq; David Budden; Diego de las Casas; Josh Merel; Martin Riedmiller; Timothy Lillicrap; Tom Erez; Yazhe Li

DeepMind Control Suite

T0 review · 0 major / 2 minor · reviewed 2026-05-13 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The DeepMind Control Suite offers a standardized set of continuous control tasks to benchmark reinforcement learning agents.

desk verdict This is mainly a practical code release for a standardized MuJoCo-based RL benchmark suite with baselines, useful for comparisons but not a conceptual leap. read the letter →

arxiv 1801.00690 v1 pith:IUSU7BX2 submitted 2018-01-02 cs.AI

Yuval Tassa , Yotam Doron , Alistair Muldal , Tom Erez , Yazhe Li , Diego de Las Casas , David Budden , Abbas Abdolmaleki

show 4 more authors

Josh Merel Andrew Lefrancq Timothy Lillicrap Martin Riedmiller

This is my paper · ORCID

classification cs.AI

keywords reinforcementlearningcontinuouscontrolbenchmarksuiteMuJoCoroboticssimulationpolicy

checked against Foundation.DAlembert.Inevitability

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces the DeepMind Control Suite, a collection of continuous control tasks designed with a standardized structure and interpretable rewards. The tasks are written in Python and use the MuJoCo physics engine, making them straightforward to use and customize. By providing these benchmarks along with performance data for several algorithms, the suite aims to facilitate fair comparisons between different reinforcement learning methods. A sympathetic reader would care because consistent benchmarks can accelerate progress in the field by reducing the need for researchers to create their own test environments.

What carries the argument

The Control Suite, a set of continuous control tasks with standardized structure and interpretable rewards.

What would settle it

Demonstrating that top-performing agents on the Control Suite perform poorly on a new set of similar control tasks not included in the suite would falsify its value as a general benchmark.

Watch

Extended reading notes

Core claim

The authors present the Control Suite as a publicly available set of continuous control tasks with standardized structure and interpretable rewards, powered by MuJoCo and implemented in Python, intended to serve as performance benchmarks for reinforcement learning agents.

Load-bearing premise

The selected tasks and their reward functions adequately represent real-world continuous control challenges so that performance generalizes.

Editorial extensions

If this is right

Algorithms can be evaluated and compared using the same tasks and rewards.
Researchers can easily modify the tasks due to the Python implementation.
The suite includes initial benchmarks for several learning algorithms.
The tasks are accessible to the public via the provided repository.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Widespread adoption could lead to more reproducible results in continuous control research.
Success on these tasks may suggest potential for real-world applications, though further validation would be needed.
The design choices might influence how future control benchmarks are structured.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

read the letter

The paper's main value is the release of dm_control, a Python wrapper around MuJoCo that gives a consistent interface, task structure, and interpretable rewards across a set of continuous control problems. They ship the full codebase, clear task definitions, and baseline runs for a handful of algorithms, which makes it immediately usable for running experiments and comparing results across papers. That addresses a real pain point in the field where everyone was rolling their own environments or using inconsistent setups. The video and GitHub link are straightforward additions that help adoption. What is actually new is the curation and standardization rather than any individual task or algorithm; many of the underlying physics problems existed in prior MuJoCo examples or other suites. The baselines are solid enough to get people started but not exhaustive, which fits the scope. No load-bearing math claims or derivations appear, so nothing to poke holes in there. The main soft spot is that the tasks remain somewhat artificial, and the paper does not provide evidence that success here transfers to messier real-world control; it simply offers the benchmark without overclaiming generalization. Citation patterns are clean and point to relevant prior work on MuJoCo and RL without padding. This is for RL researchers who need a common testbed for continuous control agents, especially those doing incremental algorithm work or benchmarking. A serious referee should see it because the release is reproducible, the code works, and it fills a practical gap even if the intellectual novelty is modest. I would send it to peer review rather than desk reject.

Referee Report

0 major / 2 minor

Summary. The manuscript introduces the DeepMind Control Suite, a collection of continuous control tasks implemented in Python and powered by the MuJoCo physics engine. The tasks feature a standardized structure and interpretable rewards, are intended to serve as performance benchmarks for reinforcement learning agents, and the paper supplies baseline results for several algorithms along with a public code release at https://www.github.com/deepmind/dm_control.

Significance. The release of a standardized, open-source benchmark suite with working code, clear task definitions, and reported baseline numbers constitutes a useful contribution to the RL community by enabling reproducible comparisons on continuous control problems. The absence of free parameters or invented entities in the central claim, combined with the provision of executable environments, strengthens the practical value if the suite sees adoption.

minor comments (2)

[Baselines] § on baseline experiments: specify the exact number of random seeds and the precise hyperparameter settings used for each algorithm to allow exact reproduction of the reported scores.
[Task descriptions] Figure 1 (task illustrations): ensure all panels use consistent axis scaling and label units explicitly so that reward magnitudes are immediately comparable across tasks.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their positive review of the manuscript and their recommendation to accept. We are pleased that the standardized benchmark suite and its public release are viewed as a useful contribution to the reinforcement learning community.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity detected

full rationale

The paper presents the DeepMind Control Suite as a collection of standardized continuous-control environments with interpretable rewards, implemented in Python atop MuJoCo, together with baseline runs of several existing RL algorithms. No derivation chain, predictive claim, or uniqueness theorem is advanced; the central contribution is the release of the task definitions and code at the cited GitHub repository, whose correctness is directly verifiable by inspection and execution rather than by any reduction to author-defined parameters or self-citations. Baseline numbers are simply reported outcomes of running published algorithms on the released tasks and do not constitute fitted predictions that loop back to the paper's own inputs.

Assumptions & free parameters 0 free parameters · 0 assumptions · 0 invented entities

This is a software and benchmarking contribution rather than a mathematical derivation. No free parameters are fitted to produce a central claim, no new axioms are introduced, and no invented entities are postulated.

how reviews work

0 comments

Cite this review

Pith. "Pith review of DeepMind Control Suite." pith.science (2026). https://pith.science/paper/IUSU7BX2

@misc{pith2026180100690,
  author       = {Pith},
  title        = {Pith review of: DeepMind Control Suite},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IUSU7BX2}},
  note         = {Machine review of arXiv:1801.00690}
}

read the original abstract

The DeepMind Control Suite is a set of continuous control tasks with a standardised structure and interpretable rewards, intended to serve as performance benchmarks for reinforcement learning agents. The tasks are written in Python and powered by the MuJoCo physics engine, making them easy to use and modify. We include benchmarks for several learning algorithms. The Control Suite is publicly available at https://www.github.com/deepmind/dm_control . A video summary of all tasks is available at http://youtu.be/rAai4QzcYbs .

Discussion (0). Continue with ORCID to comment.

Forward citations

Showing 60 of 175 Pith papers that cite this

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 524 citations worldwide. See all 175 Pith citations

Adaptive Compute in Latent World Models: When Depth Helps, Hurts, or Doesn't Matter
cs.LG 2026-07 unverdicted novelty 8.0 of 10

Predictor depth survives autoregressive composition on 6/9 DMC tasks but inverts on 2/9 because per-step deep supervision trains shallow exits to out-roll the full stack (routability catch-22).
From Reward-Free Representations to Preferences: Rethinking Offline Preference-Based Reinforcement Learning
cs.LG 2026-05 unverdicted novelty 8.0 of 10

A reward-free representation learning pipeline for offline PbRL achieves better preference efficiency than standard two-stage baselines by connecting RFRL concepts to preference data.
When Does LeJEPA Learn a World Model?
stat.ML 2026-05 unverdicted novelty 8.0 of 10

LeJEPA achieves linear identifiability of latent variables uniquely when the latents are Gaussian in worlds with stationary additive-noise transitions.
Language Game: Talking to Non-Human Systems
cs.LG 2026-05 unverdicted novelty 8.0 of 10

A language-game framework enables dialogue with dynamical systems such as GRNs by treating their frozen dynamics as an RL policy core, using an LM to route prompts so the system responds through its own behavior witho...
BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulation
cs.RO 2024-03 accept novelty 8.0 of 10

BEHAVIOR-1K introduces a benchmark of 1,000 human everyday activities in realistic simulated scenes together with the OMNIGIBSON physics simulator to evaluate embodied AI.
Factorized Spectral Representations for Reinforcement Learning
cs.LG 2026-07 conditional novelty 7.0 of 10

Factorizing the transition kernel as a CP (Hadamard) product of state, action, and next-state encoders yields a smaller hypothesis class and faster certified representation learning in high-dimensional continuous control.
Qantara: Bridge-Flow Training for Multi-Paradigm JEPA Control
cs.LG 2026-07 conditional novelty 7.0 of 10

A single ~21M JEPA checkpoint trained with Brownian-bridge state flow and edge-aligned action-state noise sampling serves planning, behaviour cloning, and inverse dynamics without retraining.
MPC-Injection: Biasing Off-Policy Locomotion RL Toward Controller-Induced Behavior Basins
cs.RO 2026-06 unverdicted novelty 7.0 of 10

MPC-Injection biases off-policy RL locomotion policies toward controller-induced behavior basins by injecting MPC transitions into the replay buffer.
Stealthy World Model Manipulation via Data Poisoning
cs.LG 2026-06 unverdicted novelty 7.0 of 10

SWAAP is the first two-stage poisoning framework that identifies a harmful target world model via bilevel optimization and realizes it through stealth-constrained gradient matching on a limited fraction of fine-tuning...
Expected Free Energy-based Planning as Variational Inference
cs.AI 2026-06 unverdicted novelty 7.0 of 10

EFE-based planning is formulated as variational free energy minimization with epistemic priors, decomposing into expected plan costs plus a complexity term.
WMAttack: Automated Attack Search for Adversarial Evaluation of World-Model Agents
cs.LG 2026-05 unverdicted novelty 7.0 of 10

WMAttack automates finite-budget attack search for world-model agents via SCAS and RGAR, reporting higher normalized reward drops than baselines on Atari and DMC tasks.
ARC-RL: A Reinforcement Learning Playground Inspired by ARC Raiders
cs.RO 2026-05 unverdicted novelty 7.0 of 10

ARC-RL provides four new MuJoCo continuous-control environments with hexapod and quadruped morphologies inspired by ARC Raiders, a unified multi-component reward without motion capture, CPG expert demonstrators, and e...
Randomized Advantage Transformation (RAT): Computing Natural Policy Gradients via Direct Backpropagation
cs.LG 2026-05 unverdicted novelty 7.0 of 10

RAT reformulates regularized natural policy gradients as vanilla gradients with a transformed advantage, computed efficiently via randomized block Kaczmarz iterations on on-policy data.
Agentick: A Unified Benchmark for General Sequential Decision-Making Agents
cs.AI 2026-05 unverdicted novelty 7.0 of 10

Agentick is a new unified benchmark for sequential decision-making agents that evaluates RL, LLM, VLM, hybrid, and human agents across 37 tasks, showing no single approach dominates.
Flow Matching Policy Gradients
cs.LG 2025-07 conditional novelty 7.0 of 10

FPO trains flow-based policies with PPO by replacing the likelihood ratio with an exponentiated flow matching loss difference.
Self-Predictive Dynamics for Generalization of Vision-based Reinforcement Learning
cs.CV 2025-06 conditional novelty 7.0 of 10

A self-supervised auxiliary loss combining weak and strong augmentations, an adversarial discriminator, and inverse-then-forward latent dynamics improves both data efficiency and zero-shot generalization in vision-based RL.
Understanding Behavioral Metric Learning: A Large-Scale Study on Distracting Reinforcement Learning Environments
cs.LG 2025-05 conditional novelty 7.0 of 10

Across noisy DeepMind Control tasks, explicit bisimulation-metric losses add little denoising benefit beyond plain self-prediction and feature normalization, which dominate performance.
Maximum Total Correlation Reinforcement Learning
cs.LG 2025-05 conditional novelty 7.0 of 10

A reinforcement learning regularizer that maximizes trajectory-level total correlation produces simpler, more compressible policies that are more robust to perturbations.
ImagineBench: Evaluating Reinforcement Learning with Large Language Model Rollouts
cs.LG 2025-05 conditional novelty 7.0 of 10

A benchmark for offline RL with LLM-generated imaginary rollouts shows existing algorithms reach only 35.44% success on hard tasks versus 64.37% when trained on real rollouts.
Wasserstein Policy Optimization
cs.LG 2025-05 conditional novelty 7.0 of 10

WPO derives a closed-form policy update from Wasserstein gradient flows, which for Gaussian policies coincides with the standard policy gradient in expectation but with lower variance, and works for arbitrary stochast...
MuJoCo Playground
cs.RO 2025-02 conditional novelty 7.0 of 10

An open-source, MJX-based robot learning framework with integrated batch rendering that provides fast training and demonstrates sim-to-real transfer on six robot platforms.
LMAct: A Benchmark for In-Context Imitation Learning with Long Multimodal Demonstrations
cs.AI 2024-12 conditional novelty 7.0 of 10

LMAct is an open benchmark showing that frontier AI models mostly fail to improve at interactive tasks even when given up to 512 expert demonstrations in a million-token context.
Learning Interactive Real-World Simulators
cs.AI 2023-10 conditional novelty 7.0 of 10

UniSim learns a universal real-world simulator from orchestrated diverse datasets, enabling zero-shot deployment of policies trained purely in simulation.
Benchmarking Model-Based Reinforcement Learning
cs.LG 2019-07 accept novelty 7.0 of 10

Introduces a benchmark suite of over 18 MBRL environments, evaluates multiple algorithms under consistent settings, and identifies three core challenges: dynamics bottleneck, planning horizon dilemma, and early-termin...
GeMPO: Generalized Measure Matching for Online Diffusion Reinforcement Learning
cs.LG 2026-03 conditional novelty 6.5 of 10

GeMPO unifies diffusion RL reweighting as measure matching to a regularized target, enabling flexible and negative weights that improve exploration and performance.
VIScore: Diagnosing Planning-Relevant Quality in Latent World Models
cs.RO 2026-08 conditional novelty 6.0 of 10

VIScore, a product of veracity, influence, and sobriety factors, predicts planning success in latent world models better than existing single-component diagnostics.
V-Simba: Unleashing the Architectural Potential of RL in Visual Continuous Control
cs.LG 2026-08 conditional novelty 6.0 of 10

V-Simba, a visual RL architecture combining layer normalization, weight decay, and a distributional critic, matches or outperforms complex baselines on 29 continuous control tasks while using less compute.
Beyond Isolation: Unlocking Reinforcement Learning Component Synergy for Sample-Efficient Continuous Control
cs.LG 2026-08 conditional novelty 6.0 of 10

Coordinating optimization stability, model-based representation, and scheduled prioritized replay yields larger sample-efficiency gains than naively stacking the same components in continuous-control RL.
Dueling World Models: Advantage-Style Action Channels for Common-Mode Distractor Rejection
cs.LG 2026-08 conditional novelty 6.0 of 10

Subtracting the action-marginal mean from a world model's action-conditioned predictions yields a vector action channel that exactly cancels common-mode distraction and works post hoc on frozen predictors.
TaskSense: Focusing on What Matters in World Models
cs.AI 2026-08 conditional novelty 6.0 of 10

TaskSense filters visual observations with latent-conditioned stochastic spatial attention before encoding, improving world-model control under visual distractions relative to DreamerV3.
ProDVI: Programmatic Dynamics Priors for Value Network Initialization
cs.LG 2026-08 conditional novelty 6.0 of 10

LLM-generated dynamics programs, used only to pretrain a value network's state-action encoder, improve sample efficiency of model-free RL on continuous control tasks.
Observation-Grounded Self-Predictive Reinforcement Learning for Visual Continuous Control
cs.LG 2026-08 conditional novelty 6.0 of 10

A model-free visual RL agent that jointly trains latent self-prediction and next-observation prediction, mediated by two adapters, improves aggregate DMControl scores over prior methods.
Collaborative Weighting with Pessimistic Critic for Mitigating Overestimation in Off-Policy Reinforcement Learning
cs.LG 2026-07 conditional novelty 6.0 of 10

A new actor-critic variant that reweights samples by TD-error and uncertainty and uses pessimistic sampled values improves continuous-control RL benchmark performance.
Koopman Dreamer: Spectrally Constrained Latent Dynamics for Stable World-Model Imagination
cs.LG 2026-07 conditional novelty 6.0 of 10

A spectrally constrained Koopman-style latent transition with teacher and rollout objectives improves long-horizon imagination and closed-loop control in Dreamer-style world models.
DWM: Separating World Effects from Actions in Latent World Models
cs.AI 2026-07 conditional novelty 6.0 of 10

A training-time 'world head' that is action-invariant, plus an orthogonality constraint, improves CEM planning in latent world models when environments have persistent action-independent dynamics (average +13.1 pp on ...
Deconstructing Actor-Critic: A Large-scale Empirical Study of Design Components for Practitioners
cs.LG 2026-07 conditional novelty 6.0 of 10

In a large empirical study on a water-treatment PID control task, bounded beta policies with adaptive critic updates were the most reliable actor-critic configuration, while common defaults like Gaussian policies with...
Operator-on-F complements value-equivalence: a planning-time diagnostic for latent world models
cs.LG 2026-07 conditional novelty 6.0 of 10

Operator-on-F—a probe-based comparison of a model's k-step latent pushforward to the environment's—tracks planning return on TD-MPC2 cheetah-run (Spearman -0.90) and separates architectures where reward checks fail.
ACID: Action Consistency via Inverse Dynamics for Planning with World Models
cs.RO 2026-07 unverdicted novelty 6.0 of 10

ACID improves decision-time planning in world models by adding per-step action consistency residuals from an inverse dynamics model to the planning cost via an adaptive weight, yielding better performance with less co...
Coachable agents for interactive gameplay
cs.AI 2026-07 unverdicted novelty 6.0 of 10

A framework combining universal value function approximators with targeted training scenarios and data augmentation produces RL agents that adapt to user-specified styles in real time across video games and humanoid d...
Delta-JEPA: Learning Action-Sensitive World Models via Latent Difference Decoding
cs.AI 2026-06 unverdicted novelty 6.0 of 10

Delta-JEPA augments latent forward prediction with a Latent Difference Action Decoder that reconstructs actions from embedding displacements, yielding action-sensitive world models that improve planning on four visual...
Dual-Flow Reinforcement Learning with State-Aware Exploration
cs.LG 2026-06 unverdicted novelty 6.0 of 10

Dual-Flow RL jointly models return distributions and multimodal policies via conditional flow matching with an added ECER for exploration, claiming SOTA results on control benchmarks.
Fast LeWorldModel
cs.LG 2026-06 unverdicted novelty 6.0 of 10

Fast-LeWM uses action-prefix encoding and parallel latent prediction to replace sequential rollout, improving success rates and cutting planning time in LeWorldModel tasks.
NASDAQ: Normalized Observation Space Dynamics-Augmented Q-Learning
cs.LG 2026-06 unverdicted novelty 6.0 of 10

NASDAQ normalizes observations in an online RL setting so that dynamics prediction losses are balanced across dimensions, yielding competitive performance with lower wall-time than prior model-based and self-predictiv...
CRAX: Fast Safe Reinforcement Learning Benchmarking
cs.LG 2026-06 unverdicted novelty 6.0 of 10

CRAX is a new fast benchmark suite for constrained RL built on MJX, with six environment suites and tasks across difficulty levels, showing no single safe RL method dominates and benefits from curriculum learning.
Sensorimotor World Models: Perception for Action via Inverse Dynamics
cs.LG 2026-06 unverdicted novelty 6.0 of 10

SMWM trains end-to-end latent world models from offline reward-free data using inverse dynamics regularization to prevent collapse and align states with controllable actions for planning.
Sampling-Based Safe Reinforcement Learning
cs.LG 2026-05 conditional novelty 6.0 of 10

SBSRL approximates worst-case safety optimization over uncertain dynamics via finite sampling, adds epistemic-uncertainty-constrained exploration, and supplies high-probability safety guarantees plus finite-time sampl...
Optimal airfoils in the intermediate Reynolds number range
physics.flu-dyn 2026-05 unverdicted novelty 6.0 of 10

Zero-thickness cambered airfoils are globally optimal within the Joukowski family for glide ratio and endurance factor across Re 1-3000, with optimal camber peaking near Re 40-50 and sensitivity to shape increasing with Re.
PH-Dreamer: A Physics-Driven World Model via Port-Hamiltonian Generative Dynamics
cs.LG 2026-05 unverdicted novelty 6.0 of 10

PH-Dreamer integrates a port-Hamiltonian framework into generative world models to enforce physical priors, yielding tighter imagined-real reward alignment and reduced latent space volume on visual control benchmarks.
R2R2: Robust Representation for Intensive Experience Reuse via Redundancy Reduction in Self-Predictive Learning
cs.LG 2026-05 unverdicted novelty 6.0 of 10

R2R2 introduces a non-centered regularization objective for SPL that addresses conflicts with spectral properties, leading to better performance on continuous control tasks at high UTD ratios.
When Does Non-Uniform Replay Matter in Reinforcement Learning?
cs.LG 2026-05 unverdicted novelty 6.0 of 10

Non-uniform replay improves RL sample efficiency mainly in low replay-volume regimes, with high-entropy sampling being key even at comparable recency.
Mean Flow Policy Optimization
cs.LG 2026-04 conditional novelty 6.0 of 10

Mean Flow Policy Optimization (MFPO) uses few-step flow-based models for RL policies and achieves performance on par with or better than diffusion-based methods while substantially lowering training and inference time...
FlashSAC: Fast and Stable Off-Policy Reinforcement Learning for High-Dimensional Robot Control
cs.LG 2026-04 unverdicted novelty 6.0 of 10

FlashSAC scales up Soft Actor-Critic with fewer updates, larger models, higher data throughput, and norm bounds to deliver faster, more stable training than PPO on high-dimensional robot control tasks across dozens of...
LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels
cs.LG 2026-03 unverdicted novelty 6.0 of 10

LeWM is the first end-to-end trainable JEPA from pixels that uses only two loss terms for stable training and fast planning on 2D/3D control tasks.
Latent Matters: Learning Deep State-Space Models
cs.LG 2026-02 conditional novelty 6.0 of 10

A constrained-optimisation objective plus an extended-Kalman VAE architecture improves system identification and prediction accuracy in deep state-space models.
Can We Really Learn One Representation to Optimize All Rewards?
cs.LG 2026-02 conditional novelty 6.0 of 10

Finite-dimensional FB representations cannot exactly encode all rewards in continuous control; a new one-step FB variant that fits the behavioral policy converges better and beats FB on average.
How Does the Lagrangian Guide Safe Reinforcement Learning through Diffusion Models?
cs.LG 2026-02 unverdicted novelty 6.0 of 10

ALGD augments the Lagrangian to locally convexify the energy landscape in diffusion models, stabilizing safe RL training and generation without changing optimal policies.
Noisy Pairwise-Comparison Random Search for Smooth Nonconvex Optimization
math.OC 2026-01 conditional novelty 6.0 of 10

Noisy-comparison random search reaches ε-stationarity in O(k/(p²ε²)) comparisons for smooth nonconvex objectives with k-dimensional active subspace.
Reverse Flow Matching: A Unified Framework for Online Reinforcement Learning with Diffusion and Flow Policies
cs.LG 2026-01 conditional novelty 6.0 of 10

Reverse flow matching trains diffusion and flow policies for online RL via posterior mean estimation and control variates, unifying noise-expectation and Q-gradient estimators as special cases.
Training Diffusion Policies via Prior-Mapping Co-Evolution
cs.LG 2025-12 conditional novelty 6.0 of 10

GoRL outperforms Gaussian and generative RL baselines on continuous control by optimizing a Gaussian latent policy while a separately trained diffusion or flow decoder maps fixed noise to actions.
A Comprehensive Survey on World Models for Embodied AI
cs.CV 2025-10 conditional novelty 6.0 of 10

A unified three-axis taxonomy — functionality, temporal modeling, spatial representation — organizes the world-model literature for embodied AI.

See all 175 Pith citations

Reference graph

Works this paper leans on

13 extracted references · 13 canonical work pages · cited by 175 Pith papers (see all)

[1]

Layer Normalization

Anonymous. Distributed prioritized experience replay.Under submission, 2017a. Anonymous. Distributional policy gradients.Under submission, 2017b. Jimmy Lei Ba, Jamie Ryan Kiros, and Geoﬀrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450,

work page Pith review arXiv
[2]

Barto, Richard S

ISSN 0018-9472. doi: 10.1109/TSMC.1983.6313077. Marc G Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling. The arcade learning environment: An evaluation platform for general agents.Journal of Artiﬁcial Intelligence Research,

work page doi:10.1109/tsmc.1983.6313077 1983
[3]

A Distributional Perspective on Reinforcement Learning

Marc G Bellemare, Will Dabney, and Rémi Munos. A distributional perspective on rein- forcement learning.arXiv preprint arXiv:1707.06887,

work page Pith review arXiv
[4]

Simulation tools for model-based robotics: Comparison of bullet, havok, mujoco, ode and physx

Tom Erez, Yuval Tassa, and Emanuel Todorov. Simulation tools for model-based robotics: Comparison of bullet, havok, mujoco, ode and physx. In Robotics and Automation (ICRA), 2015 IEEE International Conference on, pages 4397–4404. IEEE,

work page 2015
[5]

Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Control

Riashat Islam, Peter Henderson, Maziar Gomrokchi, and Doina Precup. Reproducibility of benchmarked deep reinforcement learning tasks for continuous control.arXiv preprint arXiv:1708.04133,

work page Pith review arXiv
[6]

Adam: A Method for Stochastic Optimization

Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980,

work page Pith review arXiv
[7]

Continuous control with deep reinforcement learning

Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971,

work page Pith review arXiv
[8]

Learning human behaviors from motion capture by adversarial imitation

Josh Merel, Yuval Tassa, TB Dhruva, Sriram Srinivasan, Jay Lemmon, Ziyu Wang, Greg Wayne, and Nicolas Heess. Learning human behaviors from motion capture by adversarial imitation. arXiv preprint arXiv:1707.02201,

work page Pith review arXiv

Show all 13 references

[9]

Asynchronous methods for deep reinforcement learning.arXiv preprint arXiv:1602.01783,

Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy P Lilli- crap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning.arXiv preprint arXiv:1602.01783,
[10]

ISSN 0028-0836. Letter. Ivaylo Popov, Nicolas Heess, Timothy Lillicrap, Roland Hafner, Gabriel Barth-Maron, Matej Vecerik, Thomas Lampe, Yuval Tassa, Tom Erez, and Martin Riedmiller. Data-eﬃcient deepreinforcementlearningfordexterousmanipulation. arXiv preprint arXiv:1704.03073,
[11]

Prioritized experience replay

Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver. Prioritized experience replay. arXiv preprint arXiv:1511.05952,
[12]

Synthesis and stabilization of complex be- haviors through online trajectory optimization

Yuval Tassa, Tom Erez, and Emanuel Todorov. Synthesis and stabilization of complex be- haviors through online trajectory optimization. InIntelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on, pages 4906–4913. IEEE,

2012
[13]

Mujoco: A physics engine for model- based control

Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model- based control. InIntelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on, pages 5026–5033. IEEE,

2012

Pith tools

Reviewed May 13, 2026 · model on record in the stance chip above.

[1] [1]

Layer Normalization

Anonymous. Distributed prioritized experience replay.Under submission, 2017a. Anonymous. Distributional policy gradients.Under submission, 2017b. Jimmy Lei Ba, Jamie Ryan Kiros, and Geoﬀrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450,

work page Pith review arXiv

[2] [2]

Barto, Richard S

ISSN 0018-9472. doi: 10.1109/TSMC.1983.6313077. Marc G Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling. The arcade learning environment: An evaluation platform for general agents.Journal of Artiﬁcial Intelligence Research,

work page doi:10.1109/tsmc.1983.6313077 1983

[3] [3]

A Distributional Perspective on Reinforcement Learning

Marc G Bellemare, Will Dabney, and Rémi Munos. A distributional perspective on rein- forcement learning.arXiv preprint arXiv:1707.06887,

work page Pith review arXiv

[4] [4]

Simulation tools for model-based robotics: Comparison of bullet, havok, mujoco, ode and physx

Tom Erez, Yuval Tassa, and Emanuel Todorov. Simulation tools for model-based robotics: Comparison of bullet, havok, mujoco, ode and physx. In Robotics and Automation (ICRA), 2015 IEEE International Conference on, pages 4397–4404. IEEE,

work page 2015

[5] [5]

Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Control

Riashat Islam, Peter Henderson, Maziar Gomrokchi, and Doina Precup. Reproducibility of benchmarked deep reinforcement learning tasks for continuous control.arXiv preprint arXiv:1708.04133,

work page Pith review arXiv

[6] [6]

Adam: A Method for Stochastic Optimization

Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980,

work page Pith review arXiv

[7] [7]

Continuous control with deep reinforcement learning

Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971,

work page Pith review arXiv

[8] [8]

Learning human behaviors from motion capture by adversarial imitation

Josh Merel, Yuval Tassa, TB Dhruva, Sriram Srinivasan, Jay Lemmon, Ziyu Wang, Greg Wayne, and Nicolas Heess. Learning human behaviors from motion capture by adversarial imitation. arXiv preprint arXiv:1707.02201,

work page Pith review arXiv