REVIEW 29 cited by
Temporal Difference Learning for Model Predictive Control
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Data-driven model predictive control has two key advantages over model-free methods: a potential for improved sample efficiency through model learning, and better performance as computational budget for planning increases. However, it is both costly to plan over long horizons and challenging to obtain an accurate model of the environment. In this work, we combine the strengths of model-free and model-based methods. We use a learned task-oriented latent dynamics model for local trajectory optimization over a short horizon, and use a learned terminal value function to estimate long-term return, both of which are learned jointly by temporal difference learning. Our method, TD-MPC, achieves superior sample efficiency and asymptotic performance over prior work on both state and image-based continuous control tasks from DMControl and Meta-World. Code and video results are available at https://nicklashansen.github.io/td-mpc.
Forward citations
Cited by 29 Pith papers
-
When Is a Learned Command Adapter Worth It? Closed-Loop Identification and Counterfactual Auditing of Frozen Locomotion Policies
A counterfactual audit separates same-state headroom from recoverable state-allocation gain, returning NO-GO or ABSTAIN for learned command adapters on frozen Go2 and H1 locomotion policies at 1% thresholds.
-
Beyond Test-Time Memory: State-Space Optimal Control for LLM Reasoning
A symplectic LQR layer inserted as an adapter into pretrained LLMs yields large gains on MATH-500, AMC and AIME by solving a latent optimal-control problem at inference time.
-
M$^3$PC: Test-time Model Predictive Control for Pretrained Masked Trajectory Model
M3PC runs model predictive control at test time on a pretrained masked trajectory Transformer, improving offline RL returns and enabling goal reaching without extra model training.
-
ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World Models
ContactGuard uses an action-conditioned latent world model to predict a post-contact visual embedding from a planned action chunk and abort before the gripper closes, improving failure prediction over current-latent a...
-
Beyond Myopic World Models: Long-Horizon End-to-End Training for Direct Future Prediction
A direct endpoint-prediction world model trained on long-horizon objectives substantially beats recursively-rolled-out baselines, and the objective, not the backbone, drives the improvement.
-
TaskSense: Focusing on What Matters in World Models
TaskSense filters visual observations with latent-conditioned stochastic spatial attention before encoding, improving world-model control under visual distractions relative to DreamerV3.
-
Weights or Skills? A Survey of Robot-Learning Techniques: from Action-Predicting Weights to Robots that Write their Own Skills
A taxonomy of robot learning on a weights-versus-skills axis, with a five-rung self-improvement ladder whose top cell (feedback plus memory plus search) holds only a few recent systems.
-
Explore Beyond the Boundary Using Entropic Information
ENTINEX identifies exploration boundaries using the entropy of a novelty-based action distribution and outperforms baseline exploration methods on sparse and delayed MuJoCo tasks.
-
World Action Planner: Generalizable Decision-Making with Action-Conditioned World Models
Pairing VLM-generated action proposals with rollouts from a pose-image-conditioned video world model yields high success rates in novel simulated manipulation tasks without end-to-end policy retraining.
-
Toward Goal-Agnostic Joint-Embedding Predictive Control of Partial Differential Equations
A goal-agnostic latent-dynamics controller for 2D Navier-Stokes improves tracking by planning against a learned kinetic-energy probe rather than raw latent-space distance.
-
DriftWorld: Fast World Modeling through Drifting
An action-conditioned world model trained with drifting generates robot rollout videos in one forward pass, matching diffusion quality while running 2.8-478x faster per-table, and raises GPC-RANK Push-T IoU from 0.635...
-
Next-Latent Prediction Transformers Learn Compact World Models
NextLat augments next-token prediction with latent next-state prediction, theoretically converging latents to belief states and showing empirical gains in world modeling, reasoning, planning, and faster inference via ...
-
Arnold: a generalist muscle transformer policy
A single transformer policy with a compositional sensorimotor vocabulary achieves expert or super-expert performance on 14 musculoskeletal control tasks spanning four embodiments.
-
Latent Policy Barrier: Learning Robust Visuomotor Policies by Staying In-Distribution
Latent Policy Barrier improves behavior-cloned visuomotor policies by using a latent dynamics model trained on expert and rollout data to guide actions back toward in-distribution expert states.
-
WoMAP: World Models For Embodied Open-Vocabulary Object Localization
WoMAP generates training data from Gaussian Splatting scenes, distills detector confidence into a latent world model, and uses that model to refine vision-language action proposals for open-vocabulary object localization.
-
Solving New Tasks by Adapting Internet Video Knowledge
Inverse Probabilistic Adaptation, a score-composition variant that keeps the large video model as the base and consults a small in-domain model, achieves 68.3% average success on MetaWorld policy supervision and stays...
-
TD-M(PC)$^2$: Improving Temporal Difference MPC Through Policy Constraint
TD-M(PC)2 adds a TD3-BC-style policy constraint to TD-MPC2's policy update, reducing out-of-distribution value queries caused by planner-data mismatch and improving performance on high-dimensional control tasks.
-
Latent Action Learning Requires Supervision in the Presence of Distractors
Latent action models need at least a small amount of action supervision to learn useful actions when observations contain distractors, as shown on the Distracting Control Suite.
-
When Should We Prefer State-to-Visual DAgger Over Visual Reinforcement Learning?
State-to-Visual DAgger outperforms visual RL on hard manipulation tasks and is more stable and faster in wall-clock time, but offers little sample-efficiency benefit on easy tasks.
-
The Matrix: Infinite-Horizon World Generation with Real-Time Moving Control
A 2.7B parameter diffusion model trained on game and internet footage generates control-responsive 720p video streams, but the paper's 'infinite, real-time, zero-shot' claims are not backed by public benchmarks or rel...
-
LUCID: Latent-Skill Unified Control via Imagined Dynamics for Long-Horizon Humanoid Loco-Manipulation
LUCID composes reusable latent skills with a learned macro-dynamics model so that a high-level policy can plan through imagined rollouts, improving simulated humanoid loco-manipulation task success.
-
CoRL-MPPI: Enhancing MPPI With Learnable Behaviours For Efficient And Provably-Safe Multi-Robot Collision Avoidance
CoRL-MPPI injects a learned cooperative policy into MPPI's sampling distribution to speed up and make safer multi-robot navigation in dense simulations.
-
Sample-Efficient Reinforcement Learning Controller for Deep Brain Stimulation in Parkinson's Disease
A DDPG-based adaptive DBS controller with a predictive reward model and Gumbel-Softmax exploration suppresses beta power faster than standard DDPG in simulation and survives FP16 quantization.
-
Equivariant Action Sampling for Reinforcement Learning and Planning
Augmenting each sampled action with its full symmetry orbit makes finite-sample planning exactly equivariant and speeds up learning on several rotationally symmetric control tasks.
-
M3PO: Massively Multi-Task Model-Based Policy Optimization
A hybrid model-based RL algorithm combining TDMPC2-style world models, PPO clipped updates, and POME-style exploration bonuses reports strong benchmark results but omits reproducible evidence.
-
Investigating Lagrangian Neural Networks for Infinite Horizon Planning in Quadrupedal Locomotion
In Isaac Gym simulation, Lagrangian-structured dynamics models learn faster and predict motion more accurately than a plain MLP, but the headline 10x and 2-10x numbers are not fully supported.
-
Continual Learning for Generative AI: From LLMs to MLLMs and Beyond
A survey that categorizes continual learning methods for generative models into architecture-based, regularization-based, and replay-based paradigms across four model families.
-
TD-GRPC: Temporal Difference Learning with Group Relative Policy Constraint for Humanoid Locomotion
TD-GRPC trains a humanoid locomotion policy by combining softmax group-relative Q-value advantages with an MPPI-action matching loss inside TD-MPC, reporting improved sample efficiency on eight of ten HumanoidBench tasks.
-
Reinforcement Learning: From Algorithms To Foundation Models
A dissertation uniting the author's published results: non-exploitable Nash-DQN policies and the FightLadder benchmark for games, plus diffusion/consistency-model world models for RL — a compilation rather than new results.
Discussion (0). Continue with ORCID to comment.