REVIEW 19 cited by
Data-Efficient Reinforcement Learning with Self-Predictive Representations
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Data-Efficient Reinforcement Learning with Self-Predictive Representations
read the original abstract
While deep reinforcement learning excels at solving tasks where large amounts of data can be collected through virtually unlimited interaction with the environment, learning from limited interaction remains a key challenge. We posit that an agent can learn more efficiently if we augment reward maximization with self-supervised objectives based on structure in its visual input and sequential interaction with the environment. Our method, Self-Predictive Representations(SPR), trains an agent to predict its own latent state representations multiple steps into the future. We compute target representations for future states using an encoder which is an exponential moving average of the agent's parameters and we make predictions using a learned transition model. On its own, this future prediction objective outperforms prior methods for sample-efficient deep RL from pixels. We further improve performance by adding data augmentation to the future prediction loss, which forces the agent's representations to be consistent across multiple views of an observation. Our full self-supervised objective, which combines future prediction and data augmentation, achieves a median human-normalized score of 0.415 on Atari in a setting limited to 100k steps of environment interaction, which represents a 55% relative improvement over the previous state-of-the-art. Notably, even in this limited data regime, SPR exceeds expert human scores on 7 out of 26 games. The code associated with this work is available at https://github.com/mila-iqia/spr
Forward citations
Cited by 19 Pith papers
-
PAMD: Structured Adaptive Distances for Bisimulation Representations in Visual Reinforcement Learning
A pairwise-adaptive Mahalanobis distance, parameterized as a positive-definite quadratic form, improves representation learning and control performance when plugged into bisimulation-based visual RL algorithms.
-
ECHO: Terminal Agents Learn World Models for Free
ECHO is a hybrid RL objective that trains agents to predict environment observation tokens from their actions, doubling GRPO pass@1 on TerminalBench-2.0 while improving dynamics prediction on held-out trajectories.
-
Multi-scale Predictive Representations for Goal-conditioned Reinforcement Learning
Ms.PR applies multi-scale predictive supervision to enforce goal-directed alignment in latent spaces for offline GCRL, yielding improved representation quality and performance on vision and state-based tasks.
-
Predictive but Not Plannable: RC-aux for Latent World Models
RC-aux corrects spatiotemporal mismatch in reconstruction-free latent world models by adding multi-horizon prediction and reachability supervision, improving planning performance on goal-conditioned pixel-control tasks.
-
Odysseus: Scaling VLMs to 100+ Turn Decision-Making in Games via Reinforcement Learning
Odysseus adapts PPO with a turn-level critic and leverages pretrained VLM action priors to train agents achieving at least 3x average game progress over frontier models in long-horizon Super Mario Land.
-
Self-Predictive Representation for Autonomous UAV Object-Goal Navigation
AmelPredSto, a stochastic self-predictive representation model, outperforms other state representation learning approaches when combined with actor-critic RL for object-goal navigation in UAVs.
-
Abstract Sim2Real through Approximate Information States
Abstract simulators can be grounded to real tasks by making their dynamics history-dependent and correcting them with real data, enabling RL policy transfer.
-
Hierarchical Planning with Latent World Models
Hierarchical planning over multi-scale latent world models enables 70% success on real robotic pick-and-place with goal-only input where flat models achieve 0%, while cutting planning compute up to 4x in simulations.
-
Behavior-Constrained Reinforcement Learning with Receding-Horizon Credit Assignment for High-Performance Control
A behavior-constrained RL framework with receding-horizon credit assignment learns high-performance control policies that stay aligned with expert behavior in race car simulation.
-
Dreamer-CDP: Improving Reconstruction-free World Models Via Continuous Deterministic Representation Prediction
Dreamer-CDP achieves reconstruction-free world modeling via a JEPA-style predictor on continuous deterministic representations and matches Dreamer's performance on Crafter.
-
Human-Like Goalkeeping in a Realistic Football Simulation: a Sample-Efficient Reinforcement Learning Approach
A sample-efficient SAC-based method with replay-ratio resets, offline data bootstrapping, and expert-driven fine-tuning produces a goalkeeper that outperforms the built-in AI in EA SPORTS FC 25.
-
TAPO: Transition-Aware Policy Optimization for LLM Agents
Interleaving action-conditioned next-observation supervision with group RL on a shared LLM backbone consistently lifts long-horizon agent success over pure policy optimization.
-
Direct Advantage Estimation for Scalable and Sample-efficient Deep Reinforcement Learning
Extends DAE theory to POMDPs with minimal changes and introduces discrete latent dynamics to cut computational cost, with ALE experiments showing scalability and retained sample efficiency.
-
SCALE-COMM: Shared, Contrastively-Aligned Latent Embeddings for MARL Communication
SCALE-COMM uses contrastive alignment on latent embeddings to decouple and stabilize communication learning from policy optimization in decentralized MARL, showing gains on benchmarks and a warehouse task.
-
ParkourFormer: Integrating Predictive Supervision and Sequence Modeling into Parkour Locomotion
ParkourFormer achieves 93.85% average success on multi-terrain humanoid parkour by fusing Transformer sequence modeling with supervised future-state prediction.
-
Hierarchical Planning with Latent World Models
Hierarchical latent world models with macro-actions solve long-horizon visual planning (70% Franka pick-and-place vs 0% flat planning) with up to 3× less compute.
-
Self-Predictive Representations for Combinatorial Generalization in Behavioral Cloning
BYOL-γ uses self-predictive representations to approximate successor representations, improving zero-shot combinatorial generalization in goal-conditioned behavioral cloning.
-
Intention-Conditioned Flow Occupancy Models
InFOM applies flow matching to model intention-conditioned occupancy measures for RL pre-training, reporting 1.8x median return gains and 36% higher success rates on benchmarks.
-
Structured Representation Learning with Locally Linear Embeddings and Adaptive Feature Fusion
A neuro-inspired RL method uses locally linear embeddings for state structure and attention-based fusion of dynamics and reward features to improve learning efficiency.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.