REVIEW 12 cited by
Learning to Reach Goals via Iterated Supervised Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Current reinforcement learning (RL) algorithms can be brittle and difficult to use, especially when learning goal-reaching behaviors from sparse rewards. Although supervised imitation learning provides a simple and stable alternative, it requires access to demonstrations from a human supervisor. In this paper, we study RL algorithms that use imitation learning to acquire goal reaching policies from scratch, without the need for expert demonstrations or a value function. In lieu of demonstrations, we leverage the property that any trajectory is a successful demonstration for reaching the final state in that same trajectory. We propose a simple algorithm in which an agent continually relabels and imitates the trajectories it generates to progressively learn goal-reaching behaviors from scratch. Each iteration, the agent collects new trajectories using the latest policy, and maximizes the likelihood of the actions along these trajectories under the goal that was actually reached, so as to improve the policy. We formally show that this iterated supervised learning procedure optimizes a bound on the RL objective, derive performance bounds of the learned policy, and empirically demonstrate improved goal-reaching performance and robustness over current RL algorithms in several benchmark tasks.
Forward citations
Cited by 12 Pith papers
-
INTACT: Isomorphic Intent-to-Action Learning for Search-Free World Models
INTACT couples a one-step physical intent and a stop-gradient goal displacement through one shared action predictor, producing a search-free direct policy that reaches 95.33% macro success with zero candidates on the ...
-
Spinning Straw into Gold: Relabeling LLM Agent Trajectories in Hindsight for Successful Demonstrations
Relabeling LLM-agent trajectories with all goals actually achieved, plus action masking and reweighting, yields sample-efficient gains over SFT and DPO on ALFWorld, PlanCraft, and WebShop.
-
Compositional Diffusion with Guided Search for Long-Horizon Planning
CDGS adds population-based search and likelihood-based pruning to compositional diffusion, enabling long-horizon planning from short-horizon models across robot manipulation, panoramas, and video.
-
Generative Sequential Notification Optimization via Multi-Objective Decision Transformers
A Decision Transformer with quantile-regression return prompts improved notification decisions at LinkedIn, boosting sessions by 0.72% over the deployed CQL baseline in a live A/B test.
-
Equivariant Goal Conditioned Contrastive Reinforcement Learning
Equivariant Contrastive RL imposes C8 rotation symmetry on the critic and actor, improving sample efficiency and goal generalization in simulated manipulation.
-
Behavioral Exploration: Learning to Explore via In-Context Adaptation
A coverage-conditioned behavioral cloning policy adapts in-context to its own history, making robots explore new expert-like behaviors online without online reinforcement learning.
-
Efficient Skill Discovery via Regret-Aware Optimization
A regret-aware skill discovery algorithm, RSD, improves sample efficiency and zero-shot goal-reaching in high-dimensional continuous control by focusing exploration on unmastered skills.
-
Prompting Decision Transformers for Zero-Shot Reach-Avoid Policies
RADT learns reach-avoid policies from random offline trajectories and generalizes to unseen avoid-region sizes and counts via prompt conditioning.
-
Temporal-Distance JEPA: Plan-Aware Representation Learning for Latent World Model Predictive Control
Mining a directed temporal distance from demonstration logs, and choosing the planner cost per task, improves JEPA latent MPC over LeWM on four benchmarks.
-
VisualPatchWorld: Code World Models as Latent Structured Representations for Planning
A two-level induction procedure—active-probe sketch selection plus multi-step rollout fitting—recovers executable code world models that improve CEM planning over prior code baselines on four LeWM tasks.
-
Normalizing Flows are Capable Models for Continuous Control
A simple normalizing flow policy matches or outperforms diffusion and autoregressive baselines across imitation learning, offline RL, goal-conditioned RL, and unsupervised RL on 82 tasks.
-
Reinforcement Learning: From Algorithms To Foundation Models
A dissertation uniting the author's published results: non-exploitable Nash-DQN policies and the FightLadder benchmark for games, plus diffusion/consistency-model world models for RL — a compilation rather than new results.
Discussion (0). Sign in to comment.