REVIEW 9 cited by
In-context Reinforcement Learning with Algorithm Distillation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We propose Algorithm Distillation (AD), a method for distilling reinforcement learning (RL) algorithms into neural networks by modeling their training histories with a causal sequence model. Algorithm Distillation treats learning to reinforcement learn as an across-episode sequential prediction problem. A dataset of learning histories is generated by a source RL algorithm, and then a causal transformer is trained by autoregressively predicting actions given their preceding learning histories as context. Unlike sequential policy prediction architectures that distill post-learning or expert sequences, AD is able to improve its policy entirely in-context without updating its network parameters. We demonstrate that AD can reinforcement learn in-context in a variety of environments with sparse rewards, combinatorial task structure, and pixel-based observations, and find that AD learns a more data-efficient RL algorithm than the one that generated the source data.
Forward citations
Cited by 9 Pith papers
-
MimicDroid: In-Context Learning for Humanoid Robot Manipulation from Human Play Videos
Trained only on unlabeled human play videos, MimicDroid lets a GR1 humanoid perform new manipulation tasks from one to three demonstration videos, with roughly twice the real-world success of prior video-conditioned methods.
-
Multi-Turn On-Policy Distillation with Prefix Replay
ReOPD offline-distills multi-turn agentic LLMs via teacher-prefix replay plus step-decay sampling, matching online OPD accuracy at ≥4× speed with zero tool calls.
-
Learn from What We HAVE: History-Aware VErifier that Reasons about Past Interactions Online
A history-aware verifier that scores candidate actions using past interactions cuts failure rates in ambiguous robot manipulation tasks compared to using the generator alone.
-
LLM Economist: Large Population Models and Mechanism Design in Multi-Agent Generative Simulacra
The LLM Economist framework couples persona-conditioned worker agents with an in-context RL planner to search US-bracket tax schedules, yet its Saez benchmark is derived from the planner's own solution and its headlin...
-
Behavioral Exploration: Learning to Explore via In-Context Adaptation
A coverage-conditioned behavioral cloning policy adapts in-context to its own history, making robots explore new expert-like behaviors online without online reinforcement learning.
-
Robust In-Context Reinforcement Learning Under Reward Poisoning Attacks
Adversarially training a Decision-Pretrained Transformer against learned reward-poisoning attackers makes it robust to test-time reward corruption, outperforming robust bandit baselines in experiments.
-
ReBRAC-v2: The Return of the King
A fixed-recipe offline RL method combining normalizing-flow actors, categorical critics, staged training, and test-time refinement beats recent flow-based baselines by 22.5 points averaged over ten OGBench categories.
-
Interaction as Intelligence: Deep Research With Human-AI Partnership
A human-in-the-loop deep research system with transparent, interruptible interaction is claimed to outperform commercial baselines, but the evidence is weakened by small samples and biased instructions.
-
ASTRO: Teaching Language Models to Reason by Reflecting and Backtracking In-Context
ASTRO converts MCTS search trees into chain-of-thoughts with explicit self-reflection and backtracking, trains Llama-3.1-70B on them with SFT, then improves with RL, reaching 81.8% on MATH-500, 64.4% on AMC 2023, and ...
Discussion (0). Sign in to comment.