Pith. sign in

REVIEW 19 cited by

In-context Reinforcement Learning with Algorithm Distillation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2210.14215 v1 pith:NPLNJJLS submitted 2022-10-25 cs.LG cs.AI

Michael Laskin , Luyu Wang , Junhyuk Oh , Emilio Parisotto , Stephen Spencer , Richie Steigerwald , DJ Strouse , Steven Hansen
show 6 more authors
Angelos Filos Ethan Brooks Maxime Gazeau Himanshu Sahni Satinder Singh Volodymyr Mnih
This is my paper · ORCID
classification cs.LGcs.AI
keywords algorithmlearningreinforcementdistillationhistoriesin-contextcausalgenerated
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

We propose Algorithm Distillation (AD), a method for distilling reinforcement learning (RL) algorithms into neural networks by modeling their training histories with a causal sequence model. Algorithm Distillation treats learning to reinforcement learn as an across-episode sequential prediction problem. A dataset of learning histories is generated by a source RL algorithm, and then a causal transformer is trained by autoregressively predicting actions given their preceding learning histories as context. Unlike sequential policy prediction architectures that distill post-learning or expert sequences, AD is able to improve its policy entirely in-context without updating its network parameters. We demonstrate that AD can reinforcement learn in-context in a variety of environments with sparse rewards, combinatorial task structure, and pixel-based observations, and find that AD learns a more data-efficient RL algorithm than the one that generated the source data.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 19 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MimicDroid: In-Context Learning for Humanoid Robot Manipulation from Human Play Videos

    cs.RO 2025-09 conditional novelty 7.0 of 10

    Trained only on unlabeled human play videos, MimicDroid lets a GR1 humanoid perform new manipulation tasks from one to three demonstration videos, with roughly twice the real-world success of prior video-conditioned methods.

  2. ICLR: In-Context Learning of Representations

    cs.CL 2024-12 conditional novelty 7.0 of 10

    As in-context examples grow, Llama-3.1-8B reorganizes its concept representations to mirror the connectivity structure of a graph defined entirely in context.

  3. Distill Skills into Weights, Not Prompts: Abstract Skills as Privileged Signals for On-Policy Self-Distillation

    cs.LG 2026-08 conditional novelty 6.0 of 10

    SKALD shows that distilling skill-conditioned teacher predictions into a question-only student improves math reasoning more than GRPO alone, with gains concentrated on rollout groups where rewards are uniform.

  4. Multi-Turn On-Policy Distillation with Prefix Replay

    cs.LG 2026-07 conditional novelty 6.0 of 10

    ReOPD offline-distills multi-turn agentic LLMs via teacher-prefix replay plus step-decay sampling, matching online OPD accuracy at ≥4× speed with zero tool calls.

  5. Learn from What We HAVE: History-Aware VErifier that Reasons about Past Interactions Online

    cs.RO 2025-08 conditional novelty 6.0 of 10

    A history-aware verifier that scores candidate actions using past interactions cuts failure rates in ambiguous robot manipulation tasks compared to using the generator alone.

  6. LLM Economist: Large Population Models and Mechanism Design in Multi-Agent Generative Simulacra

    cs.MA 2025-07 reject novelty 6.0 of 10

    The LLM Economist framework couples persona-conditioned worker agents with an in-context RL planner to search US-bracket tax schedules, yet its Saez benchmark is derived from the planner's own solution and its headlin...

  7. Behavioral Exploration: Learning to Explore via In-Context Adaptation

    cs.LG 2025-07 conditional novelty 6.0 of 10

    A coverage-conditioned behavioral cloning policy adapts in-context to its own history, making robots explore new expert-like behaviors online without online reinforcement learning.

  8. Robust In-Context Reinforcement Learning Under Reward Poisoning Attacks

    cs.LG 2025-06 conditional novelty 6.0 of 10

    Adversarially training a Decision-Pretrained Transformer against learned reward-poisoning attackers makes it robust to test-time reward corruption, outperforming robust bandit baselines in experiments.

  9. Filtering Learning Histories Enhances In-Context Reinforcement Learning

    cs.LG 2025-05 conditional novelty 6.0 of 10

    Filtering ICRL pretraining datasets by a simple improvement-and-stability score boosts downstream in-context learning performance across AD, DICP, and DPT baselines.

  10. Disentangling Exploration of Large Language Models by Optimal Exploitation

    cs.LG 2025-01 conditional novelty 6.0 of 10

    Exploration by LLM agents can be measured separately from exploitation using an optimal exploitation oracle, and most models explore poorly, with exploration performance correlated to reasoning ability.

  11. Hierarchical Prompt Decision Transformer: Improving Few-Shot Policy Generalization with Global and Adaptive Guidance

    cs.LG 2024-12 conditional novelty 6.0 of 10

    HPDT improves few-shot policy generalization in offline meta-RL by adding a global task-level prompt and retrieval-based adaptive prompts to a decision transformer.

  12. BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games

    cs.AI 2024-11 conditional novelty 6.0 of 10

    A benchmark of six long-horizon game environments shows current LLMs and VLMs struggle on hard tasks and often do worse when given images.

  13. AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers

    cs.LG 2024-11 conditional novelty 6.0 of 10

    Using two-hot classification for value prediction and binary-filtered imitation for policy updates makes multi-task meta-RL training scale-invariant to reward magnitudes, improving performance across five benchmarks w...

  14. ReBRAC-v2: The Return of the King

    cs.LG 2026-08 conditional novelty 5.0 of 10

    A fixed-recipe offline RL method combining normalizing-flow actors, categorical critics, staged training, and test-time refinement beats recent flow-based baselines by 22.5 points averaged over ten OGBench categories.

  15. Interaction as Intelligence: Deep Research With Human-AI Partnership

    cs.CL 2025-07 reject novelty 5.0 of 10

    A human-in-the-loop deep research system with transparent, interruptible interaction is claimed to outperform commercial baselines, but the evidence is weakened by small samples and biased instructions.

  16. ASTRO: Teaching Language Models to Reason by Reflecting and Backtracking In-Context

    cs.AI 2025-07 conditional novelty 5.0 of 10

    ASTRO converts MCTS search trees into chain-of-thoughts with explicit self-reflection and backtracking, trains Llama-3.1-70B on them with SFT, then improves with RL, reaching 81.8% on MATH-500, 64.4% on AMC 2023, and ...

  17. Large Language Model-Enhanced Multi-Armed Bandits

    cs.LG 2025-02 conditional novelty 5.0 of 10

    Using an LLM as a reward predictor inside Thompson sampling and regression-oracle bandits outperforms LLM direct arm selection in the tested tasks.

  18. HVAC-DPT: A Decision Pretrained Transformer for HVAC Control

    cs.LG 2024-11 conditional novelty 5.0 of 10

    A decision-pretrained transformer that controls HVAC dampers in-context reduced simulated annual energy use by about 31% versus a fixed baseline in one unseen building.

  19. Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning

    cs.CV 2024-11 conditional novelty 5.0 of 10

    A DPO-trained VLM critic that critiques and iteratively refines a reasoning VLM improves accuracy on several multimodal benchmarks, with large gains on MathVista and RealWorldQA.

Pith tools