REVIEW 1 cited by
Transformers Can Learn Temporal Difference Methods for In-Context Reinforcement Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Traditionally, reinforcement learning (RL) agents learn to solve new tasks by updating their neural network parameters through interactions with the task environment. However, recent works demonstrate that some RL agents, after certain pretraining procedures, can learn to solve unseen new tasks without parameter updates, a phenomenon known as in-context reinforcement learning (ICRL). The empirical success of ICRL is widely attributed to the hypothesis that the forward pass of the pretrained agent neural network implements an RL algorithm. In this paper, we support this hypothesis by showing, both empirically and theoretically, that when a transformer is trained for policy evaluation tasks, it can discover and learn to implement temporal difference learning in its forward pass.
Forward citations
Cited by 1 Pith paper
-
Meta-Prompt Optimization for LLM-Based Sequential Decision Making
EXPO uses adversarial bandit weighting over LLM-generated prompt variations to optimize the meta-prompt of LLM-based sequential decision-making agents, improving performance on optimization and bandit tasks.
Discussion (0). Continue with ORCID to comment.