Pith. sign in

REVIEW 4 major objections 5 minor 61 references

Agentic Episodic Control

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Agentic Episodic Control claims 2-6x higher data efficiency by storing LLM semantic state embeddings in episodic memory and selectively retrieving them at critical states.

desk verdict Genuinely new architecture and honest prompts, but the headline claim is contradicted by the paper's own appendix: ReAct and Naive LLM already solve UnlockLocal and match FindObj without any episodic control. read the letter →

arxiv 2506.01442 v2 pith:7JDQC4ZG submitted 2025-06-02 cs.AI

Xidong Yang , Wenhao Li , Junjie Sheng , Yun Hua , Haosheng Chen , Chuyun Shen , Xiangfeng Wang This is my paper
classification cs.AI
keywords episodiccontrollargelanguagemodelsreinforcementlearningsampleefficiencysemanticstateembeddingworldgraphworkingmemorycriticalrecognitionBabyAI-Text
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes Agentic Episodic Control (AEC), an RL architecture that replaces shallow state encoders with a frozen large language model that writes semantic embeddings into per-action episodic memory, and that decides when to retrieve those memories with a separate critical-state recognizer. The claim is that this combination turns passive similarity matching into selective, context-aware recall, yielding 2-6x higher data efficiency than episodic-RL and LLM baselines on BabyAI-Text tasks. On the long-horizon UnlockLocal task the paper reports AEC reaching above 0.9 success within 40K frames while all four baselines stay below 0.15 through 70K frames. If correct, the work suggests that linguistic priors can substitute for much of the trial-and-error signal that pure RL requires.

What carries the argument

The load-bearing mechanism has four parts. First, an LLM-based semantic encoder $\phi(s) = F(p(s))$ turns raw text observations plus task instruction into embeddings that cluster by meaning rather than surface form. Second, a per-action episodic memory stores $(\phi(s), \max \hat{Q}(s,a))$ pairs, updated after each episode with observed returns and retrieved by k-nearest-neighbour lookup. Third, a World-Graph working memory $G_t = (V_t, E_t, X_t)$ tracks rooms, doors, and reachability as a spatial graph that the agent uses for exploration. Fourth, a critical-state recognizer $\Psi(\phi(s_t))$ labels states like key pickup or reaching a locked door as critical, and only then triggers memory recall; otherwise the agent explores using the world graph. The claimed contribution is that these modules work as a circuit: semantic encoding gives memory keys that generalize, critical-state gating prevents indiscriminate retrieval, and the graph supplies structure for exploration.

What would settle it

Run AEC on UnlockLocal after replacing the frozen LLM semantic encoder with a fixed, randomly initialized text embedding, such as a bag-of-words or random projection, while keeping the rest of the architecture intact. If success remains above 0.9, the LLM is not the engine of the result; if it collapses to the 0.15 baseline level, the language-model prior, rather than the episodic-control or critical-state modules, is doing the work.

Watch

Extended reading notes

Core claim

On its own terms, AEC establishes that grounding episodic-RL state representations in LLM-derived semantics, and gating memory recall to critical decision points, yields a qualitative jump on instruction-following grid tasks expressed as text. The paper's central empirical discovery is the UnlockLocal result: with LLM semantic embeddings, a world-graph working memory, and critical-state-triggered retrieval, the agent exceeds 0.9 success in about 40K frames, while EM-DRRN, NECSA-DRRN, DRRN, and GLAM all remain below 0.15 at 70K frames. It also reports that AEC is the only method studied to reach above 0.2 success on the six-room FindObj task, and that removing either the LLM encoder or the world-graph module degrades performance across tasks. The native generalization test, presenting unseen object names, leaves AEC's success essentially unchanged on UnlockLocal and reduces it only slightly elsewhere.

Load-bearing premise

The paper assumes the performance gain comes from its new episodic-control modules, but it never tests a version without the built-in language-model ability, and the appendix shows an agent with no learning or memory already solves the hardest task perfectly.

Editorial extensions

If this is right

  • On BabyAI-Text, LLM-grounded episodic control can solve the multi-step UnlockLocal task to above 0.9 success, a level no tested RL baseline reaches within 70K frames.
  • AEC's cross-task transfer results indicate that memories encoded by the LLM semantic encoder are reusable across structurally related tasks, with a richer source task transferring more than a simpler one.
  • Ablations show both the LLM semantic encoder and the world-graph working memory contribute: removing the encoder cuts GoToLocal success from 0.84 to 0.49, while removing the graph hurts most on the long-horizon FindObj task.
  • The critical-state recognizer also reduces LLM usage: AEC invokes its LLM-based policy on 22-38% of steps in the first three tasks yet still matches or beats always-querying LLM agents.
  • State clustering by the semantic encoder suggests that generalization under new object names comes from grouping surface-different states into shared semantic categories.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An untested corollary is that the frozen LLM policy, not the episodic-control modules, may drive the headline UnlockLocal result: Appendix B reports that ReAct, an LLM agent with no learning or memory, scores 1.00 there, so a decisive next experiment is replacing the LLM encoder with a fixed non-LLM embedding and re-running the same benchmarks.
  • If AEC's gains do come from the LLM prior, then the paper's real lesson extends beyond episodic control: cheaply querying a strong generalist model at critical states could boost sample efficiency in many partially observable RL domains, including ones without natural language.
  • The cross-task transfer asymmetry (PickupLocal memory helps GoToLocal more than the reverse) suggests a testable ordering hypothesis: difficulty of the source task, not just structural similarity, predicts how much semantic memory transfers; one could verify this with a ladder of increasingly complex source tasks.
  • Because the world-graph module matters most in FindObj, a plausible extension is combining graph-based exploration with semantic memory in vision-based grid environments, where the same critical-state gate could decide when to read an occupancy map versus a learned value cache.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes Agentic Episodic Control (AEC), which integrates a frozen Qwen2.5-32B model as a semantic state encoder, a world-graph working memory, an LLM-based critical-state recognizer, and an episodic memory with k-nearest-neighbor retrieval. The agent is evaluated on four BabyAI-Text tasks and compared against four RL baselines (DRRN, EM-DRRN, NECSA-DRRN, GLAM), with additional cross-task memory transfer experiments, ablation studies, and an appendix comparing against always-querying LLM agents (Naive LLM and ReAct). The paper claims 2-6x higher data efficiency than baselines, is 'the only method' to solve UnlockLocal with over 90% success, and demonstrates strong cross-task and cross-environment generalization.

Significance. If the central claims were properly supported, the paper would make a useful contribution by showing that LLM-based semantic representations can be combined with selective episodic retrieval to improve sample efficiency and reduce LLM inference cost. The manuscript is transparent about its prompts, includes a broad set of RL baselines, and reports ablation and cross-task transfer studies. However, the main causal claim—that the episodic-control architecture rather than the frozen LLM prior drives the reported gains—is not tested in the main comparison, and the strongest 'only method' and '2-6x' statements are either contradicted by the paper's own Appendix B or left undefined. These issues are fixable, but they are load-bearing for the paper's headline conclusions.

major comments (4)
  1. [Abstract; §4.2, Table 1; Appendix B, Figure 6] The abstract and §4.2 claim that AEC is 'the only method to solve complex tasks like UnlockLocal with over 90% success' and 'the only algorithm to achieve over a 0.2 success rate' on FindObj, but Appendix B reports that ReAct—an agent with no episodic memory, no world-graph module, and no RL update—reaches 1.00 on UnlockLocal and 0.36 on FindObj, and that a Naive LLM reaches 0.98 on UnlockLocal. These LLM-only agents are absent from Table 1 and Figure 3, so the main-text comparison does not control for the pretrained policy on which AEC is built. Please restrict the 'only method' claim to the four RL baselines, integrate the LLM-agent comparison into the main results, and rephrase the abstract and §4.2 accordingly.
  2. [Abstract; §4.2] The abstract's '2-6x higher data efficiency' claim is never formally defined or computed. Section 4.2 mentions 'nearly 2×' for GoToLocal but does not specify the success-rate threshold, the number of frames used to reach it, or the corresponding factors for PickupLocal, UnlockLocal, and FindObj. Please provide a precise definition of data efficiency (e.g., frames to a given success threshold) and report the factor for each task and each baseline.
  3. [§4.4, Figure 5; Appendix B] The ablation study removes the semantic encoder and the working-memory module, but it does not remove the frozen LLM policy or the critical-state recognizer. Because ReAct and Naive LLM—which use no episodic memory, no world graph, and no RL update—match or exceed AEC on UnlockLocal and FindObj, the reported ablation cannot establish that the episodic-control modules, rather than the Qwen2.5-32B prior, drive the performance. Please add an ablation that controls for the LLM prior (e.g., an LLM-only policy at a matched inference budget, or a non-LLM semantic encoder) and an ablation that toggles the critical-state recognizer.
  4. [§3.3, Eq. (4); Appendix A.3] The critical-state recognizer is a core component that decides when episodic memory is queried, yet the manuscript provides no evaluation of its correctness or sensitivity. The prompt criterion 'is there a target direction?' (Appendix A.3) appears to encode task-specific knowledge, and Figure 7 reports only the resulting proportion of LLM decisions. Please report the false-positive and false-negative rates of the recognizer against a labeled set of states, or at minimum provide an ablation that replaces it with a fixed schedule or with always/never querying, so that the selective-retrieval claim in Eq. (5) is supported.
minor comments (5)
  1. [Figure 1; throughout] There are inconsistent spellings: Figure 1 contains 'bule door' instead of 'blue door', and the text uses 'GotoLocal' and 'PickUpLocal' in some places while 'GoToLocal' and 'PickupLocal' appear elsewhere; please standardize.
  2. [Table 1 caption] The caption contains a stray 'w' at the end of the first sentence; it should be removed.
  3. [§4.2 vs. Figure 3] The statement that 'GLAM requires around 50K frames to approach similar performance (0.75)' is not clearly visible in the GoToLocal panel of Figure 3 as printed; please add explicit markers or numerical annotations to the learning curves.
  4. [§1 and §3.3] The critical-state detector is described as 'lightweight' in the introduction, while §3.3 says it is powered by an LLM; please clarify the intended meaning, since the model used is Qwen2.5-32B.
  5. [Appendix B, Figure 7] The inference-cost comparison would be easier to interpret if the manuscript reported the total number of LLM calls for the Naive LLM and ReAct baselines as well as the proportion of LLM decisions for AEC.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: AEC's results are empirical comparisons on external benchmarks; the Appendix B LLM-prior confound is a validity gap, not a circular reduction.

full rationale

Agentic Episodic Control is an empirical methods paper, not a derivation. The central claims are success rates and data-efficiency curves measured on the external BabyAI-Text benchmark against external baselines (DRRN, EM-DRRN, NECSA-DRRN, GLAM). Equations (1)-(5) define a standard episodic-control lookup (state embedding, stored maximum return, KNN retrieval, argmax action), and no quantity in them is defined in terms of the reported result or fitted to a subset and then renamed a prediction. There are no self-citations by the present authors, so no load-bearing self-citation chain exists. The genuine weakness is in Appendix B: ReAct (1.00) and Naive LLM (0.98) match or exceed AEC's 0.95 on UnlockLocal, and ReAct's 0.36 exceeds AEC's 0.23 on FindObj, yet these LLM-only policies are absent from Table 1 and Figure 3. That is an uncontrolled-variable/ablation problem about whether the frozen LLM prior, rather than AEC's memory modules, drives the gains; it is a comparative-validity concern, not a circularity, because no equation reduces to its own input by construction. The stated computational-overhead limitation in the conclusion is candid and non-circular. Score 0.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claim rests on hand-crafted LLM prompts that encode substantial task knowledge, plus a few unreported hyperparameters. No new physical or mathematical entities are introduced.

free parameters (2)
  • KNN neighborhood size k
    Episodic memory retrieval in Eq. (5) uses k-nearest-neighbor search, but the paper never specifies k or reports sensitivity to it.
  • Return computation for Q-hat
    Eq. (1) stores 'maximum observed return' but does not define how future returns are accumulated or whether discounting is applied, which directly affects memory values and action selection.
assumptions (5)
  • domain assumption The LLM semantic encoder correctly parses raw text observations into state embeddings that cluster semantically similar states.
    Section 3.1 relies on F producing reliable embeddings for KNN retrieval; no parser accuracy or embedding quality metric is reported.
  • domain assumption The LLM world-graph builder produces factually correct and complete graphs from partial observations.
    Section 3.2 trusts the LLM to maintain room labels, object inventories, and door triplets; no graph fidelity verification is provided.
  • ad hoc to paper The critical state recognizer's yes/no answer to 'is there a target direction?' is a valid proxy for when episodic retrieval should be used.
    Section 3.3 and Appendix A.3 define Ψ as a hand-designed LLM prompt; this mapping is not learned or validated against an external notion of state criticality.
  • domain assumption Maximum observed return stored in episodic memory is a reliable Q-value estimate for action selection.
    This is the standard episodic control premise inherited from NEC [19], invoked in Section 3.1 without re-derivation or correction for stochasticity.
  • domain assumption The LLM's pretrained knowledge covers the vocabulary and concepts in BabyAI-Text, such as object names, colors, keys, and doors.
    The entire method depends on the LLM understanding these concepts; the paper does not test out-of-vocabulary objects beyond the 'New Object' setting, which still uses known color-item names.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Agentic Episodic Control." pith.science (2026). https://pith.science/paper/7JDQC4ZG

@misc{pith2026250601442,
  author       = {Pith},
  title        = {Pith review of: Agentic Episodic Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7JDQC4ZG}},
  note         = {Machine review of arXiv:2506.01442}
}
read the original abstract

Reinforcement learning (RL) remains fundamentally limited by poor data efficiency and weak generalization. Prior episodic RL methods attempt to alleviate this via external memory modules, yet they suffer from two key limitations: a representation bottleneck caused by shallow encoders, and a retrieval dilemma where episodic memory is accessed indiscriminately. To address these challenges, we propose Agentic Episodic Control (AEC), a novel architecture that integrates large language models (LLMs) into episodic RL. AEC uses an LLM-based semantic augmenter to generate semantic representations from raw observations, and a critical state recognizer to selectively retrieve valuable experiences. This transforms memory usage from passive similarity matching into strategic, context-aware recall. Across five BabyAI-Text environments, AEC achieves 2-6x higher data efficiency than baselines and is the only method to solve complex tasks like UnlockLocal with over 90% success. It further demonstrates strong cross-task and cross-environment generalization, maintaining performance even under distribution shifts. AEC shows that combining LLM-derived priors with reinforcement learning yields more sample-efficient and adaptable agents.

Figures

Figures reproduced from arXiv: 2506.01442 by the authors.

Figure 1
Figure 1. The overview of Agentic Episodic Control framework. In the Agentic Episodic Control [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. During interactions, the agent stores state embeddings, actions, and rewards. When an [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Learning curves (success rate vs. training frames) for four BabyAI-Text tasks comparing [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Ablation study results across four BabyAI-Text tasks. on superficial observational features but is instead capturing deeper semantic structures. Although these states differ in surface-level details, they exhibit strong similarity in higher-level aspects, such as the r…
Figure 6
Figure 6. Figure 6: Comparison of success rates with LLM agents on four BabyAI-Text tasks. GoToLocal PickupLocal UnlockLocal FindObj 0.0 0.2 0.4 0.6 0.8 1.0 Proportion of LLM Decisions [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

61 extracted references · 50 canonical work pages

  1. [1]

    Mas- tering the game of go with deep neural networks and tree search

    David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driess- che, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. Mas- tering the game of go with deep neural networks and tree search. nature, 529(7587):484–489, 2016

    2016

  2. [2]

    Mastering the game of go without human knowledge

    David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastering the game of go without human knowledge. nature, 550(7676):354–359, 2017

    2017

  3. [3]

    Dy- namic robot routing optimization: State–space decomposition for operations research-informed reinforcement learning

    Marlon Löppenberg, Steve Yuwono, Mochammad Rizky Diprasetya, and Andreas Schwung. Dy- namic robot routing optimization: State–space decomposition for operations research-informed reinforcement learning. Robotics and Computer-Integrated Manufacturing, 90:102812, 2024

    work page 2024

  4. [4]

    Transferring policy of deep reinforcement learning from simulation to reality for robotics

    Hao Ju, Rongshun Juan, Randy Gomez, Keisuke Nakamura, and Guangliang Li. Transferring policy of deep reinforcement learning from simulation to reality for robotics. Nature Machine Intelligence, 4(12):1077–1087, 2022

    work page 2022

  5. [5]

    Embodied intelligence via learning and evolution

    Agrim Gupta, Silvio Savarese, Surya Ganguli, and Li Fei-Fei. Embodied intelligence via learning and evolution. Nature communications, 12(1):5721, 2021

    work page 2021

  6. [6]

    Ruiz, Julian Schrittwieser, Grzegorz Swirszcz, et al

    Alhussein Fawzi, Matej Balog, Aja Huang, Thomas Hubert, Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Francisco J R. Ruiz, Julian Schrittwieser, Grzegorz Swirszcz, et al. Discovering faster matrix multiplication algorithms with reinforcement learning. Nature, 610(7930):47–53, 2022

    work page 2022

  7. [7]

    Faster sorting algorithms discovered using deep reinforcement learning

    Daniel J Mankowitz, Andrea Michi, Anton Zhernov, Marco Gelmi, Marco Selvi, Cosmin Paduraru, Edouard Leurent, Shariq Iqbal, Jean-Baptiste Lespiau, Alex Ahern, et al. Faster sorting algorithms discovered using deep reinforcement learning. Nature, 618(7964):257–263, 2023

    work page 2023

  8. [8]

    Highly accurate protein structure prediction for the human proteome

    Kathryn Tunyasuvunakool, Jonas Adler, Zachary Wu, Tim Green, Michal Zielinski, Augustin Žídek, Alex Bridgland, Andrew Cowie, Clemens Meyer, Agata Laydon, et al. Highly accurate protein structure prediction for the human proteome. Nature, 596(7873):590–596, 2021

    work page 2021

Show all 61 references
  1. [9]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leik...

    2022

  2. [10]

    Reft: Reasoning with reinforced fine-tuning

    Luong Quoc Trung, Xinbo Zhang, Zhanming Jie, Peng Sun, Xiaoran Jin, and Hang Li. Reft: Reasoning with reinforced fine-tuning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, 2024

    2024

  3. [11]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  4. [12]

    Devon Hjelm, Philip Bachman, and Aaron C

    Max Schwarzer, Nitarshan Rajkumar, Michael Noukhovitch, Ankesh Anand, Laurent Charlin, R. Devon Hjelm, Philip Bachman, and Aaron C. Courville. Pretraining representations for data-efficient reinforcement learning. In Advances in Neural Information Processing Systems, 2021

    2021

  5. [13]

    Ball, Laura M

    Philip J. Ball, Laura M. Smith, Ilya Kostrikov, and Sergey Levine. Efficient online reinforcement learning with offline data. In International Conference on Machine Learning, 2023

    2023

  6. [14]

    A survey analyzing generalization in deep reinforcement learning

    Ezgi Korkmaz. A survey analyzing generalization in deep reinforcement learning. arXiv preprint arXiv:2401.02349, 2024

    2024 arXiv

  7. [15]

    Adams, and Sergey Levine

    Dibya Ghosh, Jad Rahme, Aviral Kumar, Amy Zhang, Ryan P. Adams, and Sergey Levine. Why generalization in RL is difficult: Epistemic pomdps and implicit partial observability. In Advances in Neural Information Processing Systems, 2021

    2021

  8. [16]

    The neurobiology of semantic memory

    Jeffrey R Binder and Rutvik H Desai. The neurobiology of semantic memory. Trends in cognitive sciences, 15(11):527–536, 2011

    2011

  9. [17]

    Hippocampus: Cognitive processes and neural representations that underlie declarative memory

    Howard Eichenbaum. Hippocampus: Cognitive processes and neural representations that underlie declarative memory. Neuron, 44(1):109–120, 2004. 10

    2004

  10. [18]

    B. A. Kuhl and A. D. Wagner. Strategic Control of Memory, pages 437–444. Elsevier Ltd, 2009

    2009

  11. [19]

    Neural episodic control

    Alexander Pritzel, Benigno Uria, Sriram Srinivasan, Adria Puigdomenech Badia, Oriol Vinyals, Demis Hassabis, Daan Wierstra, and Charles Blundell. Neural episodic control. In International conference on machine learning, 2017

    2017

  12. [20]

    Been there, done that: Meta-learning with episodic recall

    Samuel Ritter, Jane X Wang, Zeb Kurth-Nelson, Siddhant M Jayakumar, Charles Blundell, Razvan Pascanu, and Matthew M Botvinick. Been there, done that: Meta-learning with episodic recall. In International Conference on Machine Learning, 2018

    2018

  13. [21]

    Hippocampal contributions to control: The third way

    Máté Lengyel and Peter Dayan. Hippocampal contributions to control: The third way. In Advances in Neural Information Processing Systems, 2008

    2008

  14. [22]

    Optimizing agent behavior over long time scales by transporting value

    Steven C Hung, Timothy P Lillicrap, Josh Abramson, Greg Wayne, Shakir Mohamed, and V olodymyr Mnih. Optimizing agent behavior over long time scales by transporting value. In International Conference on Learning Representations, 2019

    2019

  15. [23]

    A simple neural attentive meta-learner

    Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel. A simple neural attentive meta-learner. In International Conference on Learning Representations, 2018

    2018

  16. [24]

    Prefrontal cortex as a meta- reinforcement learning system

    Jane X Wang, Zeb Kurth-Nelson, Dhruva Tirumala, Hubert Soyer, Joel Z Leibo, Rémi Munos, Charles Blundell, Dharshan Kumaran, and Matthew Botvinick. Prefrontal cortex as a meta- reinforcement learning system. Nature neuroscience, 21(6):860–868, 2018

    2018

  17. [25]

    The neural representations underlying human episodic memory

    Gui Xue. The neural representations underlying human episodic memory. Trends in Cognitive Sciences, 22(6):544–561, 2018

    2018

  18. [26]

    Ramey, John M

    Michelle M. Ramey, John M. Henderson, and Andrew P. Yonelinas. Episodic memory processes modulate how schema knowledge is used in spatial memory decisions. Cognition, 225:105111, 2022

    2022

  19. [27]

    Structure and deterioration of semantic memory: a neuropsychological and computational investigation

    Timothy T Rogers, Matthew A Lambon Ralph, Peter Garrard, Sasha Bozeat, James L McClel- land, John R Hodges, and Karalyn Patterson. Structure and deterioration of semantic memory: a neuropsychological and computational investigation. Psychological review, 111(1):205, 2004

    2004

  20. [28]

    Episodic memory: From mind to brain.Annual review of psychology, 53(1):1–25, 2002

    Endel Tulving. Episodic memory: From mind to brain.Annual review of psychology, 53(1):1–25, 2002

    2002

  21. [29]

    Working memory 2.0

    Earl K Miller, Mikael Lundqvist, and André M Bastos. Working memory 2.0. Neuron, 100(2):463–475, 2018

    2018

  22. [30]

    Multi-logieval: Towards evaluating multi-step logical reasoning ability of large language models

    Nisarg Patel, Mohith Kulkarni, Mihir Parmar, Aashna Budhiraja, Mutsumi Nakamura, Neeraj Varshney, and Chitta Baral. Multi-logieval: Towards evaluating multi-step logical reasoning ability of large language models. In Proceedings of the 2024 Conference on Empirical Methods in N...

    2024

  23. [31]

    Gemini in reasoning: Unveiling commonsense in multimodal large language models

    Yuqing Wang and Yun Zhao. Gemini in reasoning: Unveiling commonsense in multimodal large language models. arXiv preprint arXiv:2312.17661, 2023

    2023 arXiv

  24. [32]

    Mathprompter: Mathematical reasoning using large language models

    Shima Imani, Liang Du, and Harsh Shrivastava. Mathprompter: Mathematical reasoning using large language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, 2023

    2023

  25. [33]

    Model-free episodic control

    Charles Blundell, Benigno Uria, Alexander Pritzel, Yazhe Li, Avraham Ruderman, Joel Z Leibo, Jack Rae, Daan Wierstra, and Demis Hassabis. Model-free episodic control. arXiv preprint arXiv:1606.04460, 2016

    2016 arXiv

  26. [34]

    Episodic memory deep q- networks

    Zichuan Lin, Tianqi Zhao, Guangwen Yang, and Lintao Zhang. Episodic memory deep q- networks. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, 2018

    2018

  27. [35]

    Fast deep reinforcement learning using online adjustments from the past

    Steven Hansen, Alexander Pritzel, Pablo Sprechmann, André Barreto, and Charles Blundell. Fast deep reinforcement learning using online adjustments from the past. In Advances in Neural Information Processing Systems, 2018

    2018

  28. [36]

    Deep reinforcement learning with parametric episodic memory

    Kangkang Chen, Zhongxue Gan, Siyang Leng, and Chun Guan. Deep reinforcement learning with parametric episodic memory. In 2022 International Joint Conference on Neural Networks, 2022

    2022

  29. [37]

    Episodic reinforcement learning with expanded state-reward space

    Dayang Liang, Yaru Zhang, and Yunlong Liu. Episodic reinforcement learning with expanded state-reward space. In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems, 2024. 11

    2024

  30. [38]

    Sample-efficient deep reinforcement learning via episodic backward update

    Su Young Lee, Sung-Ik Choi, and Sae-Young Chung. Sample-efficient deep reinforcement learning via episodic backward update. In Advances in Neural Information Processing Systems, 2019

    2019

  31. [39]

    Episodic reinforcement learning with associative memory

    Guangxiang Zhu, Zichuan Lin, Guangwen Yang, and Chongjie Zhang. Episodic reinforcement learning with associative memory. In International Conference on Learning Representations, 2020

    2020

  32. [40]

    Neural episodic control with state abstraction

    Zhuo Li, Derui Zhu, Yujing Hu, Xiaofei Xie, Lei Ma, Yan Zheng, Yan Song, Yingfeng Chen, and Jianjun Zhao. Neural episodic control with state abstraction. In International Conference on Learning Representations, 2023

    2023

  33. [41]

    Hypothesis, verification, and induction: grounding large language models with self-driven skill learning

    Shaohui Peng, Xing Hu, Qi Yi, Rui Zhang, Jiaming Guo, Di Huang, Zikang Tian, Ruizhi Chen, Zidong Du, Qi Guo, et al. Hypothesis, verification, and induction: grounding large language models with self-driven skill learning. In Proceedings of the AAAI Conference on Artificial Int...

    2024

  34. [42]

    Lgts: Dynamic task sampling using llm-generated sub-goals for reinforcement learning agents

    Yash Shukla, Wenchang Gao, Vasanth Sarathy, Alvaro Velasquez, Robert Wright, and Jivko Sinapov. Lgts: Dynamic task sampling using llm-generated sub-goals for reinforcement learning agents. In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent ...

    2024

  35. [43]

    Reward design with language models

    Minae Kwon, Sang Michael Xie, Kalesha Bullard, and Dorsa Sadigh. Reward design with language models. arXiv preprint arXiv:2303.00001, 2023

    2023 arXiv

  36. [44]

    Read and reap the rewards: learning to play atari with the help of instruction manuals

    Yue Wu, Yewen Fan, Paul Pu Liang, Amos Azaria, Yuanzhi Li, and Tom Mitchell. Read and reap the rewards: learning to play atari with the help of instruction manuals. In Advances in Neural Information Processing Systems, 2023

    2023

  37. [45]

    Auto mc-reward: Automated dense reward design with large language models for minecraft

    Hao Li, Xue Yang, Zhaokai Wang, Xizhou Zhu, Jie Zhou, Yu Qiao, Xiaogang Wang, Hongsheng Li, Lewei Lu, and Jifeng Dai. Auto mc-reward: Automated dense reward design with large language models for minecraft. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patter...

    2024

  38. [46]

    Efficient reinforcement learning via large language model-based search

    Siddhant Bhambri, Amrita Bhattacharjee, Subbarao Kambhampati, et al. Efficient reinforcement learning via large language model-based search. In NeurIPS 2024 Workshop on Open-World Agents, 2024

    2024

  39. [47]

    Ask more, know better: Reinforce-learned prompt questions for decision making with large language models

    Xue Yan, Yan Song, Xinyu Cui, Filippos Christianos, Haifeng Zhang, David Henry Mguni, and Jun Wang. Ask more, know better: Reinforce-learned prompt questions for decision making with large language models. arXiv preprint arXiv:2310.18127, 2023

    arXiv

  40. [48]

    Unleashing the power of pre-trained language models for offline reinforcement learning

    Ruizhe Shi, Yuyao Liu, Yanjie Ze, Simon Shaolei Du, and Huazhe Xu. Unleashing the power of pre-trained language models for offline reinforcement learning. In International Conference on Learning Representations, 2024

    2024

  41. [49]

    Pre-trained language models for interactive decision-making

    Shuang Li, Xavier Puig, Chris Paxton, Yilun Du, Clinton Wang, Linxi Fan, Tao Chen, De-An Huang, Ekin Akyürek, Anima Anandkumar, et al. Pre-trained language models for interactive decision-making. In Advances in Neural Information Processing Systems, 2022

    2022

  42. [50]

    Grounding large language models in interactive environments with online reinforcement learning

    Thomas Carta, Clément Romac, Thomas Wolf, Sylvain Lamprier, Olivier Sigaud, and Pierre- Yves Oudeyer. Grounding large language models in interactive environments with online reinforcement learning. In International Conference on Machine Learning, 2023

    2023

  43. [51]

    Twosome: An efficient online framework to align llms with embodied environments via reinforcement learning

    Weihao Tan, Wentao Zhang, Shanqi Liu, Longtao Zheng, Xinrun Wang, and Bo An. Twosome: An efficient online framework to align llms with embodied environments via reinforcement learning. International Journal of Artificial Intelligence and Robotics Research, 01(02):2450004, 2024

    2024

  44. [52]

    Prompt to transfer: Sim-to-real transfer for traffic signal control with prompt learning

    Longchao Da, Minquan Gao, Hao Mei, and Hua Wei. Prompt to transfer: Sim-to-real transfer for traffic signal control with prompt learning. In Thirty-Eighth AAAI Conference on Artificial Intelligence, 2024

    2024

  45. [53]

    Suttle, Carl E

    Chak Lam Shek, Xiyang Wu, Wesley A. Suttle, Carl E. Busart, Erin G. Zaroukian, Dinesh Manocha, Pratap Tokekar, and Amrit Singh Bedi. LANCAR: leveraging language for context- aware robot locomotion in unstructured environments. In IEEE/RSJ International Conference on Intelligen...

    2024

  46. [54]

    Llm-state: Open world state representation for long-horizon task planning with large language model

    Siwei Chen, Anxing Xiao, and David Hsu. Llm-state: Open world state representation for long-horizon task planning with large language model. arXiv preprint arXiv:2311.17406, 2023. 12

    2023 arXiv

  47. [55]

    Llm-empowered state representation for reinforcement learning

    Boyuan Wang, Yun Qu, Yuhang Jiang, Jianzhun Shao, Chang Liu, Wenming Yang, and Xi- angyang Ji. Llm-empowered state representation for reinforcement learning. In International Conference on Machine Learning, 2024

    2024

  48. [56]

    Morris, Richard J

    Vinamra Benara, Chandan Singh, John X. Morris, Richard J. Antonello, Ion Stoica, Alexander Huth, and Jianfeng Gao. Crafting interpretable embeddings for language neuroscience by asking llms questions. In Advances in Neural Information Processing Systems, 2024

    2024

  49. [57]

    Generalization of rein- forcement learners with working and episodic memory

    Meire Fortunato, Melissa Tan, Ryan Faulkner, Steven Hansen, Adrià Puigdomènech Badia, Gavin Buttimore, Charles Deck, Joel Z Leibo, and Charles Blundell. Generalization of rein- forcement learners with working and episodic memory. In Advances in Neural Information Processing Sy...

    2019

  50. [58]

    Babyai: A platform to study the sample efficiency of grounded language learning

    Maxime Chevalier-Boisvert, Dzmitry Bahdanau, Salem Lahlou, Lucas Willems, Chitwan Sa- haria, Thien Huu Nguyen, and Yoshua Bengio. Babyai: A platform to study the sample efficiency of grounded language learning. In International Conference on Learning Representations, 2019

    2019

  51. [59]

    Deep reinforcement learning with a combinatorial action space for predicting popular reddit threads

    Ji He, Mari Ostendorf, Xiaodong He, Jianshu Chen, Jianfeng Gao, Lihong Li, and Li Deng. Deep reinforcement learning with a combinatorial action space for predicting popular reddit threads. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2016

    2016

  52. [60]

    Jansen, Marc-Alexandre Côté, and Prithviraj Ammanabrolu

    Ruoyao Wang, Peter A. Jansen, Marc-Alexandre Côté, and Prithviraj Ammanabrolu. Science- world: Is your agent smarter than a 5th grader? In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022

    2022

  53. [61]

    red ball

    Qwen Team. Qwen2.5: A party of foundation models, September 2024. 13 Appendix In the Appendix, we introduce more details along with additional experimental results and discussions: • Section A: Prompt Engineering • Section B: Comparison with LLM agents A Prompt Engineering We ...

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.