Pith. sign in

REVIEW 3 major objections 7 minor 3 cited by

A Survey On Enhancing Reinforcement Learning in Complex Environments: Insights from Human and LLM Feedback

T0 review · 3 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This survey groups the work on making RL agents work in complex environments into two research directions—human or LLM feedback, and attention mechanisms for large observation spaces—and identifies six gaps the field has not yet closed.

desk verdict A usable but under-audited survey of human/LLM feedback for RL; the taxonomy is sensible, but the ChatGPT-generated summaries and missing selection protocol mean it is not yet a dependable reference. read the letter →

arxiv 2411.13410 v1 pith:CRM3VLRO submitted 2024-11-20 cs.LG

Alireza Rashidi Laleh , Majid Nili Ahmadabadi This is my paper · ORCID
classification cs.LG
keywords reinforcementlearninghumanfeedbacklargelanguagemodelscurseofdimensionalityattentionmechanismsnaturalsurveysampleinefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is a survey that tries to organize the growing literature on improving reinforcement learning agents in complex environments. Its central claim is that the field's main fixes fall into two complementary buckets: supplementing the agent with feedback or assistance from humans or large language models, and using attention mechanisms to cope with large observation spaces. The survey builds a taxonomy of recent work within each bucket, distinguishing natural-language from other feedback, real-time from static guidance, and external from integrated language models. It then lists six open problems, including the lack of multi-granularity feedback, limited agent autonomy, and the absence of standard datasets. A reader would care because the paper offers a structured map of what has been tried and where the field is thin.

What carries the argument

The central object is the survey's taxonomy itself: a two-part classification scheme that divides the literature into human/LLM feedback (Sections 2 and 3) and attention for large observation spaces (Section 4). The taxonomy does the work of the paper by sorting papers into nested clusters, defined by feedback modality, timing, granularity, and the role the language model plays. It is the lens through which the authors identify the six gaps in Section 5, and it is the deliverable a reader would reuse to position future work.

What would settle it

Select a random sample of the cited papers and read their abstracts; if a substantial share of the survey's one-line descriptions misstates the method or contribution stated in the original abstract, then the taxonomy's categories are not a faithful map of the field.

Watch

Extended reading notes

Core claim

The authors claim that the challenges of reinforcement learning—sample inefficiency, poor generalization, and the curse of dimensionality in large observation spaces—can be attacked along two largely separate research lines. The first line augments the RL agent with external guidance: human feedback in forms ranging from natural language instructions to demonstrations, and, more recently, feedback from large language models used as reward designers, planners, or even complete agents. The second line builds attention mechanisms directly into the agent so it can focus on the relevant parts of a large observation space. The survey organizes dozens of papers into a hierarchical taxonomy with finer clusters such as natural-language instructions in simulated versus robotic environments, real-time feedback loops, abstraction and description, LLM as a component versus LLM as an agent, and attention for visual navigation. Its conclusion is that these two directions are largely complementary and that six specific limitations currently constrain progress.

Load-bearing premise

The entire taxonomy depends on the correctness of the one-sentence summaries of each cited paper, which the authors disclose were produced with an AI-based tool; if any of those summaries misrepresents the original work, the classification built on them is unreliable.

Editorial extensions

If this is right

  • New researchers can use the taxonomy as an entry point: first decide whether an approach channels human or LLM feedback, or attacks the observation space with attention, then locate it within the finer clusters.
  • The six identified gaps—multi-granularity feedback, agent autonomy, long descriptions and instructions, human awareness of tasks, dynamic communication, and the need for datasets—name concrete places where the field is thin.
  • Combining external feedback with attention mechanisms is a natural next step, since the survey frames the two as complementary responses to the same core challenges.
  • The sub-categorization by feedback format (natural language versus other, static versus real-time, extrinsic versus integrated) provides a shared vocabulary for comparing otherwise disparate papers.
  • The emphasis on sample efficiency suggests that benchmark comparisons should report not only final performance but also training-time savings when feedback or attention is added.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The taxonomy could double as a benchmark selection guide: by clustering papers according to feedback type and environment, it implicitly suggests which baseline systems to compare against when testing a new human-in-the-loop or LLM-assisted RL method.
  • Because the paper's one-sentence summaries of cited works were produced with an AI-based tool and no systematic selection protocol is reported, the finer claims of each cluster would need independent verification against the original papers before being used to support a new research program.
  • If the identified gaps are real, the field may converge on multi-granularity, adaptive feedback systems that decide when to ask for help—a direction that is only implicit in the survey's discussion.
  • The survey's split between human feedback and LLM feedback, while useful, may blur as LLM-based systems increasingly stand in for human annotators; a testable extension would examine whether the two clusters yield the same per-paper categorizations when the LLM feedback is itself generated from human demonstrations.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. This paper is a survey of two bodies of RL literature: (i) augmenting RL agents with human or LLM feedback (§2–3), and (ii) attention mechanisms for RL in large observation spaces (§4). The survey proposes a taxonomy, with Section 2 dividing human-feedback work by natural-language versus other feedback, Section 3 dividing LLM-for-RL work into post-ChatGPT and pre-ChatGPT categories, and Section 4 clustering attention-based work into visual attention, curse of dimensionality, interpretability, and visual-language navigation. It closes with a discussion of research gaps and a conclusion. The authors disclose in the Acknowledgment that ChatGPT was used to create summaries of the cited works.

Significance. If reliable, the survey's taxonomy would provide a useful map of the growing human/LLM-feedback literature and the attention-for-large-observation-space literature. The paper does a service by collecting a broad set of recent preprints, by explicitly separating external LLM feedback from LLMs embedded as components, and by flagging open problems such as multi-granular feedback and agent autonomy. However, the contribution rests on the accuracy of per-paper summaries and on the completeness and traceability of the selection process; the disclosed use of ChatGPT for summaries, the absence of selection criteria, and several demonstrable misclassifications mean the survey cannot currently be used as a trusted reference.

major comments (3)
  1. [Section 4 and overall survey scope] The survey gives no stated inclusion/exclusion criteria, search protocol, or selection rationale. The abstract promises a survey 'dedicated to addressing the intricacies of environments characterized by large observation space,' but §4.2 contains only four papers, several of which (e.g., [76] vehicle routing, [77] sequence modeling) are not primarily framed around the curse of dimensionality, while §4.4 includes RT-2 [82], a vision-language-action manipulation model rather than a VLN navigation agent. In addition, Section 2.2.4 on 'Human Feedback as Reward' omits the canonical preference-based RLHF line (e.g., Christiano et al. 2017, Stiennon et al. 2020, Ouyang et al. 2022) even though the paper cites survey [9] on this topic. Without a documented selection process, the reader cannot distinguish a representative survey from an ad-hoc sample.
  2. [Acknowledgment; §2.1.2; §4.4] The Acknowledgment states that ChatGPT 'create[d] summaries of the cited works.' A spot-check of the summaries against the cited papers reveals clear misclassifications that affect the taxonomy. In §2.1.2, the summary of [25] ('Yell at Your Robot') describes an 'RL agent' performing action selection and behavior cloning, but that paper is a language-conditioned imitation/diffusion-policy method and does not use reinforcement learning. In §4.4, [82] (RT-2) is presented as part of the Visual-Language-Navigation cluster, but RT-2 is a vision-language-action model for robotic manipulation trained by supervised learning. These are not cosmetic issues; they change category assignments and therefore the survey's substantive claims about families of methods. The authors should audit every summary, correct or remove inaccurate attributions, and describe the verification procedure.
  3. [§2.2.5] The subsection label 'Human Feedback as Demonstrations' and the blanket statement 'A term for these approaches is called inverse RL' do not fit all papers placed there. For example, [47] uses a Gaussian-process teacher-advice mechanism built on demonstrations, and [46] relies on human intervention and mentoring rather than demonstration-based inverse RL. The criteria for membership in this category should be stated explicitly, and the category name or the placements should be adjusted so that the taxonomy boundaries are testable.
minor comments (7)
  1. [Section 1] The sentence 'The remainder of this paper is organized as follows: Section ?? provides fundamental concepts surrounding RL and LLMs' contains an unresolved placeholder; no such section exists in the manuscript, so either add the promised background section or delete the sentence.
  2. [Figure 6] Figure 6, which is placed in §2.2.4, is captioned 'Abstract idea and architecture of papers in Section 2.1.3'; the caption should refer to Section 2.2.4.
  3. [Section 3.1.1] The bullet list says these papers are 'similar to the papers from Sections 2.1.4 2.1.2 and 2.1.2'; the section numbers are garbled and the duplication suggests it should read '2.1.1, 2.1.2, and 2.1.4.'
  4. [Throughout] There are numerous typos and grammatical errors, including 'preforms' (§2.1.2), 'behaior' (§2.2.1), 'imrpove' (§2.2.1), 'alongisde' (§2.2.1), 'fucntion' (§2.2.5), 'appraoch' (§2.1.3), 'costy' (§3.1.1), 'autnomously' (§3.1.1), 'pretained' (§3.1.2), and 'granularitites' (§5.1); the manuscript needs a careful proofreading pass.
  5. [Section 5.4] The sentence 'Since, humans may not always know the task, environment and they might know the optimal behavior and decision-making' is not grammatical; the final clause should be revised to express that humans may not know the optimal behavior.
  6. [Section 1] The introduction cites existing surveys [9] and [10] but does not explain what this survey adds beyond them; a short positioning paragraph would clarify the intended contribution.
  7. [Section 2.1.4] The summary of [31] says the approach uses a 'contextual banding algorithm'; this appears to be a typo for 'contextual bandit algorithm.'

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the survey's taxonomy is an organizational summary of external papers, with no fitted inputs, no predictive derivation, and no load-bearing self-citation.

full rationale

This paper is a literature survey with no fitted parameters, no predictive model, and no stated derivation from prior author results. The central contribution is a categorization of existing work on human/LLM feedback for RL and attention in large observation spaces, which is an external factual organizing claim about the cited papers rather than a result derived from the survey's own inputs. The only self-citation is reference [73] (Borji, Ahmadabadi, Araabi, and Hamidi), a paper reviewed in Section 4.1; it is one item in a list of surveyed works and is not used as a load-bearing premise or to justify the paper's structure or conclusions. The Acknowledgment disclosure that ChatGPT 'create[d] summaries of the cited works' and the editing artifacts such as 'Section ??', 'Sections 2.1.4 2.1.2 and 2.1.2', and the Figure 6 caption reading 'Section 2.1.3' are accuracy and quality risks for the survey's summaries, not circularity: the survey does not define any output in terms of those summaries or fit a parameter and then relabel it as a prediction. There is no equation, theorem, or derivation chain to walk, and no claim is shown to reduce to its own inputs by construction. Verdict: no significant circularity, score 0.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

The survey introduces no free parameters, axioms, or invented entities of its own. Its only load-bearing premise is the accuracy of the summarized literature, which is an assumption about external facts rather than a mathematical or physical postulate.

assumptions (1)
  • domain assumption The ChatGPT-generated summaries accurately represent the content of the cited papers.
    The survey's utility depends entirely on the fidelity of the paper-by-paper summaries to the original works. The authors disclosed using ChatGPT for these summaries but provided no verification process.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Survey On Enhancing Reinforcement Learning in Complex Environments: Insights from Human and LLM Feedback." pith.science (2026). https://pith.science/paper/CRM3VLRO

@misc{pith2026241113410,
  author       = {Pith},
  title        = {Pith review of: A Survey On Enhancing Reinforcement Learning in Complex Environments: Insights from Human and LLM Feedback},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CRM3VLRO}},
  note         = {Machine review of arXiv:2411.13410}
}
read the original abstract

Reinforcement learning (RL) is one of the active fields in machine learning, demonstrating remarkable potential in tackling real-world challenges. Despite its promising prospects, this methodology has encountered with issues and challenges, hindering it from achieving the best performance. In particular, these approaches lack decent performance when navigating environments and solving tasks with large observation space, often resulting in sample-inefficiency and prolonged learning times. This issue, commonly referred to as the curse of dimensionality, complicates decision-making for RL agents, necessitating a careful balance between attention and decision-making. RL agents, when augmented with human or large language models' (LLMs) feedback, may exhibit resilience and adaptability, leading to enhanced performance and accelerated learning. Such feedback, conveyed through various modalities or granularities including natural language, serves as a guide for RL agents, aiding them in discerning relevant environmental cues and optimizing decision-making processes. In this survey paper, we mainly focus on problems of two-folds: firstly, we focus on humans or an LLMs assistance, investigating the ways in which these entities may collaborate with the RL agent in order to foster optimal behavior and expedite learning; secondly, we delve into the research papers dedicated to addressing the intricacies of environments characterized by large observation space.

Figures

Figures reproduced from arXiv: 2411.13410 by the authors.

Figure 1
Figure 1. Hierarchy for Section 2. 2.1 Natural Language Feedback/Instruction In this category, we discuss papers which utilize human natural language feedback. The feedback may be utilized in different forms or approaches by the RL agent. This category is divided into these sub-sections: • Section 2.1.1: papers which discuss natural language feedback/instruction in simulated RL environments. 2 [PITH_FULL_IMAGE:figures/full_f… view at source ↗
Figure 2
Figure 2. Abstract idea and architecture of papers in Sections 2.1.1 and 2.1.2. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Abstract idea and architecture of papers in Section 2.1.3. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Abstract idea and architecture of papers in Section 2.1.4. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Abstract idea and architecture of papers in Sections 2.2.3. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Abstract idea and architecture of papers in Section 2.1.3. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Hierarchy for Section 3. 3.1 Post-ChatGPT Papers Which Utilize Language for RL In this category, we discuss papers which utilize LLM’s natural language capabilities for RL agents. This category is divided into these sub-sections: • Section 3.1.1: papers which provide n…
Figure 8
Figure 8. Figure 8: Abstract idea and architecture of papers in Sections 3.1.2. [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Abstract idea and architecture of papers in Section 3.1.3. [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Abstract idea and architecture of papers in Sections 3.2.3. [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LLM-Guided Reinforcement Learning: Addressing Training Bottlenecks through Policy Modulation

    cs.AI 2025-05 conditional novelty 6.0 of 10

    An LLM identifies critical states, suggests corrective actions, and assigns shaped rewards to refine an existing RL policy, beating several baselines in Pong and MuJoCo.

  2. MARCO: Meta-Reflection with Cross-Referencing for Code Reasoning

    cs.CL 2025-05 conditional novelty 6.0 of 10

    MARCO combines cross-problem knowledge accumulation with cross-agent lesson sharing to improve LLM code reasoning at inference time.

  3. Evaluation of LLMs for mathematical problem solving

    cs.AI 2025-05 reject novelty 3.0 of 10

    A three-model, three-dataset LLM math evaluation using a multi-dimensional reasoning rubric, undermined by contradictory accuracy tables.

Reference graph

Works this paper leans on

83 extracted references · 45 canonical work pages · cited by 3 Pith papers

  1. [76]

    Deep reinforcement learning for the dynamic and uncertain vehicle routing problem

    Weixu Pan and Shi Qiang Liu. Deep reinforcement learning for the dynamic and uncertain vehicle routing problem. Applied Intelligence, 53(1):405–422, 2023

    work page 2023

  2. [77]

    Optimizing attention for sequence modeling via reinforcement learning

    Hao Fei, Yue Zhang, Yafeng Ren, and Donghong Ji. Optimizing attention for sequence modeling via reinforcement learning. IEEE Transactions on Neural Networks and Learning Systems, 33(8):3612–3621, 2021

    work page 2021

  3. [82]

    Rt-2: Vision-language-action models transfer web knowledge to robotic control

    Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pages 2165–2183. PMLR, 2023

    2023

  4. [9]

    A survey of reinforcement learning from human feedback

    Timo Kaufmann, Paul Weng, Viktor Bengs, and Eyke Hüllermeier. A survey of reinforcement learning from human feedback. arXiv preprint arXiv:2312.14925, 2023

    arXiv 2023

  5. [25]

    Yell at your robot: Improving on-the-fly from language corrections

    Lucy Xiaoyang Shi, Zheyuan Hu, Tony Z Zhao, Archit Sharma, Karl Pertsch, Jianlan Luo, Sergey Levine, and Chelsea Finn. Yell at your robot: Improving on-the-fly from language corrections. arXiv preprint arXiv:2403.12910, 2024

    arXiv 2024

  6. [47]

    Tag: Teacher-advice mechanism with gaussian process for reinforcement learning

    Ke Lin, Duantengchuan Li, Yanjie Li, Shiyu Chen, Qi Liu, Jianqi Gao, Yanrui Jin, and Liang Gong. Tag: Teacher-advice mechanism with gaussian process for reinforcement learning. IEEE Transactions on Neural Networks and Learning Systems, 2023

    work page 2023

  7. [46]

    HAIM-DRL: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Driving

    Zilin Huang, Zihao Sheng, Chengyuan Ma, and Sikai Chen. Human as ai mentor: Enhanced human-in-the-loop reinforcement learning for safe and efficient autonomous driving. arXiv preprint arXiv:2401.03160, 2024

    work page Pith review arXiv 2024

  8. [1]

    Reinforcement learning in healthcare: A survey

    Chao Yu, Jiming Liu, Shamim Nemati, and Guosheng Yin. Reinforcement learning in healthcare: A survey. ACM Computing Surveys (CSUR), 55(1):1–36, 2021

    2021

Show all 83 references
  1. [2]

    Survey on reinforcement learning for language processing

    Victor Uc-Cetina, Nicolas Navarro-Guerrero, Anabel Martin-Gonzalez, Cornelius Weber, and Stefan Wermter. Survey on reinforcement learning for language processing. Artificial Intelligence Review, 56(2):1543–1575, 2023

    2023

  2. [3]

    Recent advances in reinforcement learning in finance.Mathematical Finance, 33(3):437–503, 2023

    Ben Hambly, Renyuan Xu, and Huining Yang. Recent advances in reinforcement learning in finance.Mathematical Finance, 33(3):437–503, 2023

    2023

  3. [4]

    Open problems and fundamental limitations of reinforcement learning from human feedback

    Stephen Casper, Xander Davies, Claudia Shi, Thomas Krendl Gilbert, Jérémy Scheurer, Javier Rando, Rachel Freedman, Tomasz Korbak, David Lindner, Pedro Freire, et al. Open problems and fundamental limitations of reinforcement learning from human feedback. arXiv preprint arXiv:2...

    2023 arXiv

  4. [5]

    A review on interactive reinforcement learning from human social feedback

    Jinying Lin, Zhen Ma, Randy Gomez, Keisuke Nakamura, Bo He, and Guangliang Li. A review on interactive reinforcement learning from human social feedback. IEEE Access, 8:120757–120765, 2020

    2020

  5. [6]

    Assessing generalization in deep reinforcement learning

    Charles Packer, Katelyn Gao, Jernej Kos, Philipp Krähenbühl, Vladlen Koltun, and Dawn Song. Assessing generalization in deep reinforcement learning. arXiv preprint arXiv:1810.12282, 2018

    2018 arXiv

  6. [7]

    Latent exploration for reinforcement learning

    Alberto Silvio Chiappa, Alessandro Marin Vargas, Ann Huang, and Alexander Mathis. Latent exploration for reinforcement learning. Advances in Neural Information Processing Systems, 36, 2024

    2024

  7. [8]

    Reinforcement learning: A tutorial survey and recent advances

    Abhijit Gosavi. Reinforcement learning: A tutorial survey and recent advances. INFORMS Journal on Computing, 21(2):178–192, 2009

    2009

  8. [10]

    The rl/llm taxonomy tree: Reviewing synergies between reinforcement learning and large language models

    Moschoula Pternea, Prerna Singh, Abir Chakraborty, Yagna Oruganti, Mirco Milletari, Sayli Bapat, and Kebei Jiang. The rl/llm taxonomy tree: Reviewing synergies between reinforcement learning and large language models. arXiv preprint arXiv:2402.01874, 2024

    2024 arXiv

  9. [11]

    A conceptual framework for externally-influenced agents: An assisted reinforcement learning review

    Adam Bignold, Francisco Cruz, Matthew E Taylor, Tim Brys, Richard Dazeley, Peter Vamplew, and Cameron Foale. A conceptual framework for externally-influenced agents: An assisted reinforcement learning review. Journal of Ambient Intelligence and Humanized Computing, 14(4):3621–...

    2023

  10. [12]

    Language as an abstraction for hierarchical deep reinforcement learning

    Yiding Jiang, Shixiang Shane Gu, Kevin P Murphy, and Chelsea Finn. Language as an abstraction for hierarchical deep reinforcement learning. Advances in Neural Information Processing Systems, 32, 2019

    2019

  11. [13]

    Beating atari with natural language guided reinforcement learning

    Russell Kaplan, Christopher Sauer, and Alexander Sosa. Beating atari with natural language guided reinforcement learning. arXiv preprint arXiv:1704.05539, 2017

    2017 arXiv

  12. [14]

    The arcade learning environment: An evaluation platform for general agents

    Marc G Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling. The arcade learning environment: An evaluation platform for general agents. Journal of Artificial Intelligence Research, 47:253–279, 2013

    2013

  13. [15]

    Human instruction-following with deep reinforcement learning via transfer-learning from text

    Felix Hill, Sona Mokra, Nathaniel Wong, and Tim Harley. Human instruction-following with deep reinforcement learning via transfer-learning from text. arXiv preprint arXiv:2005.09382, 2020

    2005 arXiv

  14. [16]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  15. [17]

    Deep reinforcement learning for instruction following visual navigation in 3d maze-like environments

    Alessandro Devo, Gabriele Costante, and Paolo Valigi. Deep reinforcement learning for instruction following visual navigation in 3d maze-like environments. IEEE Robotics and Automation Letters, 5(2):1175–1182, 2020

    2020

  16. [18]

    Learning to follow directions in street view

    Karl Moritz Hermann, Mateusz Malinowski, Piotr Mirowski, Andras Banki-Horvath, Keith Anderson, and Raia Hadsell. Learning to follow directions in street view. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 11773–11781, 2020. 17

    2020

  17. [19]

    Language instructed reinforcement learning for human-ai coordination

    Hengyuan Hu and Dorsa Sadigh. Language instructed reinforcement learning for human-ai coordination. arXiv preprint arXiv:2304.07297, 2023

    2023 arXiv

  18. [20]

    Ask your humans: Using human instructions to improve generalization in reinforcement learning

    Valerie Chen, Abhinav Gupta, and Kenneth Marino. Ask your humans: Using human instructions to improve generalization in reinforcement learning. arXiv preprint arXiv:2011.00517, 2020

    2011 arXiv

  19. [21]

    Meta-reinforcement learning via language instructions

    Zhenshan Bing, Alexander Koch, Xiangtong Yao, Kai Huang, and Alois Knoll. Meta-reinforcement learning via language instructions. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 5985–5991. IEEE, 2023

    2023

  20. [22]

    Guiding multi-step rearrangement tasks with natural language instructions

    Elias Stengel-Eskin, Andrew Hundt, Zhuohong He, Aditya Murali, Nakul Gopalan, Matthew Gombolay, and Gregory Hager. Guiding multi-step rearrangement tasks with natural language instructions. In Conference on Robot Learning, pages 1486–1501. PMLR, 2022

    2022

  21. [23]

    Interactive language: Talking to robots in real time

    Corey Lynch, Ayzaan Wahid, Jonathan Tompson, Tianli Ding, James Betker, Robert Baruch, Travis Armstrong, and Pete Florence. Interactive language: Talking to robots in real time. IEEE Robotics and Automation Letters, 2023

    2023

  22. [24]

    Correcting robot plans with natural language feedback

    Pratyusha Sharma, Balakumar Sundaralingam, Valts Blukis, Chris Paxton, Tucker Hermans, Antonio Tor- ralba, Jacob Andreas, and Dieter Fox. Correcting robot plans with natural language feedback. arXiv preprint arXiv:2204.05186, 2022

    2022 arXiv

  23. [26]

    Continual learning for instruction following from realtime feedback

    Alane Suhr and Yoav Artzi. Continual learning for instruction following from realtime feedback. Advances in Neural Information Processing Systems, 36, 2024

    2024

  24. [27]

    Incorporating voice instructions in model-based reinforcement learning for self-driving cars

    Mingze Wang, Ziyang Zhang, and Grace Hui Yang. Incorporating voice instructions in model-based reinforcement learning for self-driving cars. arXiv preprint arXiv:2206.10249, 2022

    2022 arXiv

  25. [28]

    Correct me if i’m wrong: Using non-experts to repair reinforcement learning policies

    Sanne Van Waveren, Christian Pek, Jana Tumova, and Iolanda Leite. Correct me if i’m wrong: Using non-experts to repair reinforcement learning policies. In 2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pages 493–501. IEEE, 2022

    2022

  26. [29]

    Natural language specification of reinforcement learning policies through differentiable decision trees

    Pradyumna Tambwekar, Andrew Silva, Nakul Gopalan, and Matthew Gombolay. Natural language specification of reinforcement learning policies through differentiable decision trees. IEEE Robotics and Automation Letters, 2023

    2023

  27. [30]

    Ella: Exploration through learned language abstraction

    Suvir Mirchandani, Siddharth Karamcheti, and Dorsa Sadigh. Ella: Exploration through learned language abstraction. Advances in Neural Information Processing Systems, 34:29529–29540, 2021

    2021

  28. [31]

    How to talk so ai will learn: Instructions, descriptions, and autonomy

    Theodore Sumers, Robert Hawkins, Mark K Ho, Tom Griffiths, and Dylan Hadfield-Menell. How to talk so ai will learn: Instructions, descriptions, and autonomy. Advances in Neural Information Processing Systems , 35:34762–34775, 2022

    2022

  29. [32]

    Human-in-the-loop reinforcement learning in continuous-action space

    Biao Luo, Zhengke Wu, Fei Zhou, and Bing-Chuan Wang. Human-in-the-loop reinforcement learning in continuous-action space. IEEE Transactions on Neural Networks and Learning Systems, 2023

    2023

  30. [33]

    Toward human-in-the-loop ai: Enhancing deep reinforcement learning via real-time human guidance for autonomous driving

    Jingda Wu, Zhiyu Huang, Zhongxu Hu, and Chen Lv. Toward human-in-the-loop ai: Enhancing deep reinforcement learning via real-time human guidance for autonomous driving. Engineering, 21:75–91, 2023

    2023

  31. [34]

    Deep reinforcement learning with interactive feedback in a human–robot environment

    Ithan Moreira, Javier Rivas, Francisco Cruz, Richard Dazeley, Angel Ayala, and Bruno Fernandes. Deep reinforcement learning with interactive feedback in a human–robot environment. Applied Sciences, 10(16):5574, 2020

    2020

  32. [35]

    Deploying offline reinforcement learning with human feedback

    Ziniu Li, Ke Xu, Liu Liu, Lanqing Li, Deheng Ye, and Peilin Zhao. Deploying offline reinforcement learning with human feedback. arXiv preprint arXiv:2303.07046, 2023

    2023 arXiv

  33. [36]

    Explanation augmented feedback in human-in-the-loop reinforcement learning

    Lin Guan, Mudit Verma, and Subbarao Kambhampati. Explanation augmented feedback in human-in-the-loop reinforcement learning. arXiv preprint arXiv:2006.14804, 2020

    2006 arXiv

  34. [37]

    The expertise problem: Learning from specialized feedback

    Oliver Daniels-Koch and Rachel Freedman. The expertise problem: Learning from specialized feedback. arXiv preprint arXiv:2211.06519, 2022

    2022 arXiv

  35. [38]

    Interactive reinforcement learning with bayesian fusion of multimodal advice

    Susanne Trick, Franziska Herbert, Constantin A Rothkopf, and Dorothea Koert. Interactive reinforcement learning with bayesian fusion of multimodal advice. IEEE Robotics and Automation Letters, 7(3):7558–7565, 2022

    2022

  36. [39]

    Policy shaping: Integrating human feedback with reinforcement learning

    Shane Griffith, Kaushik Subramanian, Jonathan Scholz, Charles L Isbell, and Andrea L Thomaz. Policy shaping: Integrating human feedback with reinforcement learning. Advances in neural information processing systems, 26, 2013. 18

    2013

  37. [40]

    Learning from unreliable human action advice in interactive reinforcement learning

    Lisa Scherf, Cigdem Turan, and Dorothea Koert. Learning from unreliable human action advice in interactive reinforcement learning. In 2022 IEEE-RAS 21st International Conference on Humanoid Robots (Humanoids), pages 895–902. IEEE, 2022

    2022

  38. [41]

    Improving deep reinforcement learning in minecraft with action advice

    Spencer Frazier and Mark Riedl. Improving deep reinforcement learning in minecraft with action advice. In Proceedings of the AAAI conference on artificial intelligence and interactive digital entertainment, volume 15, pages 146–152, 2019

    2019

  39. [42]

    Advice-guided reinforce- ment learning in a non-markovian environment

    Daniel Neider, Jean-Raphael Gaglione, Ivan Gavran, Ufuk Topcu, Bo Wu, and Zhe Xu. Advice-guided reinforce- ment learning in a non-markovian environment. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 9073–9080, 2021

    2021

  40. [43]

    Few-shot preference learning for human-in-the-loop rl

    Donald Joseph Hejna III and Dorsa Sadigh. Few-shot preference learning for human-in-the-loop rl. In Conference on Robot Learning, pages 2014–2025. PMLR, 2023

    2014

  41. [44]

    Effect of human guidance and state space size on interactive reinforcement learning

    Halit Bener Suay and Sonia Chernova. Effect of human guidance and state space size on interactive reinforcement learning. In 2011 Ro-Man, pages 1–6. IEEE, 2011

    2011

  42. [45]

    Interactive reinforcement learning from demon- stration and human evaluative feedback

    Guangliang Li, Bo He, Randy Gomez, and Keisuke Nakamura. Interactive reinforcement learning from demon- stration and human evaluative feedback. In 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pages 1156–1162. IEEE, 2018

    2018

  43. [48]

    Guiding pretraining in reinforcement learning with large language models

    Yuqing Du, Olivia Watkins, Zihan Wang, Cédric Colas, Trevor Darrell, Pieter Abbeel, Abhishek Gupta, and Jacob Andreas. Guiding pretraining in reinforcement learning with large language models. arXiv preprint arXiv:2302.06692, 2023

    2023 arXiv

  44. [49]

    Reinforcement learning from llm feedback to counteract goal misgeneralization

    Houda Nait El Barj and Théophile Sautory. Reinforcement learning from llm feedback to counteract goal misgeneralization. arXiv preprint arXiv:2401.07181, 2024

    2024 arXiv

  45. [50]

    Mutual enhancement of large language and reinforcement learning models through bi-directional feedback mechanisms: A case study

    Shangding Gu. Mutual enhancement of large language and reinforcement learning models through bi-directional feedback mechanisms: A case study. arXiv preprint arXiv:2401.06603, 2024

    2024 arXiv

  46. [51]

    Lagr-seq: Language-guided reinforcement learning with sample-efficient querying

    Thommen George Karimpanal, Laknath Buddhika Semage, Santu Rana, Hung Le, Truyen Tran, Sunil Gupta, and Svetha Venkatesh. Lagr-seq: Language-guided reinforcement learning with sample-efficient querying. arXiv preprint arXiv:2308.13542, 2023

    2023 arXiv

  47. [52]

    Reward design with language models

    Minae Kwon, Sang Michael Xie, Kalesha Bullard, and Dorsa Sadigh. Reward design with language models. arXiv preprint arXiv:2303.00001, 2023

    2023 arXiv

  48. [53]

    Enabling intelligent interactions between an agent and an llm: A reinforcement learning approach

    Bin Hu, Chenyang Zhao, Pu Zhang, Zihao Zhou, Yuanhang Yang, Zenglin Xu, and Bin Liu. Enabling intelligent interactions between an agent and an llm: A reinforcement learning approach. arXiv preprint arXiv:2306.03604, 2023

    2023 arXiv

  49. [54]

    Instruction-following agents with multimodal transformer

    Hao Liu, Lisa Lee, Kimin Lee, and Pieter Abbeel. Instruction-following agents with multimodal transformer. arXiv preprint arXiv:2210.13431, 2022

    2022 arXiv

  50. [55]

    Eureka: Human-level reward design via coding large language models

    Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Eureka: Human-level reward design via coding large language models. arXiv preprint arXiv:2310.12931, 2023

    arXiv

  51. [56]

    Inherently explainable reinforcement learning in natural language

    Xiangyu Peng, Mark Riedl, and Prithviraj Ammanabrolu. Inherently explainable reinforcement learning in natural language. Advances in Neural Information Processing Systems, 35:16178–16190, 2022

    2022

  52. [57]

    Hi-core: Hierarchical knowledge transfer for continual reinforcement learning

    Chaofan Pan, Xin Yang, Hao Wang, Wei Wei, and Tianrui Li. Hi-core: Hierarchical knowledge transfer for continual reinforcement learning. arXiv preprint arXiv:2401.15098, 2024

    2024 arXiv

  53. [58]

    Motif: Intrinsic motivation from artificial intelligence feedback

    Martin Klissarov, Pierluca D’Oro, Shagun Sodhani, Roberta Raileanu, Pierre-Luc Bacon, Pascal Vincent, Amy Zhang, and Mikael Henaff. Motif: Intrinsic motivation from artificial intelligence feedback. arXiv preprint arXiv:2310.00166, 2023

    arXiv

  54. [59]

    Building open-ended embodied agent via language-policy bidirectional adaptation

    Shaopeng Zhai, Jie Wang, Tianyi Zhang, Fuxian Huang, Qi Zhang, Ming Zhou, Jing Hou, and Yu Liu. Building open-ended embodied agent via language-policy bidirectional adaptation. arXiv preprint arXiv:2401.00006, 2023

    2023 arXiv

  55. [60]

    Llm augmented hierarchical agents

    Bharat Prakash, Tim Oates, and Tinoosh Mohsenin. Llm augmented hierarchical agents. arXiv preprint arXiv:2311.05596, 2023. 19

    2023 arXiv

  56. [61]

    Exploiting contextual structure to generate useful auxiliary tasks

    Benedict Quartey, Ankit Shah, and George Konidaris. Exploiting contextual structure to generate useful auxiliary tasks. arXiv preprint arXiv:2303.05038, 2023

    2023 arXiv

  57. [62]

    Worldcoder, a model-based llm agent: Building world models by writing code and interacting with the environment

    Hao Tang, Darren Key, and Kevin Ellis. Worldcoder, a model-based llm agent: Building world models by writing code and interacting with the environment. arXiv preprint arXiv:2402.12275, 2024

    2024 arXiv

  58. [63]

    Natural language reinforcement learning

    Xidong Feng, Ziyu Wan, Mengyue Yang, Ziyan Wang, Girish A Koushiks, Yali Du, Ying Wen, and Jun Wang. Natural language reinforcement learning. arXiv preprint arXiv:2402.07157, 2024

    2024 arXiv

  59. [64]

    Llm-grounder: Open-vocabulary 3d visual grounding with large language model as an agent

    Jianing Yang, Xuweiyi Chen, Shengyi Qian, Nikhil Madaan, Madhavan Iyengar, David F Fouhey, and Joyce Chai. Llm-grounder: Open-vocabulary 3d visual grounding with large language model as an agent. arXiv preprint arXiv:2309.12311, 2023

    2023 arXiv

  60. [65]

    Interactive planning using large language models for partially observable robotics tasks

    Lingfeng Sun, Devesh K Jha, Chiori Hori, Siddarth Jain, Radu Corcodel, Xinghao Zhu, Masayoshi Tomizuka, and Diego Romeres. Interactive planning using large language models for partially observable robotics tasks. In NeurIPS 2023 Workshop on Instruction Tuning and Instruction F...

    2023

  61. [66]

    Grounding language to entities and dynamics for generalization in reinforcement learning

    Austin W Hanjie, Victor Y Zhong, and Karthik Narasimhan. Grounding language to entities and dynamics for generalization in reinforcement learning. In International Conference on Machine Learning, pages 4051–4062. PMLR, 2021

    2021

  62. [67]

    An end-to-end approach to natural language object retrieval via context- aware deep reinforcement learning

    Fan Wu, Zhongwen Xu, and Yi Yang. An end-to-end approach to natural language object retrieval via context- aware deep reinforcement learning. arXiv preprint arXiv:1703.07579, 2017

    2017 arXiv

  63. [68]

    Read, watch, and move: Reinforce- ment learning for temporally grounding natural language descriptions in videos

    Dongliang He, Xiang Zhao, Jizhou Huang, Fu Li, Xiao Liu, and Shilei Wen. Read, watch, and move: Reinforce- ment learning for temporally grounding natural language descriptions in videos. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 8393–8400, 2019

    2019

  64. [69]

    Follownet: Robot navigation by following natural language directions with deep reinforcement learning

    Pararth Shah, Marek Fiser, Aleksandra Faust, J Chase Kew, and Dilek Hakkani-Tur. Follownet: Robot navigation by following natural language directions with deep reinforcement learning. arXiv preprint arXiv:1805.06150, 2018

    2018 arXiv

  65. [70]

    Toward collaborative reinforcement learning agents that commu- nicate through text-based natural language

    Kevin M Eloff and Herman A Engelbrecht. Toward collaborative reinforcement learning agents that commu- nicate through text-based natural language. In 2021 Southern African Universities Power Engineering Confer- ence/Robotics and Mechatronics/Pattern Recognition Association of ...

    2021

  66. [71]

    Language understanding for text-based games using deep reinforcement learning

    Karthik Narasimhan, Tejas Kulkarni, and Regina Barzilay. Language understanding for text-based games using deep reinforcement learning. arXiv preprint arXiv:1506.08941, 2015

    2015 arXiv

  67. [72]

    Towards end-to-end learning for dialog state tracking and management using deep reinforcement learning

    Tiancheng Zhao and Maxine Eskenazi. Towards end-to-end learning for dialog state tracking and management using deep reinforcement learning. arXiv preprint arXiv:1606.02560, 2016

    2016 arXiv

  68. [73]

    Online learning of task-driven object-based visual attention control

    Ali Borji, Majid Nili Ahmadabadi, Babak Nadjar Araabi, and Mandana Hamidi. Online learning of task-driven object-based visual attention control. Image and Vision Computing, 28(7):1130–1145, 2010

    2010

  69. [74]

    Visual navigation with spatial attention

    Bar Mayo, Tamir Hazan, and Ayellet Tal. Visual navigation with spatial attention. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16898–16907, 2021

    2021

  70. [75]

    An actor-critic-attention mechanism for deep reinforcement learning in multi-view environments

    Elaheh Barati and Xuewen Chen. An actor-critic-attention mechanism for deep reinforcement learning in multi-view environments. arXiv preprint arXiv:1907.09466, 2019

    1907 arXiv

  71. [78]

    Attention-based curiosity-driven exploration in deep reinforcement learning

    Patrik Reizinger and Márton Szemenyei. Attention-based curiosity-driven exploration in deep reinforcement learning. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3542–3546. IEEE, 2020

    2020

  72. [79]

    Towards interpretable reinforcement learning using attention augmented agents

    Alexander Mott, Daniel Zoran, Mike Chrzanowski, Daan Wierstra, and Danilo Jimenez Rezende. Towards interpretable reinforcement learning using attention augmented agents. Advances in neural information processing systems, 32, 2019

    2019

  73. [80]

    Dynamic interaction between reinforcement learning and attention in multidimensional environments

    Yuan Chang Leong, Angela Radulescu, Reka Daniel, Vivian DeWoskin, and Yael Niv. Dynamic interaction between reinforcement learning and attention in multidimensional environments. Neuron, 93(2):451–463, 2017

    2017

  74. [81]

    Machine versus human attention in deep reinforcement learning tasks

    Ruohan Zhang, Sihang Guo, Bo Liu, Yifeng Zhu, Mary Hayhoe, Dana Ballard, and Peter Stone. Machine versus human attention in deep reinforcement learning tasks. arXiv preprint arXiv:2010.15942, 2020. 20

    2010 arXiv

  75. [83]

    Mlanet: Multi-level attention network with sub-instruction for continuous vision-and-language navigation

    Zongtao He, Liuyi Wang, Shu Li, Qingqing Yan, Chengju Liu, and Qijun Chen. Mlanet: Multi-level attention network with sub-instruction for continuous vision-and-language navigation. arXiv preprint arXiv:2303.01396, 2023. 21

    2023 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.