Teaching Embodied Reinforcement Learning Agents: Informativeness and Diversity of Language Use

Jiajun Xi; Jianing Yang; Joyce Chai; Yinong He; Yinpei Dai

RECORD

Teaching Embodied Reinforcement Learning Agents: Informativeness and Diversity of Language Use

Jiajun Xi, Yinong He, Jianing Yang, Yinpei Dai, Joyce Chai

Computation and Language · Submitted 31 Oct 2024

abstract compiled · 2026-09-23 · no language-model review

The authors' main claim Our empirical results based on four RL benchmarks demonstrate that agents trained with diverse and informative language feedback can achieve enhanced generalization and fast adaptation to new tasks.

how it is stated · asserted 1.00 read the record →

Comment (0) Read on arXiv ↗

Cite ▾

arXiv ID2410.24218 (version 1)

Pith IDXCHOEBJT, Pith's permanent record of this paper and its review

CitationJiajun Xi, Jianing Yang, Joyce Chai, Yinong He, Yinpei Dai. "Teaching Embodied Reinforcement Learning Agents: Informativeness and Diversity of Language Use." arXiv:2410.24218 (2024). https://pith.science/paper/XCHOEBJT

Claim paper

The compiled record

The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.

Open the compiled record

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 6,136 input tokens.

What the abstract itself states, as probabilities: claims first 0.05 · states limitation 0.08 · human participants 0.09 · controlled comparison 0.30 · randomized assignment 0.02 · states quantitative result 0.14 · compares with prior results 0.32 · mentions code or data availability 0.87.

The abstract, sentence by sentence

arXiv listing abstract v1. Results and the main claim are underlined; show every role.

In real-world scenarios, it is desirable for embodied agents to have the ability to leverage human language to gain explicit or implicit knowledge for learning tasks. Despite recent progress, most previous approaches adopt simple low-level instructions as language inputs, which may not reflect natural human communication. It's not clear how to incorporate rich language use to facilitate task learning. To address this question, this paper studies different types of language inputs in facilitating reinforcement learning (RL) embodied agents. More specifically, we examine how different levels of language informativeness (i.e., feedback on past behaviors and future guidance) and diversity (i.e., variation of language expressions) impact agent learning and inference. Our empirical results based on four RL benchmarks demonstrate that agents trained with diverse and informative language feedback can achieve enhanced generalization and fast adaptation to new tasks. These findings highlight the pivotal role of language use in teaching embodied agents new tasks in an open world. Project website: https://github.com/sled-group/Teachable_RL

background contribution method result interpretation main claim

What the classifier reports for each sentence. The number is the probability it gave the label.
#	sentence	role	the authors' own work	states the main conclusion	strength
s0	In real-world scenarios, it is desirable for embodied agents to have the ability to leverage human language to gain explicit or implicit knowledge for learning tasks.	background 1.00	0.04	0.02	asserted 0.81
s1	Despite recent progress, most previous approaches adopt simple low-level instructions as language inputs, which may not reflect natural human communication.	background 0.62	0.04	0.02	hedged possibility 0.53
s2	It's not clear how to incorporate rich language use to facilitate task learning.	gap 1.00	0.04	0.02	asserted 0.52
s3	To address this question, this paper studies different types of language inputs in facilitating reinforcement learning (RL) embodied agents.	contribution 1.00	0.68	0.02	asserted 0.79
s4	More specifically, we examine how different levels of language informativeness (i.e., feedback on past behaviors and future guidance) and diversity (i.e., variation of language expressions) impact agent learning and inference.	contribution 0.79	0.56	0.04	unclear 0.64
s5	Our empirical results based on four RL benchmarks demonstrate that agents trained with diverse and informative language feedback can achieve enhanced generalization and fast adaptation to new tasks.	result 1.00	0.96	0.88	asserted 1.00
s6	These findings highlight the pivotal role of language use in teaching embodied agents new tasks in an open world.	interpretation 1.00	0.58	0.49	asserted 0.99
s7	Project website: https://github.com/sled-group/Teachable_RL	future work 0.61	0.09	0.01	unclear 0.99

What kind of paper the classifier reports this is: empirical study 0.99 · evidence from observation or experiment 0.84.

What was examined, and what was not

sources read

arXiv listing abstract v1

not examined

full text
figures and tables
references and citing sentences
relations to other papers

levels run

P abstract

not run

S sections
T body sentences
C citations
R cross-paper relations

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. Every number in the measurements table is the authors', checked by code. Record as JSON

The paper

Original abstract

In real-world scenarios, it is desirable for embodied agents to have the ability to leverage human language to gain explicit or implicit knowledge for learning tasks. Despite recent progress, most previous approaches adopt simple low-level instructions as language inputs, which may not reflect natural human communication. It's not clear how to incorporate rich language use to facilitate task learning. To address this question, this paper studies different types of language inputs in facilitating reinforcement learning (RL) embodied agents. More specifically, we examine how different levels of language informativeness (i.e., feedback on past behaviors and future guidance) and diversity (i.e., variation of language expressions) impact agent learning and inference. Our empirical results based on four RL benchmarks demonstrate that agents trained with diverse and informative language feedback can achieve enhanced generalization and fast adaptation to new tasks. These findings highlight the pivotal role of language use in teaching embodied agents new tasks in an open world. Project website: https://github.com/sled-group/Teachable_RL

Human review

How Pith checks papers → · Full audit of this paper →

How the literature uses it

1 arXiv paper cite this in 1 sentence

Each citing sentence is taken from the citing paper's LaTeX source and typed by Pith's classifier. A label appears at probability 0.8 or more (0.95 for support), a contest at 0.7 or more; the quoted words are the citing authors'. How each label was checked · JSON

Papers citing it

Cites it as background 1 paper

Vision-language-action (VLA) models have become a practical paradigm for general-purpose robotic manipulation, translating open-ended semantic instructions into visuomotor policies [ref] .

VP-VLA: Visual Prompting as an Interface for Vision-Language-Action Models · Related Work · 1.00

What it cites · 18 sentences without a confident label

Uses the method or tool of 4 papers

We adopt the hindsight and foresight language templates from LLF-ALFWorld introduced in [ref] , which adds an extra language wrapper to the original ALFWorld environment.

LLF-Bench: Benchmark for Interactive Learning from Language Feedback · Environments · 0.86
Efficiency is measured by a path-weighted reward, as introduced in ALFRED [ref] .

ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks · Ablation Study · 0.88
We use language embeddings from a frozen Sentence-BERT model [ref] in all environments.

Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks · Model · 1.00

Cites as background 9 papers

Recent work also explores generalization across tasks [ref] , the use of exploratory data [ref] , and integrating large language models (LLMs) [ref] .

A Generalist Dynamics Model for Control · Related Work · 0.99
Recent studies have explored various methods for incorporating language feedback in RL, such as the LTC paradigm [ref] , lifelong robot learning with human-assisted language planners [ref] , and frameworks for rich information requests [ref] .

Adapting LLM Agents with Universal Feedback in Communication · Related Work · 0.98
Initial works utilized language for feedback and task instructions [ref] .

Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback · Related Work · 0.98

Discussion

Know something about this paper? Leave a comment, or highlight any passage above to comment on it.