Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

Learning Instruction-Following Policies through Open-Ended Instruction Relabeling with Large Language Models

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An LLM that reads failed trajectories and relabels them with accomplished subtasks turns sparse failures into training signal, and the resulting policy beats baselines on Craftax and generalizes to unseen instruction variants.

desk verdict A useful new combination that repurposes failed trajectories with LLM-generated instructions, but the unvalidated reward proxy and missing HIR baseline keep it from being fully convincing. read the letter →

arxiv 2506.20061 v1 pith:AUFJ5RN5 submitted 2025-06-24 cs.LG cs.CL

Zhicheng Zhang , Ziyan Wang , Yali Du , Fei Fang This is my paper
classification cs.LGcs.CL
keywords instruction-followingreinforcementlearninghindsightinstructionrelabelinglargelanguagemodelssparserewardopen-endedenvironmentsCraftaxsemanticprioritizedreplaybuffer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a reinforcement-learning agent's failed rollouts are not wasted data: a large language model can read a text rendering of any collected trajectory, including ones that missed their original goal, and generate open-ended instructions for subtasks the agent did complete. These relabeled instructions supply reward signals through embedding-based cosine similarity (Eq. 8), converting sparse, unsuccessful episodes into dense training batches for a single instruction-following policy. On the Craftax benchmark, with all built-in rewards removed, the proposed method, open-ended instruction relabeling (OIR), outperforms parallelised Q-network (PQN) variants and the LLM-exploration baseline ELLM in sample efficiency, number of instructions completed, success rate, and generalization to simple and complex linguistic variants of the training instructions. If the claim holds, LLM-guided hindsight relabeling removes the human-annotation bottleneck in instruction-following RL and turns exploration waste into curriculum.

What carries the argument

The central mechanism is LLM-guided hindsight instruction relabeling: each rollout is converted by a function $h$ into a temporally structured text prompt, from which a pretrained LLM samples $K$ candidate instructions for subtasks the agent fully accomplished (Eq. 4). Each candidate is scored by the embedding-based reward $r_t = \mathrm{cosim}\big(f_{state}(o_t,a_t,o_{t+1}),\, f_{instr}(i)\big)$ (Eq. 8), where $f_{state}$ embeds the transition and $f_{instr}$ is a pretrained sentence-embedding encoder such as SBERT, and an episode is declared successful and terminated the first time the reward exceeds threshold $\delta$. The acceptance criterion is formalized as the oracle value comparison of Eq. (7), $V^{\pi_{oracle}}_{i'}(s_{t+1}) > V^{\pi_{oracle}}_{i_{orig}}(s_{t+1})$, and the prioritized instruction replay buffer (Section 4.3), inspired by Prioritized Level Replay, sorts instructions into failing, learning-boundary, and mastered categories, evicting by priority so that the uniform sampling of instructions at reset concentrates training on tasks near the agent's competence frontier.

What would settle it

On a held-out set of rollouts, compare the success labels produced by the training reward (cosine similarity above $\delta$, Eq. 8) against the environment's own ground-truth achievement flags for the relabeled instructions: if a large fraction of cosine-declared successes are not real completions, the reward is rewarding spurious behavior and OIR's measured gains are not evidence of genuine instruction following. A complementary check is to relabel trajectories with deliberately irrelevant instructions while keeping the same cosine-similarity machinery, and to see whether the gap over baselines persists; if it does, the reward proxy, not the LLM's semantic relabeling, is carrying the result.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that hindsight can operate at the level of natural-language instructions rather than states or goals. Given a trajectory rendered as a temporally ordered text prompt, the LLM returns candidate instructions describing what the agent accomplished, including in episodes that failed their originally assigned instruction, and each candidate is scored by the cosine similarity between the transition embedding and the instruction embedding (Eq. 8), with a threshold $\delta$ marking success and episode termination. The acceptance ideal is that a relabeled instruction yields higher oracle value than the original instruction (Eq. 7), and a prioritized instruction buffer, modeled on Prioritized Level Replay, keeps the policy training on tasks at its learning frontier. Empirically, OIR completes roughly ten of the Craftax instructions versus fewer than four for other baselines, reaches an average success rate of 33.10%, masters hard multi-step tasks such as 'Defeat Zombie' (87.78%), 'Place Table' (95.83%), and 'Make Wood Pickaxe' (76.59%), and retains its edge on unseen simple and complex instruction variants; the paper also records a trade-off in which infrequently sampled instructions such as 'Wake Up' lose proficiency to the ground-truth-reward baseline.

Load-bearing premise

Everything rests on the reward proxy of Eq. 8: an episode is treated as successfully completing an LLM-generated instruction when the cosine similarity between the trajectory's transition embeddings and the instruction's embedding exceeds threshold $\delta$, and the paper never verifies that this proxy corresponds to the agent actually doing what the instruction describes.

Editorial extensions

If this is right

  • Failed and sparse trajectories become reusable training data: without any human annotation, a single policy learns to follow many open-ended instructions.
  • The policy generalizes to unseen instruction variants, both simple rewrites and semantically enriched compositions, because the relabeled instruction space is wider than the environment's predefined achievements.
  • Hard multi-step tasks that reward-free baselines never master ('Defeat Zombie', 'Make Wood Pickaxe', 'Place Table') become learnable, since hindsight relabeling supplies intermediate semantic goals.
  • Sample efficiency improves on every reported metric: higher aggregate score, more completed instructions, and higher average success rate at the same environment-step budget.
  • Because OIR needs only a textual rendering of observations rather than environment-specific reward logic, the same recipe applies in principle to any environment that can describe its states in text.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not measure how often a cosine-declared success (reward above $\delta$ in Eq. 8) matches a ground-truth accomplishment, so a fair share of the relabeled signal may reward trajectories that merely resemble the instruction in embedding space; one testable consequence is that OIR's advantage should shrink in environments where textual observations are less informative about actual stat
  • There is a closed loop worth probing: the instruction encoder that scores the reward is also the encoder conditioning the policy, and the LLM generates instructions from the same textual observations that get embedded; this coupling could make measured instruction diversity (the t-SNE coverage) grow faster than genuine behavioral diversity.
  • Because the discoverable instruction space is bounded by what the LLM can recognize from text, OIR's ceiling is set by the model's world knowledge rather than by the environment; a human-in-the-loop verification of relabeled instructions, which the paper itself lists as future work, would directly test how much of the relabeled signal is true task completion.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes Open-Ended Instruction Relabeling (OIR), an off-policy instruction-conditioned RL algorithm that uses an LLM to generate open-ended natural-language instructions from collected trajectories, including unsuccessful ones. These instructions are embedded and used to compute dense cosine-similarity rewards (Eq. 8); success is declared when the similarity exceeds a threshold δ, and a prioritized replay buffer manages the growing instruction set. Experiments on Craftax-Classic compare OIR against PQN with a cosine-similarity reward, PQN with the ground-truth reward, and ELLM, and the authors report improvements in aggregate score, completed instructions, average success rate, generalization to simple and complex instruction variants, and semantic coverage, with an ablation over δ.

Significance. If the results hold, the paper makes a useful contribution by reducing reliance on human-annotated instruction datasets and by turning failed trajectories into instruction-following data. The idea of using LLMs to reinterpret trajectories retrospectively is timely, the algorithm is clearly presented, and evaluation on a public benchmark with external ground-truth success signals is a strength. However, the evidence as presented is not yet convincing: the most directly relevant baseline, HIR, is omitted; the cosine-similarity training reward is never validated against true task completion; and several numerical claims in the text contradict the figures. The contribution is potentially significant, but the current manuscript requires additional verification before the claims can be accepted.

major comments (5)
  1. [§5, Baselines] The paper does not compare against Hindsight Instruction Relabeling (HIR) [Zhang et al., 2023], which is the most relevant prior method and is cited in Section 2. Because OIR is explicitly a hindsight instruction relabeling method, the claimed advantage over state-of-the-art methods cannot be assessed without an HIR baseline, for example HIR using template-based or non-LLM relabeling. Please add such a comparison or temper the claim.
  2. [§4.2, Eq. (8)] The training reward is defined entirely by the cosine similarity between transition and instruction embeddings, and Section 5 states that all environment rewards and achievement signals are removed during training. The paper never measures the correlation between this proxy and ground-truth completion on the same trajectories. If high cosine similarity can occur without task completion (e.g., proximity to trees without an inventory change), the dense reward may reward spurious behavior, and the reported gains over PQN and ELLM could be a reward-density artifact. Please validate the proxy, for example by reporting precision and recall of the threshold rule against ground-truth success on relabeled trajectories, or include an oracle-relabeling ablation that uses true success signals.
  3. [§5.1, Figure 2(d)] The numerical claims in the text are not consistent with the figure. The text reports 'Defeat Zombie' (87.78%), 'Place Table' (95.83%), and 'Make Wood Pickaxe' (76.59%), but Figure 2(d) displays values 3.40, 29.93, and 1.82 for these tasks. Likewise, the text reports approximately 10 completed instructions and an average success rate of 33.10%, while Figure 2(b) and (c) show final values of 7.28 and 18.20. These discrepancies must be reconciled before the empirical claims can be evaluated.
  4. [§4.1, Algorithm 1] Section 4.1 states that rule-based instructions are incorporated as an additional source of supervision and that candidate instructions are subject to further filtering or selection, but no rule-based instructions, filtering criteria, or selection step appear in Algorithm 1 or in the implementation details. Moreover, the prompt template in Section 4.1 ('What instruction is this trajectory following?') differs from the prompt in Appendix B.5, which asks for 'Completed Instructions' and prohibits movement instructions. The method as described is therefore not fully specified; please align the description and the implementation.
  5. [Table 1, Appendix B.2] Several hyperparameters required by Algorithm 1 are missing from Table 1, including K (number of LLM candidates), the buffer thresholds τ_low and τ_high, and the exact δ values used for the PQN baselines (Table 1 lists '0.5&0.9' for PQN, while Figures 2 and 3 report separate PQN (δ=0.5) and PQN (δ=0.9) curves). In addition, ELLM uses 1024 parallel environments while OIR and PQN use 64; because total timesteps are matched, this may be acceptable, but the potential effect of parallelism on exploration and LLM query batching should be justified. Please provide complete hyperparameters and discuss the fairness of the ELLM setup.
minor comments (6)
  1. [§4.2, Eq. (8)] The equation writes fstate(ot) but the expression uses fstate(ot, at, ot+1); please fix the notation.
  2. [§4.2] The sentence 'an episode is deemed successful the first time the reward exceeds a predefined threshold δ' should specify that the reward is computed on transitions and that termination is applied only once per episode.
  3. [Algorithm 1, line 9] The reward rcand is computed from the full trajectory τe and instruction i′, but Eq. (8) requires a single transition; please clarify how the trajectory-level reward is aggregated over timesteps.
  4. [Abstract and Introduction] The claim that the method 'completely eliminates' dependence on manual annotations is too strong, since the prompt template, instruction-generation principles, and hyperparameters are manually designed.
  5. [§5.3, Figure 5] The diversity claim is based on a visual t-SNE plot; please include a quantitative coverage metric, such as the number of distinct semantic clusters or an embedding-volume estimate.
  6. [Throughout] Minor typos include 'acessible' in Section 4, 'extual' in the Figure 1 caption, and '1S-2046640' in Section 7; please proofread the manuscript.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: OIR's surrogate reward is an explicit modeling choice, the headline claims are evaluated against external ground-truth signals, and the only self-citation is contextual and non-load-bearing.

full rationale

I find no circular step in the claimed derivation chain. OIR defines a surrogate reward (Eq. 8) as cosine similarity between a transition embedding and an instruction embedding, and the instructions used for relabeling are generated by an LLM from the same trajectories; this is an explicit modeling choice, not a result derived from the evaluation. Crucially, the paper removes environment rewards during training ('we completely remove all built-in rewards and environment-provided achievement signals during training') and evaluates with external ground-truth signals ('These signals are used only at evaluation to measure true instruction-following capabilities'). The headline comparisons in Figures 2 and 3 therefore do not reduce to the training proxy: no fitted parameter is renamed as a prediction, and the threshold δ is a hand-set hyperparameter whose ablation (Fig. 4) is not used to select the reported evaluation outcomes. The admitted limitation that LLM candidates can be inaccurate (Sec. 4.1) and the dependence on LLM quality (Sec. 6) are validity and robustness risks, not instances of logical circularity. The only self-citation, [Wang et al., 2025] in Section 2, appears in a contextual list of LLM action-guidance works and is not load-bearing. Accordingly, there are no circular steps to report.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method rests on the proxy validity of cosine-similarity rewards, the ability of the LLM to read trajectories, and the completeness of the environment's ground-truth evaluation. These assumptions are plausible but not independently verified; the paper provides no calibration of the reward proxy against ground-truth success.

free parameters (3)
  • Reward threshold δ = 0.9 (ablation over 0.3, 0.5, 0.7, 0.9)
    Episode success is declared when cosine similarity exceeds δ; the paper selects δ=0.9 after ablation (Figure 4), making it a hand-chosen threshold that influences the training signal.
  • Instruction buffer capacity Bmax = 10
    The prioritized instruction buffer holds only 10 instructions, chosen without ablation; this constrains instruction diversity and may cause forgetting, as the paper acknowledges.
  • Number of LLM candidates K = not reported
    Algorithm 1 samples K candidate instructions per trajectory, but K is not listed in the hyperparameter table, leaving the instruction generation budget unspecified.
assumptions (4)
  • domain assumption Cosine similarity between trajectory and instruction embeddings is a valid proxy for instruction completion (Eq. 8)
    The entire training reward is based on this proxy; the paper provides no validation that cosim > δ correlates with ground-truth achievement, which is load-bearing for the method.
  • domain assumption The LLM can reliably identify subtasks the agent actually accomplished from textual trajectories (Section 4.1)
    The relabeling quality depends on LLM hindsight accuracy; the paper notes LLMs can generate inaccurate candidates, so this assumption is partially acknowledged.
  • domain assumption SBERT instruction embeddings provide a semantically meaningful similarity space for open-ended instructions
    The reward function relies on pretrained SBERT embeddings; if the embedding space does not align with task semantics, the rewards are misleading.
  • domain assumption Craftax-Classic with rewards removed is a representative testbed for open-ended instruction-following (Section 5)
    The empirical claim is demonstrated only on this single environment; generality to other instruction-following domains is asserted, not shown.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning Instruction-Following Policies through Open-Ended Instruction Relabeling with Large Language Models." pith.science (2026). https://pith.science/paper/AUFJ5RN5

@misc{pith2026250620061,
  author       = {Pith},
  title        = {Pith review of: Learning Instruction-Following Policies through Open-Ended Instruction Relabeling with Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AUFJ5RN5}},
  note         = {Machine review of arXiv:2506.20061}
}
read the original abstract

Developing effective instruction-following policies in reinforcement learning remains challenging due to the reliance on extensive human-labeled instruction datasets and the difficulty of learning from sparse rewards. In this paper, we propose a novel approach that leverages the capabilities of large language models (LLMs) to automatically generate open-ended instructions retrospectively from previously collected agent trajectories. Our core idea is to employ LLMs to relabel unsuccessful trajectories by identifying meaningful subtasks the agent has implicitly accomplished, thereby enriching the agent's training data and substantially alleviating reliance on human annotations. Through this open-ended instruction relabeling, we efficiently learn a unified instruction-following policy capable of handling diverse tasks within a single policy. We empirically evaluate our proposed method in the challenging Craftax environment, demonstrating clear improvements in sample efficiency, instruction coverage, and overall policy performance compared to state-of-the-art baselines. Our results highlight the effectiveness of utilizing LLM-guided open-ended instruction relabeling to enhance instruction-following reinforcement learning.

Figures

Figures reproduced from arXiv: 2506.20061 by the authors.

Figure 1
Figure 1. Overview of OIR framework. → Blue flow → illustrates standard reinforcement learning: an instruction-conditioned policy samples rollouts from the environment using instructions sampled from the instruction buffer, after which the policy parameters are updated with the collected samples. → Green flow → highlights our novel relabeling mechanism: converting collected trajectories into extual observations and then promp… view at source ↗
Figure 2
Figure 2. (a-c) Performance comparison of OIR against baseline methods measured by (a) aggregate score, (b) number of completed instructions, and (c) average success rate across all original instructions. (d) Success rates of OIR compared to baselines for individual instructions. OIR consistently outperforms baseline methods across all evaluation metrics and nearly all individual tasks. Results are averaged over three random … view at source ↗
Figure 3
Figure 3. Generalization performance of OIR compared to baseline methods measured by aggregated score evaluation on (a) original instructions, (b) simple variant instructions, and (c) complex variant instructions. OIR demonstrates superior generalization capabilities, significantly outperforming baseline methods across all three variants. Results are averaged across three random seeds, with shaded areas denoting standard erro… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Sensitivity of OIR to the different cosine-similarity threshold δ 5.2 RQ2 (Generalization) According to [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: t-SNE visualization of se￾mantic diversity of instructions gener￾ated by OIR compared to ELLM and environment-provided achievements. To address RQ3, we visually interpret generated instructions in semantic embedding space using a two-dimensional t-SNE embedding in [PI…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Learning More from Less: Reinforcement Learning from Hindsight

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Hindsight language-and-reward relabeling of failed VLA rollouts by a VLM yields roughly 5× sample-efficiency gains over standard GRPO on OOD LIBERO-PRO and real Franka tasks.

Reference graph

Works this paper leans on

26 extracted references · 9 canonical work pages · cited by 1 Pith paper

  1. [1]

    Universal value function approximators

    Tom Schaul, Daniel Horgan, Karol Gregor, and David Silver. Universal value function approximators. In International conference on machine learning, pages 1312--1320. PMLR, 2015

    work page 2015

  2. [2]

    Hindsight experience replay

    Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob McGrew, Josh Tobin, OpenAI Pieter Abbeel, and Wojciech Zaremba. Hindsight experience replay. Advances in neural information processing systems, 30, 2017

    2017

  3. [3]

    Human instruction-following with deep reinforcement learning via transfer-learning from text

    Felix Hill, Sona Mokra, Nathaniel Wong, and Tim Harley. Human instruction-following with deep reinforcement learning via transfer-learning from text. arXiv preprint arXiv:2005.09382, 2020

    arXiv 2005

  4. [4]

    Grounding language for transfer in deep reinforcement learning

    Karthik Narasimhan, Regina Barzilay, and Tommi Jaakkola. Grounding language for transfer in deep reinforcement learning. Journal of Artificial Intelligence Research, 63: 0 849--874, 2018

    work page 2018

  5. [5]

    Craftax: A lightning-fast benchmark for open-ended reinforcement learning

    Michael Matthews, Michael Beukman, Benjamin Ellis, Mikayel Samvelyan, Matthew Jackson, Samuel Coward, and Jakob Foerster. Craftax: A lightning-fast benchmark for open-ended reinforcement learning. arXiv preprint arXiv:2402.16801, 2024

    arXiv 2024

  6. [6]

    Goal-conditioned reinforcement learning: Problems and solutions

    Minghuan Liu, Menghui Zhu, and Weinan Zhang. Goal-conditioned reinforcement learning: Problems and solutions. arXiv preprint arXiv:2201.08299, 2022

    arXiv 2022

  7. [7]

    Maximum entropy gain exploration for long horizon multi-goal reinforcement learning

    Silviu Pitis, Harris Chan, Stephen Zhao, Bradly Stadie, and Jimmy Ba. Maximum entropy gain exploration for long horizon multi-goal reinforcement learning. In International Conference on Machine Learning, pages 7750--7761. PMLR, 2020

    work page 2020

  8. [8]

    Curriculum-guided hindsight experience replay

    Meng Fang, Tianyi Zhou, Yali Du, Lei Han, and Zhengyou Zhang. Curriculum-guided hindsight experience replay. Advances in neural information processing systems, 32, 2019

    2019

Show all 26 references
  1. [9]

    Exploration via hindsight goal generation

    Zhizhou Ren, Kefan Dong, Yuan Zhou, Qiang Liu, and Jian Peng. Exploration via hindsight goal generation. Advances in Neural Information Processing Systems, 32, 2019

    2019

  2. [10]

    Visual reinforcement learning with imagined goals

    Ashvin V Nair, Vitchyr Pong, Murtaza Dalal, Shikhar Bahl, Steven Lin, and Sergey Levine. Visual reinforcement learning with imagined goals. Advances in neural information processing systems, 31, 2018

    2018

  3. [11]

    Unsupervised control through non-parametric discriminative rewards

    David Warde-Farley, Tom Van de Wiele, Tejas Kulkarni, Catalin Ionescu, Steven Hansen, and Volodymyr Mnih. Unsupervised control through non-parametric discriminative rewards. arXiv preprint arXiv:1811.11359, 2018

    2018 arXiv

  4. [12]

    A survey of reinforcement learning informed by natural language

    Jelena Luketina, Nantas Nardelli, Gregory Farquhar, Jakob Foerster, Jacob Andreas, Edward Grefenstette, Shimon Whiteson, and Tim Rockt \"a schel. A survey of reinforcement learning informed by natural language. arXiv preprint arXiv:1906.03926, 2019

    1906 arXiv

  5. [13]

    The wisdom of hindsight makes language models better instruction followers

    Tianjun Zhang, Fangchen Liu, Justin Wong, Pieter Abbeel, and Joseph E Gonzalez. The wisdom of hindsight makes language models better instruction followers. In International Conference on Machine Learning, pages 41414--41428. PMLR, 2023

    2023

  6. [14]

    Text2reward: Automated dense reward function generation for reinforcement learning

    Tianbao Xie, Chen Li, and Hao Zhang. Text2reward: Automated dense reward function generation for reinforcement learning. arXiv preprint arXiv:2305.09889, 2023

    2023 arXiv

  7. [15]

    Eureka: Human-level reward design via coding large language models

    Yujia Ma, Weitang Liu, Xingxu Yao, and Ilya Mordatch. Eureka: Human-level reward design via coding large language models. arXiv preprint arXiv:2310.12931, 2023

    2023 arXiv

  8. [16]

    Language models as zero-shot planners: Extracting actionable knowledge for embodied agents

    Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. In International conference on machine learning, pages 9118--9147. PMLR, 2022

    2022

  9. [17]

    Inferring rewards from language in context

    Jessy Lin, Daniel Fried, Dan Klein, and Anca Dragan. Inferring rewards from language in context. arXiv preprint arXiv:2204.02515, 2022

    2022 arXiv

  10. [18]

    Guiding pretraining in reinforcement learning with large language models

    Yuqing Du, Olivia Watkins, Zihan Wang, C \'e dric Colas, Trevor Darrell, Pieter Abbeel, Abhishek Gupta, and Jacob Andreas. Guiding pretraining in reinforcement learning with large language models. In International Conference on Machine Learning, pages 8657--8677. PMLR, 2023

    2023

  11. [19]

    React: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023

    2023

  12. [20]

    Minedojo: Building open-ended embodied agents with internet-scale knowledge

    Linxi Fan, Guanzhi Wang, Yunfan Jiang, Ajay Mandlekar, Yuncong Yang, Haoyi Zhu, Andrew Tang, De-An Huang, Yuke Zhu, and Anima Anandkumar. Minedojo: Building open-ended embodied agents with internet-scale knowledge. Advances in Neural Information Processing Systems, 35: 0 18343...

    2022

  13. [21]

    M3hf: Multi-agent reinforcement learning from multi-phase human feedback of mixed quality

    Ziyan Wang, Zhicheng Zhang, Fei Fang, and Yali Du. M3hf: Multi-agent reinforcement learning from multi-phase human feedback of mixed quality. arXiv preprint arXiv:2503.02077, 2025

    2025 arXiv

  14. [22]

    Puterman

    Martin L. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley-Interscience, 1994

    1994

  15. [23]

    Sutton and Andrew G

    Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. MIT Press, 2nd edition, 2018

    2018

  16. [24]

    Sentence-bert: Sentence embeddings using siamese bert-networks

    Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084, 2019

    1908 arXiv

  17. [25]

    Simplifying deep temporal difference learning

    Matteo Gallici, Mattie Fellows, Benjamin Ellis, Bartomeu Pou, Ivan Masmitja, Jakob Nicolaus Foerster, and Mario Martin. Simplifying deep temporal difference learning. arXiv preprint arXiv:2407.04811, 2024

    2024 arXiv

  18. [26]

    Prioritized level replay

    Minqi Jiang, Edward Grefenstette, and Tim Rockt \"a schel. Prioritized level replay. In International Conference on Machine Learning, pages 4940--4950. PMLR, 2021

    2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.