Pith. sign in

REVIEW 4 major objections 5 minor 36 references

LLM-Based Offline Learning for Embodied Agents via Consistency-Guided Reward Ensemble

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read CoREN shows that a large language model can be used only as a training-time reward teacher, distilling its commonsense into a 117M-parameter offline RL agent that matches online LLM agents with 8B parameters on household…

desk verdict A promising but under-supported method: the consistency-guided reward ensemble is new, but the orchestrator's per-step grounding is not demonstrated and the cross-domain test is flawed. read the letter →

arxiv 2411.17135 v1 pith:BKRTVBFF submitted 2024-11-26 cs.AI cs.CL

Yujeong Lee , Sangwoo Shin , Wei-Jin Park , Honguk Woo This is my paper · ORCID
classification cs.AIcs.CL
keywords offlinereinforcementlearninglargelanguagemodelsrewardestimationembodiedagentsinstructionfollowingconsistencyensembleVirtualHomeshaping
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether a large language model can be used not as an embodied agent but as a teacher: it scores each action in an offline dataset, and those scores are distilled into a small policy network via offline reinforcement learning. The difficulty is that LLM scores reflect commonsense, not the specific layout of the target house, so they can reward actions that make sense in general but are wrong here. CoREN addresses this with three consistency filters — agreement across different prompts, correct answers to object-relevance questions, and backward verification of the LLM's own high-value action sequence — followed by a learned blend of the filtered rewards that is aligned to the trajectory's binary success flag. The claim is that this two-stage process yields dense rewards grounded enough to train a 117M-parameter agent that matches online LLM agents with 8B parameters on VirtualHome instruction-following, and that degrades only mildly when objects move to new locations.

What carries the argument

The central object is the consistency-guided reward ensemble, defined by three consistency-constrained reward estimates r_C, r_S, r_T computed from an LLM, and a learned reward orchestrator Ψ_θ that blends them into a unified stepwise reward r̂(o,l|i) = ⟨r(o,l|i), w(o,l|i)⟩. Contextual consistency takes a majority vote over N differently prompted LLM reward estimates; structural consistency only lets a reward count if the LLM correctly answers MDP-specific queries such as which visible objects are relevant to the instruction; temporal consistency drops rewards for actions the LLM rated as high-value when backward verification says the resulting sequence cannot accomplish the instruction. The orchestrator is a small network that outputs per-step weights (w_C, w_S, w_T) and is trained to minimize the squared difference between the discounted return of the unified rewards and the trajectory's binary success flag. This machinery carries the argument because the weights turn three individually biased commonsense score streams into one reward that is aligned with the actual outcome labels, and only the orchestrator needs retraining when the environment domain shifts.

What would settle it

In the cross-domain setting, replace relabeled old trajectories with trajectories actually executed in the new layout and retrain the orchestrator; if CoREN's SR drops to the level of the RL baselines, the claim that relabeling faithfully simulates the new environment is false.

Watch

Extended reading notes

Core claim

The central discovery, as the authors state it, is that consistency-guided reward ensemble (CoREN) makes LLM-based reward estimation work in the offline setting, where the LLM never interacts with the environment. The paper reports that CoREN outperforms all offline RL agents it is compared with and achieves performance comparable to state-of-the-art LLM-based agents using 8B parameters, despite the agent policy having only 117M parameters and the LLM being used only for training. The mechanism attributed for this is the ensemble of spatio-temporally consistent rewards: contextual (majority vote over diverse prompts), structural (gating rewards by the LLM's answers to object-relevance queries about the environment), and temporal (holding the LLM's high-value actions to a backward verification that they can accomplish the instruction), merged by a reward orchestrator trained to align the trajectory return with the sparse success flag.

Load-bearing premise

The method assumes that a binary success flag on each trajectory, together with the LLM's commonsense action scores, is enough to learn a dense reward that is truly grounded in the environment, and that reusing old trajectories with relabeled flags fairly simulates a new environment layout.

Editorial extensions

If this is right

  • On VirtualHome, CoREN reaches 66.4% fine-grained SR and 57.6% abstract SR, beating the strongest RL baseline (Self-Consistency) by 20 percentage points in SR and coming within a few points of SayCan-Gemini and ProgPrompt-Gemini.
  • Under cross-domain shifts, CoREN retains most of its performance (fine-grained SR 60.0 versus 66.4 in the single domain) by only retraining the orchestrator on relabeled trajectories, while the RL baselines drop to as low as 2.5 SR.
  • Each consistency mechanism contributes: ablations that remove temporal, structural, or contextual consistency lower fine-grained SR to 53.6–64.8 from 66.4.
  • The learned ensemble outperforms averaging (53.6 SR) and majority voting (60.8 SR), so the sparse-reward alignment step is doing real work.
  • On ALFRED (appendix), CoREN achieves 72.0 fine-grained SR versus the best RL baseline's 46.4, showing the recipe transfers to a different household benchmark.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If CoREN's orchestrator really captures domain grounding from sparse labels alone, the same scheme could densify rewards for any long-horizon instruction-following benchmark that ships only binary success flags, not just embodied household simulators.
  • The cross-domain experiment relabels old trajectories rather than collecting new ones; a stronger test would be retraining the orchestrator on trajectories actually executed in the new layout, since the reported robustness may be partly an artifact of the relabeling shortcut if those relabeled trajectories contain actions infeasible in the new layout.
  • The large gap between LLaMA3-8B (SR 12.0) and GPT-4 Turbo/Gemini (SR 65.6/66.4) as reward estimators suggests CoREN is a distillation channel whose ceiling tracks the teacher's commonsense and spatial reasoning, so stronger or domain-tuned LLMs should raise the ceiling proportionally.
  • One immediately testable extension: apply CoREN to a dataset with corrupted success flags (e.g., 20% flipped) and measure SR; if performance barely moves, the orchestrator's alignment is not what grounds the reward, contradicting the paper's mechanism.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes CoREN, an offline RL framework for embodied instruction-following. Instead of using an LLM as an online agent, CoREN uses an LLM to generate dense stepwise rewards for offline trajectories, then combines three reward estimates (contextual, structural, temporal consistency) via a learned 'reward orchestrator' that is trained to make discounted trajectory returns match sparse success labels. The resulting reward-augmented dataset is used to train a compact 117M GPT2-based policy with CQL/DDQN. Experiments on VirtualHome (and ALFRED in the appendix) report that CoREN outperforms offline RL baselines and is comparable to online LLM agents with much larger policy models.

Significance. If the results hold, the paper demonstrates a useful direction: LLM commonsense can be distilled into a small offline policy, avoiding online LLM inference cost and latency. The framework is original in combining multiple consistency filters with a learned ensemble, and the authors provide extensive ablations (consistency types, LLM choices, ensemble scheme, number of prompts) plus an additional ALFRED experiment. The appendix includes full prompt templates and hyperparameters, which aids reproducibility. The main limitations are evidential: no error bars or seeds are reported, and the cross-domain protocol relabels old trajectories rather than executing in a genuinely new environment. These issues must be addressed before the central claims are fully supported.

major comments (4)
  1. [Section 4.2, Tables 1 and 2] All results are single point estimates with no standard deviations, confidence intervals, or number of seeds for any RL agent. The claims of 'significantly outperforms' and 'comparable performance' are therefore not backed by statistical evidence. This is especially problematic for the small margins in Table 5 (e.g., 66.4 vs 60.8 fine-grained SR, and 57.6 vs 55.2 abstract SR). Please report mean and standard deviation over at least 5-10 seeds and, for headline comparisons, provide a significance test.
  2. [Section 4.2, cross-domain evaluation] The cross-domain protocol relabels the sparse success flags of existing trajectories from the original environment and retrains only the reward orchestrator. Since the observations, actions, and dynamics remain those of the original layout, a trajectory that was feasible in the old layout is not guaranteed to be feasible or goal-achieving in the new layout; relabeling it as successful does not simulate a new environment. The claim of 'relatively robust performance across domain shifts' (Table 2) is therefore not established. The paper should execute the relabeled trajectories in the VirtualHome simulator with the new object locations, or explicitly reframe the experiment as a label-shift robustness test and not as a domain shift.
  3. [Section 3.3, Eq. (10)] The orchestrator is trained only on a per-trajectory aggregate loss: the discounted sum of stepwise rewards is matched to the scalar success flag. Because Psi_theta outputs per-timestep weights, there are many weight assignments that satisfy the aggregate constraint for a given trajectory, and the loss does not directly reward per-step accuracy. The paper should provide either an identifiability argument or a per-step reward quality evaluation (e.g., correlation with human or expert stepwise scores), or an ablation that separates the contribution of per-step weighting from a simple trajectory-level offset or scaling. The modest gain over majority voting in Table 5 makes this distinction important.
  4. [Section 4.1 / Appendix A.2] The offline dataset appears to contain only one expert trajectory per task labeled as success, with random perturbations labeled as failure. If the success flag is effectively 'this is the expert trajectory' rather than a verified goal-completion label, then the reward orchestrator may learn to identify the expert trajectory instead of generalizable task success. Please clarify whether success labels are verified by environment execution, and if not, discuss the consequence for the claimed 'domain-grounded' reward.
minor comments (5)
  1. [Throughout] The framework name is inconsistent: 'COREN' appears in the abstract and tables, 'CORE N' in Figure 1 and Algorithm 1, and 'CoREN' in several places. Please use a single spelling consistently.
  2. [Algorithm 1] The arrow symbols '← −' in lines 7-9 and 20-21 appear to be formatting artifacts; please use standard assignment arrows.
  3. [Appendix B.1.3] The text states fs(i, tau) in {-1, 1}, while Section 4.1 defines success/failure flags as 1/0. Please clarify the normalization and the role of alpha in Eq. (10).
  4. [Table 7] In Query 2, the answer lists 'cat' although 'cat' is not among the visible objects listed in the query. Please correct this example.
  5. [Appendix D.2] The sentence 'Table Table 14' contains a duplicated word; fix the cross-reference.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reward ensemble is supervised by sparse labels and the final policy is evaluated externally.

full rationale

The paper's derivation chain is a standard supervised pipeline: (1) an LLM produces stepwise reward estimates from prompts; (2) consistency mechanisms (contextual, structural, temporal) filter these estimates using LLM self-queries; (3) a reward orchestrator \Psi_\theta is trained via Eq. (10) to combine the filtered rewards so that each trajectory's discounted return matches its sparse success flag f_s(i,\tau); and (4) an offline RL policy is trained on the resulting dense rewards and evaluated in the VirtualHome and ALFRED simulators. The only fitted component is \Psi_\theta, and it is fitted against a known input label (the success flag), not against the downstream evaluation metric. The final performance figures are measured on held-out instructions in an external simulator, so the central empirical claim (117M-parameter policy performs comparably to 8B LLM-based agents) is not derived from the training data or from the fitted orchestrator. The self-referential consistency checks (the same LLM generates rewards and verifies them via MDP-specific queries and backward verification) are heuristic filtering steps, not a circular derivation of the target result. There are no load-bearing self-citations, no imported uniqueness theorems, and no renamed known result presented as a prediction. The concern that Eq. (10) underdetermines per-step weights is a correctness/identifiability issue, not an instance of circularity: even if the dense reward were a global rescaling, the policy evaluation would still be an external benchmark. Accordingly, no specific circular step can be exhibited from the paper's equations or text.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central mechanism rests on several fitted quantities (orchestrator weights, prompt count, rubric, thresholds) and on the assumption that LLM commonsense plus sparse binary labels are enough to ground rewards in the environment. The method is a supervised fitting pipeline rather than a derivation from first principles, and it introduces no new physical or ontological entities.

free parameters (6)
  • Reward orchestrator weights Ψθ = learned
    Trained via Eq. (10) to minimize the discrepancy between the cumulative unified reward and the sparse success flag; these weights are fitted to the training dataset and are central to producing the final dense reward.
  • Scaling hyperparameter α = not specified
    Introduced in Eq. (10) to scale the sparse reward target; no value or selection procedure is reported.
  • Number of prompts N = 5 (main), up to 7 in ablation
    Chosen from ablation (Table 15) that reports test performance; increasing N improves results, so the choice appears tuned on the evaluation metric.
  • LLM temperature = 0.7 (Gemini, PaLM, LLaMA3); 0.5 (GPT4 Turbo)
    Hand-set per model in Table 6; affects the stochasticity of reward estimates and thus the majority-voted rewards.
  • Embedding similarity threshold = 0.5
    Used to judge whether the LLM's answer to an MDP query matches the GPT4-generated ground truth; hand-set threshold in Appendix B.1.2.
  • Reward rubric values in prompts = -2, -1, 0, 1, 2
    The prompt rubric in Tables 19-22 assigns these scores to action categories; this hand-designed prior heavily shapes the LLM's reward estimates.
assumptions (4)
  • domain assumption LLM commonsense knowledge transfers to VirtualHome reward estimation
    The entire method assumes Gemini 1.0 Pro can assign meaningful dense rewards to household actions from text observations, as used throughout Section 3.2 and the prompts in Appendix B.1.1.
  • domain assumption Sparse success flag is a sufficient grounding signal
    The orchestrator is trained to align trajectory return with the binary success flag (Eq. 10); the method assumes this alignment yields genuinely domain-grounded rewards rather than a reward that merely memorizes the training labels.
  • domain assumption GPT4-generated QA answers are correct ground truth for structural consistency
    DQA in Appendix B.1.2 is generated by GPT4 and treated as ground truth for judging Gemini's object-relevance answers; no environment-based verification of these QA labels is provided.
  • ad hoc to paper Relabeling trajectories simulates a new domain
    Cross-domain evaluation in Section 4.2 uses the same trajectories and observations from the original environment and only changes the success flags; this assumes that old observations and action sequences remain meaningful after object relocation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LLM-Based Offline Learning for Embodied Agents via Consistency-Guided Reward Ensemble." pith.science (2026). https://pith.science/paper/BKRTVBFF

@misc{pith2026241117135,
  author       = {Pith},
  title        = {Pith review of: LLM-Based Offline Learning for Embodied Agents via Consistency-Guided Reward Ensemble},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BKRTVBFF}},
  note         = {Machine review of arXiv:2411.17135}
}
read the original abstract

Employing large language models (LLMs) to enable embodied agents has become popular, yet it presents several limitations in practice. In this work, rather than using LLMs directly as agents, we explore their use as tools for embodied agent learning. Specifically, to train separate agents via offline reinforcement learning (RL), an LLM is used to provide dense reward feedback on individual actions in training datasets. In doing so, we present a consistency-guided reward ensemble framework (CoREN), designed for tackling difficulties in grounding LLM-generated estimates to the target environment domain. The framework employs an adaptive ensemble of spatio-temporally consistent rewards to derive domain-grounded rewards in the training datasets, thus enabling effective offline learning of embodied agents in different environment domains. Experiments with the VirtualHome benchmark demonstrate that CoREN significantly outperforms other offline RL agents, and it also achieves comparable performance to state-of-the-art LLM-based agents with 8B parameters, despite CoREN having only 117M parameters for the agent policy network and using LLMs only for training.

Figures

Figures reproduced from arXiv: 2411.17135 by the authors.

Figure 1
Figure 1. COREN, a framework for LLM-based reward estimation and offline learning. In (i), an LLM estimates rewards based on spatio-temporal (i.e., contextual, struc￾tural, and temporal) consistencies; In (ii), these rewards are integrated into a single domain-grounded reward via an ensemble. Using the reward-augmented dataset, offline RL can be conducted effectively to achieve em￾bodied agents with resource efficiency and lo… view at source ↗
Figure 2
Figure 2. Two-staged reward estimation in COREN. In (i), spatio-temporally consistent rewards, constrained by contextual, structural, and temporal consistencies, are calculated. (a) Contextual consistency is achieved through majority voting across the responses from different prompts Pn, resulting in contextually consistent rewards r C . (b) Structural consistency is achieved by presenting MDP-specific queries to the LLM. If … view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 13 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Dai, Anja Hauth, Katie Millican, David Silver, Slav Petrov, Melvin Johnson, Ioannis Antonoglou, Julian Schrittwieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy P

    Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean - Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katie Millican, David Silver, Slav Petrov, Melvin Johnson, Ioannis Antonoglou, Julian Schrittwieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy P. Lillicrap, Angeliki Lazaridou, Orhan Firat, James Molloy, Michael ...

    arXiv doi:10.48550/arxiv.2312.11805 2023

  4. [4]

    Kate Baumli, Satinder Baveja, Feryal M. P. Behbahani, Harris Chan, Gheorghe Comanici, Sebastian Flennerhag, Maxime Gazeau, Kristian Holsheimer, Dan Horgan, Michael Laskin, Clare Lyle, Hussain Masoom, Kay McKinney, Volodymyr Mnih, Alexander Neitz, Fabio Pardo, Jack Parker - Holder, John Quan, Tim Rockt \" a schel, Himanshu Sahni, Tom Schaul, Yannick Schroe...

    arXiv doi:10.48550/arxiv.2312.09187 2023

  5. [5]

    Kun Chu, Xufeng Zhao, Cornelius Weber, Mengdi Li, and Stefan Wermter. 2023. https://doi.org/10.48550/ARXIV.2311.02379 Accelerating reinforcement learning of robotic manipulations via feedback from large language models . CoRR, abs/2311.02379

    arXiv doi:10.48550/arxiv.2311.02379 2023

  6. [6]

    Jacob Devlin, Ming - Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/V1/N19-1423 BERT: pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, ...

    doi:10.18653/v1/n19-1423 2019

  7. [7]

    Yuqing Du, Olivia Watkins, Zihan Wang, C \' e dric Colas, Trevor Darrell, Pieter Abbeel, Abhishek Gupta, and Jacob Andreas. 2023. https://proceedings.mlr.press/v202/du23f.html Guiding pretraining in reinforcement learning with large language models . In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA , volum...

    work page 2023

  8. [8]

    Dylan Hadfield - Menell, Stuart Russell, Pieter Abbeel, and Anca D. Dragan. 2016. https://proceedings.neurips.cc/paper/2016/hash/c3395dd46c34fa7fd8d729d8cf88b7a8-Abstract.html Cooperative inverse reinforcement learning . In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10,...

    work page 2016

Show all 36 references
  1. [9]

    Maryam Hashemzadeh, Elias Stengel - Eskin, Sarath Chandar, and Marc - Alexandre C \^ o t \' e . 2024. https://doi.org/10.48550/ARXIV.2405.02749 Sub-goal distillation: A method to improve small language agents . CoRR, abs/2405.02749

    2024 doi arXiv

  2. [10]

    Mengkang Hu, Yao Mu, Xinmiao Yu, Mingyu Ding, Shiguang Wu, Wenqi Shao, Qiguang Chen, Bin Wang, Yu Qiao, and Ping Luo. 2023. https://doi.org/10.48550/ARXIV.2310.08582 Tree-planner: Efficient close-loop task planning with large language models . CoRR, abs/2310.08582

    2023 doi arXiv

  3. [11]

    Joshi, Kyle Jeffrey, Rosario Jauregui Ruano, Jasmine Hsu, Keerthana Gopalakrishnan, Byron David, Andy Zeng, and Chuyuan Kelly Fu

    Brian Ichter, Anthony Brohan, Yevgen Chebotar, Chelsea Finn, Karol Hausman, Alexander Herzog, Daniel Ho, Julian Ibarz, Alex Irpan, Eric Jang, Ryan Julian, Dmitry Kalashnikov, Sergey Levine, Yao Lu, Carolina Parada, Kanishka Rao, Pierre Sermanet, Alexander Toshev, Vincent Vanho...

    2022

  4. [12]

    Edouard Klein, Matthieu Geist, Bilal Piot, and Olivier Pietquin. 2012. https://proceedings.neurips.cc/paper/2012/hash/559cb990c9dffd8675f6bc2186971dc2-Abstract.html Inverse reinforcement learning through structured classification . In Advances in Neural Information Processing ...

    2012

  5. [13]

    Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. 2020 a . https://proceedings.neurips.cc/paper/2020/hash/0d2b2061826a5df3221116a5085a6052-Abstract.html Conservative q-learning for offline reinforcement learning . In Advances in Neural Information Processing Systems...

    2020

  6. [14]

    Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. 2020 b . https://proceedings.neurips.cc/paper/2020/hash/0d2b2061826a5df3221116a5085a6052-Abstract.html Conservative q-learning for offline reinforcement learning . In Advances in Neural Information Processing Systems...

    2020

  7. [15]

    Minae Kwon, Sang Michael Xie, Kalesha Bullard, and Dorsa Sadigh. 2023. https://openreview.net/pdf?id=10uNUgI5Kl Reward design with language models . In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . OpenReview.net

    2023

  8. [16]

    Jieh - Sheng Lee. 2024. https://doi.org/10.48550/ARXIV.2406.16897 Instructpatentgpt: Training patent language models to follow instructions with human feedback . CoRR, abs/2406.16897

    2024 doi arXiv

  9. [17]

    Hao Li, Xue Yang, Zhaokai Wang, Xizhou Zhu, Jie Zhou, Yu Qiao, Xiaogang Wang, Hongsheng Li, Lewei Lu, and Jifeng Dai. 2023. https://doi.org/10.48550/ARXIV.2312.09238 Auto mc-reward: Automated dense reward design with large language models for minecraft . CoRR, abs/2312.09238

    2023 doi arXiv

  10. [18]

    Lajanugen Logeswaran, Yao Fu, Moontae Lee, and Honglak Lee. 2022. Few-shot subgoal planning with language models. arXiv preprint arXiv:2205.14288

    2022 arXiv

  11. [19]

    Yecheng Jason Ma, William Liang, Guanzhi Wang, De - An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023. https://doi.org/10.48550/ARXIV.2310.12931 Eureka: Human-level reward design via coding large language models . CoRR, abs/2310.12931

    2023 doi arXiv

  12. [20]

    Yecheng Jason Ma, Jason Yan, Dinesh Jayaraman, and Osbert Bastani. 2022. http://papers.nips.cc/paper\_files/paper/2022/hash/022a39052abf9ca467e268923057dfc0-Abstract-Conference.html Offline goal-conditioned reinforcement learning via f -advantage regression . In Advances in Ne...

    2022

  13. [21]

    Aishwarya Padmakumar, Mert Inan, Spandana Gella, Patrick L Lange, and Dilek Hakkani-Tur. 2023. Multimodal embodied plan prediction augmented with synthetic embodied dialogue. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6114--6131

    2023

  14. [22]

    Georgios Pantazopoulos, Malvina Nikandrou, Amit Parekh, Bhathiya Hemanthage, Arash Eshghi, Ioannis Konstas, Verena Rieser, Oliver Lemon, and Alessandro Suglia. 2023. Multitask multimodal prompted training for interactive embodied task completion. arXiv preprint arXiv:2311.04067

    2023 arXiv

  15. [23]

    Seohong Park, Dibya Ghosh, Benjamin Eysenbach, and Sergey Levine. 2023. http://papers.nips.cc/paper\_files/paper/2023/hash/6d7c4a0727e089ed6cdd3151cbe8d8ba-Abstract-Conference.html HIQL: offline goal-conditioned RL with latent states as actions . In Advances in Neural Informat...

    2023

  16. [24]

    Xavier Puig, Kevin Ra, Marko Boben, Jiaman Li, Tingwu Wang, Sanja Fidler, and Antonio Torralba. 2018. https://doi.org/10.1109/CVPR.2018.00886 Virtualhome: Simulating household activities via programs . In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 20...

    2018

  17. [25]

    Nils Reimers and Iryna Gurevych. 2019. https://doi.org/10.18653/V1/D19-1410 Sentence-bert: Sentence embeddings using siamese bert-networks . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on ...

    2019 doi

  18. [26]

    Juan Rocamonde, Victoriano Montesinos, Elvis Nava, Ethan Perez, and David Lindner. 2023. https://doi.org/10.48550/ARXIV.2310.12921 Vision-language models are zero-shot reward models for reinforcement learning . CoRR, abs/2310.12921

    2023 doi arXiv

  19. [27]

    Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. 2020. https://doi.org/10.1109/CVPR42600.2020.01075 ALFRED: A benchmark for interpreting grounded instructions for everyday tasks . In 2020 IEEE/CVF Conf...

    2020

  20. [28]

    Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, and Animesh Garg. 2023. https://doi.org/10.1109/ICRA48891.2023.10161317 Progprompt: Generating situated robot task plans using large language models . In IEEE ...

    2023

  21. [30]

    Sadler, Jiaman Wu, Wei - Lun Chao, Clayton Washington, and Yu Su

    Chan Hee Song, Brian M. Sadler, Jiaman Wu, Wei - Lun Chao, Clayton Washington, and Yu Su. 2023 b . https://doi.org/10.1109/ICCV51070.2023.00280 Llm-planner: Few-shot grounded planning for embodied agents with large language models . In IEEE/CVF International Conference on Comp...

    2023

  22. [31]

    Jiayang Song, Zhehua Zhou, Jiawei Liu, Chunrong Fang, Zhan Shu, and Lei Ma. 2023 c . https://doi.org/10.48550/ARXIV.2309.06687 Self-refined large language model as automated reward function designer for deep reinforcement learning in robotics . CoRR, abs/2309.06687

    2023 doi arXiv

  23. [32]

    Hado van Hasselt, Arthur Guez, and David Silver. 2016. https://doi.org/10.1609/AAAI.V30I1.10295 Deep reinforcement learning with double q-learning . In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12-17, 2016, Phoenix, Arizona, USA , pages ...

    2016 doi

  24. [33]

    Le, Ed H

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. https://openreview.net/pdf?id=1PL1NIMMrw Self-consistency improves chain of thought reasoning in language models . In The Eleventh International Conference...

    2023

  25. [34]

    Yufei Wang, Zhanyi Sun, Jesse Zhang, Zhou Xian, Erdem Biyik, David Held, and Zackory Erickson. 2024. https://doi.org/10.48550/ARXIV.2402.03681 RL-VLM-F: reinforcement learning from vision language foundation model feedback . CoRR, abs/2402.03681

    2024 doi arXiv

  26. [35]

    Tianbao Xie, Siheng Zhao, Chen Henry Wu, Yitao Liu, Qian Luo, Victor Zhong, Yanchao Yang, and Tao Yu. 2024. https://openreview.net/forum?id=tUM39YTRxH Text2reward: Reward shaping with language models for reinforcement learning . In The Twelfth International Conference on Learn...

    2024

  27. [36]

    Cheng-Fu Yang, Yen-Chun Chen, Jianwei Yang, Xiyang Dai, Lu Yuan, Yu-Chiang Frank Wang, and Kai-Wei Chang. 2023. Lacma: Language-aligning contrastive learning with meta-actions for embodied instruction following. arXiv preprint arXiv:2310.12344

    2023 arXiv

  28. [37]

    Tian Yun, Zilai Zeng, Kunal Handa, Ashish V Thapliyal, Bo Pang, Ellie Pavlick, and Chen Sun. 2023. Emergence of abstract state representations in embodied sequence modeling. arXiv preprint arXiv:2311.02171

    2023 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.