Pith. sign in

REVIEW 3 major objections 7 minor 1 cited by

RE-GoT automates reward design for robot manipulation by having a vision-language model watch rollout videos and a large language model refine rewards through a text-attributed task graph, beating expert-written rewards on several benchmark

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

RE-GoT combines graph-of-thoughts planning in LLMs with VLM feedback from rollout videos to automatically write and refine RL reward functions, beating prior LLM-based reward design on RoboGen and ManiSkill2.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection A promising reward-design integration with strong sim results, but the VLM feedback loop is never ablated and the abstract overclaims versus the results. the 3 major comments →

arxiv 2509.16136 v5 pith:VPQV2IUR submitted 2025-09-19 cs.RO

Reward Evolution with Graph-of-Thoughts: A Bi-Level Language Model Framework for Reinforcement Learning

Changwei Yao , Xinzi Liu , Chen Li , Marios Savvides This is my paper
classification cs.RO
keywords reward designgraph-of-thoughtslarge language modelsvision-language modelsreinforcement learningrobot manipulationclosed-loop refinementreward evolution
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that reward engineering—usually a manual, expert-driven bottleneck in reinforcement learning—can be automated by pairing a large language model with a vision-language model in a two-level loop. At the upper level, a VLM watches videos of the agent's rollouts and describes what goes wrong; at the lower level, an LLM first decomposes the task into a text-attributed graph of sub-goals and transitions, then uses the VLM feedback to refine the reward code and its component weights. The authors argue the graph decomposition is what stops LLMs from hallucinating reward functions for long-horizon, multi-step manipulation tasks. On 10 RoboGen tasks the refined rewards raise average success by 32.25%, and on 4 ManiSkill2 tasks RE-GoT reaches 93.73% average success, outperforming the Text2Reward and RewardSelfAlign baselines and, on some tasks, the expert-designed oracle rewards. If true, this would make reward design much more scalable and less dependent on human expertise.

Core claim

RE-GoT's central claim is that reward functions for complex robot manipulation can be evolved without human intervention by structuring the task as a graph of thoughts and closing the loop with visual feedback. The framework formalizes reward learning as a bi-level objective: minimize a loss between expected and actual behavior, where the upper level is VLM evaluation of rollout videos and the lower level is RL training under the current reward. Because the loss and observation model are not differentiable, RE-GoT replaces the gradient with a gradient-free refinement: VLM feedback (video description, potential problems, possible improvements) is combined with the text-attributed graph and cu

What carries the argument

Text-attributed Graph-of-Thoughts: a graph G=(V,E,Tv,Te) whose nodes are task sub-goals described in text, edges are robot behaviors for transitioning between sub-goals, and both carry textual attributes. It is generated by a single LLM call from the task description and environment abstraction, then used together with VLM rollout feedback as the context for reward refinement. The graph is the mechanism that turns a vague, long-horizon task into structured constraints that the LLM can reason over instead of guessing a reward function directly.

Load-bearing premise

Closed-loop improvement depends on the VLM reading rollout videos correctly and giving feedback that actually matches the task goal; the paper itself reports a performance decline on OpenCabinetDrawer after step 1e6 and suspects the VLM failed to analyze the rollout videos correctly.

What would settle it

Swap the VLM for a fixed random-feedback controller that returns arbitrary 'potential problems' text for the same rollout videos, keeping the LLM graph unchanged. If success rates still climb to the reported levels, the loop is not being driven by visual feedback; if they collapse, the VLM's video reading is the causal mechanism the paper claims.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If RE-GoT works as reported, reward design for new manipulation tasks can start from a task description plus environment code and close the loop with videos alone, removing human reward tuning.
  • The 32.25% average improvement on RoboGen suggests LLM-generated dense rewards are systematically weak on multi-step tasks and that structured task decomposition recovers most of the gap.
  • The ManiSkill2 results imply the same pipeline transfers across simulators and task suites, since RE-GoT uses the same bi-level loop on both PyBullet and ManiSkill2 environments.
  • The ablation studies imply both components matter: without the graph, performance barely improves, and without in-context examples the graph is much less effective.
  • The iteration curve implies that continuing the feedback loop for around 8 rounds monotonically improves success rates for the tested tasks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to treat VLM feedback as a learned surrogate reward model, which could make the loop differentiable and let standard RL gradients propagate through the reward parameters instead of relying on the current gradient-free refinement.
  • Since the authors identify VLM misanalysis as the suspected cause of the OpenCabinetDrawer decline, a robustness layer that scores VLM feedback confidence or aggregates multiple VLMs could stabilise the loop in exactly the regime where it currently degrades.
  • The graph is generated once per task; fine-tuning a smaller LLM on a dataset of task graphs (the authors' stated future plan) could reduce dependence on API-scale models and the one-shot graph quality.
  • The same bi-level pattern may apply to other code-search problems in RL, not just rewards—for example, automated shaping of observation encodings or environment curricula—wherever a visual evaluator can judge rollouts.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes RE-GoT, a framework for automated reward-function design in RL. An LLM first constructs a text-attributed graph of task subgoals, then generates or refines dense reward code. A VLM evaluates rollout videos and provides textual feedback, which is fed back into the LLM to refine the reward. Experiments on 10 RoboGen and 4 ManiSkill2 tasks report success rates: Table I shows an average 32.25% improvement over RoboGen's original rewards, and Fig. 5 shows gains over Text2Reward and RewardSelfAlign. Ablation studies in Fig. 6 indicate that the GoT structure and few-shot examples help. The paper frames the method as a bi-level optimization and claims a closed-loop, human-free reward-evolution mechanism.

Significance. If the empirical results are reproducible, RE-GoT would be a useful practical recipe for reducing human reward engineering. The success rates are measured by the environment simulator, not by the learned reward function, so the headline improvements are not definitionally circular. The GoT/example ablation is informative, and the range of 14 tasks gives reasonable breadth. However, the paper's central novelty—the VLM visual-feedback loop—is never isolated, and the bi-level optimization framing is notional rather than operational. The contribution is promising but not yet fully supported; with an added VLM ablation and corrected claims, it could become a solid empirical paper.

major comments (3)
  1. [§IV-B, §IV-D, §V-D] The VLM feedback loop is never ablated. The ablations in §V-D remove GoT and in-context examples, but never disable or corrupt the VLM's video-based feedback. Since the headline contribution is 'autonomous reward evolution via visual feedback', the reported gains could in principle come entirely from iterative LLM re-weighting of reward components using the stored statistics (§IV-D) plus GoT prompting. The only direct reported evidence about VLM quality is negative: performance on OpenCabinetDrawer declines after step 1e6, and the authors attribute this to VLM failure (§V-C.2, Fig. 5). Please add a condition with no VLM feedback, or with corrupted/random VLM feedback, and report per-iteration VLM output quality. Without this, the closed-loop claim is unsupported.
  2. [Abstract vs. §V-C.2] The abstract states that RE-GoT 'even exceed[s] expert-designed rewards' on ManiSkill2, but §V-C.2 says the method 'achieves performance comparable to the oracle.' These are different claims. Please clarify which tasks exceed the oracle and at which checkpoints. In particular, the sentence 'it notably improves the success rate for PickCube from 28.60% to 86.20%' appears to be a comparison with a baseline, not with the oracle. The abstract overstates the result and should be revised to match the evidence.
  3. [§III, Eq. (3)-(4)] The 'bi-level optimization' is not operationalized. The loss L in Eq. (3) is never explicitly defined, and Eq. (4) is purely symbolic; the paper then states that it uses a 'gradient-free alternative' in which LLMs/VLMs replace differentiation. This is a legitimate design choice, but describing RE-GoT as a bi-level framework is misleading because no upper-level loss is optimized and no lower-level gradient is computed. Please either define a concrete bi-level objective that the iterative procedure approximates or change the terminology to 'iterative two-stage refinement' to avoid overclaiming.
minor comments (7)
  1. [§I] Typo/grammar: 'we introduce the first to leverage GoT' should be 'we introduce the first method to leverage GoT' or similar.
  2. [§V-A.1] H3 and H4 overlap: both ask whether in-context learning improves GoT ability. Please merge or differentiate them.
  3. [§V-C.2] 'Best success rates' is not defined. Report the checkpoint/iteration at which these values are taken, and clarify whether they are averaged over seeds or are the best random seed.
  4. [§IV-D] The procedure for selecting reward weights θ is under-specified. State whether the LLM proposes new weights, whether any search is performed, and what the update rule is.
  5. [§II/References] Reference [33] appears to be cited as a bi-level programming work, but the connection to the proposed method is not explained. Please clarify.
  6. [Eq. (2)] The notation πRθ is informal: π is a policy, and the subscript Rθ is nonstandard. Please rewrite the trajectory distribution and the optimal-policy notation.
  7. [Table I] The number of random seeds (4) is stated in the text but not in the table caption. Add it to the caption.

Circularity Check

0 steps flagged

No significant circularity: reported success rates are external environment metrics, and no derivation reduces to its own inputs.

full rationale

The paper's central results are empirical: success rates are computed by the ManiSkill2 and RoboGen environment checkers, not by the LLM/VLM reward-generation or evaluation pipeline. Section III's bi-level loss (Eq. 3) and gradient (Eq. 4) are formal notation; the method explicitly replaces them with a gradient-free heuristic (Sec. IV.A), so no mathematical derivation is forced by construction. The reward-refinement loop (Sec. IV.C.3) uses VLM feedback to modify reward code and component weights, but the final evaluation is independent of that feedback. There are no load-bearing self-citations: references to GoT, RoboGen, T2R, and SA are all external prior work. The absence of a VLM-feedback ablation and the noted OpenCabinetDrawer degradation (Sec. V.C.2) are correctness/robustness concerns, not circularity; an unablated component is not evidence that the component defines the target metric. The limitations section (Sec. VI) explicitly acknowledges dependence on model quality and expert-provided task descriptions, reinforcing that the authors claim an empirical, not definitional, improvement. I find no step where a 'prediction' reduces by construction to a fitted input or to an imported self-citation.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The framework uses no new physical entities; it relies on existing LLM, VLM, and GoT techniques. The main implicit entities are the 'text-attributed graph' representation, which is a data structure, not a new entity.

free parameters (4)
  • reward component weights = not reported (searched by LLM in each iteration)
    The LLM is prompted to search the best weights for each reward component (Section IV-C.3).
  • number of iterations = 8 (peak in Table II)
    Iteration count is a hyperparameter chosen by the authors; performance peaks at n=8 in their ablation.
  • LLM/VLM models = gpt-4o-2024-08-06, gemini-1.5-pro
    Choice of foundation models is a free design choice that affects all results.
  • in-context examples = a few examples (not specified)
    The few-shot GoT examples are hand-selected; the ablation shows they matter.
axioms (4)
  • domain assumption The VLM's visual feedback is a reliable proxy for task progress.
    The whole upper-level loop depends on this; the paper even notes a case where it likely failed (Section V-C.2).
  • domain assumption The text-attributed graph generated by the LLM accurately decomposes the task.
    Section IV-C.2 states LLMs may generate incorrect outputs and heuristic rules are used, but correctness is not verified.
  • standard math The MDP formulation and the bi-level chain rule motivate the design.
    Section III presents a formal bi-level objective, but it is never actually optimized; it is a metaphor.
  • domain assumption LLM-generated reward code is syntactically and semantically valid.
    The framework relies on generated Python code running without error; no verification is described.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Reward Evolution with Graph-of-Thoughts: A Bi-Level Language Model Framework for Reinforcement Learning." pith.science (2026). https://pith.science/paper/VPQV2IUR

@misc{pith2026250916136,
  author       = {Pith},
  title        = {Pith review of: Reward Evolution with Graph-of-Thoughts: A Bi-Level Language Model Framework for Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VPQV2IUR}},
  note         = {Machine review of arXiv:2509.16136}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Designing effective reward functions remains a major challenge in reinforcement learning (RL), often requiring considerable human expertise and iterative refinement. Recent advances leverage Large Language Models (LLMs) for automated reward design, but these approaches are limited by hallucinations, reliance on human feedback, and challenges with handling complex, multi-step tasks. In this work, we introduce Reward Evolution with Graph-of-Thoughts (RE-GoT), a novel bi-level framework that enhances LLMs with structured graph-based reasoning and integrates Visual Language Models (VLMs) for automated rollout evaluation. RE-GoT first decomposes tasks into text-attributed graphs, enabling comprehensive analysis and reward function generation, and then iteratively refines rewards using visual feedback from VLMs without human intervention. Extensive experiments on 10 RoboGen and 4 ManiSkill2 tasks demonstrate that RE-GoT consistently outperforms existing LLM-based baselines. On RoboGen, our method improves average task success rates by 32.25%, with notable gains on complex multi-step tasks. On ManiSkill2, RE-GoT achieves an average success rate of 93.73% across four diverse manipulation tasks, significantly surpassing prior LLM-based approaches and even exceeding expert-designed rewards. Our results indicate that combining LLMs and VLMs with graph-of-thoughts reasoning provides a scalable and effective solution for autonomous reward evolution in RL.

Figures

Figures reproduced from arXiv: 2509.16136 by Changwei Yao, Chen Li, Marios Savvides, Xinzi Liu.

Figure 1
Figure 1. Figure 1: Conceptual illustration of GoT. (a) Three general GoT examples, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the RE-GoT framework. The upper-level evaluates rollout videos using VLMs to provide visual feedback, while the lower-level [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Evaluation environments. Ten tasks from RoboGen on the left: [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Example of the text-attributed graph for Press the Start Button, [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Success Rate & Average Episode Length vs Exploration Steps on four ManiSkill2 tasks. The solid lines represent the mean, while the shaded [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Effect of different prompting strategies on performance. LLMs [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. SolarChain-Eval: A Physics-Constrained Benchmark for Trustworthy Economic Agents in Decentralized Energy Markets

    cs.AI 2026-07 conditional novelty 6.0

    A physics-constrained benchmark for decentralized energy-market governance shows that RL agents exploit invalid generation when physics penalties are removed, and LLM-based oversight improves auditability but cannot c...

Reference graph

Works this paper leans on

34 extracted references · 16 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Learning to walk in minutes using massively parallel deep reinforcement learning,

    N. Rudin, D. Hoeller, P. Reist, and M. Hutter, “Learning to walk in minutes using massively parallel deep reinforcement learning,” in Conference on Robot Learning. PMLR, 2022, pp. 91–100

    2022

  2. [2]

    Learning complex dexterous manipulation with deep reinforcement learning and demonstrations,

    A. Rajeswaran, V . Kumar, A. Gupta, G. Vezzani, J. Schulman, E. Todorov, and S. Levine, “Learning complex dexterous manipulation with deep reinforcement learning and demonstrations,”arXiv preprint arXiv:1709.10087, 2017

    Pith/arXiv arXiv 2017

  3. [3]

    Scalable deep reinforcement learning for vision-based robotic manipulation,

    D. Kalashnikov, A. Irpan, P. Pastor, J. Ibarz, A. Herzog, E. Jang, D. Quillen, E. Holly, M. Kalakrishnan, V . Vanhouckeet al., “Scalable deep reinforcement learning for vision-based robotic manipulation,” in Conference on robot learning. PMLR, 2018, pp. 651–673

    2018

  4. [4]

    Maximum entropy deep inverse reinforcement learning,

    M. Wulfmeier, P. Ondruska, and I. Posner, “Maximum entropy deep inverse reinforcement learning,”arXiv preprint arXiv:1507.04888, 2015

    Pith/arXiv arXiv 2015

  5. [5]

    Generative adversarial imitation learning,

    J. Ho and S. Ermon, “Generative adversarial imitation learning,” Advances in neural information processing systems, vol. 29, 2016

    2016

  6. [6]

    Pebble: Feedback-efficient interac- tive reinforcement learning via relabeling experience and unsupervised pre-training,

    K. Lee, L. Smith, and P. Abbeel, “Pebble: Feedback-efficient interac- tive reinforcement learning via relabeling experience and unsupervised pre-training,”arXiv preprint arXiv:2106.05091, 2021

    Pith/arXiv arXiv 2021

  7. [7]

    Deep reinforcement learning from human preferences,

    P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei, “Deep reinforcement learning from human preferences,” Advances in neural information processing systems, vol. 30, 2017

    2017

  8. [8]

    Eureka: Human- level reward design via coding large language models,

    Y . J. Ma, W. Liang, G. Wang, D.-A. Huang, O. Bastani, D. Ja- yaraman, Y . Zhu, L. Fan, and A. Anandkumar, “Eureka: Human- level reward design via coding large language models,”arXiv preprint arXiv:2310.12931, 2023

    Pith/arXiv arXiv 2023

  9. [9]

    Text2reward: Reward shaping with language models for reinforcement learning,

    T. Xie, S. Zhao, C. H. Wu, Y . Liu, Q. Luo, V . Zhong, Y . Yang, and T. Yu, “Text2reward: Reward shaping with language models for reinforcement learning,”arXiv preprint arXiv:2309.11489, 2023

    Pith/arXiv arXiv 2023

  10. [10]

    Llms will always hallucinate, and we need to live with this,

    S. Banerjee, A. Agarwal, and S. Singla, “Llms will always hallucinate, and we need to live with this,” inIntelligent Systems Conference. Springer, 2025, pp. 624–648

    2025

  11. [11]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V . Le, D. Zhouet al., “Chain-of-thought prompting elicits reasoning in large language models,”Advances in neural information processing systems, vol. 35, pp. 24 824–24 837, 2022

    2022

  12. [12]

    Chain of thought empowers transformers to solve inherently serial problems,

    Z. Li, H. Liu, D. Zhou, and T. Ma, “Chain of thought empowers transformers to solve inherently serial problems,”arXiv preprint arXiv:2402.12875, vol. 1, 2024

    Pith/arXiv arXiv 2024

  13. [13]

    Compositional chain- of-thought prompting for large multimodal models,

    C. Mitra, B. Huang, T. Darrell, and R. Herzig, “Compositional chain- of-thought prompting for large multimodal models,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, 2024, pp. 14 420–14 431

    2024

  14. [14]

    Graph of thoughts: Solving elaborate problems with large language models,

    M. Besta, N. Blach, A. Kubicek, R. Gerstenberger, M. Podstawski, L. Gianinazzi, J. Gajda, T. Lehmann, H. Niewiadomski, P. Nyczyk et al., “Graph of thoughts: Solving elaborate problems with large language models,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 16, 2024, pp. 17 682–17 690

    2024

  15. [15]

    Got: Effective graph-of-thought reasoning in language models,

    Y . Yao, Z. Li, and H. Zhao, “Got: Effective graph-of-thought reasoning in language models,” inFindings of the Association for Computational Linguistics: NAACL 2024, 2024, pp. 2901–2921

    2024

  16. [16]

    Auto mc-reward: Automated dense reward design with large language models for minecraft,

    H. Li, X. Yang, Z. Wang, X. Zhu, J. Zhou, Y . Qiao, X. Wang, H. Li, L. Lu, and J. Dai, “Auto mc-reward: Automated dense reward design with large language models for minecraft,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 16 426–16 435

    2024

  17. [17]

    Unpack- ing reward shaping: Understanding the benefits of reward engineering on sample complexity,

    A. Gupta, A. Pacchiano, Y . Zhai, S. Kakade, and S. Levine, “Unpack- ing reward shaping: Understanding the benefits of reward engineering on sample complexity,”Advances in Neural Information Processing Systems, vol. 35, pp. 15 281–15 295, 2022

    2022

  18. [18]

    A. D. Laud,Theory and application of reward shaping in reinforce- ment learning. University of Illinois at Urbana-Champaign, 2004

    2004

  19. [19]

    Where do rewards come from,

    S. Singh, R. L. Lewis, and A. G. Barto, “Where do rewards come from,” inProceedings of the annual conference of the cognitive science society. Cognitive Science Society, 2009, pp. 2601–2606

    2009

  20. [20]

    Algorithms for inverse reinforcement learning

    A. Y . Ng, S. Russellet al., “Algorithms for inverse reinforcement learning.” inIcml, vol. 1, no. 2, 2000, p. 2

    2000

  21. [21]

    Maximum entropy inverse reinforcement learning

    B. D. Ziebart, A. L. Maas, J. A. Bagnell, A. K. Deyet al., “Maximum entropy inverse reinforcement learning.” inAaai, vol. 8. Chicago, IL, USA, 2008, pp. 1433–1438

    2008

  22. [22]

    Learning robust rewards with adversarial inverse reinforcement learning,

    J. Fu, K. Luo, and S. Levine, “Learning robust rewards with adversarial inverse reinforcement learning,”arXiv preprint arXiv:1710.11248, 2017

    Pith/arXiv arXiv 2017

  23. [23]

    Reward learning from human preferences and demonstrations in atari,

    B. Ibarz, J. Leike, T. Pohlen, G. Irving, S. Legg, and D. Amodei, “Reward learning from human preferences and demonstrations in atari,”Advances in neural information processing systems, vol. 31, 2018

    2018

  24. [24]

    Vision-language models as success detectors,

    Y . Du, K. Konyushkova, M. Denil, A. Raju, J. Landon, F. Hill, N. de Freitas, and S. Cabi, “Vision-language models as success detectors,”arXiv preprint arXiv:2303.07280, 2023

    Pith/arXiv arXiv 2023

  25. [25]

    Guiding pretraining in reinforcement learning with large language models,

    Y . Du, O. Watkins, Z. Wang, C. Colas, T. Darrell, P. Abbeel, A. Gupta, and J. Andreas, “Guiding pretraining in reinforcement learning with large language models,” inInternational Conference on Machine Learning. PMLR, 2023, pp. 8657–8677

    2023

  26. [26]

    Minedojo: Building open- ended embodied agents with internet-scale knowledge,

    L. Fan, G. Wang, Y . Jiang, A. Mandlekar, Y . Yang, H. Zhu, A. Tang, D.-A. Huang, Y . Zhu, and A. Anandkumar, “Minedojo: Building open- ended embodied agents with internet-scale knowledge,”Advances in Neural Information Processing Systems, vol. 35, pp. 18 343–18 362, 2022

    2022

  27. [27]

    Language-driven representation learning for robotics,

    S. Karamcheti, S. Nair, A. S. Chen, T. Kollar, C. Finn, D. Sadigh, and P. Liang, “Language-driven representation learning for robotics,” arXiv preprint arXiv:2302.12766, 2023

    Pith/arXiv arXiv 2023

  28. [28]

    Reward design with language models,

    M. Kwon, S. M. Xie, K. Bullard, and D. Sadigh, “Reward design with language models,”arXiv preprint arXiv:2303.00001, 2023

    Pith/arXiv arXiv 2023

  29. [29]

    Language to rewards for robotic skill synthesis,

    W. Yu, N. Gileadi, C. Fu, S. Kirmani, K.-H. Lee, M. G. Arenas, H.- T. L. Chiang, T. Erez, L. Hasenclever, J. Humpliket al., “Language to rewards for robotic skill synthesis,”arXiv preprint arXiv:2306.08647, 2023

    Pith/arXiv arXiv 2023

  30. [30]

    A large lan- guage model-driven reward design framework via dynamic feedback for reinforcement learning,

    S. Sun, R. Liu, J. Lyu, J.-W. Yang, L. Zhang, and X. Li, “A large lan- guage model-driven reward design framework via dynamic feedback for reinforcement learning,”arXiv preprint arXiv:2410.14660, 2024

    Pith/arXiv arXiv 2024

  31. [31]

    Learning reward for robot skills using large language models via self-alignment,

    Y . Zeng, Y . Mu, and L. Shao, “Learning reward for robot skills using large language models via self-alignment,”arXiv preprint arXiv:2405.07162, 2024

    Pith/arXiv arXiv 2024

  32. [32]

    Robogen: Towards unleashing infinite data for automated robot learning via generative simulation,

    Y . Wang, Z. Xian, F. Chen, T.-H. Wang, Y . Wang, K. Fragkiadaki, Z. Erickson, D. Held, and C. Gan, “Robogen: Towards unleashing infinite data for automated robot learning via generative simulation,” arXiv preprint arXiv:2311.01455, 2023

    Pith/arXiv arXiv 2023

  33. [33]

    Language-model- assisted bi-level programming for reward learning from internet videos,

    H. Mahesheka, Z. Xie, Z. Wang, and W. Jin, “Language-model- assisted bi-level programming for reward learning from internet videos,”arXiv preprint arXiv:2410.09286, 2024

    Pith/arXiv arXiv 2024

  34. [34]

    Maniskill2: A unified benchmark for generalizable manipulation skills,

    J. Gu, F. Xiang, X. Li, Z. Ling, X. Liu, T. Mu, Y . Tang, S. Tao, X. Wei, Y . Yaoet al., “Maniskill2: A unified benchmark for generalizable manipulation skills,”arXiv preprint arXiv:2302.04659, 2023

    Pith/arXiv arXiv 2023

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.