Pith. sign in

REVIEW 20 cited by

Text2Reward: Reward Shaping with Language Models for Reinforcement Learning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2309.11489 v3 pith:XQU2KGED submitted 2023-09-20 cs.LG cs.AIcs.CLcs.RO

Tianbao Xie , Siheng Zhao , Chen Henry Wu , Yitao Liu , Qian Luo , Victor Zhong , Yanchao Yang , Tao Yu This is my paper · ORCID
classification cs.LGcs.AIcs.CLcs.RO
keywords rewardtext2rewardcodesdensefunctionslanguagelocomotionpolicies
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Designing reward functions is a longstanding challenge in reinforcement learning (RL); it requires specialized knowledge or domain data, leading to high costs for development. To address this, we introduce Text2Reward, a data-free framework that automates the generation and shaping of dense reward functions based on large language models (LLMs). Given a goal described in natural language, Text2Reward generates shaped dense reward functions as an executable program grounded in a compact representation of the environment. Unlike inverse RL and recent work that uses LLMs to write sparse reward codes or unshaped dense rewards with a constant function across timesteps, Text2Reward produces interpretable, free-form dense reward codes that cover a wide range of tasks, utilize existing packages, and allow iterative refinement with human feedback. We evaluate Text2Reward on two robotic manipulation benchmarks (ManiSkill2, MetaWorld) and two locomotion environments of MuJoCo. On 13 of the 17 manipulation tasks, policies trained with generated reward codes achieve similar or better task success rates and convergence speed than expert-written reward codes. For locomotion tasks, our method learns six novel locomotion behaviors with a success rate exceeding 94%. Furthermore, we show that the policies trained in the simulator with our method can be deployed in the real world. Finally, Text2Reward further improves the policies by refining their reward functions with human feedback. Video results are available at https://text-to-reward.github.io/ .

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 20 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Online Preference-based Reinforcement Learning with Self-augmented Feedback from Large Language Model

    cs.LG 2024-12 conditional novelty 7.0 of 10

    RL-SaLLM-F uses an LLM to both label trajectory preferences and generate self-augmented imagined trajectories, achieving MetaWorld success rates comparable to privileged-reward teachers without using any privileged in...

  2. Weights or Skills? A Survey of Robot-Learning Techniques: from Action-Predicting Weights to Robots that Write their Own Skills

    cs.RO 2026-08 conditional novelty 6.0 of 10

    A taxonomy of robot learning on a weights-versus-skills axis, with a five-rung self-improvement ladder whose top cell (feedback plus memory plus search) holds only a few recent systems.

  3. EgoExo-Con: Exploring View-Invariant Video Temporal Understanding

    cs.CV 2025-10 conditional novelty 6.0 of 10

    Most Video-LLMs answer temporal questions far less consistently when the same event is shown from ego and exo views, and a GRPO variant with a reasoning-similarity reward partially closes the gap.

  4. Text2Touch: Tactile In-Hand Manipulation with LLM-Designed Reward Functions

    cs.RO 2025-09 conditional novelty 6.0 of 10

    First demonstration that LLM-generated reward functions using tactile sensing can outperform a human-engineered baseline for real-world in-hand rotation.

  5. CoCA: Step-level Reward for Free in RL-based T2I Diffusion Model Fine-tuning

    cs.CV 2025-05 conditional novelty 6.0 of 10

    CoCA redistributes a single final image reward across denoising steps using cosine similarity between intermediate and final latents, improving RL fine-tuning sample efficiency on four human preference rewards.

  6. A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewards

    cs.RO 2025-02 conditional novelty 6.0 of 10

    IKER uses VLM-generated keypoint rewards to train manipulation policies in simulation that transfer to a real robot, enabling multi-step tasks and replanning.

  7. The Synergy of LLMs & RL Unlocks Offline Learning of Generalizable Language-Conditioned Policies with Low-fidelity Data

    cs.CL 2024-12 conditional novelty 6.0 of 10

    TEDUO learns instruction-following policies from unlabeled offline transitions by combining LLM-generated reward labels, per-goal offline Q-learning, and supervised fine-tuning of a base LLM, enabling zero-shot genera...

  8. A MARL Centered Reference Architecture for Large Language Model Augmentation in Smart Manufacturing

    cs.AI 2026-08 conditional novelty 5.0 of 10

    The paper proposes a MARL-centered three-layer reference architecture for LLM augmentation in smart manufacturing, with a conditional allocation: MARL for frequent coordination, LLMs for semantic, reward, and planning roles.

  9. Meta-Learned Reward Shaping for Reinforcement Learning from Human Feedback

    cs.LG 2026-07 reject novelty 5.0 of 10

    Training a small multi-task reward-shaping network and adding it to the RLHF reward is claimed to improve LLaMA-3-8B alignment across four benchmarks, but the supporting theory is not established.

  10. LLM-as-a-Verifier: A General-Purpose Verification Framework

    cs.AI 2026-07 conditional novelty 5.0 of 10

    Expecting over scoring-token logits yields continuous, scalable verification that improves agent trajectory selection and dense RL rewards across coding, robotics, and medical benchmarks.

  11. Efficient Onboard Vision-Language Inference in UAV-Enabled Low-Altitude Economy Networks via LLM-Enhanced Optimization

    cs.LG 2025-10 conditional novelty 5.0 of 10

    A hierarchical ARPO+LLaRA framework that jointly sets image resolution, transmit power, and UAV trajectory reduces simulated latency for onboard VLM inference in low-altitude economy networks.

  12. Reward Evolution with Graph-of-Thoughts: A Bi-Level Language Model Framework for Reinforcement Learning

    cs.RO 2025-09 conditional novelty 5.0 of 10

    RE-GoT combines graph-of-thoughts planning in LLMs with VLM feedback from rollout videos to automatically write and refine RL reward functions, beating prior LLM-based reward design on RoboGen and ManiSkill2.

  13. HCRMP: A LLM-Hinted Contextual Reinforcement Learning Framework for Autonomous Driving

    cs.RO 2025-05 conditional novelty 5.0 of 10

    The HCRMP planner feeds LLM semantic hints into state representation and critic weighting instead of letting the LLM decide actions, reporting better CARLA driving metrics.

  14. TREND: Tri-teaching for Robust Preference-based Reinforcement Learning with Demonstrations

    cs.RO 2025-05 conditional novelty 5.0 of 10

    TREND combines cyclic small-loss selection among three reward models with few-shot demonstration pretraining and regularization, yielding robust policies on Meta-World tasks under up to 40 percent label noise.

  15. Learning Explainable Dense Reward Shapes via Bayesian Optimization

    cs.LG 2025-04 conditional novelty 5.0 of 10

    Reward shaping based on SHAP/LIME token attributions, with weights optimized by Bayesian optimization, improves RLHF training speed and downstream win rates while preserving the optimal policy.

  16. From Laws to Motivation: Guiding Exploration through Law-Based Reasoning and Rewards

    cs.LG 2024-11 conditional novelty 5.0 of 10

    Agents that receive LLM-extracted environment rules as context or rewards explore more purposefully and score higher in the Crafter game.

  17. Improving the Data-efficiency of Reinforcement Learning by Warm-starting with LLM

    cs.LG 2025-05 conditional novelty 4.0 of 10

    Warm-starting RL with a small LLM-generated buffer and then fine-tuning online (LORO) improves cumulative reward over pure RL in six Gym environments, though the theoretical guarantee is borrowed and the coverage assu...

  18. CLIP-RLDrive: Human-Aligned Autonomous Driving via CLIP-Based Reward Shaping in Reinforcement Learning

    cs.RO 2024-12 reject novelty 4.0 of 10

    In a simulated unsignalized intersection, CLIP-based reward shaping improves DQN success to 96%, but it degrades PPO and the evaluation lacks seeds, error bars, and released code.

  19. Reward Models in Deep Reinforcement Learning: A Survey

    cs.LG 2025-06 conditional novelty 3.0 of 10

    A structured survey of reward modeling in deep RL, proposing a three-axis taxonomy and reviewing applications and evaluation methods.

  20. Multi-agent Embodied AI: Advances and Future Directions

    cs.AI 2025-05 conditional novelty 3.0 of 10

    A survey that maps multi-agent embodied AI methods and benchmarks across control, learning, and generative-model categories, and lists open challenges.

Pith tools