REVIEW 41 cited by
Language to Rewards for Robotic Skill Synthesis
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) have demonstrated exciting progress in acquiring diverse new capabilities through in-context learning, ranging from logical reasoning to code-writing. Robotics researchers have also explored using LLMs to advance the capabilities of robotic control. However, since low-level robot actions are hardware-dependent and underrepresented in LLM training corpora, existing efforts in applying LLMs to robotics have largely treated LLMs as semantic planners or relied on human-engineered control primitives to interface with the robot. On the other hand, reward functions are shown to be flexible representations that can be optimized for control policies to achieve diverse tasks, while their semantic richness makes them suitable to be specified by LLMs. In this work, we introduce a new paradigm that harnesses this realization by utilizing LLMs to define reward parameters that can be optimized and accomplish variety of robotic tasks. Using reward as the intermediate interface generated by LLMs, we can effectively bridge the gap between high-level language instructions or corrections to low-level robot actions. Meanwhile, combining this with a real-time optimizer, MuJoCo MPC, empowers an interactive behavior creation experience where users can immediately observe the results and provide feedback to the system. To systematically evaluate the performance of our proposed method, we designed a total of 17 tasks for a simulated quadruped robot and a dexterous manipulator robot. We demonstrate that our proposed method reliably tackles 90% of the designed tasks, while a baseline using primitive skills as the interface with Code-as-policies achieves 50% of the tasks. We further validated our method on a real robot arm where complex manipulation skills such as non-prehensile pushing emerge through our interactive system.
Forward citations
Cited by 41 Pith papers
-
The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
The AI Scientist framework enables LLMs to independently conduct the full scientific process from idea generation to paper writing and review, demonstrated across three ML subfields with papers costing under $15 each.
-
ENPIRE: Agentic Robot Policy Self-Improvement in the Real World
ENPIRE supplies four modules (Environment, Policy Improvement, Rollout, Evolution) that turn real-world robot training into an autonomous optimization loop driven by coding agents.
-
Enhanced LLM Reasoning by Optimizing Reward Functions with Search-Driven Reinforcement Learning
Iterative search over reward functions with ranked feedback in GRPO training improves LLM math reasoning, achieving F1 of 0.795 on GSM8K versus 0.609 for baseline.
-
Enhanced LLM Reasoning by Optimizing Reward Functions with Search-Driven Reinforcement Learning
Iterative LLM-driven search over reward functions, screened via GRPO on GSM8K, raises F1 from 0.609 baseline to 0.795 with ensembles on Llama-3.2-3B.
-
VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models
VoxPoser uses LLMs to compose 3D value maps via VLM interaction for model-based synthesis of robust robot trajectories on open-set language-specified manipulation tasks.
-
Weights or Skills? A Survey of Robot-Learning Techniques: from Action-Predicting Weights to Robots that Write their Own Skills
A taxonomy of robot learning on a weights-versus-skills axis, with a five-rung self-improvement ladder whose top cell (feedback plus memory plus search) holds only a few recent systems.
-
LEACL: LLM-Enhanced Automatic Curriculum Learning for Reinforcement Learning in Long-Horizon Manipulation Tasks
LLMs generate subtask decompositions and ACL task spaces so sparse-reward RL solves long-horizon manipulation better than dense human rewards on five LIBERO tasks.
-
Breaking D\'ej\`a Vu: Independent Auditing of Visual Place Recognition through Vision-Language Reasoning
VLM-based post-retrieval auditing of visual place recognition raises recall@1 by 13.6% on average while cutting false accepts to 12% and holding precision above 95%.
-
Breaking D\'ej\`a Vu: Independent Auditing of Visual Place Recognition through Vision-Language Reasoning
A zero-shot VLM auditor that accepts or rejects top-1 VPR matches is reported to improve recall@1 by 13.6% on average, but the gain appears to depend on a nonstandard filtered-recall definition.
-
LocalNav: Distilling Frontier VLMs and Embodied RL for On-Device Object Goal Navigation
Distillation from frontier VLMs plus E-RLVR regularization produces a 4B local model that achieves 34.5% SR on OVON while cutting inference latency by 82.8%.
-
Beyond Pixels: Learning Invariant Rewards for Real-World Robotics From a Few Demonstrations
A framework learns invariant symbolic reward functions from few demonstrations that generalize zero-shot to variations in robotic manipulation tasks.
-
ERFSL: An Efficient Reward Function Searcher via Language Models for Custom-Environment Multi-Objective Optimization (Student Abstract)
ERFSL generates and optimizes LLM-based reward functions for custom multi-objective RL, correcting codes in one iteration and converging weights in 5.2 iterations on average even from 500x errors.
-
EvoNav: Evolutionary Reward Function Design for Robot Navigation with Large Language Models
EvoNav automates the design of reward functions for RL robot navigation by evolving LLM proposals through a three-stage cheap-to-expensive evaluation process and claims better policies than hand-crafted or prior autom...
-
Improving Zero-Shot Offline RL via Behavioral Task Sampling
Extracting task vectors from the offline dataset for policy training improves zero-shot offline RL performance by an average of 20% over random sampling baselines.
-
Sumo: Dynamic and Generalizable Whole-Body Loco-Manipulation
Test-time steering of pre-trained whole-body policies via sample-based planning lets legged robots generalize dynamic loco-manipulation to varied heavy objects and tasks without additional training or tuning.
-
PRISM-XR: Empowering Privacy-Aware XR Collaboration with Multimodal Large Language Models
PRISM-XR adds edge-based sensitive-data filtering and quick registration to MLLM-driven XR collaboration, reporting 90% request accuracy, sub-0.3s registration, and over 90% sensitive-object filtering in a 28-person study.
-
Debate2Create: Robot Co-design via Multi-Agent LLM Debate
A structured multi-agent LLM debate grounded in simulation is proposed for co-designing robot morphology and reward, but the body reports only a single Ant experiment.
-
ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution
ShinkaEvolve improves sample efficiency in LLM-driven program evolution via parent sampling, code novelty rejection-sampling, and bandit LLM ensemble selection, achieving new SOTA circle packing with 150 samples and g...
-
ZapGPT: Free-form Language Prompting for Simulated Cellular Control
ZapGPT evolves a prompt-to-intervention model against a vision-language evaluator and reports that single-prompt training generalizes to unseen free-form language prompts.
-
Exploratory Retrieval-Augmented Planning For Continual Embodied Instruction Following
ExRAP couples LLM planning with a temporal knowledge-graph memory and information-based exploration, improving success and efficiency for continual embodied instruction following.
-
Text2Touch: Tactile In-Hand Manipulation with LLM-Designed Reward Functions
First demonstration that LLM-generated reward functions using tactile sensing can outperform a human-engineered baseline for real-world in-hand rotation.
-
RoboInspector: Unveiling the Unreliability of Policy Code for LLM-enabled Robotic Manipulation
LLM-generated robot policy code is unreliable, with failures clustering into four behavior types that grow with task complexity and shrink with instruction detail; a failure-feedback retry improves success up to 35%.
-
GhostShell: Streaming LLM Function Calls for Concurrent Embodied Programming
A streaming XML function-token interface with multi-channel scheduling lets robots execute concurrent speech and motion while the LLM is still generating, reportedly beating native function calling 15/15 vs 6/15 on co...
-
CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models
CoT-VLA is a 7B VLA that generates future visual frames autoregressively as planning goals before actions, outperforming prior VLAs by 17% on real-world tasks and 6% in simulation.
-
Agent Workflow Memory
AWM induces reusable workflows from agent experiences and provides them selectively to improve success rates by 24.6% on Mind2Web and 51.1% on WebArena while reducing steps taken.
-
Language Models as Efficient Reward Function Searchers for Custom-Environment Multi-Objective Reinforcement
ERFSL uses LLMs to create per-requirement reward components, correct their code via a critic, and optimize weights with genetic-algorithm-style mutation and crossover driven by training logs, succeeding in a zero-shot...
-
Is Inter-Seed Cross-Play Enough? Evaluating the Robustness of Zero-Shot Coordination Algorithms to Implementation Details
For Other-Play in Yokai, agents trained with different implementation details coordinate across implementations about as well as across seeds, supporting inter-seed cross-play as a proxy for cross-implementation evaluation.
-
LLM-as-a-Verifier: A General-Purpose Verification Framework
Expecting over scoring-token logits yields continuous, scalable verification that improves agent trajectory selection and dense RL rewards across coding, robotics, and medical benchmarks.
-
LLM-as-a-Verifier: A General-Purpose Verification Framework
Computing the expectation over scoring-token logits instead of taking argmax enables verification to scale along granularity, repetition, and criteria decomposition, achieving state-of-the-art on four agentic benchmar...
-
VLM-AR3L: Vision-Language Models for Absolute and Relative Rewards in Reinforcement Learning
VLM-AR3L combines VLM-derived absolute and relative reward models to improve RL agent performance on control, manipulation, and Minecraft tasks over prior VLM reward methods.
-
VLM-AR3L: Vision-Language Models for Absolute and Relative Rewards in Reinforcement Learning
VLM-AR3L learns absolute and relative reward models from VLM preference labels to improve RL on control, manipulation, and Minecraft tasks.
-
Sakana Fugu Technical Report
Sakana Fugu trains LLM orchestrators using fine-tuning, evolutionary algorithms, and RL to build query-adaptive multi-agent scaffolds, claiming SOTA results on benchmarks including SWE-Bench Pro and GPQA-Diamond.
-
Improving Zero-Shot Offline RL via Behavioral Task Sampling
Extracting task vectors from offline data to define training task distributions improves zero-shot offline RL performance by an average of 20%.
-
ReinforceGen: Hybrid Skill Policies with Automated Data Generation and Reinforcement Learning
ReinforceGen uses imitation learning, RL fine-tuning of skill policies, and real-time pose replanning to reach over 80% success on five long-horizon Robosuite tasks from only 10 human demonstrations.
-
Reward Evolution with Graph-of-Thoughts: A Bi-Level Language Model Framework for Reinforcement Learning
RE-GoT combines graph-of-thoughts planning in LLMs with VLM feedback from rollout videos to automatically write and refine RL reward functions, beating prior LLM-based reward design on RoboGen and ManiSkill2.
-
EmbodiedUS-FS: Fast Slow Intelligence for Ultrasound Robotics
Introduces EmbodiedUS-FS, a fast-slow hierarchical controller for robotic ultrasound that uses intent parsing, task graphs, multimodal feedback, and a safety shield to raise success rates and cut violations under moti...
-
PIRS: Physics-Informed Reward Shaping for SAC-Based Building Energy Management
PIRS replaces ad-hoc temperature-deviation terms with the ISO 7730 PMV model inside a weighted multi-objective reward for SAC, yielding cost and carbon KPIs comparable to a manual baseline while outperforming non-phys...
-
Large Model Empowered Embodied AI: A Survey on Decision-Making and Embodied Learning
A review that categorizes large-model-empowered embodied AI into hierarchical and end-to-end decision-making, imitation and reinforcement learning, and world models.
-
Agent AI: Surveying the Horizons of Multimodal Interaction
The paper defines Agent AI as interactive multimodal systems that perceive grounded data and generate embodied actions, arguing this approach can mitigate hallucinations in foundation models.
-
Safe Embodied AI for Long-horizon Tasks: A Cross-layer Analysis of Robotic Manipulation
A structured literature survey of safety mechanisms in long-horizon robotic manipulation organized by intervention timing and strength of supporting evidence.
-
A Comprehensive Overview of Large Language Models
A survey paper providing an overview of Large Language Models, their background, and recent advances in the field.
Discussion (0). Sign in to comment.