REVIEW 9 cited by
Correcting Robot Plans with Natural Language Feedback
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
When humans design cost or goal specifications for robots, they often produce specifications that are ambiguous, underspecified, or beyond planners' ability to solve. In these cases, corrections provide a valuable tool for human-in-the-loop robot control. Corrections might take the form of new goal specifications, new constraints (e.g. to avoid specific objects), or hints for planning algorithms (e.g. to visit specific waypoints). Existing correction methods (e.g. using a joystick or direct manipulation of an end effector) require full teleoperation or real-time interaction. In this paper, we explore natural language as an expressive and flexible tool for robot correction. We describe how to map from natural language sentences to transformations of cost functions. We show that these transformations enable users to correct goals, update robot motions to accommodate additional user preferences, and recover from planning errors. These corrections can be leveraged to get 81% and 93% success rates on tasks where the original planner failed, with either one or two language corrections. Our method makes it possible to compose multiple constraints and generalizes to unseen scenes, objects, and sentences in simulated environments and real-world environments.
Forward citations
Cited by 9 Pith papers
-
Freeform Preference Learning for Robotic Manipulation
Language-conditioned multi-axis human preferences yield denser rewards and steerable robot policies that outperform sparse and binary-preference baselines by 38 points on long-horizon manipulation.
-
RoboStream: Weaving Spatio-Temporal Reasoning with Memory in Vision-Language Models for Robotics
Training-free STF-Tokens plus a Causal Spatio-Temporal Graph let VLMs keep object permanence and action history, raising long-horizon robotic manipulation success far above reactive baselines.
-
FailSafe: Reasoning and Recovery from Failures in Vision-Language-Action Models
A VLM trained on auto-generated failure trajectories with executable correction actions helps VLA models detect and fix manipulation errors, lifting success rates by up to 22.6 percentage points.
-
OVITA: Open-Vocabulary Interpretable Trajectory Adaptations
OVITA uses LLM-generated Python code, a QP safety module, and user feedback to adapt robot trajectories from open-vocabulary natural language instructions, with an 81.4% user-study success rate.
-
"Stack It Up!": 3D Stable Structure Generation from 2D Hand-drawn Sketch
StackItUp converts 2D hand-drawn sketches into stable 3D block arrangements using a symbolic relation graph and diffusion-based block pose generation.
-
A Human-in-the-loop Approach to Robot Action Replanning through LLM Common-Sense Reasoning
A human-in-the-loop system lets users refine vision-generated robot behavior trees through natural-language requests to GPT-4o, correcting errors and adapting plans before execution.
-
Language-Conditioned Open-Vocabulary Mobile Manipulation with Pretrained Models
A robot system that combines GPT-4, vision-language maps, and a CLIPort-style network follows free-form household commands across rooms in simulation, reaching 10.2% average success on unseen tasks and beating two bas...
-
Prompt Informed Reinforcement Learning for Visual Coverage Path Planning
Adding GPT-3.5 semantic recommendations as an auxiliary reward term to PPO improves visual coverage and reduces redundancy for simulated aerial coverage path planning, according to reported experiments.
-
VerifyLLM: LLM-Based Pre-Execution Task Plan Verification for Robots
An LLM plus LTL-based verification module that reorders, inserts, and removes steps in household robot plans, reporting reduced ordering errors but with weak experimental support.
Discussion (0). Sign in to comment.