Pith. sign in

REVIEW 4 major objections 6 minor 60 references

A robot can bootstrap its own manipulation skills from zero human demos by practicing, reusing, and consolidating experience into muscle memory.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-30 20:24 UTC pith:6MQMFKDZ

load-bearing objection Solid real-robot systems integration of VLM primitives, one-shot transfer, and π0.5 under a monitor/replan loop; the zero-demo claim is real for data collection but rests on human-designed primitives and four curated tasks. the 4 major comments →

arxiv 2607.26809 v1 pith:6MQMFKDZ submitted 2026-07-29 cs.RO

Practice Makes Policies: Bootstrapping and Consolidating Robotic Capabilities from Zero Human Demonstrations

Jialiang Li , Yuhan Wang , Haojun Li , Gaojing Zhang , Yangtian Ye , Qipeng Liu , Haotian Liang , Wenzhao Lian This is my paper
classification cs.RO
keywords robotic manipulationself-improving agentshierarchical orchestrationzero-demonstration learningvisuomotor policiesexperience reuseembodied AI
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that general-purpose robot manipulation should not be learned as a static skill set but should evolve the way human practice turns deliberate effort into muscle memory. It introduces HERO, a hierarchical agent that starts with vision-language reasoning over a small set of primitive actions, reuses successful trajectories by transferring them to new scenes, and eventually trains closed-loop visuomotor policies from the data it collected itself. An orchestrator plans subtasks, chooses which capability to use, monitors outcomes, synthesizes reverse tasks to reset the scene, and decides when to fall back or replan. On four real-world tabletop tasks the system gathers hundreds of successful episodes with only about a second of human help per subtask on average, then deploys the evolved stack at 86% mean success—higher than any single capability or partial combination. The claim that matters is practical: robots can close the loop from reasoning to data to policy without a human demonstration pipeline, if reasoning, reuse, and reflex are scheduled together.

Core claim

HERO shows that a robot can autonomously evolve manipulation capability from zero human demonstrations by orchestrating three layers—heuristic VLM-grounded primitives, one-shot exemplar motion transfer, and distilled closed-loop visuomotor policies—coupled with reverse-task scene reset and adaptive scheduling, so experience is collected, reused, and consolidated into reliable real-world behavior.

What carries the argument

The HERO hierarchical orchestrator: it runs a H→E→R evolution loop (heuristic bootstrap → exemplar accelerator → reflexive policy) for autonomous data collection with reverse-task reset, and an R→E→H execution loop that prefers learned policies, falls back to transfer, then to zero-shot reasoning, with continuous monitoring and replanning.

Load-bearing premise

The system assumes a fixed, human-designed set of primitive skills and reliable reverse-task scene resets are enough to bootstrap and keep feeding the experience distribution outside a few curated tasks.

What would settle it

Run the same zero-demo pipeline on a new long-horizon task whose needed contacts are not expressible as grasp/place/push/pull with the current keypoint and geometric constraints, or where reverse reset routinely fails: if data collection stalls or success collapses without new human-designed primitives, the central claim fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Robotic data pipelines can shift from teleoperation-first to self-play with sparse human recovery.
  • Once enough successful trajectories exist, closed-loop policies should handle most subtasks while reasoning and transfer cover the long tail.
  • Orchestrated reverse tasks become a practical mechanism for continuous real-world collection without constant manual reset.
  • Capability scheduling (policy first, then reuse, then zero-shot) is a concrete recipe for balancing speed and robustness at deployment.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If reverse-task synthesis generalizes, the same loop could support multi-robot shared exemplar libraries that accelerate each new machine’s H→E→R climb.
  • The latency gap between VLM grounding and policy execution implies that open-world systems will stay hybrid for a long time: reasoning buys coverage, policies buy tempo.
  • Removing the fixed primitive vocabulary is the natural next stress test—open-ended skill discovery would turn HERO from a closed skill ladder into a truly open-ended practice engine.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents HERO, a hierarchical embodied agent that bootstraps and consolidates real-robot manipulation capabilities from zero human demonstrations. It organizes three capability layers—L1 heuristic VLM-guided primitive skills (grasp/place/push/pull with 3D grounding), L2 one-shot exemplar motion transfer (MT3-style registration), and L3 closed-loop visuomotor policies (π0.5)—under an agentic orchestrator. Two coupled loops are claimed: H→E→R autonomous evolution (bootstrap with L1, accelerate with L2, consolidate into L3, with orchestrator-synthesized reverse tasks for scene reset) and R→E→H adaptive execution (prefer L3, fall back to L2 then L1, with monitoring and replanning). On four real Franka tabletop tasks, the system collects 664 successful subtask episodes with ~1.03 s average human intervention per subtask, reports 94.5% orchestrator accept/reject agreement, and achieves 86.0% mean task success over 30 trials/task, outperforming single- and partial-layer ablations and no-replan variants (Table 1), with supporting analyses of collection efficiency, capability evolution (Fig. 5), latency (Table 2), and failure modes (Fig. 6).

Significance. If the results hold under the stated scope, this is a solid systems contribution to self-improving real-robot manipulation. The main strengths are (i) a concrete zero-demonstration data-collection loop with reverse-task reset and very low measured human intervention, (ii) a clear three-layer capability spectrum with dual scheduling directions (H→E→R vs R→E→H), and (iii) thorough internal empirical support: layer and replan ablations, monitoring confusion-style matrix, L1/L2 collection efficiency, evolution curves, latency breakdown, and a failure taxonomy. These go beyond many agentic robotics papers that stop at modular orchestration without closed-loop experience consolidation. The work does not introduce a new learning algorithm or a theory of skill discovery; its value is the integrated real-world pipeline and the empirical demonstration that reasoning, transfer, and reflexive policies can be scheduled as experience accumulates. That is significant for the self-improving robotics agenda, provided claims are bounded to the primitive vocabulary and reversible tabletop settings actually evaluated.

major comments (4)
  1. [Method §Heuristic Bootstrapper; Limitations; Abstract] Central claim vs. fixed primitive vocabulary (Method, Heuristic Bootstrapper / Fig. 2; Supplementary skill constraints; Limitations). The zero-demo evolution narrative is load-bearing on a human-designed L1 skill set (grasp/place/push/pull with VLM 2D keypoints, Depth-Anything-3 lifting, fixed TCP offsets, pre-approach/retract distances, and transition constraints) plus orchestrator-synthesized reverse tasks. All four evaluation tasks are covered exactly by these primitives and admit simple reversible resets. The Limitations section correctly notes that evolution starts from a predefined primitive space requiring human expertise, but the Abstract/Introduction frame open-world capability evolution from zero human demonstrations without that bound. Please either (a) tighten the claim to “zero teleoperated demonstrations within a fixed primitive vocabulary on reversible tabletop tasks,” or
  2. [Experiments; Table 1; Related Work] Missing external baselines (Experiments; Table 1). Table 1 and related ablations only compare HERO configurations (L1/L2/L3 subsets and replan on/off). Related Work cites RoboClaw, ASPIRE, ENPIRE, modular VLM agents, and end-to-end VLAs, but none appear as experimental baselines. For a systems paper whose contribution is autonomous evolution and reduced human intervention, at least one of the following is needed: (i) human-demonstration imitation learning with matched data volume/time, (ii) a static modular agent (L1-style) without evolution, or (iii) a published self-improving pipeline under comparable hardware. Internal ablations show the full stack helps; they do not show that HERO advances the external state of the art. Please add comparable baselines or explicitly reframe contributions as an internal orchestration study.
  3. [Method §H2E2R Autonomous Capability Evolution; Experiments §Autonomous Data Collection] Autonomous collection loop depends on reverse-task success that is only partially characterized (Method §H2E2R; Autonomous Data Collection). Reverse tasks achieve 84.3% average success and enable up to 46 consecutive cycles, which is a genuine strength, but failure modes of reverse synthesis/execution, distribution shift after failed resets, and human recovery protocol are not broken down by task. The 1.03 s human intervention per subtask is reported as an average without variance, intervention type taxonomy, or what fraction of cycles would halt without a human. Because continual H→E→R evolution assumes a restored scene distribution, please report reverse-task SR and intervention statistics per task, define the human recovery interface, and state how failed resets affect subsequent exemplar quality and L3 training data.
  4. [Experiments; Table 1; Figure 5] Statistical reporting and trial protocol (Table 1; Fig. 5; 30 trials/task). Success rates are point estimates without confidence intervals, standard errors, or multi-seed variation for L3 training. Fig. 5’s evolution curves likewise lack uncertainty. With n=30 per task and binary outcomes, intervals are material (e.g., distinguishing 86% full HERO from 82.7% L1+L3). Please add binomial CIs or bootstrap intervals for SR, clarify whether the 30 evaluation trials are fully held-out from the 30 collected trajectories per task, and state whether L3 was trained once or across seeds. This is load-bearing for the claim that full orchestration “substantially” outperforms partial stacks.
minor comments (6)
  1. [Method §Reflexive Policy (L3); Supplementary Training and Inference] Supplementary language-injection pathway for π0.5 (linear projection into Action Expert) is a non-trivial architectural change relative to the cited base model. Mention it briefly in the main L3 subsection and note whether ablations without injection were run, especially for color-conditioned tasks (Package Picking, Block Stacking).
  2. [Experiments §Execution Setup; Method §Exemplar Accelerator] LCS retrieval threshold is set to 1.0 after stop-word removal (Execution Setup), i.e., exact instruction match. Discuss how this limits “one-shot fast transfer” under paraphrased commands and whether softer thresholds were tried.
  3. [Figure 1; Figure 4] Figure 1 and several multi-panel figures are dense; ensure axis labels and the H2E2R/R2E2H acronyms are defined in captions on first use. Figure 4’s confusion-style matrix would benefit from explicit class definitions (successful vs failed vs call_human).
  4. [Throughout] Typos and spacing artifacts appear throughout (e.g., “performdiversetasks,” “Openadrawer,” missing spaces after periods in the compiled text). A full proofread pass is needed before camera-ready.
  5. [Experiments §Hardware Setup; Limitations] Hardware uses four RealSense D435i cameras with a specific split (three exterior for L1/L2 reconstruction; front+wrist for L3). State calibration procedure and whether depth for DA3 is monocular estimated or sensor depth, since Limitations cite sensitivity to depth noise.
  6. [Related Work §Embodied Agentic Self-Improvement] Related Work could more sharply contrast HERO’s reverse-task real-world reset with simulation-reset skill libraries (e.g., ASPIRE) in one sentence each, rather than listing systems.

Circularity Check

0 steps flagged

No significant circularity: empirical systems paper with measured success rates and ablations, not a self-defining derivation.

full rationale

HERO is an embodied systems architecture (L1 VLM primitives → L2 motion transfer → L3 π0.5 policies, coordinated by an orchestrator with reverse-task reset). Its central claims are experimental: autonomous collection of 664 episodes with ~1.03s human intervention per subtask, 86.0% mean SR across four real-robot tasks, and ablations showing full orchestration beats single/partial layers. Success is measured on held-out execution trials; L3 is trained on collected data then evaluated separately; monitoring accuracy is checked against actual outcomes (94.5%). There is no mathematical derivation in which a quantity is defined from the same quantity it purports to predict, no fitted parameter renamed as an independent prediction, no uniqueness theorem imported from overlapping authors, and no ansatz smuggled in via self-citation. External components (π0.5, MT3, SAM3, Depth-Anything-3, Gemini) are third-party. Closed-loop quality scoring of exemplars and orchestrator accept/reject logging are ordinary online bookkeeping, not circular reduction of the result to its inputs. Weaknesses (human-designed primitive vocabulary, reverse-task design, limited task suite) are assumption/generalization risks, not circularity. Score 0; steps empty.

Axiom & Free-Parameter Ledger

9 free parameters · 6 axioms · 2 invented entities

The central claim rests on engineering assumptions and external models more than on free physical constants. Capability evolution is only as strong as VLM keypoint grounding, depth/point-cloud fidelity, the closed four-skill interface, reverse-task reset quality, and the decision thresholds that gate retrieval, retries, and L3 termination. Invented structure is the HERO orchestration hierarchy itself; independent evidence is the robot trial table, not an external formal proof.

free parameters (9)
  • LCS exemplar retrieval threshold = 1.0
    Normalized word-level LCS must equal 1.0 after stopword removal for reuse; directly controls when L2 fires vs L1.
  • TCP/gripper z-offset for pose execution = 0.15 m
    Hand-measured 0.15 m offset along −z_ee to avoid collisions; changes all L1 end-effector targets.
  • Pre-approach / retract distance d = 0.15 m
    Fixed 0.15 m geometric pre-condition offset for skill safety and transitions.
  • Depth-ambiguity trigger for push/pull = 5 cm
    Ray-plane correction activates when keypoint depth difference exceeds 5 cm.
  • L2 quality score update rule = init 1.0; +0.1 / −0.3
    Exemplar q ∈ (0,2], +0.1 on successful reuse, −0.3 on failure, drop at q≤0; ranks which demos transfer.
  • GICP / registration gates = as listed in supplement
    Max correspondence 0.10 m, 50 iters, 1.0 s timeout, ≥500 correspondences, yaw deviation cap 15°, ±2 cm restarts; reject/accept transferred trajectories.
  • Layer retry limit before fallback = 3
    At most 3 consecutive retries per layer before orchestrator falls back.
  • L3 termination (convergence / max rounds) = 0.02 / 8 chunks / 25 rounds
    Stop when max consecutive joint diff <0.02 over latest 8 chunks, or after 25 inference rounds; defines successful L3 subtask completion.
  • π0.5 fine-tune schedule and language-injection head = 10k steps; W_lang ~2.1M params
    10k steps, batch 32, LR 5e-5→1e-6, new W_lang 2048×1024 trained from scratch; policy competence depends on these training choices and collected data volume (~30 success trajectories/task).
axioms (6)
  • domain assumption A contemporary VLM (Gemini-3-Flash) can decompose free-form language tasks and annotate reliable 2D semantic keypoints for grasp/place/push/pull on real RGB.
    Core of L1 zero-shot bootstrapping; failures WVA appear in the failure analysis.
  • domain assumption Multi-view RGB + Depth-Anything-3 + calibrated extrinsics yield 3D workspace geometry accurate enough for open-loop primitive execution and L2 pointclouds.
    Stated in Heuristic Bootstrapper; Limitations admits sensitivity to depth noise/occlusion; WGD failures counted.
  • ad hoc to paper Four human-specified primitive skills with fixed transition constraints span the manipulation behaviors needed for the evaluated open-world tasks.
    Method defines only grasp/place/push/pull; Limitations says this limits autonomous discovery of new skill abstractions.
  • ad hoc to paper After a forward task, the orchestrator can synthesize and execute a reverse task that restores a usable scene distribution for continual collection.
    H2E2R evolution loop; reported 84.3% reverse success among reverse cases—assumed adequate for autonomy claims.
  • domain assumption Rigid 4-DoF object-centric warp of a successful trajectory (centroid + PointNet++ init + GICP) is a valid one-shot transfer for new instances of the same instruction class.
    L2 builds on MT3; WMT/WOS failures show the assumption is approximate.
  • domain assumption Filtered successful L1/L2 trajectories are i.i.d.-enough training signal for π0.5 to become the default reliable executor (R→E→H).
    Reflexive Policy and evolution figures; WPE remains the largest execution failure mode.
invented entities (2)
  • HERO hierarchical capability spectrum (L1 Heuristic Bootstrapper, L2 Exemplar Accelerator, L3 Reflexive Policy) under one agentic orchestrator no independent evidence
    purpose: Unify zero-shot bootstrapping, one-shot transfer, and closed-loop policies with dual H→E→R evolution and R→E→H execution schedules.
    Primary named framework; not a physical entity but the paper’s organizing invention. Evidence is internal robot experiments, not independent external measurement.
  • H2E2R / R2E2H dual operational loops with automatic reverse-task reset no independent evidence
    purpose: Couple autonomous data collection to deployment-time capability fallback without human teleop demonstrations.
    Scheduling pattern is the paper’s process-level contribution; success metrics are self-reported on four tasks.

pith-pipeline@v1.2.0-daily-grok45 · 20032 in / 4708 out tokens · 102405 ms · 2026-07-30T20:24:52.393236+00:00 · methodology

0 comments
read the original abstract

General-purpose robotic manipulation requires robots to perform diverse tasks in open-world environments while improving their skills over time. Despite recent progress in robotic manipulation, existing systems still primarily acquire manipulation skills in a static manner, where capabilities are learned for specific tasks or settings rather than adaptively evolving through physical interaction. Resembling how repeated practice enables humans to develop muscle memory, advanced manipulation proficiency requires an autonomous capability evolution mechanism that allows robots to progressively transform interaction experiences into increasingly effective manipulation abilities. To this end, we propose HERO, a self-improving hierarchical embodied agent that enables autonomous capability evolution from zero human demonstrations. HERO organizes heuristic reasoning, exemplar reuse, and reflexive execution into a unified orchestration framework, allowing robots to autonomously bootstrap manipulation experience, rapidly accumulate reusable behaviors through experience transfer, and progressively consolidate recurring interactions into efficient closed-loop visuomotor policies. By tightly coupling autonomous data collection with task execution, HERO continuously expands and dynamically schedules manipulation capabilities according to different stages of experience accumulation and execution requirements. Extensive experiments demonstrate that HERO substantially reduces human intervention during robotic data collection while achieving robust manipulation across diverse tasks, providing a promising path toward self-improving robotic systems.

Figures

Figures reproduced from arXiv: 2607.26809 by Gaojing Zhang, Haojun Li, Haotian Liang, Jialiang Li, Qipeng Liu, Wenzhao Lian, Yangtian Ye, Yuhan Wang.

Figure 1
Figure 1. Figure 1: Overview of the HERO framework. HERO enables continuous capability evolution from zero human demonstration to [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Visualization of the four primitive action skills [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Illustration of the real-world task settings. All tasks [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Capability evolution of HERO via autonomous [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Failure Statistics in Task Execution [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

60 extracted references · 15 linked inside Pith

  1. [1]

    Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education

    Clancey, William J. Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education. Proceedings of the Eighth International Joint Conference on Artificial Intelligence (IJCAI-83)

  2. [2]

    Classification Problem Solving

    Clancey, William J. Classification Problem Solving. Proceedings of the Fourth National Conference on Artificial Intelligence

  3. [3]

    , title =

    Robinson, Arthur L. , title =. 1980 , doi =. https://science.sciencemag.org/content/208/4447/1019.full.pdf , journal =

    1980

  4. [4]

    New Ways to Make Microcircuits Smaller---Duplicate Entry

    Robinson, Arthur L. New Ways to Make Microcircuits Smaller---Duplicate Entry. Science

  5. [5]

    Clancey and Glenn Rennels , abstract =

    Diane Warner Hasling and William J. Clancey and Glenn Rennels , abstract =. Strategic explanations for a diagnostic consultation system , journal =. 1984 , issn =. doi:https://doi.org/10.1016/S0020-7373(84)80003-6 , url =

    doi:10.1016/s0020-7373(84)80003-6 1984

  6. [6]

    and Rennels, Glenn R

    Hasling, Diane Warner and Clancey, William J. and Rennels, Glenn R. and Test, Thomas. Strategic Explanations in Consultation---Duplicate. The International Journal of Man-Machine Studies

  7. [7]

    Poligon: A System for Parallel Problem Solving

    Rice, James. Poligon: A System for Parallel Problem Solving

  8. [8]

    Transfer of Rule-Based Expertise through a Tutorial Dialogue

    Clancey, William J. Transfer of Rule-Based Expertise through a Tutorial Dialogue

  9. [9]

    The Engineering of Qualitative Models

    Clancey, William J. The Engineering of Qualitative Models

  10. [10]

    2023 , eprint=

    Attention Is All You Need , author=. 2023 , eprint=

    2023

  11. [11]

    Pluto: The 'Other' Red Planet

    NASA. Pluto: The 'Other' Red Planet

  12. [14]

    2023 IEEE International conference on robotics and automation (ICRA) , pages=

    Code as policies: Language model programs for embodied control , author=. 2023 IEEE International conference on robotics and automation (ICRA) , pages=. 2023 , organization=

    2023

  13. [20]

    Conference on Robot Learning , pages=

    Rt-2: Vision-language-action models transfer web knowledge to robotic control , author=. Conference on Robot Learning , pages=. 2023 , organization=

    2023

  14. [22]

    Black, Kevin and Brown, Noah and Darpinian, James and Dhabalia, Karan and Driess, Danny and Esmail, Adnan and Equi, Michael and Finn, Chelsea and Fusai, Niccolo and others , journal=. _

  15. [24]

    2023 IEEE International Conference on Robotics and Automation (ICRA) , pages=

    ProgPrompt: Generating Situated Robot Task Plans using Large Language Models , author=. 2023 IEEE International Conference on Robotics and Automation (ICRA) , pages=. 2023 , organization=

    2023

  16. [25]

    Conference on Robot Learning , pages=

    Inner Monologue: Embodied Reasoning through Planning with Language Models , author=. Conference on Robot Learning , pages=. 2023 , organization=

    2023

  17. [26]

    Conference on Robot Learning , pages=

    VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models , author=. Conference on Robot Learning , pages=. 2023 , organization=

    2023

  18. [28]

    2024 , organization=

    Dreureka: Language model guided sim-to-real transfer , author=. 2024 , organization=

    2024

  19. [29]

    International conference on learning Representations , volume=

    Eureka: Human-level reward design via coding large language models , author=. International conference on learning Representations , volume=

  20. [30]

    International Conference on Learning Representations , volume=

    Hamster: Hierarchical action models for open-world robot manipulation , author=. International Conference on Learning Representations , volume=

  21. [31]

    International Conference on Machine Learning , pages=

    Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Models , author=. International Conference on Machine Learning , pages=. 2025 , organization=

    2025

  22. [37]

    Science Robotics , volume=

    Learning a thousand tasks in a day , author=. Science Robotics , volume=. 2025 , publisher=

    2025

  23. [39]

    Advances in neural information processing systems , volume=

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space , author=. Advances in neural information processing systems , volume=

  24. [40]

    , author=

    Generalized-icp. , author=. Robotics: science and systems , volume=. 2009 , organization=

    2009

  25. [41]

    ACM Transactions on Knowledge Discovery from Data (TKDD) , volume=

    Semantic text similarity using corpus-based word similarity and string similarity , author=. ACM Transactions on Knowledge Discovery from Data (TKDD) , volume=. 2008 , publisher=

    2008

  26. [45]

    2024 , publisher =

    Remi Cadene and Simon Alibert and Alexander Soare and Quentin Gallouedec and Adil Zouitine and Steven Palma and Pepijn Kooijmans and Michel Aractingi and Mustafa Shukor and Dana Aubakirova and Martino Russi and Francesco Capuano and Caroline Pascal and others , title =. 2024 , publisher =

    2024

  27. [46]

    Ahn, M.; Brohan, A.; Brown, N.; Chebotar, Y.; Cortes, O.; David, B.; Finn, C.; Fu, C.; Gopalakrishnan, K.; Hausman, K.; et al. 2022. Do as i can, not as i say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691

    Pith/arXiv arXiv 2022

  28. [47]

    S.; Kolesnikov, A.; Wang, X.; Salz, D

    Beyer, L.; Steiner, A.; Pinto, A. S.; Kolesnikov, A.; Wang, X.; Salz, D. M.; Neumann, M.; Alabdulmohsin, I. M.; Tschannen, M.; Bugliarello, E.; Unterthiner, T.; Keysers, D.; Koppula, S.; Liu, F.; Grycner, A.; Gritsenko, A. A.; Houlsby, N.; Kumar, M.; Rong, K.; Eisenschlos, J. M.; Kabra, R.; Bauer, M.; Bo s njak, M.; Chen, X.; Minderer, M.; Voigtlaender, P...

    Pith/arXiv arXiv 2024

  29. [48]

    Black, K.; Brown, N.; Darpinian, J.; Dhabalia, K.; Driess, D.; Esmail, A.; Equi, M.; Finn, C.; Fusai, N.; et al. 2025. _ 0.5 : a Vision-Language-Action Model with Open-World Generalization. arXiv preprint arXiv:2504.16054

    Pith/arXiv arXiv 2025

  30. [49]

    Cadene, R.; Alibert, S.; Soare, A.; Gallouedec, Q.; Zouitine, A.; Palma, S.; Kooijmans, P.; Aractingi, M.; Shukor, M.; Aubakirova, D.; Russi, M.; Capuano, F.; Pascal, C.; et al. 2024. LeRobot: State-of-the-art Machine Learning for Real-World Robotics in Pytorch. https://github.com/huggingface/lerobot

    2024

  31. [50]

    V.; Khedr, H.; Huang, A.; et al

    Carion, N.; Gustafson, L.; Hu, Y.-T.; Debnath, S.; Hu, R.; Suris, D.; Ryali, C.; Alwala, K. V.; Khedr, H.; Huang, A.; et al. 2025. Sam 3: Segment anything with concepts. arXiv preprint arXiv:2511.16719

    Pith/arXiv arXiv 2025

  32. [51]

    A.; Song, C

    Chen, S.; Hadfield, H.; Zook, A.; Uy, M. A.; Song, C. H.; Coumans, E.; Yang, X.; Ladhak, F.; Qu, Q.; Birchfield, S.; et al. 2026. VoLo: A Physical Orchestrator for Open-Vocabulary Long-Horizon Manipulation. arXiv preprint arXiv:2606.07723

    Pith/arXiv arXiv 2026

  33. [52]

    Dreczkowski, K.; Vitiello, P.; Vosylius, V.; and Johns, E. 2025. Learning a thousand tasks in a day. Science Robotics, 10(108): eadv7594

    2025

  34. [53]

    Fu, M.; Yu, J.; El-Refai, K.; Kou, E.; Xue, H.; Huang, H.; Xiao, W.; Wang, G.; Li, F.-F.; Shi, G.; et al. 2026. CaP-X: A framework for benchmarking and improving coding agents for robot manipulation. arXiv preprint arXiv:2603.22435

    Pith/arXiv arXiv 2026

  35. [54]

    Gemini Team, Google . 2023. Gemini: A family of highly capable multimodal models. arXiv preprint arXiv:2312.11805

    Pith/arXiv arXiv 2023

  36. [55]

    Huang, W.; Wang, C.; Zhang, R.; Li, Y.; Wu, J.; and Fei-Fei, L. 2023 a . VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models. In Conference on Robot Learning, 540--562. PMLR

    2023

  37. [56]

    Huang, W.; Xia, F.; Xiao, T.; Chan, H.; Liang, J.; Florence, P.; Zeng, A.; Tompson, J.; Mordatch, I.; Chebotar, Y.; et al. 2023 b . Inner Monologue: Embodied Reasoning through Planning with Language Models. In Conference on Robot Learning, 1769--1782. PMLR

    2023

  38. [57]

    Huo, D.; Liu, H.; Liu, G.; Qi, D.; Sun, Z.; Gao, M.; He, J.; Yang, Y.; Chang, X.; Xiong, F.; et al. 2026. ABot-Claw: A foundation for persistent, cooperative, and self-evolving robotic agents. arXiv preprint arXiv:2604.10096

    Pith/arXiv arXiv 2026

  39. [58]

    Islam, A.; and Inkpen, D. 2008. Semantic text similarity using corpus-based word similarity and string similarity. ACM Transactions on Knowledge Discovery from Data (TKDD), 2(2): 1--25

    2008

  40. [59]

    J.; Pertsch, K.; Karamcheti, S.; Xiao, T.; Balakrishna, A.; Nair, S.; Rafailov, R.; Foster, E.; Lam, G.; Sanketi, P.; et al

    Kim, M. J.; Pertsch, K.; Karamcheti, S.; Xiao, T.; Balakrishna, A.; Nair, S.; Rafailov, R.; Foster, E.; Lam, G.; Sanketi, P.; et al. 2024. Openvla: An open-source vision-language-action model. arXiv preprint arXiv:2406.09246

    Pith/arXiv arXiv 2024

  41. [60]

    Li, R.; Zhou, Y.; Zhu, Y.; Chen, K.; Wang, J.; Wang, S.; Hu, K.; Yu, M.; Jiang, B.; Su, Z.; et al. 2026. Roboclaw: An agentic framework for scalable long-horizon robotic tasks. arXiv preprint arXiv:2603.11558

    arXiv 2026

  42. [61]

    Li, Y.; Deng, Y.; Zhang, J.; Jang, J.; Memmel, M.; Garrett, C.; Ramos, F.; Fox, D.; Li, A.; Gupta, A.; et al. 2025. Hamster: Hierarchical action models for open-world robot manipulation. In International Conference on Learning Representations, volume 2025, 24040--24068

    2025

  43. [62]

    Liang, J.; Huang, W.; Xia, F.; Xu, P.; Hausman, K.; Ichter, B.; Florence, P.; and Zeng, A. 2023. Code as policies: Language model programs for embodied control. In 2023 IEEE International conference on robotics and automation (ICRA), 9493--9500. IEEE

    2023

  44. [63]

    Y.; Li, Z.; Shi, G.; Feng, J.; and Kang, B

    Lin, H.; Chen, S.; Liew, J.; Chen, D. Y.; Li, Z.; Shi, G.; Feng, J.; and Kang, B. 2025 a . Depth anything 3: Recovering the visual space from any views. arXiv preprint arXiv:2511.10647

    Pith/arXiv arXiv 2025

  45. [64]

    Lin, Z.; Duan, J.; Fang, H.; Fox, D.; Krishna, R.; Tan, C.; and Wen, B. 2025 b . Failsafe: Reasoning and recovery from failures in vision-language-action models. arXiv preprint arXiv:2510.01642

    Pith/arXiv arXiv 2025

  46. [65]

    Lu, R.; Wu, Y.; Kou, E.; Fu, L.; Xiao, W.; Mandlekar, A.; Xu, Y.; Shi, G.; Goldberg, K.; Chen, A.; et al. 2026. ASPIRE: Agentic/Skills Discovery for Robotics. arXiv preprint arXiv:2607.00272

    Pith/arXiv arXiv 2026

  47. [66]

    Ma, J.; Liang, W.; Wang, H.-J.; Zhu, Y.; Fan, L.; Bastani, O.; and Jayaraman, D. 2024 a . Dreureka: Language model guided sim-to-real transfer. RSS

    2024

  48. [67]

    J.; Liang, W.; Wang, G.; Huang, D.-A.; Bastani, O.; Jayaraman, D.; Zhu, Y.; Fan, J.; et al

    Ma, Y. J.; Liang, W.; Wang, G.; Huang, D.-A.; Bastani, O.; Jayaraman, D.; Zhu, Y.; Fan, J.; et al. 2024 b . Eureka: Human-level reward design via coding large language models. In International conference on learning Representations, volume 2024, 26516--26560

    2024

  49. [68]

    Morin, S.; Gupta, K.; Sandhu, M.; Gauthier, C.; Argenziano, F.; Ellis, K.; and Paull, L. 2025. Agentic Scene Policies: Unifying Space, Semantics, and Affordances for Robot Action. arXiv preprint arXiv:2509.19571

    arXiv 2025

  50. [69]

    R.; Yi, L.; Su, H.; and Guibas, L

    Qi, C. R.; Yi, L.; Su, H.; and Guibas, L. J. 2017. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems, 30

    2017

  51. [70]

    Segal, A.; Haehnel, D.; Thrun, S.; et al. 2009. Generalized-icp. In Robotics: science and systems, volume 2, 435. Seattle, WA

    2009

  52. [71]

    X.; Ichter, B.; Equi, M

    Shi, L. X.; Ichter, B.; Equi, M. R.; Ke, L.; Pertsch, K.; Vuong, Q.; Tanner, J.; Walling, A.; Wang, H.; Fusai, N.; et al. 2025. Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Models. In International Conference on Machine Learning, 54919--54933. PMLR

    2025

  53. [72]

    Singh, I.; Blukis, V.; Mousavian, A.; Goyal, A.; Xu, D.; Tremblay, J.; Fox, D.; Thomason, J.; and Garg, A. 2023. ProgPrompt: Generating Situated Robot Task Plans using Large Language Models. In 2023 IEEE International Conference on Robotics and Automation (ICRA), 11523--11530. IEEE

    2023

  54. [73]

    Wang, G.; Xie, Y.; Jiang, Y.; Mandlekar, A.; Xiao, C.; Zhu, Y.; Fan, L.; and Anandkumar, A. 2023. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291

    Pith/arXiv arXiv 2023

  55. [74]

    Wang, W.; Song, J.; Liu, C.; Ma, J.; Feng, S.; Wang, J.; Jiang, Y.; Chen, K.; Zhan, S.; Wang, Y.; et al. 2025 a . Genie centurion: Accelerating scalable real-world robot training with human rewind-and-refine guidance. arXiv preprint arXiv:2505.18793

    arXiv 2025

  56. [75]

    Wang, W.; Ye, K.; Zhou, X.; Chen, T.; Min, C.; Zhu, Q.; Yang, X.; Luo, P.; Shen, Y.; Yang, Y.; et al. 2025 b . FieldGen: From Teleoperated Pre-Manipulation Trajectories to Field-Guided Data Generation. arXiv preprint arXiv:2510.20774

    arXiv 2025

  57. [76]

    Xiao, W.; Xie, J.; Zhang, T.; Lin, H.; Fu, L.; Xue, H.; Lu, J.; Yang, Y.; Dai, C.; Wang, Z.; et al. 2026. ENPIRE: Agentic Robot Policy Self-Improvement in the Real World. arXiv preprint arXiv:2606.19980

    Pith/arXiv arXiv 2026

  58. [77]

    Zhou, P.; Chen, L.; Chen, S.; Chen, D.; Zhao, W.; Jin, R.; Ren, G.; and Luo, J. 2025. Act2Goal: From World Model To General Goal-conditioned Policy. arXiv preprint arXiv:2512.23541

    arXiv 2025

  59. [78]

    Zhou, X.; Liu, L.; Xiao, T.; Feng, W.; Fu, F.; Meng, X.; Wang, X.; Han, J.; Yu, B.; Du, Y.; et al. 2026. HoloAgent-0: A Unified Embodied Agent Framework with 3D Spatial Memory. arXiv preprint arXiv:2606.23565

    Pith/arXiv arXiv 2026

  60. [79]

    Zitkovich, B.; Yu, T.; Xu, S.; Xu, P.; Xiao, T.; Xia, F.; Wu, J.; Wohlhart, P.; Welker, S.; Wahid, A.; et al. 2023. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, 2165--2183. PMLR

    2023