Pith. sign in

REVIEW 3 major objections 5 minor 35 references

The paper argues that object-state divergence (the gap between the state used to act and the state used to verify) is the core failure in long-horizon humanoid manipulation, and that one shared, refreshed 3D object memory closes it.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 16:21 UTC pith:SSI3Y72Z

load-bearing objection Real integration with a real robot and a transparent but unresolved confound: the headline gain may be extra fine-tuning, not the shared object tokens. the 3 major comments →

arxiv 2607.18016 v2 pith:SSI3Y72Z submitted 2026-07-20 cs.RO

Closing the Loop in Humanoid VLA: Persistent 3D Object Tokens for Verifiable Loco-Manipulation

Peng Ren , Haoyang Ge , Jiang Zhao , Cong Huang , Yukun Shi , Pei Chi , Kai Chen This is my paper
classification cs.RO
keywords humanoid loco-manipulationvision-language-action policyobject-centric grounding3D object tokensexecution verificationgeometric predicatesclosed-loop controlobject-state divergence
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that the central problem in long-horizon humanoid loco-manipulation is object-state divergence: the state used to condition an action can drift from the state used to verify it, causing false completions and failed recoveries. It proposes Persistent Object Tokenization (POT), a role-indexed 3D object memory built from RGB-D observations, serialized into tokens for a whole-body vision-language-action policy, and reused after each chunk to run geometric predicate checks. On a real humanoid across eight task families, this closed loop raises task success from 39/80 to 71/80 against a matched direct baseline, with the largest gains on tasks that require maintaining 3D relations across multiple action chunks. The broader claim is that persistent, object-centered, verifiable state is a useful abstraction for general humanoid VLA execution, without needing a simulator or learned dynamics model.

Core claim

POT-VLA conditions a whole-body action expert on Persistent 3D Object Tokens—fixed-slot tensors carrying semantic role, grounding confidence, 3D centroid and extent, and relational features for up to eight task entities—and then verifies the same refreshed object memory with geometric predicates such as containment, support, proximity, displacement, and handover distance. Because action generation and verification draw on identical object records, a subtask advances only when the intended metric relation actually holds in refreshed 3D state. Across eight real-world task families on a real humanoid robot, POT-VLA completes 71/80 trials versus 39/80 for a matched direct baseline; the largest g

What carries the argument

The central mechanism is the role-indexed object memory M, which binds task entities to persistent role slots (TARGET, DESTINATION, SUPPORT, HANDOVER_PARTNER) and refreshes their metric 3D evidence from RGB-D after each executed chunk. This memory is serialized into Persistent 3D Object Tokens: K=8 slots by F=33 features, projected through a LayerNorm→Linear→GELU→Dropout→Linear branch and inserted into the action head's self-attention sequence as state, object, and action tokens. The same memory feeds the geometric predicate supervisor, whose predicates p=(kappa, alpha, op, nu, n) (type, grounded arguments, comparison, threshold, stability window) decide in_progress, done, blocked, failed, o

Load-bearing premise

The evaluation assumes the direct baseline is a matched control that isolates the object-token loop, but the compared variants differ in which checkpoint they use (object-token fine-tuned versus direct) and in hand-specified verification thresholds, retry budgets, and timeouts, so the measured gains cannot be cleanly attributed to persistent 3D object tokens alone.

What would settle it

Run the four-task ablation subset with the object-token fine-tuned checkpoint but with object tokens masked out throughout execution (while keeping predicate verification disabled); if success stays near the full POT-VLA level rather than falling toward the direct baseline, the gain is carried by the fine-tuned checkpoint rather than by the token signal.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the central claim holds, VLA policies for long-horizon manipulation should expose task-object state explicitly rather than leaving it implicit inside visual-language features.
  • Sharing one refreshed object memory between action generation and verification should reduce false task completions, since a subtask finishes only when the intended 3D relation is measured, not merely attempted.
  • The approach should generalize without a simulator or physics engine: POT is built from measurement-grounded RGB-D records, so it applies wherever reliable depth, masks, and calibration are available.
  • The largest gains on multi-chunk relational tasks suggest that persistent object tokens help most where objects move, become occluded, or change support through contact and transport.
  • Recovery becomes less expensive: local re-observation, regrounding, and retry can fix object-state errors before any full language-level replan is needed.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A clean testable extension would isolate the token signal from the fine-tuning effect: run the object-token fine-tuned checkpoint with the object tokens zeroed or masked, and compare with the same checkpoint using live tokens; if the gap persists, the improvement comes from the token content rather than from checkpoint adaptation.
  • If persistent role-indexed memory is the operative abstraction, similar gains should appear when the same object-token schema is attached to a different whole-body action expert, provided the action head supports token insertion—an explicit cross-backbone transfer test the paper does not run.
  • The predicate supervisor's hand-specified thresholds and retry budgets are currently calibrated per subtask; one could infer that auto-tuning these from measured success or variance would make the system less dependent on manual calibration and more portable across robots and scenes.
  • A further implication is that the same refreshed object memory could power higher-level failure explanations or a learned residual correction model, since it already localizes failures as missed grasp, outside-goal placement, unstable support, or excessive handover distance.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Persistent Object Tokenization (POT), a role-indexed 3D object memory that is refreshed from RGB-D observations and serialized into fixed-slot tokens for a GR00T-N1.7 whole-body action head. The same memory is also read by a geometric predicate supervisor that verifies subtask completion and triggers recovery. The system, POT-VLA, is evaluated on a Unitree G1 across eight real-world task families, reporting 71/80 successes versus 39/80 for a direct baseline, and 44/50 on a Being-0-aligned service-task reference versus 37/50 reported by the Being-0 paper. The authors conclude that persistent object-centered state reduces object-state divergence and enables verifiable humanoid loco-manipulation.

Significance. If the causal claim is correct, the paper makes a useful contribution: it shows concrete gains from making object state explicit, persistent, and shared between action generation and verification in a real humanoid system. The strengths are the real-robot evaluation, the use of a fixed action backbone and embodiment across conditions, the separate ablations of token conditioning and verification, and the generalization tests under object-state shifts. However, the main comparison is confounded because the POT variants use an additional object-token fine-tuned checkpoint while the direct baseline does not. The stated success criterion is also the same geometric predicate set used by the verifier to gate completion and trigger recovery. These issues mean the reported improvements are plausible but not yet cleanly attributable to the persistent object-token abstraction. The paper's central technical idea is defensible, but the experimental design needs strengthening before the headline claim can be accepted.

major comments (3)
  1. [§4.3, Table 1A] The headline comparison is not a matched control. The direct and verifier-only variants use the direct action-expert checkpoint, while the POT-token variants use the object-token fine-tuned checkpoint, demonstration sidecars, fine-tuning budget, and checkpoint-selection rule. Thus the 39/80 → 71/80 improvement includes at least two coupled factors: the presence of persistent object tokens and an additional fine-tuning stage with sidecar-paired data and new projection parameters. The claim that POT-VLA improves a 'matched direct baseline' is therefore not supported. Please add a control that fine-tunes the direct action expert with the same demonstrations and budget but does not receive object tokens at inference, or otherwise equalizes the amount of fine-tuning across conditions.
  2. [§4.1 and §3.3] Trial success is defined by whether the final geometric predicates remain satisfied over a temporal window, and those same predicates are used by the supervisor to gate completion and trigger recovery. For POT-VLA, a failed predicate can lead to retry/recovery before success is recorded; the direct baseline is evaluated without this feedback. The metric is therefore aligned with the mechanism being added, which can inflate the apparent gain even if the action policy itself is unchanged in quality. Please report first-attempt success (before recovery), the number of recovery attempts per trial, and an independent human or externally defined verification of final states to separate improved action generation from improved recovery gating.
  3. [§4.3, Table 2A] The ablation subset is explicitly chosen as 'four tasks with large state-divergence gaps in Table 1.' Selecting tasks based on the outcome variable biases the estimated contribution of token conditioning upward. The claim that 'POT tokens provide the largest gain' (15/40 to 31/40) is based on this non-random subset. Please run the ablation on the full eight-task suite or pre-register the subset selection before seeing Table 1 results. The small per-setting sample sizes (10 trials) also warrant confidence intervals or a simple significance test.
minor comments (5)
  1. [§4.1] The text contains an unresolved 'Table??' reference. It should be replaced with the actual table number.
  2. [§4.2] The sentence 'The matched comparison supports the state-divergence hypothesis' overstates the evidence: the experiment does not directly measure object-state divergence, and the comparison is confounded as noted above.
  3. [§3.1, §3.3] Per-subtask predicate thresholds ν, stability windows n, retry budgets, timeouts, and chunk horizons are hand-specified. A sensitivity analysis over these values would help establish that the reported gains are not driven by favorable calibration.
  4. [§4.3] The fine-tuning details are underspecified: demonstration count, number of gradient steps, learning rate, and checkpoint-selection criterion are not reported. This is needed for reproducibility and for judging whether the fine-tuned control would be fair.
  5. [§4.2, Table 1B] The Being-0 comparison uses paper-reported numbers on 'aligned' tasks rather than a local reproduction. This is acknowledged, but the alignment procedure should be described in more detail, and the comparison should not be presented as evidence of equivalence to a matched baseline.

Circularity Check

1 steps flagged

Minor measurement circularity: trial success is defined as the same geometric predicates the verifier uses to declare completion; the checkpoint confound is a control flaw, not circularity.

specific steps
  1. self definitional [Section 3.3 (Predicate-Level Verification and Recovery); Section 4.1 (Setup and Protocol)]
    "After execution from t to t+ = t+H, completion is decided on M^{τ_i}_{t+}. A subtask is therefore complete only when the intended metric relation holds in the refreshed 3D object state, not simply when the policy attempted the corresponding action. / A trial succeeds only if the final geometric predicates remain satisfied over the required temporal window within the predefined timeout and retry budget, without human intervention."

    The trial success criterion is identical in content to the verifier's completion condition. Section 3.3 says completion is decided on the refreshed object memory and a subtask is complete only when the intended metric relation holds there; Section 4.1 counts a trial as successful only if the same geometric predicates remain satisfied over the temporal window. POT-VLA's supervisor uses these predicates to return done and trigger re-observation or retry, so the system is measured on the exact signal it is designed to maximize. The direct baseline is not given this internal completion loop, making the 39/80 vs 71/80 comparison partly by construction. This is partial, since retry budgets are finite and the tokens-only variant (31/40) improves without the verifier.

full rationale

No mathematical derivation exists here, so circularity can only enter through the evaluation protocol. The one self-definitional element is that trial success is defined as geometric-predicate satisfaction, which is the same condition POT-VLA's verifier uses to declare a subtask done and to trigger retries; the headline gain is therefore partly built into the loop. This is a real but partial measurement circularity, not a full reduction, because retry budgets are finite, the direct baseline is also judged by the same predicates, and the tokens-only ablation shows gains without verification. The larger control problem in Section 4.3 is a confound, not circularity: POT-token variants use a separate object-token fine-tuned checkpoint while direct/verifier-only variants use an un-fine-tuned checkpoint, so the matched comparison and the token ablation do not isolate persistent 3D object tokens from extra fine-tuning. I do not count this as circularity because the improvement could come from additional training rather than being true by definition. There is no self-citation chain, no imported uniqueness theorem, and no ansatz smuggled via citation. The paper is self-contained on real-robot trials, so the circularity score is low.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 1 invented entities

The central success claims rest on assumptions about SAM3 segmentation quality, depth back-projection accuracy, GR00T-N1.7's ability to use object tokens, correct role binding from VLM/human planners, and the sufficiency of geometric predicates as success criteria. The token schema, predicate thresholds, retry budgets, calibration values, and projection-network weights are hand-chosen or learned parameters, and none are released. These are inputs the reader must accept before the reported gains can be interpreted.

free parameters (5)
  • Per-subtask predicate thresholds ν and stability windows n = not reported (hand-specified per task)
    §3.3: 'Thresholds and stability windows are specified by the typed subtask and robot calibration, rather than being learned.' These directly determine when a subtask is declared done and when recovery is triggered, so the success rates depend on them.
  • Token schema dimensions K=8, F=33 and feature set = K=8 slots, F=33 features per slot
    §3.1: the default schema allocates K=8 slots and F=33 features; these hand-chosen dimensions bound what object state the policy can see and the verifier can check.
  • Calibrated workspace bounds, camera intrinsics, camera-to-base transform = calibrated values not reported
    §3.1: valid depth pixels are back-projected through camera intrinsics and the camera-to-base transform and filtered by calibrated workspace bounds; 3D record accuracy depends on these calibration values.
  • Object-token projection network parameters f_obj_theta = learned; values not released
    §3.2: the projector is a LayerNorm→Linear→GELU→Dropout→Linear network fine-tuned with the action loss; the projection is fitted to demos and no checkpoint is released.
  • Retry budget, timeout, action chunk horizon H per subtask = not reported
    §3.1/§3.3: typed subtasks specify chunk horizon, timeout, and retry budget; these parameters affect how many recovery attempts count toward a successful trial.
axioms (5)
  • domain assumption SAM3 masks correctly and consistently identify task objects in the RGB-D observations used for grounding.
    §3.1: 'online or cached SAM3 masks provide role/query masks'; if segmentation fails, object records degrade to uncertainty and the whole pipeline relies on re-observation.
  • domain assumption Back-projected depth plus camera-to-base transform yields metric-accurate 3D centroids and extents in the robot base frame.
    §3.1: valid depth pixels are back-projected and summarized in the robot base frame; the geometric verifier measures containment, support, and displacement from these coordinates.
  • domain assumption The GR00T-N1.7 backbone can effectively consume object tokens inserted into the action-head self-attention sequence after fine-tuning.
    §3.2: the action head sequence is [Z_state, Z_obj, Z_action] with visual-language features in cross-attention; the paper does not show systematic analysis of how the backbone handles the new token modality.
  • domain assumption The typed-subtask plan (from a VLM planner or human-authored task file) binds the correct object roles and grounding queries.
    §3.1: the plan provides roles, grounding queries, and success predicates; the paper does not evaluate the planner's role-binding accuracy, and a human-authored plan is an input, not a learned contribution.
  • domain assumption Geometric predicates are sufficient to represent the physical success criterion for each task.
    §3.3 and §4.1: a trial succeeds only if the final geometric predicates remain satisfied; semantic conditions like category ambiguity or handover acceptance are only optionally checked by a VLM and are not prominent in the reported results.
invented entities (1)
  • Role-indexed object memory M_t^{tau_i} / Persistent 3D Object Tokens no independent evidence
    purpose: Shared, refreshed 3D object state that conditions whole-body action chunks and simultaneously grounds geometric predicate checks for verification and recovery.
    All evidence is the internal 80/50-trial real-robot evaluation in this paper; there is no external benchmark, no released artifact, and no falsifiable prediction outside the paper that would independently confirm the representation's causal role. The contribution may be confounded by the extra fine-tuning applied to token variants.

pith-pipeline@v1.3.0-alltime-deepseek · 9785 in / 13625 out tokens · 144521 ms · 2026-08-01T16:21:33.235616+00:00 · methodology

0 comments
read the original abstract

Vision-language-action policies are a promising foundation for general robot control, but long-horizon humanoid loco-manipulation requires the robot to treat task objects as persistent physical entities across movement, contact, occlusion, and recovery. We study this problem as object-state divergence: the object state used to condition a whole-body action can differ from the state used to decide whether the action achieved the intended physical relation. We propose \emph{Persistent Object Tokenization} (POT), which maintains role-indexed 3D object records from RGB-D observations and converts them into object tokens for a whole-body action expert. Instantiated as \emph{POT-VLA}, the same object records condition action generation and support geometric predicate checks, yielding a closed-loop execution system in which object state is both actionable and verifiable. On a Unitree G1, POT-VLA improves a matched direct GR00T-N1.7 baseline from 39/80 to 71/80 successes over eight real-world task families. In an external Being-0-aligned reference, POT-VLA achieves 44/50 successes on aligned service tasks, compared with the 37/50 success reported by the Being-0 paper. The largest gains occur on tasks requiring maintained 3D relations, suggesting that persistent object-centered state is a useful abstraction for verifiable humanoid VLA execution.

Figures

Figures reproduced from arXiv: 2607.18016 by Cong Huang, Haoyang Ge, Jiang Zhao, Kai Chen, Pei Chi, Peng Ren, Yukun Shi.

Figure 1
Figure 1. Figure 1: POT-VLA system overview. POT maintains a role-indexed object memory [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Real-world task rollouts. Each task family is shown with three representative frames from [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references · 15 linked inside Pith

  1. [1]

    Black, N

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al.π 0: A vision-language-action flow model for general robot control.arXiv preprint arXiv:2410.24164, 2024

    Pith/arXiv arXiv 2024

  2. [2]

    Intelligence, K

    P. Intelligence, K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, et al.π 0.5: a vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054, 2025

    Pith/arXiv arXiv 2025

  3. [3]

    Brohan, N

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Haus- man, A. Herzog, J. Hsu, et al. Rt-1: Robotics transformer for real-world control at scale.arXiv preprint arXiv:2212.06817, 2022

    Pith/arXiv arXiv 2022

  4. [4]

    Zitkovich, T

    B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pages 2165–2183. PMLR, 2023

    2023

  5. [5]

    X. Zhao, M. Li, C. Weber, M. B. Hafez, and S. Wermter. Chat with the environment: Inter- active multimodal perception using large language models. In2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3590–3596. IEEE, 2023

    2023

  6. [6]

    Q. Liao, T. E. Truong, X. Huang, Y . Gao, G. Tevet, K. Sreenath, and C. K. Liu. Beyondmimic: From motion tracking to versatile humanoid control via guided diffusion.arXiv preprint arXiv:2508.08241, 2025

    Pith/arXiv arXiv 2025

  7. [7]

    X. B. Peng, P. Abbeel, S. Levine, and M. Van de Panne. Deepmimic: Example-guided deep re- inforcement learning of physics-based character skills.ACM Transactions On Graphics (TOG), 37(4):1–14, 2018

    2018

  8. [8]

    X. Lin, Y . Huang, T. Fu, X. Xiong, and C. Wang. iwalker: Imperative visual planning for walking humanoid robot. In2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2865–2872. IEEE, 2025

    2025

  9. [9]

    Y . Ze, Z. Chen, W. Wang, T. Chen, X. He, Y . Yuan, X. B. Peng, and J. Wu. Generalizable hu- manoid manipulation with 3d diffusion policies. In2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2873–2880. IEEE, 2025

    2025

  10. [10]

    O’Neill, A

    A. O’Neill, A. Rehman, A. Gupta, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, et al. Open x-embodiment: Robotic learning datasets and rt-x models. arXiv preprint arXiv:2310.08864, 2023

    Pith/arXiv arXiv 2023

  11. [11]

    Ghosh, H

    D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xu, J. Luo, et al. Octo: An open-source generalist robot policy.arXiv preprint arXiv:2405.12213, 2024

    Pith/arXiv arXiv 2024

  12. [12]

    Jiang, A

    Y . Jiang, A. Gupta, Z. Zhang, G. Wang, Y . Dou, Y . Chen, L. Fei-Fei, A. Anandkumar, Y . Zhu, and L. Fan. Vima: General robot manipulation with multimodal prompts. InInternational Conference on Machine Learning, pages 14975–15022. PMLR, 2023

    2023

  13. [13]

    Huang, C

    W. Huang, C. Wang, R. Zhang, Y . Li, J. Wu, and L. Fei-Fei. V oxposer: Composable 3d value maps for robotic manipulation with language models.arXiv preprint arXiv:2307.05973, 2023

    Pith/arXiv arXiv 2023

  14. [14]

    Shridhar, L

    M. Shridhar, L. Manuelli, and D. Fox. Cliport: What and where pathways for robotic manipu- lation. InConference on robot learning, pages 894–906. PMLR, 2022

    2022

  15. [15]

    Shridhar and L

    M. Shridhar and L. Manuelli. Perceiver-actor: A multi-task transformer for robotic manipula- tion. InConference on Robot Learning, pages 785–799. PMLR, 2023. 10

    2023

  16. [16]

    Goyal, J

    A. Goyal, J. Xu, Y . Guo, V . Blukis, Y .-W. Chao, and D. Fox. Rvt: Robotic view transformer for 3d object manipulation. InConference on Robot Learning, pages 694–710. PMLR, 2023

    2023

  17. [17]

    W. Liu, Y . Du, T. Hermans, S. Chernova, and C. Paxton. Structdiffusion: Language-guided creation of physically-valid structures using unseen objects. InRobotics: Science and Systems, 2023

    2023

  18. [18]

    Radosavovic, T

    I. Radosavovic, T. Xiao, B. Zhang, T. Darrell, J. Malik, and K. Sreenath. Real-world humanoid locomotion with reinforcement learning.Science Robotics, 9(89):eadi9579, 2024

    2024

  19. [19]

    H. Yuan, Y . Bai, Y . Fu, B. Zhou, Y . Feng, X. Xu, Y . Zhan, B. F. Karlsson, and Z. Lu. Being-0: A humanoid robotic agent with vision-language models and modular skills.arXiv preprint arXiv:2503.12533, 2025

    Pith/arXiv arXiv 2025

  20. [20]

    H. Xue, X. Huang, D. Niu, Q. Liao, T. Kragerud, J. T. Gravdahl, X. B. Peng, G. Shi, T. Dar- rell, K. Sreenath, et al. Leverb: Humanoid whole-body control with latent vision-language instruction.arXiv preprint arXiv:2506.13751, 2025

    arXiv 2025

  21. [21]

    J. Wang, A. Laurenzi, and N. Tsagarakis. Autonomous behavior planning for humanoid loco- manipulation through grounded language model. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 10856–10863. IEEE, 2024

    2024

  22. [22]

    Schakkal, B

    A. Schakkal, B. Zandonati, Z. Yang, and N. Azizan. Hierarchical vision-language planning for multi-step humanoid manipulation.arXiv preprint arXiv:2506.22827, 2025

    Pith/arXiv arXiv 2025

  23. [23]

    Jiang, J

    H. Jiang, J. Chen, Q. Bu, L. Chen, M. Shi, Y . Zhang, D. Li, C. Suo, C. Wang, Z. Peng, et al. Wholebodyvla: Towards unified latent vla for whole-body loco-manipulation control.arXiv preprint arXiv:2512.11047, 2025

    arXiv 2025

  24. [24]

    Zhang, J

    H. Zhang, J. Sun, M. Caprio, J. Tang, S. Zhang, Q. Zhang, and W. Pan. Humanoidverse: A versatile humanoid for vision-language guided multi-object rearrangement.arXiv preprint arXiv:2508.16943, 2025

    Pith/arXiv arXiv 2025

  25. [25]

    Brohan, Y

    A. Brohan, Y . Chebotar, C. Finn, K. Hausman, A. Herzog, D. Ho, J. Ibarz, A. Irpan, E. Jang, R. Julian, et al. Do as i can, not as i say: Grounding language in robotic affordances. In Conference on robot learning, pages 287–318. PMLR, 2023

    2023

  26. [26]

    Liang, W

    J. Liang, W. Huang, F. Xia, P. Xu, K. Hausman, B. Ichter, P. Florence, and A. Zeng. Code as policies: Language model programs for embodied control. In2023 IEEE International conference on robotics and automation (ICRA), pages 9493–9500. IEEE, 2023

    2023

  27. [27]

    Huang, P

    W. Huang, P. Abbeel, D. Pathak, and I. Mordatch. Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. InInternational conference on machine learning, pages 9118–9147. PMLR, 2022

    2022

  28. [28]

    Huang, F

    W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Florence, A. Zeng, J. Tompson, I. Mordatch, Y . Chebotar, et al. Inner monologue: Embodied reasoning through planning with language models.arXiv preprint arXiv:2207.05608, 2022

    Pith/arXiv arXiv 2022

  29. [29]

    D. S. Grigorev, A. K. Kovalev, and A. I. Panov. Verifyllm: Llm-based pre-execution task plan verification for robots. In2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 18489–18496. IEEE, 2025

    2025

  30. [30]

    C. R. Yu, D. Chae, D. Seo, S. Lee, H. Im, and J. Kim. Scene graph-guided proactive replanning for failure-resilient embodied agent.arXiv preprint arXiv:2508.11286, 2025

    Pith/arXiv arXiv 2025

  31. [31]

    Mei, G.-N

    A. Mei, G.-N. Zhu, H. Zhang, and Z. Gan. Replanvlm: Replanning robotic tasks with visual language models.IEEE Robotics and Automation Letters, 9(11):10201–10208, 2024. 11

    2024

  32. [32]

    P. Guo, Z. Mai, Z. Xu, K. Zhang, H. Zhang, Z. Miao, A. Ajoudani, Z. Kingston, Q. Qiu, and Y . She. Agenticlab: A real-world robot agent platform that can see, think, and act.arXiv preprint arXiv:2602.01662, 2026

    Pith/arXiv arXiv 2026

  33. [33]

    S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. R. Narasimhan, and Y . Cao. ReAct: Synergiz- ing reasoning and acting in language models. InThe Eleventh International Conference on Learning Representations, 2023

    2023

  34. [34]

    Carion, L

    N. Carion, L. Gustafson, Y .-T. Hu, S. Debnath, R. Hu, D. Suris, C. Ryali, K. V . Alwala, H. Khedr, A. Huang, et al. Sam 3: Segment anything with concepts.arXiv preprint arXiv:2511.16719, 2025

    Pith/arXiv arXiv 2025

  35. [35]

    Bjorck, F

    J. Bjorck, F. Castaneda, N. Cherniadev, et al. GR00T N1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025. doi:10.48550/arXiv. 2503.14734. 12

    Pith/arXiv arXiv doi:10.48550/arxiv.2503.14734 2025