Robotics · arXiv:2610.28259 v1 · submitted 20 Oct 2026 · 66 references
Large Multimodal Models for Direct Robot Control: GPT-6 as a Beginning
The paper says
“Recent advances in visuospatial understanding and agentic reasoning enable direct robotic manipulation with general-purpose multimodal models, but their embodied capabilities remain poorly understood. This paper systematically evaluate GPT-6 Astra for closed-loop robotic manipulation using a lightweight agent execution framework.”
more of the abstract
We first probe three fundamental embodied capabilities-depth perception, embodied in-context learning, and tactile understanding-and demonstrate that the model can exploit depth information, acquire simple manipulation strategies from contextual demonstrations, and use tactile feedback to correct its actions. We then evaluate GPT-6 Astra across five established manipulation benchmarks along six capability dimensions. Despite competitive performance on several benchmarks-26.92% success on RoboDojo and 76.0% on RoboTwin 2.0-our analysis reveals a pronounced gap between semantic reasoning and physical execution, particularly in consequence-aware action planning, precision, and complex motion coordination. To further examine these limitations, we introduce LIBERO-GeoMotion, a new benchmark of geometry-constrained tasks that require precise pose alignment, coordinated translation and rotation, and anticipation of physical action consequences. GPT-6 Astra achieves only 11.1% zero-shot success, exposing the difficulty of translating spatial understanding into physically feasible motion. Real-world experiments corroborate these findings: zero-shot execution is feasible for simple tasks, yet persistent deficits remain in motion feasibility, precise control, and physical safety, while substantial inference latency further hinders practical deployment. Overall, our results characterize the emerging embodied capabilities of general-purpose multimodal models and delineate the key challenges toward safe, efficient, and reliable robotic manipulation.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 21 Oct 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. The most systematic look yet at GPT-6 Astra as a direct robot controller; the semantic-vs-execution gap is real, but the precision/coordination part is partly an interface artifact the paper never isolates. This line is wrong
3 major objections · 7 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 66 references say.
The paper says, at Sections 3.6 and 4.2
The review did not quote this passage.
Pith says: Objection 1 of 3 · major
The paper's headline deficit attribution—that GPT-6 Astra itself lacks precision and complex-motion coordination—is partially confounded by the GPT-GEAR action interface through which all execution evidence passes. Act accepts a single relative pose target bounded at 0.20 m and pi/2 per call, tracked at 0.01 m and 0.05 rad per native control step with a default limit of 10 steps per call, and the controller performs no trajectory planning or collision checking beyond kinematic feasibility.
more of objection 1
Under these constraints a pi/2 rotation requires roughly 32 native steps, so any sustained coordinated translation-rotation motion must be decomposed by the model into many sequential decisions, and there is no interface for expressing a continuous trajectory or correcting it mid-call; the narrow-passage transport (Table 6, family (a); Figure 17(b)), upright book placement (family (b)), and real-world block-in-cup insertion (Figure 18(a)) failures are therefore not clean evidence about the model's geometric reasoning, since the same failures could plausibly arise from the command granularity and the absence of a planner. The concern is explicitly partial: the vase knock-over (Figure 17(a)), the joint-limit wrist rotation in block-in-cup, and the drawer obstruction are decisions the model makes, and they do support the consequence-aware-planning deficit. The acknowledgment in Section 3.6 that "the controller's feasibility checks do not guarantee collision-free motion or successful grasping" marks the confound rather than resolving it; the paper should add an interface ablation (larger per-call bounds, a waypoint or planner tool, or higher step limits) and show the deficits persist, or restrict the precision and coordination conclusions to the model-plus-harness system that the data actually measure.
The paper says, at Tables 1-3, 6, and 7
The review did not quote this passage.
Pith says: Objection 2 of 3 · major
The fine-grained comparative claims rest on small trial counts reported as point estimates without any uncertainty quantification. Table 1 has n=40 per condition, and the headline contrast of 47.5% versus 60.0% has overlapping exact binomial 95% intervals (approximately 32-63% and 43-75%), so "Camera Parameters Improve Manipulation" is not established at the stated strength; the analogous 64% to 78% ICL contrast in Table 2 (n=50) also has overlapping intervals.
more of objection 2
LIBERO-GeoMotion's per-family cells are n=4 (Table 6), where a delta such as 0/4 to 3/4 corresponds to a two-sided Fisher exact p of about 0.14, and the real-world per-task cells are n=5 (Table 7), for example mango grasping 0/5 to 3/5 (p about 0.17); the overall real-world contrast of 10/25 to 20/25 is significant, so the aggregate direction holds. The central gap claim is not in jeopardy because Table 5 aggregates hundreds of trials per dimension, but the paper's capability-level contributions (the depth, ICL, and tactile ablations, and the per-task real-world ICL gains) require exact binomial confidence intervals on every reported success-rate cell, and ideally more trials for the n=4 and n=5 cells, before those claims can be stated at their current strength.
The paper says, at Section 6 (LIBERO-GeoMotion)
The review did not quote this passage.
Pith says: Objection 3 of 3 · major
As a new benchmark contribution, LIBERO-GeoMotion is not calibrated: no trained VLA or WAM baseline, scripted oracle, or human performance is reported on the 36 variants, and the success predicates are author-defined with no implementation details, so the 11.1% zero-shot figure is hard to interpret in absolute terms.
more of objection 3
The internal gradient in Table 6 mitigates part of this concern—with demonstrations the model reaches 75-100% on four families, showing the tasks are not infeasible for the tested harness—and the persistent 0% on package storage (family (h)) is informative precisely because other families improve. To support the claim that the benchmark "exposes critical limitations that conventional pick-and-place benchmarks cannot reveal," at least one strong trained-policy baseline and per-variant (rather than per-family) results should be reported, together with a precise specification of the success predicates used for the 36 variants.
7 minor objections
The paper says, at Section 4.5
Pith says: minor objection 1 of 7
The main benchmark evaluation provides the model with task-specific in-context demonstrations (video frames, textual step descriptions, and action trajectories), so describing the LIBERO-Plus and RoboTwin 2.0 results as "competitive zero-shot transfer" conflates zero-shot (no examples) with "no gradient updates"; the phrase should be qualified wherever it appears.
The paper says, at Section 4.2
Pith says: minor objection 2 of 7
The relationship between the per-call bounds (0.20 m, pi/2 rad) and the default 10 native control steps is unclear—at 0.01 m and 0.05 rad per step the default cap covers at most 0.10 m and 0.5 rad per call—and the paper never states the overall per-episode execution budget used in Algorithm 1, which matters for the long-horizon interpretation; please specify whether the model can raise the step limit via the M argument and state the episode-level Act or step budget.
show 5 more
The paper says, at Abstract
Pith says: minor objection 3 of 7
"This paper systematically evaluate GPT-6 Astra" should read "evaluates."
The paper says, at Figure 4
Pith says: minor objection 4 of 7
The depth MAE values are computed from only 8-14 observations across three task episodes; the text should label these as illustrative case studies rather than a quantitative estimate of the model's depth-estimation accuracy, and ideally report more episodes.
The paper says, at Section 5.2.3
Pith says: minor objection 5 of 7
"Superior memory" rests on 6 RoboDojo tasks (60 trials) with a 56.67% success rate; "competitive" or "adequate" memory would be a more measured characterization.
The paper says, at Tables 4 and 5
Pith says: minor objection 6 of 7
The task-to-dimension assignments are author-defined and not released, and the "Overall" row is dominated by RoboDojo (for example, 300 of 660 long-horizon episodes), so the aggregate gap partly reflects benchmark composition; the text acknowledges difficulty confounds, but releasing the full assignment mapping would materially aid reproducibility and interpretation.
The paper says, at Figure 8
Pith says: minor objection 7 of 7
The comparison mixes officially reported baseline numbers with protocol-reevaluated numbers and gives GPT-6 Astra in-context demonstrations, so the caption should restate that the comparison is indicative rather than strictly apples-to-apples; the disclosure in the Section 4.5 text is appreciated but the caption alone is insufficient.
What would settle it
Run the LIBERO-GeoMotion zero-shot condition twice: once with the current GPT-GEAR Act executor, and once with a variant that keeps GPT-6's outputs, prompts, and success predicates identical but replaces Act with a waypoint planner that interpolates the same relative targets at 1 mm/0.005 rad without a 10-step cap and with the same kinematic checks plus a simple collision checker. If overall success rises materially, for example by at least 20 percentage points driven by the currently failing precision and rotation families, then the reported precision and coordination deficits are substantially harness-mediated and the paper should be reframed as an evaluation of GPT-GEAR plus GPT-6 rather than of GPT-6 alone; if success stays near 11.1%, the model-level attribution is supported.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 21 Oct 2026
- Reader holds up, with conditions
- Skeptic agrees
- Editor wrote the verdict above
- Referee major revision: 3 major objections, 7 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
Jim—
Two things to know about this one. It is the most complete empirical picture I have seen of GPT-6 Astra as a direct robot controller: capability probes, five benchmarks, a new geometry-constrained benchmark, and real hardware, all through one lightweight harness. And the headline result—strong task understanding with a real gap to physical execution—is credible, but the paper’s version of the gap needs a caveat: part of the precision/coordination deficit is probably the interface, not the model.
What is genuinely new: GPT-GEAR, a tool-mediated framework that keeps the model as the only planner, and LIBERO-GeoMotion, a benchmark that isolates geometric execution demands while keeping semantics simple. The design is thoughtful. The ablation story is the best part of the paper: depth maps beat point queries, textual force values beat tactile images, and action trajectories added to video/text give the biggest ICL gains. These are real, interpretable findings, and the real-world ICL jump from 40% to 80% is the most practical result in the paper.
The soft spots are proportionate. First, the harness confound is real. Act calls are discretized to 0.01 m / 0.05 rad increments with a 10-step cap, and the controller does no trajectory planning or collision checking. The paper acknowledges the controller’s limits (§3.6) but never isolates model from interface. So failures in LIBERO-GeoMotion precision tasks and the real-world block insertion could partly be an artifact of the command granularity and the lack of planning support. That said, the stress-test concern is not uniform: the vase knock-over, the wrist joint-limit error, and the drawer obstruction are decisions the model makes, so consequence-aware planning failures survive the confound. The paper would be stronger with a control condition—same interface with a finer discretization, or a simple planner on top—to see how much the gap closes.
Second, statistics are thin. Ten or twenty trials per task, five per real-world condition, no confidence intervals. The paper’s protocol check—reevaluating a few baselines and getting close numbers—is reasonable but not a substitute for error bars. The six capability dimensions are author-assigned; Memory has only six tasks, all from RoboDojo, so the aggregate profile is partly benchmark difficulty.
Third, the LIBERO-GeoMotion success predicates are not specified in the text. For a new benchmark, the release needs to define them clearly; the paper should at least sketch the verification mechanism.
None of this undermines the core claim. The semantic-vs-execution gap appears consistently across simulation, the targeted benchmark, and real hardware. This is a useful baseline for the field, and it deserves a serious referee. I would send it to review with a request for a harness-control experiment and tighter statistics.
The strongest objection
The central claim is that GPT-6 Astra itself shows a semantic-vs-execution gap, with precision and complex motion coordination as model-level deficits. All execution evidence passes through GPT-GEAR (Sections 3.6 and 4.2). Act accepts only a relative pose target; the controller decomposes it into increments capped at 0.01 m and 0.05 rad, with at most 10 native control steps per call, and explicitly performs no trajectory planning or collision checking. Under these bounds a single call can realize at most roughly 0.10 m and 0.5 rad, so any large reorientation or constrained motion must be decomposed into many sequential model decisions; there is no interface for expressing a smooth coordinated trajectory. Failures in LIBERO-GeoMotion (Table 6, e.g., narrow-passage transport and upright book placement) and in real-world block-in-cup and mango tasks are therefore not clean evidence about GPT-6's geometric reasoning: the same failures could arise from the harness forcing pose increments into a small discrete command budget with no way to plan around obstacles. The authors acknowledge that the controller does not guarantee collision-free motion, but that acknowledgment marks the confound rather than resolving it. The concern is not uniform: consequence-aware failures such as the vase knock-over, joint-limit wrist rotation, and drawer obstruction are decisions the model makes, so those parts of the gap survive. The un-isolated part is the paper's headline claim about precision and complex coordination.
The reader's weakest assumption identifies the most load-bearing threat: it sits exactly between the evidence (task success and failure under GPT-GEAR) and the conclusion (GPT-6's physical-execution limits). I agree with that identification. The confound does not overturn the paper, because the semantic-understanding strengths and the consequence-aware planning failures are less interface-dependent, and the findings converge across several benchmarks and real hardware. However, the precision and coordination headline is the part of the 'persistent gap' claim most exposed to the harness, and the paper provides no control condition that separates model capability from interface-mediated execution. The proposed executor-only ablation is feasible and decisive: it reuses GPT-6's outputs, changing only how their targets are executed. Because the reader's CONDITIONAL verdict already reflects the need for such checks, my assessment does not move the verdict.
The referee's report
If the central claim holds, the paper provides one of the most comprehensive capability-level characterizations of a general-purpose multimodal model as a direct manipulation controller, and it will be a useful reference point for the generalist-models-as-policies line of work. The evaluation is well triangulated: five official benchmarks with externally defined success predicates (which keep the main empirical claim non-circular), a purpose-built geometry benchmark, and real hardware with documented safety failures. The paper is commendably honest about negative results—the low RoboDojo and RoboCasa rates, the 40% real-world zero-shot success, and the joint-limit and collision incidents—and it ships code and a project page, supporting reproducibility. The six-dimension capability decomposition and the latency measurements (roughly 8-11x slower than the compared VLA baselines) are useful community assets. The main limitations—the action-interface confound for precision and coordination attributions, small trial counts in several tables, and the uncalibrated new benchmark—affect the strength of specific claims rather than the overall directional finding.
Pith's reading of the paper (a machine's summary, not the authors')
The paper asks whether a general-purpose multimodal model, never trained on robot data, can act as a closed-loop robot controller. Testing GPT-6 Astra through a lightweight tool interface called GPT-GEAR, the authors find real but partial embodied ability: the model uses depth information, picks up simple manipulation strategies from in-context demonstrations, and corrects its actions from tactile feedback. Across five benchmarks, six capability dimensions, a new geometry-constrained benchmark, and real-robot trials, a gap opens between semantic reasoning and physical execution: task understanding reaches 73.45% while long-horizon execution reaches 33.03%, with 11.1% zero-shot success on the new LIBERO-GeoMotion benchmark and 40% on real hardware. The paper's central claim is that general-purpose models are a promising but incomplete basis for direct robot control, with consequence-aware planning, precision, and coordinated motion as the specific bottlenecks.
What carries the argument. Two instruments carry the argument. The first is GPT-GEAR (GPT-Generalist End-effector Action Reasoning), a tool-mediated closed-loop harness: the model sees RGB observations, camera parameters, and optional depth maps and tactile signals, and it issues actions through an Act tool that moves the end effector by bounded increments (up to $0.01$ m translation and $0.05$ rad rotation per controller step, within a ten-step budget per call), while Observe, Reference, Status, and Finish handle history retrieval, demonstrations, completion checks, and termination. The controller checks kinematic feasibility but deliberately supplies no trajectory planning or collision checking, leaving all task-dependent motion planning to the model. The second instrument is LIBERO-GeoMotion, a benchmark of 36 scene variants across nine task families built on LIBERO assets, which isolates the paper's target failure modes — precise pose alignment, coordinated translation with rotation, and anticipation of physical consequences — by minimizing semantic difficulty. The mechanism that produces the central result is visible in the model's reasoning traces: it states the correct goal and begins a plausible motion, then loses geometric feasibility across the sequence.
Core claim. On the paper's own terms, the discovery is an uneven capability map of a general-purpose multimodal model acting as a robot controller. GPT-6 Astra, with no task-specific policy training and only five general tools, reaches 83.50% on LIBERO-Plus, 76.0% on RoboTwin 2.0, and 72.5% on UniVTAC, and it shows measurable emerging abilities: depth estimates within about 2–4 cm of ground truth, tactile-guided corrections during contact-rich insertion, and in-context adaptation that raises success whenever action trajectories accompany demonstrations. The same model falls to 26.92% on RoboDojo, 18.96% on RoboCasa GR1, 33.03% on long-horizon tasks, 36.19% on precision tasks, 11.1% zero-shot on the new LIBERO-GeoMotion benchmark, and 40% zero-shot on real hardware. The authors conclude that the bottleneck is not understanding what to do but executing it: the model initiates plausible motions yet fails to keep geometry, contact, and joint feasibility consistent across successive actions.
The premise it rests on. The paper attributes the robotic failures to the model's own limits, but the harness it uses — coarse action steps of $0.01$ m and $0.05$ rad within a tight step budget, and no collision or trajectory planning — could itself cause much of the measured imprecision, and the study never isolates the model from that interface.
If the paper is right (Pith's inferences, not the authors')
A general-purpose model with no robot-specific policy training can already act as a zero-shot controller for simple pick-and-place, so the open question shifts from whether such models can control robots to which physical demands exceed them.
The drop from 73.45% task understanding to 33.03% long-horizon execution implies that aggregate or pick-and-place-style benchmark scores overstate deployment readiness, and geometry-constrained tests are the discriminating measure.
One demonstration per task lifts real-world success from 40% to 80%, with every hard task improving, so embodiment-level in-context learning is a working adaptation mechanism while the model remains frozen.
Failures involving unintended contact and joint limits show that direct pose-based control without a collision-checking layer is not yet safe, so safe deployment requires either consequence-aware planning by the model or an external safety layer.
The roughly ten-fold latency overhead relative to specialized policies means interactive cost, not just success rate, must be counted before such models become practical for real-time robotics.
The coarse action interface ($0.01$ m steps, a ten-step budget, no trajectory planner) may itself cause part of the measured precision and coordination failures; the paper does not isolate model limits from harness limits, so a natural next test is the same model through a finer-grained or trajectory-level interface.
The fact that package storage stayed at 0% even with demonstrations suggests an ordering of difficulty — consequence-aware sequencing is the dimension least amenable to in-context help — which could guide where to invest in model capabilities versus external scaffolding.
The paper's data can be read as indirect evidence for hybrid designs that pair a strong semantic reasoner with a low-level motion planner, since that pairing targets exactly the observed failure class, even though the paper itself evaluates the pure direct-control setting.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Recent advances in visuospatial understanding and agentic reasoning enable direct robotic manipulation with general-purpose multimodal models, but their embodied capabilities remain poorly understood. This paper systematically evaluate GPT-6 Astra for closed-loop robotic manipulation using a lightweight agent execution framework. We first probe three fundamental embodied capabilities-depth perception, embodied in-context learning, and tactile understanding-and demonstrate that the model can exploit depth information, acquire simple manipulation strategies from contextual demonstrations, and use tactile feedback to correct its actions. We then evaluate GPT-6 Astra across five established manipulation benchmarks along six capability dimensions. Despite competitive performance on several benchmarks-26.92% success on RoboDojo and 76.0% on RoboTwin 2.0-our analysis reveals a pronounced gap between semantic reasoning and physical execution, particularly in consequence-aware action planning, precision, and complex motion coordination. To further examine these limitations, we introduce LIBERO-GeoMotion, a new benchmark of geometry-constrained tasks that require precise pose alignment, coordinated translation and rotation, and anticipation of physical action consequences. GPT-6 Astra achieves only 11.1% zero-shot success, exposing the difficulty of translating spatial understanding into physically feasible motion. Real-world experiments corroborate these findings: zero-shot execution is feasible for simple tasks, yet persistent deficits remain in motion feasibility, precise control, and physical safety, while substantial inference latency further hinders practical deployment. Overall, our results characterize the emerging embodied capabilities of general-purpose multimodal models and delineate the key challenges toward safe, efficient, and reliable robotic manipulation.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-21
The authors' main claim, as the classifier picked it out 0.97
“Overall, our results characterize the emerging embodied capabilities of general-purpose multimodal models and delineate the key challenges toward safe, efficient, and reliable robotic manipulation.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.83 · human participants 0.06 · controlled comparison 0.11 · randomized assignment 0.01 · states quantitative result 0.99 · compares with prior results 0.49 · mentions code or data availability 0.04
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Recent advances in visuospatial understanding and agentic reasoning enable direct robotic manipulation with general-purpose multimodal models, but their embodied capabilities remain poorly understood. | background 0.67 | 0.05 | 0.04 | asserted 1.00 |
| This paper systematically evaluate GPT-6 Astra for closed-loop robotic manipulation using a lightweight agent execution framework. | contribution 0.67 | 0.92 | 0.03 | asserted 0.97 |
| We first probe three fundamental embodied capabilities-depth perception, embodied in-context learning, and tactile understanding-and demonstrate that the model can exploit depth information, acquire simple manipulation strategies from contextual demonstrations, and use tactile feedback to correct its actions. | result 0.87 | 0.97 | 0.06 | asserted 1.00 |
| We then evaluate GPT-6 Astra across five established manipulation benchmarks along six capability dimensions. | method 0.90 | 0.95 | 0.03 | asserted 0.93 |
| Despite competitive performance on several benchmarks-26.92\% success on RoboDojo and 76.0\% on RoboTwin 2.0-our analysis reveals a pronounced gap between semantic reasoning and physical execution, particularly in consequence-aware action planning, precision, and complex motion coordination. | result 0.85 | 0.96 | 0.30 | asserted 1.00 |
| To further examine these limitations, we introduce LIBERO-GeoMotion, a new benchmark of geometry-constrained tasks that require precise pose alignment, coordinated translation and rotation, and anticipation of physical action consequences. | contribution 0.83 | 0.95 | 0.04 | asserted 0.99 |
| GPT-6 Astra achieves only 11.1\% zero-shot success, exposing the difficulty of translating spatial understanding into physically feasible motion. | result 0.99 | 0.96 | 0.11 | asserted 1.00 |
| Real-world experiments corroborate these findings: zero-shot execution is feasible for simple tasks, yet persistent deficits remain in motion feasibility, precise control, and physical safety, while substantial inference latency further hinders practical deployment. | result 0.97 | 0.94 | 0.19 | asserted 1.00 |
| Overall, our results characterize the emerging embodied capabilities of general-purpose multimodal models and delineate the key challenges toward safe, efficient, and reliable robotic manipulation. | interpretation 0.98 | 0.86 | 0.86 | asserted 1.00 |
What kind of paper the classifier reports this is: empirical study 1.00 · evidence from observation or experiment 0.99
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-21: 1 request, 7,062 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
References (66)
- [1] Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Daniel Ho, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Eric Jang, Rosario Jauregui Ruano, Kyle Jeffrey, ... on PithPith's reviewarXiv
- [2] Shuanghao Bai, Wenxuan Song, Jiayi Chen, Yuheng Ji, Zhide Zhong, Jin Yang, Han Zhao, Wanqi Zhou, Wei Zhao, Zhe Li, Pengxiang Ding, Cheng Chi, Haoang Li, Chang Xu, Xiaolong Zheng, Donglin Wang, Shanghang Zhang, and Badong Chen. Towards a unified understanding of robot manipulat... on PithPith's reviewarXiv
- [3] Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, et al. Gr00t n1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025. URLhttps://arxiv.org/abs/2503.14734 on PithPith's reviewarXiv
- [4] Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Robert Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang S... on Pith
- [5] Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, Pete Florence, Chuyuan Fu, Montse Gonzalez Arenas, Keerthana Gopalakrishnan, Kehang Han, Karol Hausman, Alexander Herzog, Jasm... on PithPith's reviewarXiv
- [6] Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Tomas Jackson, Sally Jesmonth, Nikhil J Joshi, Ryan Julian, Dmitry Kalashnikov,... on PithPith's reviewarXiv
- [7] Jun Cen, Chaohui Yu, Hangjie Yuan, Yuming Jiang, Siteng Huang, Jiayan Guo, Xin Li, Yibing Song, Hao Luo, Fan Wang, Deli Zhao, and Hao Chen. WorldVLA: Towards Autoregressive Action World Model.arXiv preprint arXiv:2506.21539, 2025. URLhttps://arxiv.org/abs/2506.21539 on PithPith's reviewarXiv
- [8] Baijun Chen, Weijie Wan, Tianxing Chen, Xianda Guo, Congsheng Xu, Yuanyang Qi, Haojie Zhang, Longyan Wu, Tianling Xu, Zixuan Li, Yizhe Wu, Rui Li, Xiaokang Yang, Ping Luo, Wei Sui, and Yao Mu. Univtac: A unified simulation platform for visuo-tactile manipulation data generatio... on PitharXiv
- [9] Boyuan Chen, Zhuo Xu, Sean Kirmani, Brian Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. SpatialVLM: Endowing vision-language models with spatial reasoning capabilities. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14... on Pith
- [10] Tianxing Chen, Zanxin Chen, Baijun Chen, Zijian Cai, Yibin Liu, Zixuan Li, Qiwei Liang, Xianliang Lin, Yiheng 33 Ge, Zhenyu Gu, et al. Robotwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation.arXiv preprint... on PithPith's reviewarXiv
- [11] Tianxing Chen, Yue Chen, Zixuan Li, Junyuan Tang, Kailun Su, Haoran Lu, Weijie Wan, Baijun Chen, Songling Liu, Haowen Yan, Honghao Su, Zhiyang Dou, Kaixuan Wang, Dandan Zhang, Yunze Liu, Yan Qin, Qiwei Liang, Qiwei Wu, Zijian Lin, Wenwei Lin, Yuran Wang, Minghua He, Tianshu Wu... on PithPith's reviewarXiv
- [12] Yanzhe Chen, Zechen Bai, Zhijun Cao, Wenzheng Zeng, Kevin Qinghong Lin, Yiqi Lin, Guoqiang Liang, Kevin Yuchen Ma, Qiming Huang, and Mike Zheng Shou. Show-Harness: Just a vlm agent can play robots.arXiv preprint arXiv:2609.10522, 2026. URLhttps://arxiv.org/abs/2609.10522 on PithPith's reviewarXiv
- [13] Dongzhou Cheng, Taoran Yi, Ye Fang, Xingwu Zhang, Fan Feng, Yixuan Li, Gengxiong Zhuang, Rongze Wang, Shuai Yang, Wei Song, Weizhi Xue, Minyan Wu, Jie Gui, Jiaqi Wang, and Tong Wu. In-context robot learning with vlm agents, 2026. URLhttps://arxiv.org/abs/2609.19138 on PithPith's reviewarXiv
- [15] Dexmal. OpenDM. Official DM0.5 code, models, and benchmark results, 2026. URLhttps://github.com/dexmal/ opendm. Accessed: 2026-10-18 on Pith
- [16] Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint... on Pith
- [17] Senyu Fei, Siyin Wang, Junhao Shi, Zihao Dai, Jikun Cai, Pengfang Qian, Li Ji, Xinzhe He, Shiduo Zhang, Zhaoye Fei, Jinlan Fu, Jingjing Gong, and Xipeng Qiu. Libero-plus: In-depth robustness analysis of vision-language-action models.arXiv preprint arXiv:2510.13626, 2025. URLht... on PithPith's reviewarXiv
- [18] Yankai Fu, Ning Chen, Junkai Zhao, Heng Zhang, Guocai Yao, Pengwei Wang, Zhongyuan Wang, and Shanghang Zhang. Decal: Towards physically-grounded dexterous vision-language-action models via contact-aware latent co-imagination.arXiv preprint arXiv:2609.09119, 2026 on PithPith's reviewarXiv
- [19] Abraham George, Selam Gano, Pranav Katragadda, and Amir Barati Farimani. Vital pretraining: Visuo-tactile pretraining for tactile and non-tactile manipulation policies. In2025 IEEE International Conference on Robotics and Automation (ICRA), pages 258–264. IEEE, 2025 on Pith
- [20] Dibya Ghosh, Homer Rich Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, Jianlan Luo, You Liang Tan, Lawrence Yunliang Chen, Quan Vuong, Ted Xiao, Pannag R Sanketi, Dorsa Sadigh, Chelsea Finn, and Sergey Levine. Octo: An open-... on Pithdoi:10.15607/rss.2024.xx.090
- [22] Bohan Hou, Jiuning Gu, Jiayan Guo, Ronghao Dang, Sicong Leng, Xin Li, Xuemeng Song, and Jianfei Yang. Interlv-search: Benchmarking interleaved multimodal agentic search.arXiv preprint arXiv:2605.07510, 2026. URL https://arxiv.org/abs/2605.07510 on PithPith's reviewarXiv
show 46 more
- [23] URLhttps://arxiv.org/abs/2609.22966 on PithPith's reviewarXiv
- [24] Wenlong Huang, Chen Wang, Ruohan Zhang, Yunzhu Li, Jiajun Wu, and Li Fei-Fei. VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models.arXiv preprint arXiv:2307.05973, 2023. URL https://arxiv.org/abs/2307.05973 on PithPith's reviewarXiv
- [25] Bohan Hou, Gen Li, Jindou Jia, Tuo An, Xinying Guo, Sicong Leng, Haoran Geng, Yanjie Ze, Tatsuya Harada, Philip Torr, et al. World model for robot learning: A comprehensive survey.The International Journal of Robotics Research, page 02783649261488980, 2026. URLhttps://doi.org/... on Pithdoi:10.1177/02783649261488980
- [26] Wenlong Huang, Chen Wang, Yunzhu Li, Ruohan Zhang, and Li Fei-Fei. ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation.arXiv preprint arXiv:2409.01652, 2024. URL https://arxiv.org/abs/2409.01652 on PithPith's reviewarXiv
- [27] Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, Pierre Sermanet, Tomas Jackson, Noah Brown, Linda Luu, Sergey Levine, Karol Hausman, and Brian Ichter. Inner monologue: Embodied reasoning th... on Pith
- [28] Dongyoung Kim, Huiwon Jang, Myungkyu Koo, Suhyeok Jang, Taeyoung Kim, et al. RLDX-1 technical report. arXiv preprint arXiv:2605.03269, 2026. URLhttps://arxiv.org/abs/2605.03269 on PithPith's reviewarXiv
- [29] Tao Jiang, Tianyuan Yuan, Yicheng Liu, Chenhao Lu, Jianning Cui, Xiao Liu, Shuiqi Cheng, Jiyang Gao, Huazhe Xu, and Hang Zhao. Galaxea open-world dataset and G0 dual-system VLA model.arXiv preprint arXiv:2509.00576, 2025. URLhttps://arxiv.org/abs/2509.00576 on PithPith's reviewarXiv
- [30] Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan P Foster, Pannag R Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. OpenVLA: An op... on Pith
- [31] Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision-language-action models: Optimizing speed and success.arXiv preprint arXiv:2502.19645, 2025. URLhttps://arxiv.org/abs/2502.19645 on PithPith's reviewarXiv
- [32] Kehan Li, Bohan Hou, Minghao Zhu, Tianyi Zhang, Zesen Cheng, Zhikai Wang, Sicong Leng, Xin Li, Xiao Lin, Biying Yao, et al. Rynnbrain 1.1: towards more capable and generalizable embodied foundation model.arXiv preprint arXiv:2607.17977, 2026. URLhttps://arxiv.org/abs/2607.17977 on PithPith's reviewarXiv
- [33] Jiajun Li, Tiecheng Guo, Yifan Ye, Rongyu Zhang, Xiaowei Chi, Qianpu Sun, Ying Li, Yunfan Lou, Yan Huang, Zhihe Lu, et al. Efficient-wam: A 1b-parameter world-action model with low-cost future imagination.arXiv preprint arXiv:2606.10040, 2026. URLhttps://arxiv.org/abs/2606.10040 on PithPith's reviewarXiv
- [34] Xin Li, Dongze Lian, Zhihe Lu, Jiawang Bai, Zhibo Chen, and Xinchao Wang. Graphadapter: Tuning vision- language models with dual knowledge graph.Advances in Neural Information Processing Systems, 36:13448–13466, 2023 on Pith
- [35] Qiang Li, Oliver Kroemer, Zhe Su, Filipe Fernandes Veiga, Mohsen Kaboli, and Helge Joachim Ritter. A review of tactile information: Perception and action through touch.IEEE Transactions on Robotics, 36(6):1619–1634, on Pith
- [36] Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning. InAdvances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track, 2023. URL https://proceedings.... on Pith
- [37] Fangcheng Liu, Yeqing Shen, Anda Cheng, Weishi Mi, Chao Tang, Chenyuan Liu, Yushun Xiang, Tingguang Li, Yong-Lu Li, and Yehui Tang. RoboICL: Embodied in-context learning with GPT-6 Astra.arXiv preprint arXiv:2609.34261, 2026. URLhttps://arxiv.org/abs/2609.34261 on PithPith's reviewarXiv
- [38] Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as policies: Language model programs for embodied control. In2023 IEEE International Conference on Robotics and Automation (ICRA), pages 9493–9500. IEEE, 2023. doi: 10... on Pithdoi:10.1109/icra48891.2023.10160591
- [39] Yunfan Lou, Yifan Ye, Yankai Fu, Jun Cen, Xiaowei Chi, Yaoxu Lyu, Peidong Jia, Sirui Han, Zhihe Lu, and Shanghang Zhang. Dream-tac: A unified tactile world action model for contact-rich robot manipulation.arXiv preprint arXiv:2606.08737, 2026. URLhttps://arxiv.org/abs/2606.08737 on PithPith's reviewarXiv
- [40] Zhihe Lu, Jiawang Bai, Xin Li, Zeyu Xiao, and Xinchao Wang. Beyond sole strength: Customized ensembles for generalized vision-language models. InInternational Conference on Machine Learning, 2024 on Pith
- [41] Haowen Liu, Xirui Li, Shaoxiong Yao, Peng Shi, Tianyi Zhou, Jia-Bin Huang, Furong Huang, and Jiayuan Mao. Guava: An effective and universal harness for embodied manipulation.arXiv preprint arXiv:2606.18363, 2026. URLhttps://arxiv.org/abs/2606.18363. 35 on PithPith's reviewarXiv
- [42] Jiangran Lyu, Kai Liu, Xuheng Zhang, Haoran Liao, Yusen Feng, Wenxuan Zhu, Tingrui Shen, Jiayi Chen, Jiazhao Zhang, Yifei Dong, Wenbo Cui, Senmao Qi, Shuo Wang, Yixin Zheng, Mi Yan, Xuesong Shi, Haoran Li, Dongbin Zhao, Ming-Yu Liu, Zhizheng Zhang, Li Yi, Yizhou Wang, and He W... on PithPith's reviewarXiv
- [43] Soroush Nasiriany, Abhiram Maddukuri, Lance Zhang, Adeet Parikh, Aaron Lo, Abhishek Joshi, Ajay Mandlekar, and Yuke Zhu. Robocasa: Large-scale simulation of everyday tasks for generalist robots. InRobotics: Science and Systems (RSS), 2024. URLhttps://www.roboticsproceedings.or... on Pith
- [44] Zhihe Lu, Jiawang Bai, Xin Li, Zeyu Xiao, and Xinchao Wang. Task-to-instance prompt learning for vision-language models at test time.IEEE Transactions on Image Processing, 34:1908–1920, 2025 on Pith
- [45] OpenAI. GPT-6 Astra: A new generation of intelligence. Official model announcement, 2026. URL https: //openai.com/index/gpt-6-astra/. Accessed: 2026-10-17 on Pith
- [46] Jiaqi Peng, Xiqian Yu, Delin Feng, Yuqiang Yang, Wenzhe Cai, Jing Xiong, Ganlin Yang, Jinliang Zheng, Jiafei Cao, Xueyuan Wei, Jiangmiao Pang, Yuan Shen, and Tai Wang. Cortex: A bidirectionally aligned embodied agent framework for long-horizon manipulation.arXiv preprint arXiv... on PithPith's reviewarXiv
- [47] NVIDIA. GR00T N1.5: An improved open foundation model for generalist humanoid robots. Official model release, June 2025. URLhttps://research.nvidia.com/labs/gear/gr00t-n1_5/. Accessed: 2026-10-19 on Pith
- [48] Lucy Xiaoyang Shi, Brian Ichter, Michael Robert Equi, Liyiming Ke, Karl Pertsch, Quan Vuong, James Tanner, Anna Walling, Haohuan Wang, Niccolo Fusai, Adrian Li-Bell, Danny Driess, Lachy Groom, Sergey Levine, and Chelsea Finn. Hi Robot: Open-ended instruction following with hie... on Pith
- [49] StarVLA Community. StarVLA: A lego-like codebase for vision-language-action model developing.arXiv preprint arXiv:2604.05014, 2026. URLhttps://arxiv.org/abs/2604.05014 on PithPith's reviewarXiv
- [50] Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Manuel Y. Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Devin Le... on PithPith's reviewarXiv
- [52] Zhiyuan Xu, Kun Wu, Junjie Wen, Jinming Li, Ning Liu, Zhengping Che, and Jian Tang. A survey on robotics with foundation models: Toward embodied AI.arXiv preprint arXiv:2402.02385, 2024. URL https: //arxiv.org/abs/2402.02385 on PithPith's reviewarXiv
- [53] Jiayi Su, Yixin Zheng, Mi Yan, Li Yi, Zhizheng Zhang, and He Wang. GPT 6 Astra as an embodied policy. Technical report and code, 2026. URLhttps://github.com/anonymous-report-421/eval-of-gpt-6-astra-as-policy on Pith
- [54] Jinhui Ye, Ning Gao, Senqiao Yang, Jinliang Zheng, Zixuan Wang, Yuxin Chen, Pengguang Chen, Yilun Chen, Shu Liu, and Jiaya Jia. Starvla-α: Reducing complexity in vision-language-action systems. InEuropean Conference on Computer Vision, pages 351–369. Springer, 2026 on Pith
- [55] URLhttps://arxiv.org/abs/2609.07398 on PithPith's reviewarXiv
- [56] Yifan Ye, Yankai Fu, Yaoxu Lv, Bohan Hou, Jun Cen, Lingdong Kong, Duo Zheng, Tianxing Chen, Jiaming Liu, Ziang Cao, et al. Data pyramid for embodied manipulation: a survey.arXiv preprint arXiv:2607.24744, 2026. URLhttps://arxiv.org/abs/2607.24744 on PithPith's reviewarXiv
- [57] Senqiao Yang, Chengyao Wang, Yuxin Chen, Zixuan Wang, Longxiang Tang, Haokun Gui, Jinhui Ye, Changsheng Lu, Xiaoyang Wu, Mingkang Zhu, Pengguang Chen, Shu Liu, Zhuotao Tian, Hengshuang Zhao, Bei Yu, and Jiaya 36 Jia. Beyond data scaling: Representation-centric continued pre-tr... on PithPith's reviewarXiv
- [58] Tao Yu, Zhihe Lu, Xin Jin, Zhibo Chen, and Xinchao Wang. Task residual for tuning vision-language models. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10899–10909. IEEE, 2023 on Pith
- [59] Yifan Ye, Jun Cen, Jing Chen, and Zhihe Lu. Self-evolved imitation learning in simulated world.IEEE Robotics and Automation Letters, 2026 on Pith
- [60] Tianyuan Yuan, Zibin Dong, Yicheng Liu, and Hang Zhao. Fast-WAM: Do world action models need test-time future imagination?arXiv preprint arXiv:2603.16666, 2026. URLhttps://arxiv.org/abs/2603.16666 on PithPith's reviewarXiv
- [61] Yifan Ye, Jiaqi Ma, Jun Cen, and Zhihe Lu. Token expand-merge: Training-free token compression for vision- language-action models.IEEE Robotics and Automation Letters, 2026 on Pith
- [62] Wenbo Zhang, Kaixuan Wang, Yutao Ouyang, Xiaoyu Huang, Liyang Li, Kailun Su, Weiyang Jin, Wenhao Chai, Haotian Liang, Zhiyang Dou, et al. An unexpected robot policy: Early evaluations of gpt-6 astra on robodojo and beyond.arXiv preprint arXiv:2609.24170, 2026. URLhttps://arxiv... on PithPith's reviewarXiv
- [63] Chengbo Yuan, Zicheng Zhang, Mingjie Zhou, Wendi Chen, Yi Wang, Zhuoyang Liu, Dantong Niu, Shuo Wang, Hui Zhang, Wenkang Zhang, Yingdong Hu, Yuanqing Gong, Wanli Xing, Chuan Wen, Cewu Lu, Kaifeng Zhang, and Yang Gao. FTP-1: A generalist foundation tactile policy across tactile... on PithPith's reviewarXiv
- [64] Tony Z. Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipulation with low-cost hardware. InProceedings of Robotics: Science and Systems, Daegu, Republic of Korea, July 2023. doi: 10.15607/RSS.2023.XIX.016. URLhttps://www.roboticsproceedi... on Pithdoi:10.15607/rss.2023.xix.016
- [65] Aiming Zhang, Tianyuan Yu, Liang Bai, Jun Tang, Yanming Guo, Yirun Ruan, Yun Zhou, and Zhihe Lu. Cola: Context-aware language-driven test-time adaptation.IEEE Transactions on Image Processing, 2025 on Pith
- [67] Yixian Zhang, Huanming Zhang, Feng Gao, Xiao Li, Zhihao Liu, Chunyang Zhu, Jiaxing Qiu, Yuchen Yan, Jiyuan Liu, Wenhao Tang, Zhengru Fang, Yi Nie, Changxu Wei, Yu Wang, Wenbo Ding, and Chao Yu. Harness VLA: Steering frozen vlas into reliable manipulation primitives via memory-... on PithPith's reviewarXiv
- [69] Jinliang Zheng, Jianxiong Li, Zhihao Wang, Dongxiu Liu, Xirui Kang, Yuchun Feng, Yinan Zheng, Jiayin Zou, Yilun Chen, Jia Zeng, et al. X-vla: Soft-prompted transformer as scalable cross-embodiment vision-language-action model. InInternational Conference on Learning Representat... on PithPith's reviewarXiv
- [2020] doi: 10.1109/TRO.2020.3003230. URLhttps://doi.org/10.1109/TRO.2020.3003230 on Pithdoi:10.1109/tro.2020.3003230
- [2026] URLhttps://arxiv.org/abs/2602.14979 on PitharXiv
Cited by 4 Pith papers
Reviewed papers on Pith that cite this one. The full citation record
- Reins: Frontier Multimodal Models Driving Real Robot Arms on Long-Horizon Tasks, Zero-Shot cs.RO · Nov 2026
- Embodied First Exam cs.RO · Nov 2026
- Flex-RSI: Benchmarking Recursive Self-Improvement of Frontier Agents cs.AI · Nov 2026
- LEASH: Stay Safe in Agentic Robotics cs.RO · Nov 2026
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment