Pith.

Robotics · arXiv:2610.41320 v1 · submitted 30 Oct 2026 · 38 references

Learning from Physical Experience:Context-Space Adaptation for Robotic Manipulation

Yihan Zeng, Yisu Li, Kaixin Huang, Marc WuDunn, Amelia Kovacs, Mikael Harseno, Kimmy Lin, Porter Hunley

show 4 more authors

Fred Shentu, Philipp Wu, Zhongyu Li, Zike Yan

The paper says

“Recent multimodal foundation models exhibit strong perception, reasoning, and in-context learning capabilities, but translating these capabilities into physical action remains challenging.”

more of the abstract

Through real-world experiments across diverse manipulation tasks, we find that ChatGPT-6-Astra, a state-of-the-art general-purpose reasoner, can serve as a physically actionable, context-responsive manipulation policy, demonstrating effective fine manipulation and spatial reasoning. Building on these findings, we introduce LEXP, a harness for learning from physical experience through context-space adaptation. LEXP combines task-level planning with an experience module that interprets execution evidence, extracts and retains lessons with applicability conditions, and reuses relevant lessons to generate contextual feedback for subsequent actions. Model weights and the action interface remain fixed throughout. Across 12 real-world manipulation tasks, LEXP achieves 23/36 successful trials (63.9%), compared with 19/36 (52.8%) with memory alone and 2/36 (5.6%) for vanilla Astra. On three tasks, bread transfer, sort trash and toiletries, and retrieve drawer, removing planning reduces success from 7/9 to 4/9, while removing retained experience reduces it to 3/9. Execution traces illustrate feedback-guided corrections and cross-episode reuse of an inspection strategy. Execution analyses illustrate how task structure and experience-derived feedback guide task progression, action correction, and experience reuse. These results support context-space adaptation as a practical mechanism for turning physical interaction into reusable guidance for robotic manipulation.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 2 Nov 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. The paper's real contribution is the memory effect (2/36 to 19/36); the LEXP-over-memory headline is a four-trial margin inside sampling noise, but the evaluation is genuine and deserves refereeing. This line is wrong

4 major objections · 6 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 38 references say.

The paper says, at §4.1-4.2, Table 2, Figure 6a

The review did not quote this passage.

Pith says: Objection 1 of 4 · major

The central incremental claim, that LEXP improves over memory alone (23/36 versus 19/36), is not supported at the reported sample size and no significance testing is reported. At the count level, the 12 per-task (Memory, LEXP) pairs contain 4 LEXP-only wins (ping-pong +1, bread +1, sort +1, drawer +2) and 1 memory-only win (cable tie -1), with 7 ties, so …

more of objection 1

… the exact two-sided sign test (equivalently McNemar's test on the discordant pairs, 4 versus 1) gives p = 0.375, and a Wilcoxon signed-rank test on the task deltas is also non-significant (p approximately 0.31). The 95% Wilson intervals around the two pooled rates (approximately 48-78% and 37-68%) overlap substantially. Since the margin is four net trials out of 36, the "further aggregate gain of 11.1 percentage points" stated in Section 4.2 is statistically indistinguishable from sampling noise. The paper should report trial-level outcomes and exact paired tests, or substantially increase the number of trials per task, before claiming that LEXP outperforms memory.

The paper says, at §4.1 (Protocol and metrics)

The review did not quote this passage.

Pith says: Objection 2 of 4 · major

The protocol matches only "similar distributions of initial configurations" across conditions rather than matched configurations or recorded configuration parameters. With three trials per task, a single easier or harder initial configuration changes a task's count by one full trial, which is exactly the scale of the four-trial aggregate margin.

more of objection 2

Configuration drift is therefore an uncontrolled channel for the LEXP-versus-memory comparison. The authors should either pair trials by initial configuration or record and report a configuration descriptor for each trial so the reader can check whether LEXP's gains and losses track configuration difficulty rather than the harness itself.

The paper says, at §4.2-4.4, Figure 7

The review did not quote this passage.

Pith says: Objection 3 of 4 · major

All of LEXP's net gain over memory is concentrated in the three tasks used for the component ablations (bread transfer +1, sort trash and toiletries +1, retrieve drawer +2); outside these tasks the comparison is exactly one LEXP win, one memory win, and ten ties.

more of objection 3

The paper does not justify how these three tasks were selected for the ablation study, and the ablation contrasts are themselves non-significant (Fisher exact two-sided p approximately 0.33 for 7/9 versus 4/9 without planning, and p approximately 0.08 for 7/9 versus 3/9 without retained experience, at n = 9 per condition). The category analysis in Figure 6c is not independent evidence either, since the long-horizon category is the same set of tasks. Given that the headline claim depends entirely on this subset, the ablation-task selection must be stated as a pre-specified criterion rather than a post hoc choice, and the corresponding caveat should accompany the abstract and the conclusions.

The paper says, at §4.1, §3.4, Figure 9

The review did not quote this passage.

Pith says: Objection 4 of 4 · major

Because LEXP carries retained lessons across trials within a task (cross-episode seeding is described in Section C.1 and its effect is documented in Figure 9), the three LEXP trials per task are serially dependent, whereas the memory and vanilla conditions are not.

more of objection 4

The pooled percentages therefore treat non-exchangeable trials as exchangeable Bernoulli draws. The paper should report per-episode outcomes (success on trials 1, 2, and 3 within each task) so the within-task learning curve is visible, and should avoid pooled-trial confidence statements without accounting for this dependence.

6 minor objections

The paper says, at §4.2, Figure 6b

Pith says: minor objection 1 of 6

The sole per-task LEXP regression (cable tie, 3/3 with memory versus 2/3 with LEXP) is not discussed; given that the aggregate margin is four trials, a sentence acknowledging per-task variability is warranted.

The paper says, at §4.1

Pith says: minor objection 2 of 6

The paper does not report how often trials ended by budget exhaustion rather than by actor completion; this matters for interpreting the zero-success conditions (e.g., drawer retrieval with memory) and should be stated per condition.

show 4 more

The paper says, at Figure 3b

Pith says: minor objection 3 of 6

The timeline labels feedback messages F1-F3 and F7-F8 but not F4-F6; please clarify whether intermediate messages were delivered and omitted from the figure, or were never issued.

The paper says, at Figure 9

Pith says: minor objection 4 of 6

The elapsed-time comparison (333 s versus 88 s and 83 s to the selected inspection action) is presented as evidence of faster reuse, but the three episodes differ in configuration and difficulty; the comparison should be explicitly caveated as illustrative.

The paper says, at §4.1, Appendix E

Pith says: minor objection 5 of 6

For reproducibility, the paper should report the exact GPT-6 Astra model snapshot and API date used, together with the sampling parameters (temperature, top-p, token budget) for the actor, planner, and experience-module calls, since the evaluated system is proprietary.

The paper says, at Abstract and Section 6

Pith says: minor objection 6 of 6

There are spacing and typographical errors in derived strings ("23/36successful trials", "4 /9", "7 /9") in the abstract and conclusion that should be cleaned before resubmission.

What would settle it

Compute the exact paired test from the published Table 2 counts: the 12 per-task (Memory, LEXP) outcome pairs have 4 discordant LEXP wins, 1 discordant memory win, and 7 ties, so the exact two-sided binomial/McNemar p-value is 0.375; since p ≥ 0.05, the 4-trial aggregate margin lies within sampling noise even before any allowance for configuration drift. If the authors want to establish the incremental claim, the decisive experimental follow-up is a matched-configuration rerun on the three gain-carrying tasks (bread transfer, sort trash and toiletries, retrieve drawer), holding initial configurations fixed across memory and LEXP conditions, counterbalancing trial order, and using more than three trials per condition; if the LEXP-over-memory margin does not survive matched configurations, the central claim reduces to the robust memory-vs-vanilla effect.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 2 Nov 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic agrees 2 min 20 s · 21,815 tokens read, 21,625 written
  3. Editor wrote the verdict above 36 s · 23,019 tokens read, 4,931 written
  4. Referee major revision: 4 major objections, 6 minor 1 min 58 s · 22,158 tokens read, 18,296 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

The short version: the memory result is the paper, and it's robust; the LEXP headline is not supported at this sample size. Frozen GPT-6 Astra goes from 2/36 to 19/36 successful trials when given interaction history, with fine manipulation jumping from 0/9 to 8/9. That is a large, believable effect. The paper's incremental claim — LEXP adds another four trials (23/36 vs 19/36) — is inside sampling noise: a sign test on the discordant per-task pairs (4 LEXP wins, 1 memory win) gives p ≈ 0.38, and the protocol matches only "similar distributions" of initial configurations, not matched ones, with no significance testing anywhere.

What's genuinely good. The evaluation is real: bimanual robot, 12 tasks, human-annotated success criteria, and every reported count checks out across the abstract, Figures 6–7, and Table 2. The harness design is thoughtful — planner subtask reference plus lessons carrying applicability, evidence, limits, and revision fields, distilled into one line of feedback for a frozen actor — and the bread-transfer trace (inspection onset dropping from 333s to 88s and 83s across episodes) is credible qualitative evidence of reuse. Reporting is honest: two task categories never succeed, Opus improves less and Sol not at all, and the authors state outright that the mechanism behind memory's benefit is open (b73). Related work is handled well; they don't oversell novelty.

Soft spots, in order. First, the incremental statistics. Three trials per task per condition, and the entire net LEXP-over-memory gain sits in exactly the three ablation tasks (bread, drawer, sort; ping-pong's +1 cancels cable-tie's −1 at the count level). Those ablation contrasts are themselves not significant (7/9 vs 4/9, Fisher p ≈ 0.33). Second, the comparison is asymmetric: memory history is per-episode while LEXP carries lessons across episodes, so cross-episode information transfer is confounded with lesson structure. The paper says carryover is identified but never quantifies it. Third, no code, data, or trial videos, and a proprietary model on a bespoke robot, so the specific numbers can't be independently checked. The memory-vs-vanilla gap is far too large for these concerns to bite; the LEXP-vs-memory margin is exactly where they bite.

Bottom line: this deserves a serious referee, not a desk reject. The platform is real, the memory finding is citable on its own, and the weak statistics are fixable — more trials, matched configurations, significance tests, and released video would settle the LEXP question. I'd send it out with exactly those requests. My own verdict is skeptical of the headline increment but convinced the paper belongs in the literature.

The strongest objection

The paper's reported arithmetic is internally consistent: the abstract, Table 2, and Figures 6–7 agree everywhere, and the memory-vs-vanilla margin (19/36 vs 2/36) is far too large to be sampling noise or configuration drift. The load-bearing step is the incremental claim that LEXP beats memory, which rests on four net trials. The 12 per-task (Memory, LEXP) pairs in Table 2 give four LEXP-only wins (ping-pong +1, bread +1, sort +1, drawer +2), one memory-only win (cable tie −1), and seven ties at the count level; trial-level outcomes are not released, so even these generous count-level ties cannot be resolved. The exact two-sided sign test (equivalently McNemar's test) on the discordant pairs, 4 versus 1, gives p = 0.375, and a Wilcoxon signed-rank test on the deltas gives p ≈ 0.31; even granting identical configurations, the headline margin is indistinguishable from sampling noise. Configuration drift is a second uncontrolled channel, because Section 4.1 matches only 'similar distributions of initial configurations' rather than matched configurations; with n = 3 per task, one easier or harder configuration moves a task by a full trial, which is the scale of the effect. The pattern of gains sharpens the problem: all of the net LEXP-over-memory improvement sits in bread transfer, sort trash and toiletries, and retrieve drawer, which are exactly the three tasks used for the Figure 7 ablations, and those ablation differences are themselves not significant (7/9 vs 4/9, Fisher exact p ≈ 0.33; 7/9 vs 3/9, p ≈ 0.15). Finally, LEXP episodes demonstrably accumulate lessons across trials (Figure 9's inspection onset: 333 s, then 88 s and 83 s), while the memory condition's history is per-episode; the paper states carryover is identified but never quantifies this context asymmetry. The memory finding is robust; the LEXP mechanism claim is not distinguishable from sampling variation at the reported sample size.

Good-faith reading: the paper's aim is to show that a frozen general-purpose reasoner can improve real manipulation by converting physical interaction into reusable context, with LEXP as the harness. For the central claim to hold, the LEXP-over-memory advantage must reflect the harness rather than sampling variation or configuration drift. I verified the arithmetic: every reported count in the abstract, Table 2, Figure 6, and Figure 7 is internally consistent (2/36, 19/36, 23/36; ablations 3/9, 4/9, 3/9, 7/9; subtask totals 13/27, 16/27, 12/27, 24/27). The memory-vs-vanilla contrast (17 trials) is robust and not in question. The fragile step is the incremental LEXP claim, and the paired test on the paper's own numbers confirms the reader's Wilson-interval intuition more cleanly: p = 0.375 for the sign/McNemar test, p ≈ 0.31 for the signed-rank analog. Because the protocol matches only 'similar distributions' of initial configurations, drift across conditions is an additional uncontrolled channel that a four-trial margin cannot absorb. The distribution of the gains also matters: they sit entirely in the three tasks later used for the component ablations, so the mechanism attribution is measured on the most favorable subset and is itself non-significant (Fisher p ≈ 0.33 and ≈ 0.15). No code, data, trial videos, or significance analysis are provided, and the model and robot are proprietary, so the concern cannot be resolved externally. I agree with the reader's weakest_assumption; this pass adds the exact paired-test computation and the gain-concentration point. My read does not change the verdict: CONDITIONAL remains the right call, because the paper's numbers are consistent and its memory finding is strong, but its central incremental claim is not statistically established as reported.

The referee's report

Taken at face value, the memory-versus-vanilla result (19/36 versus 2/36, including 0/9 to 8/9 on three fine-manipulation tasks) is a strong and internally consistent demonstration that a frozen general-purpose reasoner can use raw interaction history, and the traces in Figures 3, 8, and 9 concretely document the feedback mechanisms on which LEXP is built. The paper is in this respect genuinely transparent: the abstract, Table 2, and Figures 6-7 agree; per-cell counts are fully reported; and success is human-annotated against predefined physical criteria from Appendix A. The design separation between raw memory and retained lessons with applicability conditions is clean and well motivated. If the incremental LEXP-over-memory effect were established, this would be a valuable empirical result for context-based adaptation in manipulation. At present, however, that incremental claim rests on four net trials, is concentrated in exactly the three tasks used for the ablations, and is not statistically supported at the reported sample size; the paper's contribution is therefore currently at the level of a suggestive pilot study plus a solid prerequisite finding.

Pith's reading of the paper (a machine's summary, not the authors')

The paper claims that adapting a robot's behavior does not have to mean retraining the model: the same frozen general-purpose reasoner can get better at physical tasks if the only thing that changes is the context it is given. To test this, the authors built LEXP, a harness that plans a task into subtasks, checks each proposed action against what actually happened, stores what was learned as lessons with applicability conditions, and injects the relevant lesson as one line of feedback before the next action. Across 12 real-world manipulation tasks, LEXP succeeds in 23/36 trials (63.9%), versus 19/36 (52.8%) when the model is given only raw interaction memory and 2/36 (5.6%) when it is given no history at all. If the claim holds, the expensive ingredients of robot adaptation — demonstrations, reward engineering, and weight updates — can be partly replaced by a loop that compiles physical experience into reusable guidance.

What carries the argument. The load-bearing mechanism is the experience module, the part of LEXP that converts physical interaction records into reusable guidance. Each retained lesson carries its applicability conditions, supporting evidence, and limitations, so a finding can be revised when new evidence contradicts it and applied only when the current scene matches its conditions. The module works with the planner, which supplies the current subtask and proposed method as a task-level reference, so feedback is grounded in what the robot commanded and in what observations show rather than in the actor's self-reported intentions. The context compiler then closes the loop by assembling the actor's next input from the task instruction, current observations, history, and the generated feedback, keeping model weights and the action interface fixed.

Core claim. The central claim is that manipulation can be improved in context space rather than parameter space. With the actor $\pi_\theta$ and its control interface fixed, the only variable is the compiled context $c_t = C(l, o_t, H_t, f_t)$ that conditions each action: the task instruction, the current observation, the interaction history $H_t$, and experience-derived feedback $f_t$. The paper reports that GPT-6 Astra, given interaction history, goes from 0/9 to 8/9 on three fine-manipulation tasks and from 0/6 to 5/6 on two spatial-reasoning tasks, and that demonstrations and language corrections steer both strategy and individual motions. On top of that, LEXP adds a planner that tracks subtask progress and an experience module that compares each proposed method against observed physical outcomes, retains lessons with applicability conditions and counterevidence, and produces one line of feedback per decision. Across 12 tasks, LEXP completes 23/36 trials versus 19/36 with memory alone and 2/36 with no memory; on the three-task ablation, removing planning lowers success from 7/9 to 4/9 and removing retained experience lowers it to 3/9. These results are read as supporting context-space adaptation as a practical mechanism for turning physical interaction into reusable guidance.

The premise it rests on. The load-bearing premise is that three trials per task per condition, drawn from similar rather than matched initial configurations and with no significance testing, are enough to attribute the four-trial aggregate gap between LEXP (23/36) and memory alone (19/36) to the harness rather than to sampling variation or configuration drift.

If the paper is right (Pith's inferences, not the authors')

If the reported numbers hold, improving a manipulation policy no longer requires changing model weights: the same frozen actor gains 11.1 percentage points over raw memory, with the gain concentrated in long-horizon tasks (3/12 to 7/12).

Experience compiles into lessons that outlive the raw history window: the inspection strategy learned in one bread-transfer episode is reused in later episodes and appears earlier each time (333 seconds, then 88, then 83).

The two components are both doing measurable work: removing planning drops success from 7/9 to 4/9 on the three ablation tasks, and removing retained experience drops it to 3/9, the same level as memory alone.

Memory by itself is a strong baseline — the same actor goes from 0/9 to 8/9 on fine manipulation when given history — so the paper's contribution is specifically about turning that history into distilled, transferable guidance rather than about adding new capability.

Because adaptation happens in context, the cost structure of robot learning shifts from demonstrations, reward engineering, and physical trial infrastructure toward lesson extraction and reuse.

The paper's own per-actor data (Astra 0 to 8/9, Opus 0 to 4/9, Sol 1 to 0/9 with memory) suggest the ceiling is set by the base model's ability to use context; a direct extension would run full LEXP on Opus and Sol to see whether lesson feedback recovers what raw memory did not.

If lesson reuse is the active ingredient, a natural stress test follows: shrink the actor's history window and rely only on retained lessons, since the memory-versus-experience separation the paper draws implies performance should degrade gracefully rather than collapse.

The aggregate gain concentrates in long-horizon tasks, which predicts that LEXP's advantage over memory grows with the number of decisions a task requires — a scaling relationship that could be checked on longer task suites.

The lesson format (lesson, applicability, evidence, limits) is close to machine-checkable: one could automate applicability checks against scene state, removing the current reliance on the model judging when its own lessons apply.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Recent multimodal foundation models exhibit strong perception, reasoning, and in-context learning capabilities, but translating these capabilities into physical action remains challenging. Through real-world experiments across diverse manipulation tasks, we find that ChatGPT-6-Astra, a state-of-the-art general-purpose reasoner, can serve as a physically actionable, context-responsive manipulation policy, demonstrating effective fine manipulation and spatial reasoning. Building on these findings, we introduce LEXP, a harness for learning from physical experience through context-space adaptation. LEXP combines task-level planning with an experience module that interprets execution evidence, extracts and retains lessons with applicability conditions, and reuses relevant lessons to generate contextual feedback for subsequent actions. Model weights and the action interface remain fixed throughout. Across 12 real-world manipulation tasks, LEXP achieves 23/36 successful trials (63.9%), compared with 19/36 (52.8%) with memory alone and 2/36 (5.6%) for vanilla Astra. On three tasks, bread transfer, sort trash and toiletries, and retrieve drawer, removing planning reduces success from 7/9 to 4/9, while removing retained experience reduces it to 3/9. Execution traces illustrate feedback-guided corrections and cross-episode reuse of an inspection strategy. Execution analyses illustrate how task structure and experience-derived feedback guide task progression, action correction, and experience reuse. These results support context-space adaptation as a practical mechanism for turning physical interaction into reusable guidance for robotic manipulation.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-11-02

The authors' main claim, as the classifier picked it out 1.00

“These results support context-space adaptation as a practical mechanism for turning physical interaction into reusable guidance for robotic manipulation.”

How the main claim is stated: asserted 0.56

What the abstract itself states, as probabilities: claims first 0.05 · states limitation 0.36 · human participants 0.08 · controlled comparison 0.97 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.97 · mentions code or data availability 0.04

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Recent multimodal foundation models exhibit strong perception, reasoning, and in-context learning capabilities, but translating these capabilities into physical action remains challenging. background 0.80 0.04 0.03 asserted 1.00
Through real-world experiments across diverse manipulation tasks, we find that ChatGPT-6-Astra, a state-of-the-art general-purpose reasoner, can serve as a physically actionable, context-responsive manipulation policy, demonstrating effective fine manipulation and spatial reasoning. result 0.98 0.95 0.29 asserted 1.00
Building on these findings, we introduce LEXP, a harness for learning from physical experience through context-space adaptation. contribution 0.98 0.77 0.09 asserted 1.00
LEXP combines task-level planning with an experience module that interprets execution evidence, extracts and retains lessons with applicability conditions, and reuses relevant lessons to generate contextual feedback for subsequent actions. method 0.93 0.82 0.07 asserted 1.00
Model weights and the action interface remain fixed throughout. method 0.80 0.67 0.03 asserted 1.00
Across 12 real-world manipulation tasks, LEXP achieves $23/36$ successful trials (63.9\%), compared with $19/36$ (52.8\%) with memory alone and $2/36$ (5.6\%) for vanilla Astra. result 1.00 0.96 0.18 asserted 1.00
On three tasks, bread transfer, sort trash and toiletries, and retrieve drawer, removing planning reduces success from $7/9$ to $4/9$, while removing retained experience reduces it to $3/9$. result 1.00 0.95 0.06 asserted 1.00
Execution traces illustrate feedback-guided corrections and cross-episode reuse of an inspection strategy. result 0.83 0.91 0.04 asserted 1.00
Execution analyses illustrate how task structure and experience-derived feedback guide task progression, action correction, and experience reuse. result 0.40 0.90 0.08 asserted 0.99
These results support context-space adaptation as a practical mechanism for turning physical interaction into reusable guidance for robotic manipulation. interpretation 1.00 0.74 0.90 asserted 0.56

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 1.00

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-02: 1 request, 7,642 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (3)
Figure 2 of the paper
Figure 2. Memory supports fine manipulation and spatial reasoning. Success rates with and without memory for (a) GPT-6 Astra, (b) Opus 5.5, and (c) Sol 5.6 on three fine-manipulation tasks, and (d) GPT-6 Astra on two tasks requiring spatial reasoning. Memory consists of prior observations, actions and intention notes, and execution outcomes retained in the actor’s context. Bars show observed success rates, with successful trials over total trials indicated above each bar; error bars denote 95% Wilson confidence intervals. Memory improves Astra’s performance across the evaluated tasks.…
Figure 3 of the paper
Figure 3. Context steering changes strategies and follows spatially specific instructions. (a) Expert and recovery demonstrations lead to different grasp-checking and repositioning strategies during towel folding. (b) During barbecue transfer, online language feedback guides changes in grasp approach, lifting clearance, arm selection, and motion order. Selected observations and measured displacements illustrate the robot’s responses, including moving backward to grasp, lifting to avoid collision, moving the left arm away, and lifting before moving right. The timeline marks feedback delivery;…

Showing 2 of the paper's 3 figures. Figures are the authors', reproduced from the paper.

References (38)
  1. [1] Lakshya A Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J Ryan, Meng Jiang, Christopher Potts, Koushik Sen, Alexandros G. Dimakis, Ion Stoica, Dan Klein, Matei Zaharia, and Omar Khattab. GEPA: Reflecti... on PithPith's reviewarXivdoi:10.48550/arxiv.2507.19457
  2. [2] Michael Ahn, Anthony Brohan, Noah Brown, et al. Do as i can, not as i say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691, 2022. doi: 10.48550/arXiv.2204.01691. URLhttps://arxiv.org/ abs/2204.01691 on PithPith's reviewarXivdoi:10.48550/arxiv.2204.01691
  3. [3] Yuzheng Cai, Siqi Cai, Yuchen Shi, Zihan Xu, Lichao Chen, Yulei Qin, Xiaoyu Tan, Gang Li, Zongyi Li, Haojia Lin, Yong Mao, Ke Li, and Xing Sun. Training-free group relative policy optimization.arXiv preprintarXiv:2510.08191, on PitharXiv
  4. [4] Dongzhou Cheng, Taoran Yi, Ye Fang, Xingwu Zhang, Fan Feng, Yixuan Li, Gengxiong Zhuang, Rongze Wang, Shuai Yang, Wei Song, Weizhi Xue, Minyan Wu, Jie Gui, Jiaqi Wang, and Tong Wu. In-context robot learning with VLM agents. arXiv preprint arXiv:2609.19138, 2026. doi: 10.48550/... on PithPith's reviewarXivdoi:10.48550/arxiv.2609.19138
  5. [5] Letian Fu, Huang Huang, Gaurav Datta, Lawrence Yunliang Chen, William Chung-Ho Panitch, Fangchen Liu, Hui Li, and Ken Goldberg. In-context imitation learning via next-token prediction.arXiv preprint arXiv:2408.15980, on PithPith's reviewarXiv
  6. [6] Letian Fu, Justin Yu, Karim El-Refai, Ethan Kou, Haoru Xue, Huang Huang, Wenli Xiao, Guanzhi Wang, Dantong Niu, Fei-Fei Li, Guanya Shi, Jiajun Wu, Shankar Sastry, Yuke Zhu, Ken Goldberg, and Linxi Fan. CaP-X: A framework for benchmarking and improving coding agents for robot m... on PithPith's reviewarXivdoi:10.48550/arxiv.2603.22435
  7. [7] Wenlong Huang, Fei Xia, Ted Xiao, et al. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint arXiv:2207.05608, 2022. doi: 10.48550/arXiv.2207.05608. URLhttps://arxiv.org/abs/ 2207.05608 on PithPith's reviewarXivdoi:10.48550/arxiv.2207.05608
  8. [8] Wenlong Huang, Chen Wang, Ruohan Zhang, Yunzhu Li, Jiajun Wu, and Li Fei-Fei. VoxPoser: Composable 3d value maps for robotic manipulation with language models.arXiv preprint arXiv:2307.05973, 2023. doi: 10.48550/arXiv.2307.05973. URLhttps://arxiv.org/abs/2307.05973 on PithPith's reviewarXivdoi:10.48550/arxiv.2307.05973
  9. [9] Wenlong Huang, Chen Wang, Yunzhu Li, Ruohan Zhang, and Li Fei-Fei. ReKep: Spatio-temporal reasoning of relational keypoint constraints for robotic manipulation.arXiv preprint arXiv:2409.01652, 2024. doi: 10.48550/ arXiv.2409.01652. URLhttps://arxiv.org/abs/2409.01652 on PithPith's reviewarXivdoi:10.48550/arxiv.2409.01652
  10. [10] Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, et al. OpenVLA: An open-source vision-language-action model. arXiv preprint arXiv:2406.09246, 2024. doi: 10.48550/arXiv.2406.09246. URLhttps://arxiv.org/abs/ 2406.09246 on PithPith's reviewarXivdoi:10.48550/arxiv.2406.09246
  11. [11] Moo Jin Kim, Yihuai Gao, Tsung-Yi Lin, Yen-Chen Lin, Yunhao Ge, Grace Lam, Percy Liang, Shuran Song, Ming-Yu Liu, Chelsea Finn, and Jinwei Gu. Cosmos policy: Fine-tuning video models for visuomotor control and planning. arXiv preprint arXiv:2601.16163, 2026. doi: 10.48550/arXi... on PithPith's reviewarXivdoi:10.48550/arxiv.2601.16163
  12. [12] Bowen Li, Mayank Mishra, Y. Isabel Liu, Stone Tao, Nishanth Kumar, Alexander G. Gray, Ruwan Wickra- marachchi, Jonathan Francis, Sebastian Scherer, and Tom Silver. Recover, discover, plan: Learning skills and concepts from robot failures. arXiv preprint arXiv:2606.18328, 2026.... on PithPith's reviewarXivdoi:10.48550/arxiv.2606.18328
  13. [13] Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as policies: Language model programs for embodied control.arXiv preprint arXiv:2209.07753, 2022. doi: 10.48550/arXiv.2209.07753. URLhttps://arxiv.org/abs/2209.07753 on PithPith's reviewarXivdoi:10.48550/arxiv.2209.07753
  14. [14] Jacky Liang, Fei Xia, Wenhao Yu, et al. Learning to learn faster from human feedback with language model predictive control.arXiv preprint arXiv:2402.11450, 2024. doi: 10.48550/arXiv.2402.11450. URLhttps://arxiv. org/abs/2402.11450. 15 on PithPith's reviewarXivdoi:10.48550/arxiv.2402.11450
  15. [15] Zeyi Liu, Arpit Bahety, and Shuran Song. REFLECT: Summarizing robot experiences for failure explanation and correction. arXiv preprint arXiv:2306.15724, 2023. doi: 10.48550/arXiv.2306.15724. URLhttps://arxiv.org/ abs/2306.15724 on PithPith's reviewarXivdoi:10.48550/arxiv.2306.15724
  16. [16] Runyu Lu, Yubo Wu, Ethan Kou, Letian Fu, Wenli Xiao, Ajay Mandlekar, Yinzhen Xu, Guanya Shi, Ken Goldberg, Ang Chen, Mosharaf Chowdhury, Yuke Zhu, Linxi Fan, and Guanzhi Wang. ASPIRE: Agentic /skills discovery for robotics. arXiv preprint arXiv:2607.00272, 2026. doi: 10.48550/... on PithPith's reviewarXivdoi:10.48550/arxiv.2607.00272
  17. [17] Jianlan Luo, Zheyuan Hu, Charles Xu, You Liang Tan, Jacob Berg, Archit Sharma, Stefan Schaal, Chelsea Finn, Abhishek Gupta, and Sergey Levine. SERL: A software suite for sample-efficient robotic reinforcement learning. arXiv preprint arXiv:2401.16013, 2024. doi: 10.48550/arXiv... on PithPith's reviewarXivdoi:10.48550/arxiv.2401.16013
  18. [18] Jianlan Luo, Charles Xu, Jeffrey Wu, and Sergey Levine. Precise and dexterous robotic manipulation via human- in-the-loop reinforcement learning.Science Robotics, 10(105):eads5033, 2025. doi: 10.1126/scirobotics.ads5033. URLhttps://doi.org/10.1126/scirobotics.ads5033 on Pithdoi:10.1126/scirobotics.ads5033
  19. [19] AchuMenon, SravanthiMachcha, SabrinaZou, TzuKitChan, andJayChooi. GPT-6Astraonroboticmanipulation. RoboCurve evaluation report, September 4, 2026. URLhttps://openai.robocurve.org/gpt-6-astra/. Accessed September 25, 2026 on Pith
  20. [20] OpenAI. GPT-6 Astra model. OpenAI API documentation, 2026. URLhttps://developers.openai.com/api/ docs/models/gpt-6-astra. Accessed September 26, 2026 on Pith
show 18 more
  1. [21] Physical Intelligence, Ali Amin, Raichelle Aniceto, et al.π∗0.6: a VLA that learns from experience.arXiv preprint arXiv:2511.14759, 2025. doi: 10.48550/arXiv.2511.14759. URLhttps://arxiv.org/abs/2511.14759 on PithPith's reviewarXivdoi:10.48550/arxiv.2511.14759
  2. [22] Physical Intelligence, Kevin Black, Noah Brown, et al.π0.5: A vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054, 2025. doi: 10.48550/arXiv.2504.16054. URL https://arxiv. org/abs/2504.16054 on PithPith's reviewarXivdoi:10.48550/arxiv.2504.16054
  3. [23] Pratyusha Sharma, Balakumar Sundaralingam, Valts Blukis, Chris Paxton, Tucker Hermans, Antonio Tor- ralba, Jacob Andreas, and Dieter Fox. Correcting robot plans with natural language feedback.arXiv preprint arXiv:2204.05186, 2022. doi: 10.48550/arXiv.2204.05186. URLhttps://arx... on PithPith's reviewarXivdoi:10.48550/arxiv.2204.05186
  4. [24] Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning.arXiv preprint arXiv:2303.11366, 2023. doi: 10.48550/arXiv.2303.11366. URLhttps://arxiv.org/abs/2303.11366 on PithPith's reviewarXivdoi:10.48550/arxiv.2303.11366
  5. [25] Kaustubh Sridhar, Souradeep Dutta, Dinesh Jayaraman, and Insup Lee. RICL: Adding in-context adaptability to pre-trained vision-language-action models.arXiv preprintarXiv:2508.02062, 2025. doi: 10.48550/arXiv.2508.02062. URLhttps://arxiv.org/abs/2508.02062 on PithPith's reviewarXivdoi:10.48550/arxiv.2508.02062
  6. [26] Jiayi Su, Yixin Zheng, Mi Yan, Li Yi, Zhizheng Zhang, and He Wang. GPT 6 Astra as an embodied policy. Technical report and code, 2026. URLhttps://github.com/anonymous-report-421/GPT-as-Policy . Accessed September 25, 2026 on Pith
  7. [27] Peidong Wang, Zhiming Ma, Ying Chang, Xufang Luo, Yiqun Zhang, Zihan Wang, Xiaocui Yang, Shi Feng, Yuqing Yang, and Dongsheng Li. Self-evolving embodied agents via skill-harness evolution.arXiv preprint arXiv:2608.11350, 2026. doi: 10.48550/arXiv.2608.11350. URLhttps://arxiv.o... on PithPith's reviewarXivdoi:10.48550/arxiv.2608.11350
  8. [28] Wenli Xiao, Jia Xie, Tonghe Zhang, Haotian Lin, Letian Fu, Haoru Xue, Jalen Lu, Yi Yang, Cunxi Dai, Zi Wang, Jimmy Wu, Guanzhi Wang, S. Shankar Sastry, Ken Goldberg, Linxi Fan, Yuke Zhu, and Guanya Shi. ENPIRE: Agentic robot policy self-improvement in the real world.arXiv prep... on PithPith's reviewarXivdoi:10.48550/arxiv.2606.19980
  9. [29] Seonghyeon Ye, Yunhao Ge, Kaiyuan Zheng, et al. World action models are zero-shot policies.arXiv preprint arXiv:2602.15922, 2026. doi: 10.48550/arXiv.2602.15922. URLhttps://arxiv.org/abs/2602.15922 on PithPith's reviewarXivdoi:10.48550/arxiv.2602.15922
  10. [30] Yida Yin, Zekai Wang, Yuvan Sharma, Dantong Niu, Trevor Darrell, and Roei Herzig. In-context learning enables robot action prediction in LLMs.arXiv preprint arXiv:2410.12782, 2024. doi: 10.48550/arXiv.2410.12782. URL https://arxiv.org/abs/2410.12782. 16 on PithPith's reviewarXivdoi:10.48550/arxiv.2410.12782
  11. [31] Lihan Zha, Yuchen Cui, Li-Heng Lin, Minae Kwon, Montserrat Gonzalez Arenas, Andy Zeng, Fei Xia, and Dorsa Sadigh. Distilling and retrieving generalizable knowledge for robot manipulation via language corrections.arXiv preprint arXiv:2311.10678, 2023. doi: 10.48550/arXiv.2311.1... on PithPith's reviewarXivdoi:10.48550/arxiv.2311.10678
  12. [32] Qizheng Zhang, Changran Hu, Shubhangi Upasani, Boyuan Ma, Fenglu Hong, Vamsidhar Kamanuru, Jay Rainton, Chen Wu, Mengmeng Ji, Hanchen Li, Urmish Thakker, James Zou, and Kunle Olukotun. Agentic context engineering: Evolving contexts for self-improving language models.arXiv prep... on PithPith's reviewarXivdoi:10.48550/arxiv.2510.04618
  13. [33] Wenbo Zhang, Kaixuan Wang, Yutao Ouyang, Xiaoyu Huang, Liyang Li, Kailun Su, Weiyang Jin, Wenhao Chai, Haotian Liang, Zhiyang Dou, Yue Chen, and Tianxing Chen. An unexpected robot policy: Early evaluations of GPT-6 Astra on RoboDojo and beyond.arXiv preprint arXiv:2609.24170, ... on PithPith's reviewarXivdoi:10.48550/arxiv.2609.24170
  14. [34] Yixian Zhang, Huanming Zhang, Feng Gao, Xiao Li, Zhihao Liu, Yi Nie, Chunyang Zhu, Jiaxing Qiu, Yuchen Yan, Jiyuan Liu, Wenhao Tang, Jiaji Rao, Zhengru Fang, Changxu Wei, Yu Wang, Wenbo Ding, and Chao Yu. Harness VLA: Steering frozen VLAs into reliable manipulation primitives ... on PithPith's reviewarXivdoi:10.48550/arxiv.2607.08448
  15. [35] Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. ExpeL: LLM agents are experiential learners.arXiv preprint arXiv:2308.10144, 2023. doi: 10.48550/arXiv.2308.10144. URLhttps: //arxiv.org/abs/2308.10144 on PithPith's reviewarXivdoi:10.48550/arxiv.2308.10144
  16. [36] Jiaqiang Ye Zhu, Carla Gomez Cano, David Vazquez Bermudez, and Michal Drozdzal. InCoRo: In-context learning for robotics control with feedback loops.arXiv preprint arXiv:2402.05188, 2024. doi: 10.48550/arXiv.2402.05188. URLhttps://arxiv.org/abs/2402.05188. 17 Appendix A Task D... on PithPith's reviewarXivdoi:10.48550/arxiv.2402.05188
  17. [2024] doi: 10.48550/arXiv.2408.15980. URLhttps://arxiv.org/abs/2408.15980 on PithPith's reviewarXivdoi:10.48550/arxiv.2408.15980
  18. [2025] doi: 10.48550/arXiv.2510.08191. URLhttps://arxiv.org/abs/2510.08191 on Pithdoi:10.48550/arxiv.2510.08191

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment