Pith.

Robotics · arXiv:2610.26846 v1 · submitted 19 Oct 2026 · 32 references

GPT-6 Astra Is a Native Visuo-Tactile Policy for Contact-Rich Manipulation

Chen Yang, Linzhe Shi, Hang Zhang, Mu Xu, Jiansheng Fan, Chen Wang

The paper says

“Recent advances in multimodal models have enabled GPT-6 Astra to perform robotic manipulation through in-context learning.”

more of the abstract

Yet contact-rich control and tactile perception are often regarded as strengths of specialist policies, leaving open whether Astra can directly understand and use tactile observations. We provide, to our knowledge, the first demonstration that Astra directly interprets vision-based tactile images and uses their contact cues for manipulation. Tactile RGB images enter through its existing visual interface, enabling few-shot closed-loop control on nineteen simulated UniVTAC and OpenViTac tasks with 210 sampled scene seeds per setting. With visuo-tactile input, Astra achieves mean success rates of 93.8% and 81.8%, exceeding the strongest reported specialist baselines in our comparison by 22.9 and 13.1 percentage points, respectively. Vision-only performance also leads these baselines, reaching 76.3% and 80.6%. Trajectory analysis reveals active probing, cautious grasping, contact-guided alignment, and recovery, alongside difficulties with tangential loading and corrective execution. Our findings challenge the exclusivity of specialist policies in contact-rich manipulation and show that fine-grained vision can partially compensate for missing touch. They also identify vision-based tactile sensing as an interface for scaling contact understanding with foundation models and reveal signs of benchmark saturation. These results motivate joint visual-tactile pretraining, harder contact-rich evaluations, and physical-robot validation.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 20 Oct 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. The core claim — that a general multimodal model can read tactile RGB images and use them for closed-loop contact-rich control — is supported by internally controlled evidence; the headline margins over specialist baselines are not, because they compare unmatched samples. This line is wrong

2 major objections · 5 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 32 references say.

The paper says, at §4 'Baselines and Metrics'; Abstract; §5.1; §8

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.80):

“Cost constraints limit most tasks to ten episodes per input setting instead of the full 100-episode evaluation, so a single outcome changes the task success rate by ten percentage points.”

Pith says: Objection 1 of 2 · major

The headline margins (22.9 and 13.1 percentage points) compare Astra's ten-scene-per-task sample, drawn with a single sampling seed (42), against published means computed over the benchmarks' full evaluations. The authors disclose this protocol ('Astra is evaluated on the sampled scenes; baseline values are the corresponding reported benchmark results', Section 4) and Section 8 acknowledges the fragility (b349: one changed outcome moves a task score by ten points; b350: no variance estimate from scene selection).

more of objection 1

What remains unresolved is that the Abstract, Figure 1, and Conclusion present the margins as unqualified point estimates, and nothing establishes that the gaps survive scene-selection variance: with ten episodes per task, one flipped outcome shifts the OpenViTac mean by roughly 0.9 points, several correlated flips together with a sampled scene set that is systematically easier than the full benchmark could move the 13.1-point margin materially, and no variance estimate is provided for either Astra's scores or the baseline means. The 22.9-point UniVTAC margin is large enough to resist moderate sampling noise; the 13.1-point OpenViTac margin is not. I request a matched-seed re-run of the strongest baselines on the same sampled scenes (at least OpenVTLA on OpenViTac and SimpleTouch on UniVTAC), or, failing that, per-task variance or bootstrap intervals for Astra and a rephrasing of the Abstract and Section 5.1 so the gaps are explicitly reported as sampled-scene estimates rather than benchmark-level facts.

The paper says, at §4 'Success Criteria'; Table 2

The review did not quote this passage.

Pith says: Objection 2 of 2 · major

The Grasp Chip criterion change is disclosed in Section 4, but its effect on the comparison is never quantified. Astra's 100% on Grasp Chip is scored under an adjusted definition that 'accepts supported placement without requiring a particular release-and-withdrawal sequence', while OpenVTLA's 69% and every other baseline use the original criterion.

more of objection 2

Since Grasp Chip carries one-eleventh of the OpenViTac mean, the 13.1-point headline margin compares policies under two different success definitions, and the direction of the bias is known: restoring the original criterion can only leave Astra's score unchanged or lower it. Please re-score the sampled Grasp Chip episodes under the original criterion and report both numbers; if the original-criterion score is materially below 100%, either exclude Grasp Chip from the headline mean or report the OpenViTac margin conditionally.

5 minor objections

The paper says, at §2 Related Work

Pith says: minor objection 1 of 5

The sentence 'Tactile feedback has also been studied in Astra-based peg insertion (Tanaka, 2026)' is the only text delimiting the claimed 'first demonstration' of direct tactile-image interpretation; please add one clause stating whether TacZero's tactile feedback is vision-based or non-visual, so the boundary of the novelty claim is verifiable from the manuscript.

The paper says, at Figure 1 caption

Pith says: minor objection 2 of 5

The caption states that task-level results and evaluation scope appear in Section 5, but the sampling protocol (ten scenes per task, seed 42) is defined in Section 4; a one-sentence note that Astra bars reflect a single-seed sample while baseline bars are published full-evaluation means would prevent over-reading of the figure.

show 3 more

The paper says, at Abstract

Pith says: minor objection 3 of 5

The phrase '210 sampled scene seeds per setting' is easy to misread as 210 tasks; consider stating the breakdown (80 UniVTAC plus 130 OpenViTac seeds) at least once, as Section 4 does.

The paper says, at §5.2, §6

Pith says: minor objection 4 of 5

The trajectory analysis uses model-generated action notes as part of the evidence for the policy's interpretations; the practice of pairing notes with recorded observations is sound, but one sentence acknowledging that the notes may not reflect the model's internal state would keep the qualitative evidence calibrated.

The paper says, at Notation, throughout

Pith says: minor objection 5 of 5

The benchmark name appears as both 'UniVTAC' (Tables 1, Figure 1, Figure 3) and 'UniVT AC' (body text and references); please unify the notation.

What would settle it

Re-run SimpleTouch on Astra's 80 UniVTAC seed-42 scenes and OpenVTLA on the 130 OpenViTac seed-42 scenes, score all methods under the original Grasp Chip criterion (rescore Astra's ten recorded Grasp Chip rollouts against the release-and-withdrawal requirement), and recompute both headline margins with per-task cluster bootstrap. Decision rule: if the OpenViTac margin falls from 13.1 points toward or below roughly 10 points, or the UniVTAC margin shrinks by more than a few points, re-scope the abstract's 'exceeding ... by 22.9 and 13.1 percentage points' to a matched-evaluation comparison; if both margins survive with separating confidence intervals, the conditional can be lifted.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 20 Oct 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic agrees 1 min 26 s · 21,348 tokens read, 13,173 written
  3. Editor wrote the verdict above 26 s · 22,407 tokens read, 3,484 written
  4. Referee major revision: 2 major objections, 5 minor 1 min 36 s · 21,760 tokens read, 16,055 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Here's my read. The paper's real contribution is not the headline number; it is the internally controlled evidence that a general multimodal model can act on vision-based tactile RGB images in closed-loop manipulation. Appendix B's same-seed pairs — vision-only fails, visuo-tactile succeeds on the same scene — are the strongest part, because they isolate the tactile input as the change that flips outcomes, and the rollouts show contact-state changes rather than hidden prompt trickery. The 19-task sweep across UniVTAC and OpenViTac, the vision-versus-touch comparison, and the visual-compensation analysis are new relative to the cited literature, and the paper is candid about its failures. The tables add up, and the limitations section is genuinely informative.

The soft spots, in decreasing order: the margins in the abstract are apples-to-oranges. Astra is scored on ten sampled scenes per task with one seed; the baselines are the published full-evaluation means. The authors disclose this in Section 8, but the abstract and Figure 1 state the 22.9 and 13.1 point gaps as if the comparison were matched. With one episode worth ten points per task, those margins could shrink. The OpenViTac margin is the weaker of the two because the Grasp Chip success criterion was loosened on the Astra side, and there is no sensitivity analysis. I am not doubting the published baseline values; I am doubting the comparison design. The second soft spot is framing: the first-demonstration claim is hard to square with the cited TacZero work, which already reports training-free peg insertion with a general vision-language model and tactile feedback. The paper should state the boundary explicitly. A minor issue: the trajectory analysis leans on the model's own action notes as evidence of understanding. The paired outcomes and recorded contact changes carry the claim, so I would treat the notes as illustrative rather than load-bearing.

Overall, the capability claim holds up as a demonstration, even though the quantitative superiority claim does not yet. This paper deserves a serious referee: the unmatched-baseline problem is fixable with matched-seed reruns or variance reporting, and the visual-compensation and benchmark-saturation observations deserve scrutiny. I would not rely on the numerical gaps until the authors re-run the strongest baselines under matched scenes, but I would send this to peer review and would bring it to a reading group.

The strongest objection

Section 4 fixes the evaluation design: 'Astra is evaluated on the sampled scenes; baseline values are the corresponding reported benchmark results.' The quantitative headline (Abstract, b4; Section 5.1, b115) therefore rests on comparing an 80- or 130-episode, single-seed (42) sample against published means computed over the benchmarks' full evaluations. The paper candidly discloses the fragility in Section 8 (b349: one outcome changes a task score by ten percentage points; b350: no variance from scene selection), but the Abstract and Figure 1 state the margins without that qualification, and what remains unresolved is whether the margins survive a matched comparison. The OpenViTac margin (13.1 points) is the weakest link for two additive reasons: (i) the Grasp Chip success criterion was loosened (Section 4, original conditions required a particular release-and-withdrawal sequence), so Table 2 scores Astra's 100% under a different success definition than the OpenVTLA 69% it is compared against; (ii) the eleven-task equal-weight mean (81.8%, 130 episodes) shifts by roughly 0.9 points per episode, so a few episode-level swings plus the criterion change could move the margin toward single digits. The UniVTAC margin (22.9 points) involves no criterion change and is larger, but still lacks any matched-scene or variance check on either side. The capability claim is better supported: Appendix B's same-seed V-fail/V+T-success pairs are internally controlled evidence that tactile images change outcomes, the Tables 1-2 arithmetic is internally consistent (running the means confirms the stated gaps), and the rollouts in Figures 4-11 document contact-state reasoning. A secondary boundary issue: the 'first demonstration' framing is not explicitly reconciled with the cited TacZero (Tanaka, 2026), reported as training-free peg insertion with tactile feedback using a general-purpose vision-language model.

The central claim decomposes into a capability claim (Astra can interpret tactile RGB images through its existing visual interface and use contact cues for manipulation) and a quantitative superiority claim (mean success rates of 93.8% and 81.8%, exceeding the strongest reported specialist baselines by 22.9 and 13.1 points). The capability claim is reasonably supported: the V and V+T settings share demonstration blocks, segment starts, actions, and scene seeds (Section 3, Appendix A), so Appendix B's paired same-seed failures and successes control for scene difficulty and isolate the tactile images; the per-task arithmetic in Tables 1-2 checks out (UniVTAC V+T: 750/8 = 93.8; OpenViTac V+T: 900/11 = 81.8; gaps against SimpleTouch 70.9 and OpenVTLA 68.7 are exact), and the seeded rollouts show contact-state inference, probing, alignment, and recovery. The quantitative claim is where the evidence is thinnest, for the same reason the reader identified: Section 4 does not re-run baselines on Astra's sampled scenes, uses one sampling seed, and provides no variance estimate on either side; one episode equals 10 points per task (b349, acknowledged); and the Grasp Chip criterion change means the OpenVTLA comparison on that task mixes success definitions. The paper deserves credit for disclosing all of this in Section 8 and Section 4, and I am not treating an acknowledged limitation as a new finding; the unresolved issue is that the headline numbers in the Abstract and Figure 1 stand unqualified despite the acknowledged noise, and no sensitivity analysis quantifies how the stated margins would move under the disclosed criterion change or a matched sample. The decisive check is concrete and feasible because the baseline methods come from released benchmark frameworks (UniVTAC, OpenViTac, SimpleTouch) and Astra's rollouts are archived. Two secondary concerns, noted but not load-bearing for the verdict: the mechanism-level ambiguity that tactile-image gains concentrate in fixed-camera-only tasks and could partly reflect an additional close-up viewpoint rather than deformation-based contact inference, and the unstated novelty boundary against TacZero. Agreement with the reader is full on the weakest assumption; CONDITIONAL remains the right verdict pending the matched comparison.

The referee's report

The capability claim is significant and, in my assessment, genuinely evidenced. The same-seed paired rollouts (Appendix B) provide internally controlled demonstrations that tactile images change the policy's decisions and outcomes; the failure analysis (Section 6) shows the results were not obtained by gaming the tasks; and Section 5.3's visual-compensation analysis adds an honest check on what 'tactile understanding' means in these benchmarks. The released code, project page, and explicit cost and latency disclosure (Section 7) strengthen the paper's usefulness. Two reservations bound the significance. First, the quantitative headline rests on an unmatched comparison between a ten-scene-per-task single-seed sample and published full-evaluation baseline means (Section 4), so the precise margins are not yet established; the capability claim itself is not in question. Second, the paper's own saturation analysis (Section 5.5) shows several nominally tactile tasks are solvable from visible cues, so the results demonstrate a ceiling for these benchmarks at least as much as they isolate tactile understanding. Both reservations are addressable within the manuscript's scope.

Pith's reading of the paper (a machine's summary, not the authors')

This paper tries to establish that a general multimodal model, GPT-6 Astra, can act directly as a contact-rich manipulation policy by receiving vision-based tactile images through its ordinary image interface, with no tactile-specific training. The authors run few-shot closed-loop control on nineteen simulated tasks from two visuo-tactile benchmarks and report mean success rates of 93.8% and 81.8%, exceeding the strongest published specialist baselines in their comparison by 22.9 and 13.1 percentage points. They further show that vision alone reaches 76.3% and 80.6%, ahead of those same baselines, by exploiting subtle visual texture and slip cues in tasks intended to require touch. If correct, the result matters because it challenges the assumption that contact-rich manipulation needs specialist tactile policies, and it identifies vision-based tactile sensing as a ready-made interface for scaling tactile understanding with foundation models.

What carries the argument. The mechanism is direct tactile-image in-context control: vision-based tactile sensors such as GelSight render local contact deformation and geometry as RGB images, and GPT-6 Astra processes those images through the same visual interface as ordinary scene images. Demonstration blocks pair contact observations with action sequences, execution receipts, and resulting observations, so the model can relate a current contact pattern to what the fingers did and what happened to the object. The action command is an ordered sequence $a=(\Delta p, \Delta r, u)$ of translation, rotation, and gripper commands sampled from the posterior $A_t \sim \pi(\cdot \mid g, D, M_t, x_t, I_t, T_t)$; online interaction memory provides the latest execution feedback and distributed anchors across task stages, allowing the model to revise decisions when a command completes but the task does not.

Core claim. The central discovery, on the paper's own terms, is the first demonstration that GPT-6 Astra directly interprets vision-based tactile images and uses their contact cues for manipulation. Tactile RGB images from the two finger-contact sensors enter through the model's existing visual interface; together with scene images, robot state, few-shot demonstration blocks, and bounded interaction memory, the model generates end-effector and gripper commands in closed loop. On 8 UniVTAC tasks and 11 OpenViTac tasks, with ten sampled scene seeds per task, visuo-tactile Astra scores 93.8% and 81.8% mean success, exceeding the strongest reported specialist baselines (SimpleTouch at 70.9% and OpenVTLA at 68.7%) by 22.9 and 13.1 percentage points, while vision-only Astra scores 76.3% and 80.6%. Trajectory analysis attributes the gains to contact-state inference, active probing, cautious grasp formation, contact-guided alignment, and recovery from failed placement; it also documents specific failures in tangential-load interpretation, fine corrective execution, and probe convergence.

The premise it rests on. The headline comparison assumes that published specialist baseline means are directly comparable to Astra's scores computed on a different, much smaller ten-scene sample per task, with no matched-seed re-run and no variance estimate for either side.

If the paper is right (Pith's inferences, not the authors')

A general multimodal model with no tactile-specific training can serve as a contact-rich policy, so the presumed moat of specialist tactile policies is weaker than commonly assumed.

Vision-based tactile sensing becomes a practical interface for scaling contact understanding: any improvement in the general visual model may transfer to tactile interpretation without retraining a tactile encoder.

Fine-grained vision can partially replace touch on tasks where texture, contour, or visible slip correlates with the underlying physical property, so benchmarks must control visual correlates to measure true tactile understanding.

Current visuo-tactile benchmarks show signs of saturation for this model class: thirteen of nineteen tasks reach at least 90% sampled success, so harder tasks with visually hidden physical properties are needed.

Joint visual-tactile pretraining is a natural next step: aligned scene and tactile images with actions and outcomes could make contact understanding part of general multimodal learning rather than deployment-time inference.

If the reported gaps generalize to physical robots, the practical implication is that foundation-model APIs with vision inputs could replace expensive specialist tactile policy training for many contact-rich tasks, with the main deployment bottleneck shifting from policy learning to latency, cost, and closed-loop correction speed rather than perception.

The paper's own failure cases suggest a testable refinement: tasks that require tangential force or load inference, such as weight classification, may remain a reliable discriminator between direct tactile-image interpretation and true contact-force understanding, since visible tactile deformation under tangential load is ambiguous.

A matched-seed re-run of specialist baselines on the same ten scenes per task would directly separate the model's advantage from benchmark-sampling differences; the paper's disclosed ten-point-per-episode sensitivity makes such a comparison the natural next experiment.

Because the tactile sensor renders contact as images, the boundary between vision and touch is not fixed: richer camera views (as in OpenViTac) narrow the visuo-tactile gap, so sensor design that removes visual correlates of contact would better isolate genuine tactile reasoning.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Recent advances in multimodal models have enabled GPT-6 Astra to perform robotic manipulation through in-context learning. Yet contact-rich control and tactile perception are often regarded as strengths of specialist policies, leaving open whether Astra can directly understand and use tactile observations. We provide, to our knowledge, the first demonstration that Astra directly interprets vision-based tactile images and uses their contact cues for manipulation. Tactile RGB images enter through its existing visual interface, enabling few-shot closed-loop control on nineteen simulated UniVTAC and OpenViTac tasks with 210 sampled scene seeds per setting. With visuo-tactile input, Astra achieves mean success rates of 93.8% and 81.8%, exceeding the strongest reported specialist baselines in our comparison by 22.9 and 13.1 percentage points, respectively. Vision-only performance also leads these baselines, reaching 76.3% and 80.6%. Trajectory analysis reveals active probing, cautious grasping, contact-guided alignment, and recovery, alongside difficulties with tangential loading and corrective execution. Our findings challenge the exclusivity of specialist policies in contact-rich manipulation and show that fine-grained vision can partially compensate for missing touch. They also identify vision-based tactile sensing as an interface for scaling contact understanding with foundation models and reveal signs of benchmark saturation. These results motivate joint visual-tactile pretraining, harder contact-rich evaluations, and physical-robot validation.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-20

The authors' main claim, as the classifier picked it out 0.97

“Our findings challenge the exclusivity of specialist policies in contact-rich manipulation and show that fine-grained vision can partially compensate for missing touch.”

How the main claim is stated: asserted 0.98

What the abstract itself states, as probabilities: claims first 0.94 · states limitation 0.92 · human participants 0.02 · controlled comparison 0.95 · randomized assignment 0.03 · states quantitative result 0.99 · compares with prior results 0.98 · mentions code or data availability 0.03

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Recent advances in multimodal models have enabled GPT-6 Astra to perform robotic manipulation through in-context learning. background 1.00 0.07 0.04 asserted 0.96
Yet contact-rich control and tactile perception are often regarded as strengths of specialist policies, leaving open whether Astra can directly understand and use tactile observations. gap 0.96 0.05 0.03 asserted 0.62
We provide, to our knowledge, the first demonstration that Astra directly interprets vision-based tactile images and uses their contact cues for manipulation. contribution 1.00 0.95 0.52 asserted 1.00
Tactile RGB images enter through its existing visual interface, enabling few-shot closed-loop control on nineteen simulated UniVTAC and OpenViTac tasks with 210 sampled scene seeds per setting. method 0.99 0.94 0.05 asserted 1.00
With visuo-tactile input, Astra achieves mean success rates of 93.8% and 81.8%, exceeding the strongest reported specialist baselines in our comparison by 22.9 and 13.1 percentage points, respectively. result 1.00 0.96 0.24 asserted 1.00
Vision-only performance also leads these baselines, reaching 76.3% and 80.6%. result 1.00 0.93 0.06 asserted 1.00
Trajectory analysis reveals active probing, cautious grasping, contact-guided alignment, and recovery, alongside difficulties with tangential loading and corrective execution. result 0.94 0.95 0.06 asserted 1.00
Our findings challenge the exclusivity of specialist policies in contact-rich manipulation and show that fine-grained vision can partially compensate for missing touch. interpretation 1.00 0.92 0.67 asserted 0.98
They also identify vision-based tactile sensing as an interface for scaling contact understanding with foundation models and reveal signs of benchmark saturation. interpretation 0.99 0.91 0.21 asserted 0.87
These results motivate joint visual-tactile pretraining, harder contact-rich evaluations, and physical-robot validation. future work 1.00 0.47 0.10 hedged possibility 0.48

What kind of paper the classifier reports this is: empirical study 0.59 · evidence from simulation or computation 0.67

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-20: 1 request, 7,590 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (19)
Figure 1 of the paper
Figure 1. Benchmark-level results. Task-averaged success rates (%). Astra uses vision alone (blue) or vision with tactile RGB images (red). Grey bars show published specialist baselines. Models are ordered by their reported mean within each benchmark. Task-level results and evaluation scope appear in Section 5. 1 arXiv:2610.26846v1 [cs.RO] 19 Oct 2026
Figure 2 of the paper
Figure 2. Direct tactile-image input for contact-guided manipulation. Tactile RGB images and scene observations enter Astra through its visual input interface. Demonstrations and online interaction memory connect contact evidence to actions and outcomes. Astra generates end-effector and gripper commands; benchmark execution returns observations and a receipt for the next decision. The displayed fixed-camera and tactile images are actual inputs from Insert Hole, seed 1000028. Vision￾only evaluation omits the tactile images throughout the context. Online memory M𝑡 uses the same interaction…
Figure 3 of the paper
Figure 3. Per-task changes with tactile input. Circles denote V, diamonds denote V+T, and the right-hand numbers show V+T minus V in percentage points. Six tasks improve, nine are unchanged, and four decrease. notes with visible execution. Action notes describe the policy assessment, and the recorded observations show the resulting motion and task outcome. Appendix B presents same-seed outcomes for the six tasks that improve with touch, connecting the task-level comparison to individual trajectories. 5.2.1. Contact-State Inference A useful tactile policy must connect changing contact…
Figure 4 of the paper
Figure 4. Pull-out Key: resistance-aware extraction. Seed 1000017, V+T, success. At step 60, the action note identifies torsional deformation and proposes unloading the twist. Continued axial resistance leads to reseating and finer angular adjustment at step 80; the later frame shows extraction. The sequence alternates extraction attempts with unloading and realignment as resistance persists. Full rollout. The informative feature of the Pull-out Key sequence (
Figure 5 of the paper
Figure 5. Insert Hole: inferring the bore direction. Seed 1000028, V+T, success. Astra interprets the opposing slopes of the tactile bands together with visible shaft deflection to infer a direction different from the reference trajectory. Astra pivots the shaft while maintaining entrance contact, then advances along the inferred bore. The case illustrates adaptation to contact geometry rather than literal replay of the demonstrated motion. Full rollout. In Insert Hole (
Figure 6 of the paper
Figure 6. Hardness Classify: probing while maintaining retention. Seed 10006, V+T, success. Astra slightly relaxes the grasp after lifting, detects slipping, and re-establishes contact. The subsequent action note associates the sharply defined loaded edge with the hard-object demonstration. The probe changes the contact condition, and the subsequent correction preserves retention before classification. Full rollout. 8
Figure 7 of the paper
Figure 7. Grasp Chip: cautious convergence on a fragile grasp. Seed 10099, V+T, success. Unilateral contact triggers smaller closure increments and a commanded 0.5 mm lateral correction at step 110, between panels (a) and (b). After bilateral contact develops, Astra avoids continued tightening and tests retention through a gentle lift. The episode later completes intact placement; the panels focus on the contact-establishment phase. Full rollout. The Grasp Chip sequence (
Figure 8 of the paper
Figure 8. Gear Assembly: resolving successive assembly constraints. Seed 10034, V+T, success. When the red gear tilts, Astra unloads contact and adjusts its center. A later yaw adjustment addresses possible tooth interference before seating and rotation. The correction strategy changes from centering to tooth engagement as the observed constraint changes. Full rollout. (a) Step 45 Wrist camera Left tactile Right tactile Contact shifts; tip exposed (b) Step 55 Wrist camera Left tactile Right tactile Withdraw the blocked plug (c) Step 80 Wrist camera Left tactile Right tactile Controlled…
Figure 9 of the paper
Figure 9. Insert USB: verifying object progress. Seed 10066, V+T, success. The connector remains exposed while the tactile contacts shift, and the action note identifies finger slip along a blocked plug. Astra withdraws, recenters, and issues a submillimeter correction before insertion progresses. Hand motion is not treated as sufficient evidence that the connector has entered the socket. Full rollout. scale of correction, combining a small retreat with millimeter-scale recentering. The retreat creates an opportunity to change the approach before renewed entry. The episode links preparation…
Figure 10 of the paper
Figure 10. Insert Block v1: a local correction after rim contact. Seed 10036, cube, V+T, success. Astra prepares the hand orientation before grasping, transports the cube, and then responds to possible rim contact with a small retreat and a commanded 2 mm centering adjustment. The final frame shows the release-and-verification phase. The local retreat creates clearance for recentering before renewed insertion. Full rollout. (a) Step 100 Wrist camera Left tactile Right tactile Released part stays on rim (b) Step 120 Wrist camera Left tactile Right tactile Regrasp and lift clear (c) Step 218…
Figure 11 of the paper
Figure 11. Insert Block v2: recovering from unsuccessful placement. Seed 10019, half-cylinder, V+T, success. After release, the part remains at the opening. Astra regrasps, lifts, and makes further alignment attempts until the part settles into the opening. The behavior demonstrates outcome verification and recovery from an incomplete placement. Full rollout. 5.3 Visual Compensation for Missing Touch The gain associated with tactile input differs substantially between benchmarks: 17.5 percentage points on UniVTAC and 1.2 points on OpenViTac. Their camera configurations provide one possible…
Figure 12 of the paper
Figure 12. Visible cues in nominally tactile tasks. Three V episodes from tasks with 100% sampled success. Left: a recorded camera view; center: enlargement of the green-marked region; right: the visible cue. Grasp Classify shows cylinder texture, Roughness Classify exposes surface ridges and contour structure, and Roughness-Guided Regrasp contains visually different surfaces even in the fixed-camera view. Enlargements preserve the recorded pixels. Rollouts: Grasp Classify, Roughness Classify, and Roughness-Guided Regrasp. The three cases expose different routes to visual discrimination.…
Figure 13 of the paper
Figure 13. Limited sensitivity to tangential motion and load. Weight Classify, seed 10038, V+T, failure. Astra interprets small tactile changes during a lift as evidence of a light object and places the heavy block on the incorrect blue tray. The image pattern also depends on grasp position, aperture, and motion; a small shear response is therefore not an unambiguous weight measurement. Full rollout. interpretation without resolving that ambiguity, showing a gap between recognizing tactile variation and reliably interpreting tangential loading. 6.2 Fine Adjustments Remain Challenging (a) Step…
Figure 14 of the paper
Figure 14. Recognizing misalignment does not ensure effective correction. Insert Block v1, seed 10016, V+T, failure. Repeated rim contact prompts regrasping and leveling. From step 240 onward, the cumulative commanded pitch increments total approximately 51.6 ◦ , with compensatory translation, yet insertion remains incomplete at the action limit. Full rollout. The Insert Block v1 failure (
Figure 15 of the paper
Figure 15. Recognizing obstruction without resolving the correction direction. Insert HDMI, seed 1000062, V+T, failure. Fixed-camera crops and paired tactile images show three stages of the same unsuccessful episode. The recorded notes identify pitch-sign uncertainty at action 235, a return toward an upright interpretation at 270, and persistent obstruction at 585. The changing contact patterns do not lead to successful insertion. Full rollout. The contact evidence triggers plausible adjustments, but the sequence does not converge on a reliable relationship between the observed deformation…
Figure 16 of the paper
Figure 16. Repeated probing without timely convergence. Empty-Can Selection, seed 10034, V+T, failure. Astra compares load responses, changes the grasp after unstable contact, and repeats the comparison. It selects the correct blue can at step 370 and starts the final grasp-and-lift sequence at step 390, but exhausts the 400-action budget before transfer. Information gathering must be coordinated with a stopping rule and the remaining manipulation. Full rollout. 7 Inference Cost and Latency The two evaluation phases consume a combined 335.97 million tokens and CNY 7,505.05 in logged provider…
Figure 17 of the paper
Figure 17. Distribution of API response latency. Five-second bins, normalized by the total number of valid responses in each setting: 3,927 for V and 4,890 for V+T. The plotted range ends at 120 s; the omitted tails remain in the normalization. Failed requests and responses without output tokens are excluded from latency statistics. control actions, the next observation-dependent correction must wait for another model response. This dependency makes response latency a key factor in end-to-end deployment. 8 Limitations of This Study All experiments are conducted in simulation; robustness on…
Figure 18 of the paper
Figure 18. Paired outcomes, part I. Pull-out Key (seed 1000025), Lift Can (1000093), and Insert Hole (1000014). The V episodes fail and the corresponding V+T episodes succeed. These endpoint views complement the successful contact￾inference sequences in Figures 4 and 5. Pull-out Key. The vision-only run repeatedly turns at fixed depth. With touch, growing edge deformation prompts torsional unloading; extraction begins after the contact returns to a round shape. The contact change guides the transition to axial withdrawal. Lift Can. The vision-only run follows a tipping arc and later reports…
Figure 19 of the paper
Figure 19. Paired outcomes, part II. Insert HDMI (seed 1000063), Hardness Classify (10003), and Insert Block v1 (10008). The V+T episodes succeed where the same-seed V episodes fail. Hardness Classify requires the correct category-specific destination; the insertion tasks require the part to enter its matching opening. Complete action sequences and input views remain available in the online archive. Insert HDMI. The vision-only run ends during another attempt to regrasp the displaced connector. With touch, shifting contact without plug motion is interpreted as rim contact, prompting unloading…

Figures are the authors', reproduced from the paper.

References (32)
  1. [1] Jianxin Bi, Kevin Yuchen Ma, Ce Hao, Mike Zheng Shou, and Harold Soh. VLA-Touch: Enhancing Vision-Language-Action Models with Dual-Level Tactile Feedback , 2025. URL https://arxiv.org/abs/2507.17294 on PithPith's reviewarXiv
  2. [2] Baijun Chen, Weijie Wan, Tianxing Chen, Xianda Guo, Congsheng Xu, Yuanyang Qi, Haojie Zhang, Longyan Wu, Tianling Xu, Zixuan Li, Yizhe Wu, Rui Li, Xiaokang Yang, Ping Luo, Wei Sui, and Yao Mu. UniVTAC: A Unified Simulation Platform for Visuo-Tactile Manipulation Data Generatio... on PitharXiv
  3. [3] Tianxing Chen, Yue Chen, Zixuan Li, Junyuan Tang, Kailun Su, Haoran Lu, Weijie Wan, Baijun Chen, Songling Liu, Haowen Yan, Honghao Su, Zhiyang Dou, Kaixuan Wang, Dandan Zhang, Yunze Liu, Yan Qin, Qiwei Liang, Qiwei Wu, Zijian Lin, Wenwei Lin, Yuran Wang, Minghua He, Tianshu Wu... on PithPith's reviewarXiv
  4. [4] Dongzhou Cheng, Taoran Yi, Ye Fang, Xingwu Zhang, Fan Feng, Yixuan Li, Gengxiong Zhuang, Rongze Wang, Shuai Yang, Wei Song, Weizhi Xue, Minyan Wu, Jie Gui, Jiaqi Wang, and Tong Wu. In-Context Robot Learning with VLM Agents . arXiv preprint arXiv:2609.19138, 2026. URL https://a... on PithPith's reviewarXiv
  5. [5] Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin C. M. Burchfiel, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. In Robotics: Science and Systems XIX, 2023. doi:10.15607/RSS.2023.XIX.026. URL https://diffusion-policy.c... on Pithdoi:10.15607/rss.2023.xix.026
  6. [6] Ruoxuan Feng, Jiangyu Hu, Wenke Xia, Tianci Gao, Ao Shen, Yuhao Sun, Bin Fang, and Di Hu. AnyTouch: Learning Unified Static-Dynamic Representation across Multiple Visuo-tactile Sensors , 2025. URL https://arxiv.org/abs/2502.12191 on PithPith's reviewarXiv
  7. [7] Galbot Team , Xuchuan Chen, Xiaoqian Cheng, Yu Deng, Lihe Ding, Shaocong Dong, Xiangjun Gao, Haozhe Jia, Zekai Li, Zhoujian Li, Yunrui Lian, Sikai Liang, Chenghuai Lin, Dairu Liu, Jiahang Liu, Qingtao Liu, Yuxuan Ma, Zekun Qi, Jiayi Su, He Wang, Ruochen Xu, Tianyu Xu, Xudong X... on PithPith's reviewarXiv
  8. [8] Abraham George, Selam Gano, Pranav Katragadda, and Amir Barati Farimani. VITaL Pretraining: Visuo-Tactile Pretraining for Tactile and Non-Tactile Manipulation Policies . In 2025 IEEE International Conference on Robotics and Automation (ICRA), pp.\ 258--264. IEEE, 2025. doi:10.... on Pithdoi:10.1109/icra55743.2025.11128336
  9. [9] Meng-Hao Guo, Zhe-Han Mo, Jia-Jun Wang, Yi Zhang, Kejin Wang, Yi-Xuan Deng, Jia-Peng Zhang, Yongming Rao, and Shi-Min Hu. Transferring the Intelligence of VLMs to Robotic Control . arXiv preprint arXiv:2609.22966, 2026. URL https://arxiv.org/abs/2609.22966 on PithPith's reviewarXiv
  10. [10] Liang Heng, Haoran Geng, Kaifeng Zhang, Pieter Abbeel, and Jitendra Malik. ViTacFormer: Learning Cross-Modal Representation for Visuo-Tactile Dexterous Manipulation , 2025. URL https://arxiv.org/abs/2506.15953 on PithPith's reviewarXiv
  11. [11] Carolina Higuera, Akash Sharma, Chaithanya Krishna Bodduluri, Taosha Fan, Patrick Lancaster, Mrinal Kalakrishnan, Michael Kaess, Byron Boots, Mike Lambeta, Tingfan Wu, and Mustafa Mukadam. Sparsh: Self-supervised touch representations for vision-based tactile sensing , 2024. U... on PithPith's reviewarXiv
  12. [12] Binghao Huang, Yixuan Wang, Xinyi Yang, Yiyue Luo, and Yunzhu Li. 3D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensing , 2024. URL https://arxiv.org/abs/2410.24091 on PithPith's reviewarXiv
  13. [13] Jialei Huang, Shuo Wang, Fanqi Lin, Yihang Hu, Chuan Wen, and Yang Gao. Tactile-VLA: Unlocking Vision-Language-Action Model's Physical Knowledge for Tactile Generalization , 2025. URL https://arxiv.org/abs/2507.09160 on PithPith's reviewarXiv
  14. [14] Wenlong Huang, Chen Wang, Ruohan Zhang, Yunzhu Li, Jiajun Wu, and Li Fei-Fei. VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models . arXiv preprint arXiv:2307.05973, 2023. URL https://arxiv.org/abs/2307.05973 on PithPith's reviewarXiv
  15. [15] Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan P Foster, Pannag R Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. OpenVLA : An o... on Pith
  16. [16] Xiaoqi Li, Muhe Cai, Jiadong Xu, Juan Zhu, Hongwei Fan, Yan Shen, Guangrui Ren, and Hao Dong. AT-VLA: Adaptive Tactile Injection for Enhanced Feedback Reaction in Vision-Language-Action Models , 2026. URL https://arxiv.org/abs/2605.07308 on PithPith's reviewarXiv
  17. [17] Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as Policies: Language Model Programs for Embodied Control . In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp.\ 9493--9500. IEEE, 2023. doi:1... on Pithdoi:10.1109/icra48891.2023.10160591
  18. [18] Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning . arXiv preprint arXiv:2306.03310, 2023. URL https://arxiv.org/abs/2306.03310 on PithPith's reviewarXiv
  19. [19] Fangcheng Liu, Yeqing Shen, Anda Cheng, Weishi Mi, Chao Tang, Chenyuan Liu, Yushun Xiang, Tingguang Li, Yong-Lu Li, and Yehui Tang. RoboICL: Embodied In-Context Learning with GPT-6 Astra . arXiv preprint arXiv:2609.34261, 2026. URL https://arxiv.org/abs/2609.34261 on PithPith's reviewarXiv
  20. [20] Physical Intelligence , Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Manuel Y. Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Devin L... on PithPith's reviewarXiv
show 12 more
  1. [21] Jiayi Su, Yixin Zheng, Mi Yan, Li Yi, Zhizheng Zhang, and He Wang. GPT 6 Astra as an embodied policy. Technical report and code, 2026. URL https://github.com/anonymous-report-421/GPT-as-Policy on Pith
  2. [22] Kazutoshi Tanaka. TacZero: Training-Free Peg Insertion Using a General-Purpose Vision-Language Model with Tactile Feedback . arXiv preprint arXiv:2610.07621, 2026. URL https://arxiv.org/abs/2610.07621 on PithPith's reviewarXiv
  3. [23] Stone Tao, Fanbo Xiang, Arth Shukla, Yuzhe Qin, Xander Hinrichsen, Xiaodi Yuan, Chen Bao, Xinsong Lin, Yulin Liu, Tse-Kai Chan, Yuan Gao, Xuanlin Li, Tongzhou Mu, Nan Xiao, Arnav Gurha, Viswesh Nagaswamy Rajesh, Yong Woo Choi, Yen-Ru Chen, Zhiao Huang, Roberto Calandra, Rui Ch... on Pith
  4. [24] Enyi Wang, Mingxin Wang, Quan Shi, Hetian Guo, Hongyu Wang, Xi Wang, Bin Qian, Yupeng Zheng, Wenxuan Song, Houde Liu, Yong Xu, Cheng Chi, Wenchao Ding, Yilun Chen, and Yan Wang. TacDyn-WAM: Learning Implicit Tactile Dynamics in a Heterogeneous Visuo-Tactile World Action Model ... on PithPith's reviewarXiv
  5. [25] Yifan Wu, Qin Li, Nan Min, Guojin Zhong, Haoyu Zhao, Zhiyuan Li, Houze Xu, Shengqi Xu, Xingyao Lin, Zijie Diao, Zhaoxiang Liu, Shiguo Lian, Shunlin Lu, Shihao Zhao, Ziyi Ye, Zuxuan Wu, and Yu-Gang Jiang. OpenViTac: Learning and Benchmarking Visuo-Tactile Policies in a Unified ... on PithPith's reviewarXiv
  6. [26] Chen Yang, Linzhe Shi, Changjie Wu, Hang Zhang, Ronghan Chen, Lingjun Zhang, Xu Hu, Mu Xu, Jiansheng Fan, and Chen Wang. SimpleTouch: Can Vision-Language-Action Models Master Contact-Rich Manipulation Without Tactile Policy Pretraining? arXiv preprint arXiv:2610.02784, 2026 a ... on PithPith's reviewarXiv
  7. [27] Zhenjie Yang, Yideng Zhang, Dongjie Zhang, Chenyu Jiang, Xianshuai Liu, Yufeng Li, Zuhao Ge, Xingyu Jiao, Zheng Zhang, Kaiyu He, He Wang, Yuwen Zhong, Yi Deng, Muyun Jiang, Xianliang Huang, Haisheng Su, Donghang Zhang, Jian Zhang, Xue Yang, Hongyang Li, Zuxuan Wu, Yu-Gang Jian... on PithPith's reviewarXiv
  8. [28] Chengbo Yuan, Zicheng Zhang, Mingjie Zhou, Wendi Chen, Yi Wang, Zhuoyang Liu, Dantong Niu, Shuo Wang, Hui Zhang, Wenkang Zhang, Yingdong Hu, Yuanqing Gong, Wanli Xing, Chuan Wen, Cewu Lu, Kaifeng Zhang, and Yang Gao. FTP-1: A Generalist Foundation Tactile Policy Across Tactile... on PithPith's reviewarXiv
  9. [29] Wenzhen Yuan, Siyuan Dong, and Edward H. Adelson. GelSight : High-resolution robot tactile sensors for estimating geometry and force. Sensors, 17 0 (12): 0 2762, 2017. doi:10.3390/s17122762. URL https://doi.org/10.3390/s17122762 on Pithdoi:10.3390/s17122762
  10. [30] Wenbo Zhang, Kaixuan Wang, Yutao Ouyang, Xiaoyu Huang, Liyang Li, Kailun Su, Weiyang Jin, Wenhao Chai, Haotian Liang, Zhiyang Dou, Yue Chen, and Tianxing Chen. An Unexpected Robot Policy: Early Evaluations of GPT-6 Astra on RoboDojo and Beyond . arXiv preprint arXiv:2609.24170... on PithPith's reviewarXiv
  11. [31] Jialiang Zhao, Yuxiang Ma, Lirui Wang, and Edward Adelson. Transferable Tactile Transformers for Representation Learning Across Diverse Sensors and Tasks . In Pulkit Agrawal, Oliver Kroemer, and Wolfram Burgard (eds.), Proceedings of The 8th Conference on Robot Learning, volum... on Pith
  12. [32] Tony Z. Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware . In Robotics: Science and Systems XIX, RSS2023. Robotics: Science and Systems Foundation, July 2023. doi:10.15607/rss.2023.xix.016. URL https://www.... on Pithdoi:10.15607/rss.2023.xix.016

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment