Pith. sign in

REVIEW 4 major objections 4 minor 63 references

Adding register tokens and uncertainty-gated zooming to a vision-language-action model raises simulated manipulation success from 94.2% to 98.4%.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Inserting register tokens into a VLA encoder plus uncertainty-gated, attention-guided cropping raises π0's success from 94.2% to 98.4% on LIBERO and 46.5% to 69.0% on a real-world benchmark, at 1.4–1.6× compute.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection A plausible, well-specified VLA perception upgrade whose headline cropping benefit is the least isolated claim in the paper, but the work is worth refereeing. the 4 major comments →

arxiv 2608.02197 v1 pith:DA4XR5LG submitted 2026-08-03 cs.RO

Look Where It Matters: Adaptive Visual Refinement for Vision-Language-Action Models

Jin Cui , Yanbin Hu , Xinyue Long , Linkai Li , Boran Zhao , Pengju Ren This is my paper
classification cs.RO
keywords vision-language-action modelsregister tokensattention artifactsvisual groundinguncertainty gatingadaptive croppingrobot manipulationspatial precision
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AtVLA sets out to fix two visual weaknesses in vision-language-action (VLA) policies: corrupted attention that mislocates target objects, and missing local geometry at low resolution. It argues that embodied post-training pushes spatial knowledge such as object location, depth ordering, and local geometry into the wrong patch tokens because the original global tokens have limited capacity. The fix is to insert four learnable register tokens that absorb this excess spatial information, leaving patch tokens with clean, spatially faithful attention. A second mechanism samples multiple action chunks, measures their disagreement, and only when that uncertainty is high crops the region that the policy's own attention points to, re-encodes it at high resolution, and refines the action. The result the paper reports is a jump in average task success in standard simulation and in a real single-view robot benchmark, at roughly 1.4–1.6 times the base compute.

Core claim

The central claim is that VLA visual encoders develop attention artifacts—high-norm background tokens that hoard global information—as they learn manipulation-relevant spatial features during post-training, and that these artifacts are a cause of poor object grounding. Inserting four learnable register tokens and training them with the original action objective makes the registers the dedicated carriers of embodied spatial information, restoring clean patch attention that can localize targets and interaction regions. Because clean attention cannot recover geometric detail that was never captured at 224×224, the model couples this rectification with an uncertainty gate: when several sampled a

What carries the argument

Four learnable register tokens inserted into the visual encoder after the class token; they are trained end-to-end with the original action objective and kept as extra global context. The uncertainty gate compares K=4 action chunks sampled from a flow-matching action expert and triggers on disagreement in near-term translational actions. The locator is an attention rollout that averages action-to-image attention over all joint-attention layers and denoising steps {3,6,9}, then searches square windows with a contrastive criterion. The refinement branch crops the chosen region, re-encodes it at 224×224, adds a position embedding, and appends the tokens to the cached prefix without recomputing

Load-bearing premise

The load-bearing premise is that at deployment the action-expert attention, trained with per-observation annotated target regions, keeps localizing the task-relevant region in novel scenes, and that a high-resolution crop from the same camera view can supply the geometric detail missing in the base image.

What would settle it

On a held-out set of novel scenes with human-annotated task-relevant regions, measure the intersection-over-union between the attention-rollout crop and the annotation, and measure whether re-encoding that crop reduces near-term end-effector error. If the IoU is no better than an untrained external crop selector, or if the crop does not reduce action error, the refinement mechanism is not doing the work; similarly, if removing the register tokens leaves performance unchanged, the proposed artifact hypothesis is unsupported.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If AtVLA's mechanism is sound, VLA policies can gain spatial precision without new sensors, 3D representations, or extra visual supervision beyond embodied demonstrations.
  • Uncertainty gating means most replanning steps stay as cheap as the base policy; only about 30% pay for an extra crop encoding, giving a predictable compute-success trade-off.
  • Clean attention trained with an action objective can serve as a reliable localizer for cropping, whereas attention from a generic vision-language model does not transfer to embodied interaction regions.
  • Register tokens that carry embodied spatial context could become a standard component in VLA visual encoders, since they add only a few thousand parameters.
  • Because crops also help long-horizon tasks, the refinement mechanism is not only about local precision but about supplying decision-relevant visual context over time.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • We infer that the same register-token mechanism would transfer to other embodied policies with ViT-style encoders, provided they are trained on interaction data; the paper only demonstrates it on one base policy.
  • The uncertainty gate could be reused as a general competence signal: the policy's own multi-sample disagreement may indicate when to request additional sensing, a high-resolution camera, or human help.
  • A testable extension is to vary the trigger-rate budget; the paper fixes roughly 30%, but the calibration objective suggests a cost-benefit curve that could be traded off in deployment.
  • The dependence on annotated task-relevant regions for the attention-grounding loss raises an open question about whether the attention could be bootstrapped from action success alone, which the paper does not resolve.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes AtVLA, an extension of the π0 vision-language-action policy. The method has two components: (i) inserting four learnable register tokens into the SigLIP vision encoder, retaining their outputs as extra global context, to absorb attention artifacts and preserve clean patch-level spatial features; and (ii) an uncertainty-gated local refinement branch, in which the action expert samples K action chunks, estimates disagreement, and, when the disagreement exceeds a calibrated threshold, uses action-conditioned attention rollout to locate a task-relevant region, crop it, re-encode it at high resolution, and append the tokens to the cached prefix for a refined action generation. The paper reports gains over π0 on LIBERO (94.2→98.4 average), SimplerEnv, and a new real-world benchmark (46.5→69.0), with adaptive cropping triggered on about 30% of replanning steps at 1.4–1.6× the compute of the base model. The manuscript includes controlled variants (π0+Registers, π0+Cropping, AtVLA(Full)) that share initialization and training data, plus analyses of register capacity, linear probing of spatial information, and attention visualizations.

Significance. If the empirical claims hold, the register-token finding would be a useful and inexpensive diagnostic: it connects a well-known ViT artifact to embodied post-training and provides a parameter-light fix (4,608 parameters). The uncertainty-gated cropping idea is also appealing for deployment, as it offers a sensor-free route to precise single-view manipulation. The paper's strengths include the use of controlled π0 variants in some comparisons, the real-world benchmark, and the linear-probe evidence that register tokens carry task-relevant spatial information. However, the central contribution—adaptive visual refinement—is not isolated by a clean ablation, and several auxiliary claims (cost, annotation dependence) rest on assumptions that are not fully exposed. The paper is therefore promising but requires substantial additional evidence before the headline claims can be accepted.

major comments (4)
  1. [§Results, Table 1 and §Method (Stages 1–2)] No ablation isolates the uncertainty-gated local refinement branch, which is the paper's central contribution. AtVLA(Full) differs from π0+Registers not only by the crop branch but also by Stage 1 (ground-truth crop alignment) and Stage 2 (joint refinement with LoRA and L_ag). The π0+Cropping row is not a valid control for the proposed refinement mechanism because it uses an external VLM to choose crops, lacks register rectification, and uses a different training setup. Thus the 1.2–3.4 point gaps on LIBERO and the real-world gains cannot be attributed specifically to adaptive cropping; they may come from the additional embodied fine-tuning or from LoRA adaptation. A decisive addition is a variant that includes registers and LoRA but disables cropping at inference, and an AtVLA variant that always crops (or crops by a fixed schedule) to test the uncertainty gate itself.
  2. [Table 1 and Table 2 (statistical protocol)] The statement that baselines are 'taken from reported settings' means the π0 improvements are not measured under a shared evaluation protocol. Differences in simulator versions, rollouts, seeds, or success criteria could contribute to the gap. Moreover, no significance tests or confidence intervals are reported for the key comparisons; on LIBERO Long-10 the AtVLA average is 96.5±2.6 versus 93.1±1.7 for π0+Registers, so the 3.4-point gap is well within one standard deviation of either. Re-running the baselines under the same protocol and reporting paired trials or per-task confidence intervals is necessary to support the claimed improvements.
  3. [Eq. (25) and Eq. (13)] The 'approximately 30% trigger rate' is not an emergent property of the uncertainty estimator; Eq. (25) imposes Pr(U_t > τ) ≤ 0.3 as an explicit constraint during threshold calibration. The cost model in Eq. (13) then uses r ≈ 0.3 to obtain the 1.4–1.6× figure, so the compute claim is partly by construction. The paper should report the measured trigger rate on held-out test episodes, show the trade-off between trigger rate and success, and justify the assumed per-pass constants (10–15% for one action-expert pass, 20–25% for crop refinement) with the deployment measurements mentioned in the text.
  4. [Eq. (14) and Appendix: Training Data] The attention-grounding loss L_ag and the Stage-1 ground-truth crops require an annotated task-relevant region for every observation in the LIBERO training set (2,000 demonstrations) and in the real-world demonstrations. The manuscript does not describe how these regions were produced, what criteria were used, whether inter-annotator consistency was checked, or whether the annotations will be released. Because the refinement path depends on attention that is trained to match these annotations, the reproducibility and generalization of the localization branch are not established. This is a load-bearing missing resource for a central component of the method.
minor comments (4)
  1. [Abstract vs. Eq. (18)] The abstract says registers are 'trained end-to-end using only embodied data and the original action objective,' but in Stage 2 the register embeddings are explicitly optimized with Ljoint = Lπ0 + λ_ag L_ag (Eq. 18), i.e., with an auxiliary attention-grounding loss. The wording should be corrected to reflect the actual objective.
  2. [§Results and Analysis, Q1] In the Q1 paragraph, 'Fig. 1 shows that adding register tokens...' should refer to Fig. 4 (or a later figure); Fig. 1 is the artifact illustration. The same paragraph also appears to duplicate 'Fig. 1' for the before/after comparison.
  3. [Table 1] The abbreviation 'O/C' in the SimplerEnv column is not defined in the caption. Clarify that it denotes Open/Close Drawer. Also, the footnote for π0 + Cropping should state which VLM generated the crops and whether the variant used the same three-stage schedule.
  4. [Eq. (5) and Eq. (22)] Equations (5) and (22) are identical formulations of the uncertainty score; one of them should be removed or the second one should reference the first.

Circularity Check

1 steps flagged

Main success-rate claims are measured and not circular; however, the headline '≈30% trigger rate' is enforced by the threshold-calibration budget rather than discovered, making the compute story partly by construction.

specific steps
  1. fitted input called prediction [Abstract; Threshold Calibration (Eq. 25); cost analysis (Eqs. 12–13)]
    "Abstract: 'The additional inference cost remains limited: cropping is triggered on approximately 30% of replanning steps, resulting in only 1.4–1.6× the total computation of π0 under the representative deployment setting.' Threshold Calibration: 'We select the threshold that maximizes the expected benefit of refinement subject to an approximate 30% trigger-rate budget: τ* = arg max_τ E[Δe_t 1(U_t>τ)], Pr(U_t>τ) ≤ 0.3.'"

    The reported 30% trigger rate is not a measured property that emerges from the uncertainty criterion; τ is selected under the explicit constraint Pr(U_t>τ) ≤ 0.3. The abstract then presents cropping as triggered on approximately 30% of replanning steps as if it were a finding, and Eqs. 12–13 combine this enforced r≈0.3 with assumed per-pass cost percentages to claim 'only 1.4–1.6×' computation. Thus this specific efficiency 'prediction' reduces to the calibration budget by construction. The success-rate improvements on LIBERO, SimplerEnv, and real-world benchmarks are separate measured outcomes and are not circular.

full rationale

The central scientific claims of the paper — that VLA visual encoders exhibit attention artifacts, that register tokens can absorb excess global information, and that uncertainty-gated high-resolution cropping improves manipulation success — are evaluated on held-out benchmarks against external baselines, so they are empirical findings rather than definitional consequences. The method components (register tokens, L_ag attention grounding, contrastive window search, threshold calibration) are trained or tuned with explicit objectives, not assumed to equal the result. There is no load-bearing self-citation chain and no uniqueness theorem imported from the authors' prior work; citations to Darcet et al. and other external work provide independent grounding. The one genuine by-construction element is the trigger rate: Eq. 25 imposes a 30% budget on τ, and the paper then headlines the resulting 'approximately 30%' trigger rate and the 1.4–1.6× compute multiplier derived from it (plus assumed per-pass constants) as an outcome. That is a fitted-input-called-prediction issue, but it does not bear on the measured success-rate gains. The absence of an exact ablation holding training stages and LoRA fixed when comparing AtVLA to π0+Registers is an experimental confound and a correctness concern, not a circularity. Overall, the derivation's main content is self-contained and independently evaluated, so the circularity score is low.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 1 invented entities

The central success claims are measured outcomes, so the ledger is lighter than for a derivation paper. The main burden sits in: the per-embodiment threshold and 30% trigger budget (fitted to held-out data and then quoted as a finding), the hand-selected architecture constants (N_r=4, K=4, λ_ag=1.0, window grid), the assumption that supervised action-attention transfers to novel scenes, the monocular information-presence assumption, and the undescribed annotation pipeline that supervises the crop branch.

free parameters (6)
  • Uncertainty threshold τ = Not reported numerically; initialized at 70th percentile of validation uncertainty, adjusted to enforce ≈30% trigger rat
    Calibrated per embodiment on a held-out set to maximize E[Δe_t·1(U_t>τ)] subject to Pr(U_t>τ)≤0.3 (Threshold Calibration, Appendix, Eq. 25). The headline '~30% trigger' is imposed by this budget, not measured emergently.
  • Number of register tokens N_r = 4 = 4
    Selected by ablation over {1,2,4,6} registers (Fig. 6); fewer registers hurt LIBERO success, performance saturates at 4. A hyperparameter fit to validation performance.
  • Auxiliary attention-grounding weight λ_ag = 1.0 (and λ_cp = 0.1) = 1.0 / 0.1
    Chosen by hand (Eq. 11). L_ag supervises action-to-image attention to concentrate inside annotated task regions; this supervision — not pure emergence — produces the cropping signal used at deployment.
  • Action samples K = 4 and rollout denoising steps {3,6,9} = 4; {3,6,9}
    Design choices. K=4 defines the uncertainty estimate and already makes the confident path cost ~1.4–1.6× a single-sample π0 pass before any cropping.
  • Crop window grid: sides {84,112,140} px, stride 14, context expansion 1.25×, padding 10% = {84,112,140}, 14, 1.25, 10%
    Hand-chosen search grid for the contrastive window selection (Eqs. 8/30); no sensitivity analysis provided.
  • Cost-model constants (one action-expert pass = 10–15% of full replanning pass; crop refinement = 20–25%) = 10–15%; 20–25%
    Assumed values used in Eq. 13 to arrive at the 1.4–1.6× headline. The paper states latency was measured on an RTX 4090 but reports no measurements, so the headline cost is an estimate.
axioms (6)
  • domain assumption ViT attention artifacts (high-norm background tokens) occur in SigLIP-based VLA encoders and are causally connected to embodied spatial-knowledge spillover during post-training
    Stated in the Introduction and Q1/Q2. The causal direction ('limited global-token capacity causes information to spill into patch tokens') is inferred from correlational linear-probe evidence (Fig. 5), not demonstrated by intervention.
  • domain assumption Action-expert self-attention, regularized by L_ag (Eq. 14) on annotated task-relevant regions, yields saliency that transfers to novel scenes for crop selection
    Load-bearing for the refinement path. Training uses ground-truth crop annotations (Stages 1–2) whose generation process is undescribed; deployment relies on attention rollout alone. If transfer fails, cropping degrades performance (as shown by the π0+Cropping variant).
  • domain assumption K-sample disagreement of flow-matching ODE solves from independent Gaussian initializations measures action uncertainty relevant to spatial error
    The gate (Eq. 5) assumes high disagreement correlates with the error reduction obtainable from cropping; validity is only checked implicitly through the held-out threshold calibration, not by any reported correlation.
  • domain assumption Task-critical geometric detail absent at 224×224 is present in a same-view high-resolution crop (monocular, no new sensor)
    The entire refinement branch assumes the missing contact/geometry information is in the pixels of the same single view; if the ambiguity is in viewpoint or depth, cropping cannot recover it. The paper offers no evidence for this information-presence assumption beyond the success-rate gains.
  • standard math Attention rollout as a row-normalized product of attention matrices (Eqs. 6–7) is a valid token-influence estimator
    Standard attention-rollout approximation (Abnar & Zuidema 2020), used without derivation and without citing the origin, which weakens auditability of the saliency-mapping step.
  • ad hoc to paper Per-observation annotated task-relevant regions exist for all 2,000 LIBERO demonstrations and all real-world demos
    LIBERO provides no such annotations; the paper states each observation is 'additionally associated with an annotated task-relevant region' but does not describe who annotated, how, with what instructions, or inter-annotator agreement, and releases nothing.
invented entities (1)
  • Register tokens retained as output context slots (r_t) in the SigLIP encoder no independent evidence
    purpose: Carry embodied spatial information (object location, depth ordering, local geometry) that would otherwise spill into patch tokens; register outputs are projected into the multimodal prefix (Eq. 3) and used by the action expert
    Mechanism borrowed from Darcet et al. 2023 (where registers are an internal workspace) and repurposed as exposed output carriers. In-paper linear-probe evidence (Fig. 5) supports the spatial-information claim, but the probes are trained on the same embodied distribution that trained the registers, and there is no external falsifiable handle.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Look Where It Matters: Adaptive Visual Refinement for Vision-Language-Action Models." pith.science (2026). https://pith.science/paper/DA4XR5LG

@misc{pith2026260802197,
  author       = {Pith},
  title        = {Pith review of: Look Where It Matters: Adaptive Visual Refinement for Vision-Language-Action Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DA4XR5LG}},
  note         = {Machine review of arXiv:2608.02197}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Visual representations of VLA models remain unreliable for spatially precise robotic manipulation. We uncover that vision encoders in VLAs also exhibit attention artifacts previously documented in generic Vision Transformers, and further show that, in embodied policies, these artifacts are closely associated with spatial perception capabilities acquired during post-training. As the encoder learns task-relevant information such as object location, depth ordering, and local geometry, limited global-token capacity causes part of this information to spill into low-information patch tokens. We introduce AtVLA, a framework that inserts learnable register tokens into the visual encoder. Trained end-to-end using only embodied data and the original action objective, these registers emerge as dedicated carriers of embodied spatial information, while the remaining patch tokens recover clean and spatially faithful attention distributions crucial for precise target localization and fine-grained contact. Clean attention restores reliable localization, but cannot recover geometric details lost in low-resolution observations. AtVLA therefore couples attention rectification with uncertainty-gated local refinement. The action expert samples multiple action chunks and estimates uncertainty from their disagreement; only for uncertain predictions, action-conditioned attention rollout identifies the task-relevant region, which is cropped, re-encoded at high resolution, and appended to the cached prefix for refined action generation. Across LIBERO, SimplerEnv, and a challenging single-view real-world benchmark, AtVLA improves the average LIBERO success rate from 94.2% to 98.4% and real-world success from 46.5% to 69.0%. The cropping is triggered on approximately 30% of replanning steps, resulting in only 1.4-1.6x the total computation of the base model under the representative deployment setting.

Figures

Figures reproduced from arXiv: 2608.02197 by Boran Zhao, Jin Cui, Linkai Li, Pengju Ren, Xinyue Long, Yanbin Hu.

Figure 1
Figure 1. Figure 1: Attention artifacts in a standard VLA visual en [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of AtVLA. Register tokens absorb excess global information and retain embodied spatial context while restoring clean patch representations. The action expert estimates uncertainty from multiple sampled action chunks: confident predictions follow the base path, whereas uncertain predictions trigger attention-guided cropping, high-resolution re-encoding, and prefix extension. A three-stage curriculu… view at source ↗
Figure 3
Figure 3. Figure 3: Real-world evaluation benchmark. The Kitchen suite (top) evaluates visual grounding and reasoning in cluttered household scenes, while Building Blocks (bottom) tests small-object stacking, spatial relations, and long-horizon construction. on integrating the newly introduced register tokens, crop￾position encoding, and attention-guided localization while preserving the pretrained policy capabilities. We ado… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative effects of attention rectification and adaptive visual refinement. (a) Register-enhanced encoding pro [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Probing embodied spatial information in visual tokens. Independent linear probes predict object location, depth, and [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Effect of register-token capacity. Left: probe scores for the CLS token, patch tokens, and different numbers of registers. Right: success rates in LIBERO suites. Number of register tokens. As shown in [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Additional real-world demonstrations of AtVLA. Each row presents a temporally ordered manipulation rollout. [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

63 extracted references · 18 linked inside Pith

  1. [1]

    ArXiv , year=

    Vision Transformers Need Registers , author=. ArXiv , year=

  2. [2]

    ArXiv , year=

    Don't Blind Your VLA: Aligning Visual Representations for OOD Generalization , author=. ArXiv , year=

  3. [3]

    AAAI Conference on Artificial Intelligence , year=

    ReconVLA: Reconstructive Vision-Language-Action Model as Effective Robot Perceiver , author=. AAAI Conference on Artificial Intelligence , year=

  4. [4]

    2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year=

    What’s in the Imageƒ A Deep-Dive into the Vision of Vision Language Models , author=. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year=

    2025

  5. [5]

    2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year=

    Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens , author=. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year=

    2025

  6. [6]

    ArXiv , year=

    SlotVLA: Towards Modeling of Object-Relation Representations in Robotic Manipulation , author=. ArXiv , year=

  7. [7]

    ArXiv , year=

    Focusing on What Matters: Object-Agent-centric Tokenization for Vision Language Action models , author=. ArXiv , year=

  8. [8]

    ArXiv , year=

    SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model , author=. ArXiv , year=

  9. [9]

    Proceedings of The 9th Conference on Robot Learning , pages =

    3DS-VLA: A 3D Spatial-Aware Vision Language Action Model for Robust Multi-Task Manipulation , author =. Proceedings of The 9th Conference on Robot Learning , pages =. 2025 , editor =

    2025

  10. [10]

    ArXiv , year=

    PointVLA: Injecting the 3D World into Vision-Language-Action Models , author=. ArXiv , year=

  11. [11]

    ArXiv , year=

    MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMs , author=. ArXiv , year=

  12. [12]

    Conference on Empirical Methods in Natural Language Processing , year=

    ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Exploration , author=. Conference on Empirical Methods in Natural Language Processing , year=

  13. [13]

    ArXiv , year=

    Chain-of-Spot: Interactive Reasoning Improves Large Vision-Language Models , author=. ArXiv , year=

  14. [14]

    2024 , eprint=

    Towards Perceiving Small Visual Details in Zero-shot Visual Question Answering with Multimodal LLMs , author=. 2024 , eprint=

    2024

  15. [15]

    2025 , eprint=

    FOCUS: Internal MLLM Representations for Efficient Fine-Grained Visual Question Answering , author=. 2025 , eprint=

    2025

  16. [16]

    2026 , eprint=

    CropVLM: Learning to Zoom for Fine-Grained Vision-Language Perception , author=. 2026 , eprint=

    2026

  17. [17]

    2024 , url=

    VIP: Vision Instructed Pre-training for Robotic Manipulation , author=. 2024 , url=

    2024

  18. [18]

    Conference on Robot Learning , year=

    Robotic Control via Embodied Chain-of-Thought Reasoning , author=. Conference on Robot Learning , year=

  19. [19]

    ArXiv , year=

    GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Data , author=. ArXiv , year=

  20. [20]

    ArXiv , year=

    OpenVLA: An Open-Source Vision-Language-Action Model , author=. ArXiv , year=

  21. [21]

    ArXiv , year=

    Octo: An Open-Source Generalist Robot Policy , author=. ArXiv , year=

  22. [22]

    ArXiv , year=

    0: A Vision-Language-Action Flow Model for General Robot Control , author=. ArXiv , year=

  23. [23]

    ArXiv , year=

    0.5: a Vision-Language-Action Model with Open-World Generalization , author=. ArXiv , year=

  24. [24]

    2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year=

    CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models , author=. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year=

    2025

  25. [25]

    ArXiv , year=

    VLANeXt: Recipes for Building Strong VLA Models , author=. ArXiv , year=

  26. [26]

    ArXiv , year=

    Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success , author=. ArXiv , year=

  27. [27]

    2023 , eprint=

    RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control , author=. 2023 , eprint=

    2023

  28. [28]

    Nature Machine Intelligence , year=

    What matters in building vision–language–action models for generalist robots , author=. Nature Machine Intelligence , year=

  29. [29]

    ArXiv , year=

    TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policies , author=. ArXiv , year=

  30. [30]

    ArXiv , year=

    From Spatial to Actions: Grounding Vision-Language-Action Model in Spatial Foundation Priors , author=. ArXiv , year=

  31. [31]

    ArXiv , year=

    LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning , author=. ArXiv , year=

  32. [32]

    Conference on Robot Learning , year=

    Evaluating Real-World Robot Manipulation Policies in Simulation , author=. Conference on Robot Learning , year=

  33. [33]

    Bendikas, R.; Dijkman, D.; Peschl, M.; Haresh, S.; and Mazzaglia, P. 2025. Focusing on What Matters: Object-Agent-centric Tokenization for Vision Language Action models. ArXiv, abs/2509.23655

    arXiv 2025

  34. [34]

    X.; Tanner, J.; Vuong, Q.; Walling, A.; Wang, H.; and Zhilinsky, U

    Black, K.; Brown, N.; Driess, D.; Esmail, A.; Equi, M.; Finn, C.; Fusai, N.; Groom, L.; Hausman, K.; Ichter, B.; Jakubczak, S.; Jones, T.; Ke, L.; Levine, S.; Li-Bell, A.; Mothukuri, M.; Nair, S.; Pertsch, K.; Shi, L. X.; Tanner, J.; Vuong, Q.; Walling, A.; Wang, H.; and Zhilinsky, U. 2024. 0: A Vision-Language-Action Flow Model for General Robot Control....

    Pith/arXiv arXiv 2024

  35. [35]

    G.; Gopalakrishnan, K.; Han, K.; Hausman, K.; Herzog, A.; Hsu, J.; Ichter, B.; Irpan, A.; Joshi, N.; Julian, R.; Kalashnikov, D.; Kuang, Y.; Leal, I.; Lee, L.; Lee, T.-W

    Brohan, A.; Brown, N.; Carbajal, J.; Chebotar, Y.; Chen, X.; Choromanski, K.; Ding, T.; Driess, D.; Dubey, A.; Finn, C.; Florence, P.; Fu, C.; Arenas, M. G.; Gopalakrishnan, K.; Han, K.; Hausman, K.; Herzog, A.; Hsu, J.; Ichter, B.; Irpan, A.; Joshi, N.; Julian, R.; Kalashnikov, D.; Kuang, Y.; Leal, I.; Lee, L.; Lee, T.-W. E.; Levine, S.; Lu, Y.; Michalew...

    Pith/arXiv arXiv 2023

  36. [36]

    Carvalho, M.; Dias, H.; and Martins, B. 2026. CropVLM: Learning to Zoom for Fine-Grained Vision-Language Perception. arXiv:2511.19820

    Pith/arXiv arXiv 2026

  37. [37]

    Darcet, T.; Oquab, M.; Mairal, J.; and Bojanowski, P. 2023. Vision Transformers Need Registers. ArXiv, abs/2309.16588

    Pith/arXiv arXiv 2023

  38. [38]

    Deng, S.; Yan, M.; Wei, S.; Ma, H.-L.; Yang, Y.; Chen, J.; Zhang, Z.; Yang, T.; Zhang, X.; Cui, H.; Zhang, Z.; and Wang, H. 2025. GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Data. ArXiv, abs/2505.03233

    Pith/arXiv arXiv 2025

  39. [39]

    Hanyu, T.; Chung, N.; Le, H.; Nguyen, T.; Ikebe, Y.; Gunderman, A.; Minh, D. N. H.; Vo, K. T.; Kieu, T.; Yamazaki, K.; Rainwater, C.; Nguyen, A.; and Le, N. 2025. SlotVLA: Towards Modeling of Object-Relation Representations in Robotic Manipulation. ArXiv, abs/2511.06754

    Pith/arXiv arXiv 2025

  40. [40]

    Y.; Ghosh, D.; Groom, L.; Hausman, K.; Ichter, B.; Jakubczak, S.; Jones, T.; Ke, L.; LeBlanc, D.; Levine, S.; Li-Bell, A.; Mothukuri, M.; Nair, S.; Pertsch, K.; Ren, A

    Intelligence, P.; Black, K.; Brown, N.; Darpinian, J.; Dhabalia, K.; Driess, D.; Esmail, A.; Equi, M.; Finn, C.; Fusai, N.; Galliker, M. Y.; Ghosh, D.; Groom, L.; Hausman, K.; Ichter, B.; Jakubczak, S.; Jones, T.; Ke, L.; LeBlanc, D.; Levine, S.; Li-Bell, A.; Mothukuri, M.; Nair, S.; Pertsch, K.; Ren, A. Z.; Shi, L. X.; Smith, L.; Springenberg, J. T.; Sta...

    Pith/arXiv arXiv 2025

  41. [41]

    Jiang, Z.; Chen, J.; Zhu, B.; Luo, T.; Shen, Y.; and Yang, X. 2024. Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 25004--25014

    2024

  42. [42]

    K.; and Panov, A

    Kachaev, N.; Kolosov, M.; Zelezetsky, D.; Kovalev, A. K.; and Panov, A. I. 2025. Don't Blind Your VLA: Aligning Visual Representations for OOD Generalization. ArXiv, abs/2510.25616

    arXiv 2025

  43. [43]

    Kaduri, O.; Bagon, S.; and Dekel, T. 2024. What’s in the Imageƒ A Deep-Dive into the Vision of Vision Language Models. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 14549--14558

    2024

  44. [44]

    J.; Finn, C.; and Liang, P

    Kim, M. J.; Finn, C.; and Liang, P. 2025. Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success. ArXiv, abs/2502.19645

    Pith/arXiv arXiv 2025

  45. [45]

    J.; Pertsch, K.; Karamcheti, S.; Xiao, T.; Balakrishna, A.; Nair, S.; Rafailov, R.; Foster, E

    Kim, M. J.; Pertsch, K.; Karamcheti, S.; Xiao, T.; Balakrishna, A.; Nair, S.; Rafailov, R.; Foster, E. P.; Lam, G.; Sanketi, P. R.; Vuong, Q.; Kollar, T.; Burchfiel, B.; Tedrake, R.; Sadigh, D.; Levine, S.; Liang, P.; and Finn, C. 2024. OpenVLA: An Open-Source Vision-Language-Action Model. ArXiv, abs/2406.09246

    Pith/arXiv arXiv 2024

  46. [46]

    Li, C.; Wen, J.; Peng, Y.; Peng, Y.; Feng, F.; and Zhu, Y. 2025 a . PointVLA: Injecting the 3D World into Vision-Language-Action Models. ArXiv, abs/2503.07511

    Pith/arXiv arXiv 2025

  47. [47]

    Li, X.; Heng, L.; Liu, J.; Shen, Y.; Gu, C.; Liu, Z.; Chen, H.; Han, N.; Zhang, R.; Tang, H.; Zhang, S.; and Dong, H. 2025 b . 3DS-VLA: A 3D Spatial-Aware Vision Language Action Model for Robust Multi-Task Manipulation. In Lim, J.; Song, S.; and Park, H.-W., eds., Proceedings of The 9th Conference on Robot Learning, volume 305 of Proceedings of Machine Le...

    2025

  48. [48]

    R.; Fu, C.; Lunawat, I.; Sieh, I.; Kirmani, S.; Levine, S.; Wu, J.; Finn, C.; Su, H.; Vuong, Q

    Li, X.; Hsu, K.; Gu, J.; Pertsch, K.; Mees, O.; Walke, H. R.; Fu, C.; Lunawat, I.; Sieh, I.; Kirmani, S.; Levine, S.; Wu, J.; Finn, C.; Su, H.; Vuong, Q. H.; and Xiao, T. 2024 a . Evaluating Real-World Robot Manipulation Policies in Simulation. In Conference on Robot Learning

    2024

  49. [49]

    Li, X.; Li, P.; Qian, L.; Liu, M.; Wang, D.; Liu, J.; Kang, B.; Ma, X.; Wang, X.; Guo, D.; Kong, T.; Zhang, H.; and Liu, H. 2024 b . What matters in building vision–language–action models for generalist robots. Nature Machine Intelligence, 8: 158 -- 172

    2024

  50. [50]

    Li, Z.; Ren, L.; Yang, J.; Zhao, Y.; Wu, X.; Xu, Z.; Bai, X.; and Zhao, H. 2024 c . VIP: Vision Instructed Pre-training for Robotic Manipulation

    2024

  51. [51]

    Liu, B.; Zhu, Y.; Gao, C.; Feng, Y.; Liu, Q.; Zhu, Y.; and Stone, P. 2023. LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning. ArXiv, abs/2306.03310

    Pith/arXiv arXiv 2023

  52. [52]

    Liu, Z.; Dong, Y.; Rao, Y.; Zhou, J.; and Lu, J. 2024. Chain-of-Spot: Interactive Reasoning Improves Large Vision-Language Models. ArXiv, abs/2403.12966

    Pith/arXiv arXiv 2024

  53. [53]

    Qu, D.; Song, H.; Chen, Q.; Yao, Y.; Ye, X.; Ding, Y.; Wang, Z.; Gu, J.; Zhao, B.; Wang, D.; and Li, X. 2025. SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model. ArXiv, abs/2501.15830

    Pith/arXiv arXiv 2025

  54. [54]

    Shen, H.; Zhao, K.; Zhao, T.; Xu, R.; Zhang, Z.; Zhu, M.; and Yin, J. 2024. ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Exploration. In Conference on Empirical Methods in Natural Language Processing

    2024

  55. [55]

    Song, W.; Zhou, Z.; Zhao, H.; Chen, J.; Ding, P.; Yan, H.; Huang, Y.; Tang, F.; Wang, D.; and Li, H. 2025. ReconVLA: Reconstructive Vision-Language-Action Model as Effective Robot Perceiver. In AAAI Conference on Artificial Intelligence

    2025

  56. [56]

    M.; Ghosh, D.; Walke, H

    Team, O. M.; Ghosh, D.; Walke, H. R.; Pertsch, K.; Black, K.; Mees, O.; Dasari, S.; Hejna, J.; Kreiman, T.; Xu, C.; Luo, J.; Tan, Y. L.; Sanketi, P. R.; Vuong, Q.; Xiao, T.; Sadigh, D.; Finn, C.; and Levine, S. 2024. Octo: An Open-Source Generalist Robot Policy. ArXiv, abs/2405.12213

    Pith/arXiv arXiv 2024

  57. [57]

    Wu, X.-M.; Fan, B.; Liao, K.; Jiang, J.-J.; Yang, R.; Luo, Y.; Wu, Z.; Zheng, W.; and Loy, C. C. 2026. VLANeXt: Recipes for Building Strong VLA Models. ArXiv, abs/2602.18532

    Pith/arXiv arXiv 2026

  58. [58]

    Zawalski, M.; Chen, W.; Pertsch, K.; Mees, O.; Finn, C.; and Levine, S. 2024. Robotic Control via Embodied Chain-of-Thought Reasoning. In Conference on Robot Learning

    2024

  59. [59]

    Zhang, J.; Khayatkhoei, M.; Chhikara, P.; and Ilievski, F. 2024. Towards Perceiving Small Visual Details in Zero-shot Visual Question Answering with Multimodal LLMs. arXiv:2310.16033

    Pith/arXiv arXiv 2024

  60. [60]

    Zhang, J.; Khayatkhoei, M.; Chhikara, P.; and Ilievski, F. 2025. MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMs. ArXiv, abs/2502.17422

    Pith/arXiv arXiv 2025

  61. [61]

    J.; Fu, Z.; Zhang, Z.; Wu, Y.; Li, Z.; Ma, Q.; Han, S.; Finn, C.; Handa, A.; Liu, M.-Y.; Xiang, D.; Wetzstein, G.; and Lin, T.-Y

    Zhao, Q.; Lu, Y.; Kim, M. J.; Fu, Z.; Zhang, Z.; Wu, Y.; Li, Z.; Ma, Q.; Han, S.; Finn, C.; Handa, A.; Liu, M.-Y.; Xiang, D.; Wetzstein, G.; and Lin, T.-Y. 2025. CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 1702--1713

    2025

  62. [62]

    Zheng, R.; Liang, Y.; Huang, S.; Gao, J.; Daum'e, H.; Kolobov, A.; Huang, F.; and Yang, J. 2024. TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policies. ArXiv, abs/2412.10345

    Pith/arXiv arXiv 2024

  63. [63]

    Zhong, L.; Rosenthal, F.; Sicking, J.; Hüger, F.; Bagdonat, T.; Gottschalk, H.; and Schwinn, L. 2025. FOCUS: Internal MLLM Representations for Efficient Fine-Grained Visual Question Answering. arXiv:2506.21710

    arXiv 2025

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.