Pith. sign in

REVIEW 4 major objections 6 minor 95 references

A compact 3D Gaussian scene memory, compressed to 64 tokens, lifts real-robot manipulation success by 22.8 points over 2D and depth baselines, the authors report.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 06:31 UTC pith:CXZL3LZM

load-bearing objection A credible 3D-Gaussian VLA interface with a clean compression scheme, but the semantic-grounding attribution isn't isolated by the experiments. the 4 major comments →

arxiv 2607.12356 v2 pith:CXZL3LZM submitted 2026-07-14 cs.RO

VistaVLA: Geometry- and Semantic-Aware 3D Gaussian-Grounded VLA for Robotic Manipulation

Mohan Liu , Zhihao Gu , Xuanyu Chen , Haitian Zhang , Kaimin Mao , Yan Wu , Wei-Yun Yau , Lin Wang This is my paper
classification cs.RO
keywords 3D Gaussian splattingvision-language-action modelsrobotic manipulationsemantic scene representationtoken compressionspatial generalizationmulti-view feature distillationLIBERO benchmark
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

VistaVLA argues that vision-language-action (VLA) robots fail at precise manipulation because they see the world as flat 2D images, lacking an explicit, semantics-aware 3D scene model. The paper shows that lifting language-aligned visual features into a multi-view-consistent Gaussian field, then compressing that field to just 64 tokens, improves real-robot success by 22.8 percentage points over a strong 2D baseline across seven tasks. The key evidence that the gains come from the 3D grounding: adding more 2D camera views or raw depth improves the baseline only slightly, while the 3D-token policy is the only one to succeed at all on a position-perturbed task where every baseline scores zero. A reader should care because this is a concrete recipe for giving generalist robot policies genuine 3D spatial reasoning without blowing up their compute budget.

Core claim

The paper's central claim is that a scene-level, geometry-anchored semantic representation—Gaussian primitives whose 128-dimensional latent codes are distilled from foundation visual encoders under multi-view RGB-D supervision—serves as a better context for VLA policy learning than 2D patch tokens or raw depth. Once the dense field (about 10^5 primitives) is summarized by the Merge-then-Query mechanism into 64 query tokens, the policy can plan over spatial layouts and semantic identities together. The authors assert this is why VistaVLA outperforms the VLA-Adapter baseline by 22.8 points across seven real-world tasks, improves depth-perturbed success from 6/10 to 9/10, and is the only method

What carries the argument

The load-bearing object is the semantic Gaussian field: each 3D Gaussian primitive carries a 128-dimensional feature vector, trained so that alpha-composited rendering of these features matches a compact teacher representation distilled from language-aligned and self-supervised visual encoders. This binds semantics to concrete 3D positions, visible from any viewpoint. The second mechanism is Merge-then-Query (MtQ), which compresses roughly 10^5 primitives to 64 tokens by Morton-order spatial partitioning, similarity-based merging of redundant tokens, and a two-layer query transformer. MtQ's job is to preserve action-relevant spatial layout while cutting the token count by 99%, making 3D grou

Load-bearing premise

The load-bearing premise is that the gains come from the 3D semantic Gaussian tokens themselves, not from the extra multi-view reconstruction pipeline; the paper's own limitation note concedes that pipeline depends on fixed calibrated cameras, which complicates a fair comparison against baselines that receive the same two views more naively.

What would settle it

Give the VLA-Adapter baseline the same two calibrated views but replace its 2D patch tokens with 64 tokens produced by the identical teacher-feature distillation and MtQ compression applied to a pseudo-3D input such as depth-composited per-view features; if that baseline then matches VistaVLA's success rates, especially the 3/10 on the position-perturbed task, the Gaussian field itself is not the causal ingredient.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • VLA policies can gain explicit 3D spatial reasoning without a large increase in inference cost: 64 summary tokens replace hundreds of thousands of Gaussian primitives (a 99% token reduction) while retaining action-relevant layout and semantics.
  • Robustness to spatial perturbations improves sharply: on a position-shifted task every 2D and depth-based baseline scores 0/10 while VistaVLA scores 3/10, and depth-perturbed success rises from 6/10 to 9/10.
  • The gains are attributable to the 3D-grounded semantic interface, not to extra inputs: ablations show that adding 2D camera views (up to 768 tokens) or raw depth improves the baseline only marginally, while VistaVLA's roughly 32 tokens do far better.
  • The approach transfers to simulation: on LIBERO-Pro-Swap spatial out-of-distribution layouts, VistaVLA raises the baseline from 1.7% to 12.2%, and on standard LIBERO it reaches 96.05% average success without large-scale robot-action pretraining.
  • A 0.5B-parameter policy with the Gaussian context can surpass a 3B generalist policy on spatially demanding real-world tasks, suggesting the spatial interface partly substitutes for raw model scale.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the Gaussian field is viewpoint-invariant, a natural extension the authors do not pursue is persistent scene memory—updating the same field across time rather than rebuilding it each step—which could give robots temporally stable spatial reasoning under occlusion or camera motion.
  • Editorial inference: the MtQ compression result suggests that action-relevant 3D context is highly redundant; the same merge-then-query principle could be applied to other dense 3D representations (point clouds, voxel grids, depth maps) to test whether the benefit is specific to Gaussian primitives or generalizes.
  • Editorial inference: a cheap control experiment would feed the same distilled teacher features into the baseline as depth-augmented 2D tokens; if the margin narrows, the semantic distillation rather than the 3D field is doing the work.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. VistaVLA proposes a two-stage framework that constructs a 3D Gaussian scene representation whose primitives carry distilled 2D foundation-model semantic features, then summarizes the dense primitives with Merge-then-Query (MtQ) into 64 compact tokens injected into a VLA policy. Stage I trains a Gaussian field with RGB-D reconstruction losses and a teacher-feature rendering loss (Sec. 3.1); Stage II performs stride pooling, Morton-order-based merging, and query-based decoding to produce the final context tokens (Sec. 3.2). The method is evaluated on seven real-world tabletop tasks and on LIBERO/LIBERO-Pro-Swap, reporting a 22.8-point average real-world improvement over VLA-Adapter and a 3/10 vs 0/10 success on a position-perturbed test where all baselines fail.

Significance. The paper attacks a timely problem—giving VLA policies an explicit, compact, semantic 3D scene representation—and the proposed MtQ compression is simple, largely parameter-free, and clearly described. The real-world perturbed-task evaluation is a good direction, and the authors are candid about calibration and scope limitations. If the central attribution claim were established, this would be a useful contribution to 3D-grounded VLA. However, that claim is not yet supported: the experiments lack a matched ablation isolating semantic content, the real-world numbers rest on 10-trial point estimates, the baseline comparison may confound representation with pipeline maturity, and no code/data/parameters are released to make the token-reduction and semantic-preservation claims externally checkable.

major comments (4)
  1. [§4.3, Table 3(b); §4.1, Fig. 4] The headline attribution—'gains do not simply come from injecting geometric cues or adding more camera views, but from introducing a structured 3D Gaussian representation as a semantic 3D spatial interface'—is not supported by the experiments. No ablation removes semantic content from the same GS/MtQ pipeline: every row in Table 3(b) uses distilled SigLIP2+DINOv2 features, and VLA-Adapter+Depth is a 2D depth-augmented baseline rather than a GS pipeline without semantics. A variant with random or non-semantic Gaussian features, using the identical multi-view reconstruction and MtQ compression, is needed to show that the learned semantic teacher features, not the 3D geometry/tokenization, are load-bearing.
  2. [§4.1, Fig. 4, Table 1; §4.3, Table 3] All real-world results are point estimates from 10 trials per task, with no per-seed variance, confidence intervals, or statistical tests. Several decisive comparisons are one or two successes apart (e.g., Table 3(a) T1: 9/10 vs 8/10; Table 1 Depth: 9/10 vs 7/10), and the position-perturbation difference is 3/10 vs 0/10. Under a binomial model these differences can be within sampling noise. Reporting more trials, repeated seeds, exact confidence intervals, or at least per-trial logs is necessary to support the 22.8-point and 30-point claims.
  3. [§4.1, Setup; Fig. 4] The fairness of the comparison is under-specified. The baseline is 'equipped with the same number of 2 fixed camera views' in a naive 2D-token manner, while VistaVLA receives calibrated multi-view RGB-D reconstruction, a pretrained DepthSplat encoder, teacher-feature distillation, and an online Gaussian field. No inference latency, reconstruction failure rate, or per-method compute budget is reported. The observed 22.8-point margin could therefore reflect pipeline maturity or additional engineering rather than the 3D semantic representation. A stronger 2D baseline with the same per-view feature encoders and token budget, or a matched 3D geometric baseline, is needed to isolate the claimed cause.
  4. [Abstract; §1; §3.2] The claim of '99% token reduction while preserving action-relevant 3D layouts and semantic context' overstates what is measured. No direct evaluation of semantic fidelity or layout preservation of the compressed tokens is provided; the only evidence is downstream task success. I recommend adding a direct fidelity metric (e.g., reconstruction of the teacher features or semantic segmentation/retrieval consistency from the 64 query tokens) or softening the claim to 'without degrading policy performance in the tested tasks.'
minor comments (6)
  1. [§2] Typo: 'By constrast' should be 'By contrast.' Also, the method name alternates between 'VistaVLA' and 'Vista-VLA'; please use one consistent form.
  2. [Table 3(a)] The row 'Vista-VLA-3Cam 3209/10 7/10 7/10 7/10' is ambiguous: the token count 320 and the success count 9/10 run together. Insert a space or use a separate column.
  3. [Fig. 4] Success rates are shown without confidence intervals or error bars. For 10-trial data, at least add the raw counts or a binomial CI to make the visual comparison informative.
  4. [§4.2, Table 2(a)] The term 'zero-shot transfer on LIBERO-Pro-Swap' is not defined in the setup. Is the same standard-LIBERO checkpoint evaluated directly on Pro-Swap without any adaptation or fine-tuning? Please state the protocol explicitly.
  5. [§3.1] The teacher auto-encoder is described as 'trained once,' but the training data, loss weighting, and schedule are not specified. Please clarify, since the 2176→128 bottleneck is part of the method.
  6. [§3.2] The choice of the compression target N'=1000, the top-50% merge fraction, and the stride-4 pooling are not swept or motivated beyond the main ablations. Even a short sensitivity discussion would help, since these are free design parameters.

Circularity Check

0 steps flagged

No circularity: the method is an empirical VLA pipeline whose gains are claimed from comparative experiments, not derived from its own fitted quantities.

full rationale

VistaVLA's pipeline is a two-stage empirical system: Stage I distills SigLIP2/DINOv2 features into 3D Gaussian primitives using an RGB-D rendering loss, and Stage II compresses the resulting tokens with Merge-then-Query before injecting them into a VLA policy. The paper's central claim is that this representation improves real-world and simulated manipulation success rates. That claim is supported by external benchmarks, baselines, and ablations rather than by a derivation that reduces to the inputs. There is no equation in which a predicted quantity is defined as the fitted parameter, no fitted input renamed as a prediction, and no uniqueness theorem imported from the authors' prior work to force the design. The closest concerns are (a) the absence of a control that removes only the semantic teacher features from the same 3D-Gaussian/MtQ pipeline, and (b) references [17,19] to the authors' own prior work for the '3D semantic cognitive map' motivation. Neither is circular: (a) is an evidence/fairness gap about whether geometry or semantics drives the improvement, not an internal logical reduction; (b) is a cognitive-science framing citation that is not load-bearing for the quantitative results. The claim that 'the gains do not simply come from injecting geometric cues or adding more camera views, but from introducing a structured 3D Gaussian representation as a semantic 3D spatial interface' is an interpretation of the experimental comparison, not a result forced by construction. Accordingly, the paper does not exhibit the specific reduction required for a circularity finding.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

No new physical or ontological entities are introduced. The conceptual invention is the MtQ token pipeline and the 'cognitive map' framing; the free parameters are all standard architectural choices, but several are unablated or unreported (loss weights), which raises the reproduction burden.

free parameters (6)
  • Teacher bottleneck dimension (128-d) = 128
    Chosen to compress SigLIP2+DINOv2 2176-d features; no ablation of this capacity; the 128-d GS token is the main information channel into the policy.
  • Compression target N' = 1000 = 1000
    Intermediate post-merge token count chosen in Sec. 3.2 without a sweep.
  • Query token count Nq = 64 = 64
    Ablated in Table 3(c) on one task; 64 wins, 256 is worst; selected per task-specific data.
  • Top-50% merge fraction = 50%
    Heuristic gate in Morton-guided merging; no sensitivity analysis.
  • Action execution chunk K_exec = 4 = 4
    Inference-frequency choice described in Sec. 3.2; no sweep reported.
  • Loss weights (lambda_rgb, lambda_dep, lambda_feat) = unspecified
    Referenced in Sec. 3.1 but never given numeric values, making the reconstruction/distillation balance unreproducible.
axioms (5)
  • domain assumption Two calibrated multi-view RGB-D views at training time and calibrated poses at inference are available per control step.
    The entire Gaussian-field construction in Sec. 3.1 and the online scene build at inference depend on posed observations; the paper's own limitation section states this.
  • domain assumption DepthSplat [60] provides a feed-forward Gaussian reconstruction from the posed RGB/D inputs that is stable enough online.
    VistaVLA's Stage I uses DepthSplat as the Gaussian backbone; reconstruction quality is not independently evaluated in the paper.
  • domain assumption SigLIP2 and DINOv2-Large features, compressed to 128-d by a separately trained auto-encoder, retain the semantics needed for manipulation.
    The whole semantic grounding rests on this distillation; no evaluation of feature-reconstruction fidelity is provided.
  • domain assumption Language-conditioned cross-attention ordering [X_vis, X_lang, X_gs, X_act] is a sufficient interface for injecting 3D context.
    Used in Sec. 3.2 without comparison to other injection orders.
  • domain assumption LIBERO-Pro-Swap is a valid zero-shot spatial-transfer benchmark and the reported baseline numbers are comparable given matched training budget.
    The simulation claims depend on this benchmark's fairness; several baselines score 0.0, which is consistent with known benchmark difficulty but also with protocol differences.

pith-pipeline@v1.3.0-alltime-deepseek · 16113 in / 8793 out tokens · 70222 ms · 2026-08-02T06:31:39.075577+00:00 · methodology

0 comments
read the original abstract

Vision-Language-Action (VLA) models have emerged as a powerful end-to-end paradigm for robotic manipulation by mapping language instructions and 2D visual inputs directly to actions. However, these models lack an explicit, scene-level 3D representation, limiting their ability to reason over spatial layouts and geometric constraints. While recent efforts incorporate explicit 3D cues, such as depth maps or point clouds, to improve geometric awareness, they primarily capture low-level structures and lack high-level semantic grounding in 3D space. In human cognition, interaction with the physical world relies on a 3D semantic cognitive map - an internal mental model that integrates spatial layouts with semantic context to enable persistent, viewpoint-invariant reasoning. In light of this, we present VistaVLA, a novel two-stage framework that constructs a geometry- and semantics-aware 3D cognitive representation from 3D Gaussian primitives and grounds it as compact context tokens for VLA policy learning. Specifically, VistaVLA lifts multi-view vision-language features into 3D Gaussian primitives, forming geometry-anchored semantic tokens that align view-consistent spatial grounding with 2D visual feature spaces. To make this 3D representation computationally tractable for effective VLA control, we introduce Merge-then-Query (MtQ), a token summarization mechanism. MtQ compresses dense Gaussian primitives into a highly compact set of spatially informative tokens, achieving a 99% token reduction while preserving action-relevant 3D layouts and semantic context. Extensive evaluations in both simulated and real-world environments demonstrate the effectiveness of VistaVLA. Notably, in real-world scenarios, VistaVLA improves success rates by 22.8% across seven real-world tasks and by 30.0% over the VLA-Adapter baseline on challenging out-of-distribution tasks.

Figures

Figures reproduced from arXiv: 2607.12356 by Haitian Zhang, Kaimin Mao, Lin Wang, Mohan Liu, Wei-Yun Yau, Xuanyu Chen, Yan Wu, Zhihao Gu.

Figure 1
Figure 1. Figure 1: VistaVLA bridges 3D geometric structure and semantic grounding for VLA manip￾ulation. (a) We compare current VLA paradigms: 2D-input VLAs lack explicit 3D structure, while existing 3D-input VLAs mainly provide geometric cues without sufficient semantic grounding in 3D space. VistaVLA addresses both aspects by constructing semantically grounded 3D Gaussian scene representations for policy learning. (b) This… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of VistaVLA. Stage I: Semantic Gaussian field training distills foundation visual features under RGB-D rendering supervision. Stage II: Merge-then-Query compresses dense Gaussian primitives into compact 3D summary tokens. These tokens are finally injected into the VLA policy with instruction and wrist-image tokens for spatially grounded action prediction. For each training frame, we extract dense … view at source ↗
Figure 3
Figure 3. Figure 3: Illustration of real-world tasks. Each row shows keyframes of the corresponding manip￾ulation trajectory, covering diverse object arrangements, interaction patterns, and temporal horizons. This makes each GS token a multi-view semantic descriptor grounded in 3D space. During train￾ing, we use two calibrated RGB-D views as context and render the third view as a novel target to enforce cross-view consistency… view at source ↗
Figure 4
Figure 4. Figure 4: Real-world results. All policies are trained or fine-tuned on the same task-specific demon￾strations. Our method consistently achieves the highest success rates across all tasks, with especially large gains on spatially demanding manipulation scenarios. Results are reported as success rates (%). Query-based GS summarization. After parameter-free compression, we further summarize the compressed GS tokens in… view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

95 extracted references · 2 canonical work pages

  1. [1]

    O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xu, J. Luo, Y . L. Tan, L. Y . Chen, P. Sanketi, Q. Vuong, T. Xiao, D. Sadigh, C. Finn, and S. Levine. Octo: An open-source generalist robot policy. 2024. URL https://arxiv.org/abs/2405.12213

    Pith/arXiv arXiv 2024

  2. [2]

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, Q. Vuong, T. Kollar, B. Burchfiel, R. Tedrake, D. Sadigh, S. Levine, P. Liang, and C. Finn. Openvla: An open-source vision-language-action model. 2024. URL https://arxiv.org/abs/2406.09246

    Pith/arXiv arXiv 2024

  3. [3]

    M. J. Kim, C. Finn, and P. Liang. Fine-tuning vision-language-action models: Optimizing speed and success. 2025. URLhttps://arxiv.org/abs/2502.19645

    Pith/arXiv arXiv 2025

  4. [4]

    Z. Gu, M. Yang, D. Zou, and D. Xu. Learning diffusion policy from primitive skills for robot manipulation. InProceedings of the AAAI Conference on Artificial Intelligence, page 18262–18270, 2026

    2026

  5. [5]

    Black, N

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Haus- man, B. Ichter, S. Jakubczak, T. Jones, L. Ke, S. Levine, A. Li-Bell, M. Mothukuri, S. Nair, K. Pertsch, L. X. Shi, J. Tanner, Q. Vuong, A. Walling, H. Wang, and U. Zhilinsky.π 0: A vision-language-action flow model for general robot control, 2026. URLhttps://arxiv. o...

    Pith/arXiv arXiv 2026

  6. [6]

    Intelligence, K

    P. Intelligence, K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, M. Y . Galliker, D. Ghosh, L. Groom, K. Hausman, B. Ichter, S. Jakubczak, T. Jones, L. Ke, D. LeBlanc, S. Levine, A. Li-Bell, M. Mothukuri, S. Nair, K. Pertsch, A. Z. Ren, L. X. Shi, L. Smith, J. T. Springenberg, K. Stachowicz, J. Tanner, Q. V...

    Pith/arXiv arXiv 2025

  7. [7]

    Y . Wang, P. Ding, L. Li, C. Cui, Z. Ge, X. Tong, W. Song, H. Zhao, W. Zhao, P. Hou, S. Huang, Y . Tang, W. Wang, R. Zhang, J. Liu, and D. Wang. Vla-adapter: An effective paradigm for tiny- scale vision-language-action model, 2025. URLhttps://arxiv.org/abs/2509.09372

    arXiv 2025

  8. [8]

    Zitkovich, T

    B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Proceedings of The 7th Conference on Robot Learning, volume 229 ofProceedings of Machine Learning Research, pages 2165–2183, 2023

    2023

  9. [9]

    Brohan, N

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Haus- man, A. Herzog, J. Hsu, et al. Rt-1: Robotics transformer for real-world control at scale.arXiv preprint arXiv:2212.06817, 2022

    Pith/arXiv arXiv 2022

  10. [10]

    Bhat, Y .-H

    V . Bhat, Y .-H. Lan, P. Krishnamurthy, R. Karri, and F. Khorrami. 3d cavla: Leveraging depth and 3d context to generalize vision language action models for unseen tasks. 2025. URL https://arxiv.org/abs/2505.05800

    arXiv 2025

  11. [11]

    Y . Li, Y . Chen, M. Zhou, H. Li, Z. Zhang, and D. Zhao. Qdepth-vla: Quantized depth prediction as auxiliary supervision for vision-language-action models, 2025. URLhttps://arxiv. org/abs/2510.14836

    Pith/arXiv arXiv 2025

  12. [12]

    D. Qu, H. Song, Q. Chen, Y . Yao, X. Ye, Y . Ding, Z. Wang, J. Gu, B. Zhao, D. Wang, and X. Li. Spatialvla: Exploring spatial representations for visual-language-action model. 2025. URLhttps://arxiv.org/abs/2501.15830

    Pith/arXiv arXiv 2025

  13. [13]

    C. Li, J. Wen, Y . Peng, Y . Peng, F. Feng, and Y . Zhu. Pointvla: Injecting the 3d world into vision-language-action models. 2025. URLhttps://arxiv.org/abs/2503.07511. 9

    Pith/arXiv arXiv 2025

  14. [14]

    F. Li, W. Song, H. Zhao, J. Wang, P. Ding, D. Wang, L. Zeng, and H. Li. Spatial forcing: Implicit spatial representation alignment for vision-language-action model.arXiv preprint arXiv:2510.12276, 2025

    arXiv 2025

  15. [15]

    D. H. Clements and M. T. Battista. Geometry and spatial reasoning. 2006

    2006

  16. [16]

    R. M. Byrne and P. N. Johnson-Laird. Spatial reasoning.Journal of memory and language, 28 (5):564–575, 1989

    1989

  17. [17]

    B. D. Manh, S. Debnath, Z. Zhang, S. Damodaran, A. Kumar, Y . Zhang, L. Mi, E. Cambria, and L. Wang. Mind meets space: Rethinking agentic spatial intelligence from a neuroscience- inspired perspective.arXiv preprint arXiv:2509.09154, 2025

    Pith/arXiv arXiv 2025

  18. [18]

    Zhang, M

    W. Zhang, M. Wang, G. Liu, X. Huixin, Y . Jiang, Y . Shen, G. Hou, Z. Zheng, H. Zhang, X. Li, et al. Embodied-reasoner: Synergizing visual search, reasoning, and action for embodied interactive tasks.arXiv preprint arXiv:2503.21696, 2025

    Pith/arXiv arXiv 2025

  19. [19]

    S. Ruan, L. Wang, C. Kang, Q. Zhu, S. Liu, X. Wei, and H. Su. From reactive to cognitive: brain-inspired spatial intelligence for embodied agents.intelligence (AGI), 3(9):10, 2025

    2025

  20. [20]

    Kerbl, G

    B. Kerbl, G. Kopanas, T. Leimk ¨uhler, and G. Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM Transactions on Graphics, 42(4):1–14, 2023. doi:10.1145/ 3592433

    2023

  21. [21]

    Dong, K.-H

    P. Dong, K.-H. Hung, T. Gao, D. Sadigh, and C. Finn. Expo-ft: Sample-efficient reinforcement learning finetuning for vision-language-action models, 2026. URLhttps://arxiv.org/ abs/2605.25477

    Pith/arXiv arXiv 2026

  22. [22]

    Jin and Z

    R. Jin and Z. Zhang. Agentic-vla: Efficient online adaptation for vision-language-action mod- els, 2026. URLhttps://arxiv.org/abs/2605.22896

    Pith/arXiv arXiv 2026

  23. [23]

    Y . Fu, C. Zhang, S. Gong, Y . Deng, K. Sun, Q. Min, Q. Hou, Y . Tang, J. Wang, and D. Zhou. Stablevla: Towards robust vision-language-action models without extra data, 2026. URL https://arxiv.org/abs/2605.18287

    Pith/arXiv arXiv 2026

  24. [24]

    X. Guo, B. Xie, W. Chai, X. Deng, T. Wang, Z. Wu, and X. Chen. Priorvla: Prior-preserving adaptation for vision-language-action models, 2026. URLhttps://arxiv.org/abs/2605. 10925

    2026

  25. [25]

    Q. Li, X. Gong, X. Li, P. Li, Q. Zhou, H. Ye, J. Zhou, and Y . Mu. Rotvla: Rotational latent ac- tion for vision-language-action model, 2026. URLhttps://arxiv.org/abs/2605.13403

    Pith/arXiv arXiv 2026

  26. [26]

    Z. Tang, H. Liu, X. Chang, C. Wu, D. Huo, Y . Yang, B. Liu, Z. Cai, F. Xiong, M. Xu, J. Luo, D. Ma, Z. Ma, and G. Pan. Alam: Algebraically consistent latent action model for vision- language-action models, 2026. URLhttps://arxiv.org/abs/2605.10819

    Pith/arXiv arXiv 2026

  27. [27]

    Jiang, J

    Y . Jiang, J. Lu, X. Qin, X. Chen, K. Wang, F. Gao, and L. Zhao. Vla-gse: Boosting parameter- efficient fine-tuning in vla with generalized and specialized experts, 2026. URLhttps:// arxiv.org/abs/2605.06175

    Pith/arXiv arXiv 2026

  28. [28]

    S. Lian, B. Yu, X. Lin, Z. Shen, L. T. Yang, Y . Jin, H. Liu, C. Wu, H. Yuan, C. Huang, and K. Chen. Intentvla: Short-horizon intent modeling for aliased robot manipulation, 2026. URL https://arxiv.org/abs/2605.14712

    Pith/arXiv arXiv 2026

  29. [29]

    S. Fan, K. Wu, Z. Che, X. Wang, D. Wu, F. Liao, N. Liu, Y . Zhang, Z. Zhao, Z. Xu, M. Li, Q. Liu, S. Zhang, M. Wan, and J. Tang. Xr-1: Towards versatile vision-language-action models via learning unified vision-motion representations, 2025. URLhttps://arxiv.org/abs/ 2511.02776. 10

    Pith/arXiv arXiv 2025

  30. [30]

    M. Koo, D. Choi, T. Kim, K. Lee, C. Kim, Y . Seo, and J. Shin. Hamlet: Switch your vision- language-action model into a history-aware policy, 2025. URLhttps://arxiv.org/abs/ 2510.00695

    Pith/arXiv arXiv 2025

  31. [31]

    L. Xiao, J. Li, J. Gao, F. Ye, Y . Jin, J. Qian, J. Zhang, Y . Wu, and X. Yu. Ava-vla: Improving vision-language-action models with active visual attention, 2025. URLhttps://arxiv. org/abs/2511.18960

    Pith/arXiv arXiv 2025

  32. [32]

    Y . Li, P. Tang, W. Zhang, C. Zhu, Y . Duan, W. Shi, X. Zhang, Z. Yang, J. Ji, and Y . Zhang. Favla: A force-adaptive fast-slow vla model for contact-rich robotic manipulation, 2026. URL https://arxiv.org/abs/2602.23648

    arXiv 2026

  33. [33]

    Z. Yang, Y . Qi, T. Xie, B. Yu, S. Liu, and M. Li. Dysl-vla: Efficient vision-language-action model inference via dynamic-static layer-skipping for robot manipulation, 2026. URLhttps: //arxiv.org/abs/2602.22896

    arXiv 2026

  34. [34]

    Zhong, Y

    L. Zhong, Y . Liu, Y . Wei, Z. Xiong, M. Yao, S. Liu, and G. Ren. Acot-vla: Action chain- of-thought for vision-language-action models, 2026. URLhttps://arxiv.org/abs/2601. 11404

    2026

  35. [35]

    H. Zhen, X. Qiu, P. Chen, J. Yang, X. Yan, Y . Du, Y . Hong, and C. Gan. 3d-vla: A 3d vision-language-action generative world model. 2024. URLhttps://arxiv.org/abs/ 2403.09631

    Pith/arXiv arXiv 2024

  36. [36]

    L. Sun, B. Xie, Y . Liu, H. Shi, T. Wang, and J. Cao. Geovla: Empowering 3d representations in vision-language-action models. 2025. URLhttps://arxiv.org/abs/2508.09071

    Pith/arXiv arXiv 2025

  37. [37]

    Singh, A

    I. Singh, A. Goyal, S. Birchfield, D. Fox, A. Garg, and V . Blukis. Og-vla: Orthographic image generation for 3d-aware vision-language action model. 2025. URLhttps://arxiv.org/ abs/2506.01196

    arXiv 2025

  38. [38]

    P. Li, Y . Chen, H. Wu, X. Ma, X. Wu, Y . Huang, L. Wang, T. Kong, and T. Tan. Bridgevla: Input-output alignment for efficient 3d manipulation learning with vision-language models

  39. [39]

    T. Yuan, Y . Liu, C. Lu, Z. Chen, T. Jiang, and H. Zhao. Depthvla: Enhancing vision-language- action models with depth-aware spatial reasoning. 2025. URLhttps://arxiv.org/abs/ 2510.13375

    arXiv 2025

  40. [40]

    Z. Rao, W. Chen, L. Xie, X. Hua, D. Yin, Z. Tian, and F. R. Yu. Augvla-3d: Depth-driven feature augmentation for vision-language-action models. 2026. URLhttps://arxiv.org/ abs/2602.10698

    Pith/arXiv arXiv 2026

  41. [41]

    T. Lin, Y . Du, J. Liu, N. Zhu, Y . Li, Y . Fu, Y . Chen, H. Cai, Z. Ye, B. Cheng, K. Ye, Y . Mao, Y . Zhong, M. Dong, J. Yan, G. Li, and B. Zhao. Evo-depth: A lightweight depth-enhanced vision-language-action model, 2026. URLhttps://arxiv.org/abs/2605.14950

    Pith/arXiv arXiv 2026

  42. [42]

    Song and L

    H. Song and L. Le. Avi: Action from volumetric inference, 2025. URLhttps://arxiv. org/abs/2510.21746

    arXiv 2025

  43. [43]

    Zhang, H

    Z. Zhang, H. Li, Y . Dai, Z. Zhu, L. Zhou, C. Liu, D. Wang, F. E. H. Tay, S. Chen, Z. Liu, Y . Liu, X. Li, and P. Zhou. From spatial to actions: Grounding vision-language-action model in spatial foundation priors, 2025. URLhttps://arxiv.org/abs/2510.17439

    arXiv 2025

  44. [44]

    Huang, M

    H. Huang, M. Cen, K. Tan, X. Quan, G. Huang, and H. Zhang. Graphcot-vla: A 3d spatial- aware reasoning vision-language-action model for robotic manipulation with ambiguous in- structions, 2025. URLhttps://arxiv.org/abs/2508.07650. 11

    Pith/arXiv arXiv 2025

  45. [45]

    J. Koo, T. Cho, H. Kang, E. Pyo, T. G. Oh, T. Kim, and A. J. Choi. Retovla: Reusing register tokens for spatial reasoning in vision-language-action models, 2025. URLhttps://arxiv. org/abs/2509.21243

    arXiv 2025

  46. [46]

    C. Xu, S. Zhang, Y . Liu, B. Sun, W. Chen, B. Xu, Q. Liu, J. Wang, S. Wang, S. Luo, J. Peters, A. V . Vasilakos, S. Zafeiriou, and J. Deng. An anatomy of vision-language-action models: From modules to milestones and challenges.arXiv preprint arXiv:2512.11362, 2025

    arXiv 2025

  47. [47]

    Cheng, H

    A.-C. Cheng, H. Yin, Y . Fu, Q. Guo, R. Yang, J. Kautz, X. Wang, and S. Liu. Spatialrgpt: Grounded spatial reasoning in vision language models.arXiv preprint arXiv:2406.01584, 2024

    Pith/arXiv arXiv 2024

  48. [48]

    G. Sun, T. Du, K. Feng, C. Luo, X. Ding, Z. Shen, Z. Wang, Y . He, and A. Li. Rocket: Residual-oriented multi-layer alignment for spatially-aware vision-language-action models. arXiv preprint arXiv:2602.17951, 2026

    arXiv 2026

  49. [49]

    Zhang, S

    J. Zhang, S. Wu, X. Luo, H. Wu, L. Gao, H. T. Shen, and J. Song. Inspire: Vision-language- action models with intrinsic spatial reasoning, 2025. URLhttps://arxiv.org/abs/2505. 13888

    2025

  50. [50]

    Q. Li, Y . Deng, Y . Liang, L. Luo, L. Zhou, C. Yao, L. Zeng, Z. Feng, H. Liang, S. Xu, Y . Zhang, X. Chen, H. Chen, L. Sun, D. Chen, J. Yang, and B. Guo. Scalable vision-language-action model pretraining for robotic manipulation with real-life human activity videos, 2025. URL https://arxiv.org/abs/2510.21571

    arXiv 2025

  51. [51]

    Y . Ling, Q. Lian, J. Li, Q. Jiang, T. Zhang, X. Jiang, C. Liu, J. Liu, and L. Zhang. Guide, think, act: Interactive embodied reasoning in vision-language-action models, 2026. URLhttps: //arxiv.org/abs/2605.13632

    Pith/arXiv arXiv 2026

  52. [52]

    W. Guo, Y . Wang, R. Zhou, Y . Zhang, R. Fang, Y . Pang, W. Xu, and G. Huang. Action with visual primitives, 2026. URLhttps://arxiv.org/abs/2605.22183

    Pith/arXiv arXiv 2026

  53. [53]

    H. Liu, S. Ruan, J. Long, J. Wu, J. Hou, H. Tang, T. Jiang, W. Zhou, and W. Yao. Eva-vla: Evaluating vision-language-action models’ robustness under real-world physical variations,

  54. [54]

    Y . Zhai, S. Tong, X. Li, M. Cai, Q. Qu, Y . J. Lee, and Y . Ma. Investigating the catastrophic forgetting in multimodal large language models.arXiv preprint arXiv:2309.10313, 2023

    Pith/arXiv arXiv 2023

  55. [55]

    URLhttps://arxiv.org/abs/2509.18953

    arXiv

  56. [56]

    S. Li, R. Lin, and S. Pei. Multi-modal preference alignment remedies degradation of visual instruction tuning on language models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14188–14200. Association for Computational Linguistics, 2024. doi:10.18653/v1/2024.acl-long.765

    work page doi:10.18653/v1/2024.acl-long.765 2024

  57. [57]

    J. Wu, Y . Xiong, X. Li, Y . Xia, R. Wang, Y . Wang, T. Yu, S. Kim, R. A. Rossi, L. Yao, J. Shang, and J. McAuley. Mitigating visual knowledge forgetting in mllm instruction-tuning via modality-decoupled gradient descent. InFindings of the Association for Computational Linguistics: EMNLP 2025, pages 2282–2295, Suzhou, China, 2025. Association for Compu- t...

    work page doi:10.18653/v1/2025.findings-emnlp.123 2025

  58. [58]

    Charatan, S

    D. Charatan, S. Li, A. Tagliasacchi, and V . Sitzmann. pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction, 2024. URLhttps://arxiv.org/ abs/2312.12337. 12

    Pith/arXiv arXiv 2024

  59. [59]

    Yu and S

    Z. Yu and S. Ananiadou. Locate-then-merge: Neuron-level parameter fusion for mitigating catastrophic forgetting in multimodal llms. InFindings of the Association for Computational Linguistics: EMNLP 2025. Association for Computational Linguistics, 2025

    2025

  60. [60]

    H. Xu, S. Peng, F. Wang, H. Blum, D. Barath, A. Geiger, and M. Pollefeys. Depthsplat: Con- necting gaussian splatting and depth, 2025. URLhttps://arxiv.org/abs/2410.13862

    Pith/arXiv arXiv 2025

  61. [61]

    Y . Chen, H. Xu, C. Zheng, B. Zhuang, M. Pollefeys, A. Geiger, T.-J. Cham, and J. Cai. Mvs- plat: Efficient 3d gaussian splatting from sparse multi-view images. InEuropean conference on computer vision, pages 370–386. Springer, 2024

    2024

  62. [62]

    S. Zhou, H. Chang, S. Jiang, Z. Fan, Z. Zhu, D. Xu, P. Chari, S. You, Z. Wang, and A. Kadambi. Feature 3dgs: Supercharging 3d gaussian splatting to enable distilled feature fields. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

    2024

  63. [63]

    M. Qin, W. Li, J. Zhou, H. Wang, and H. Pfister. Langsplat: 3d language gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

    2024

  64. [64]

    A. Thai, S. Peng, K. Genova, L. Guibas, and T. Funkhouser. Splattalk: 3d VQA with gaussian splatting. InProceedings of the IEEE/CVF International Conference on Computer Vision, 2025

    2025

  65. [65]

    X. Wang, C. Lan, H. Zhu, Z. Chen, and Y . Lu. Gsemsplat: Generalizable semantic 3d gaus- sian splatting from uncalibrated image pairs, 2024. URLhttps://arxiv.org/abs/2412. 16932

    2024

  66. [66]

    Zheng, X

    Y . Zheng, X. Chen, Y . Zheng, S. Gu, R. Yang, B. Jin, P. Li, C. Zhong, Z. Wang, L. Liu, C. Yang, D. Wang, Z. Chen, X. Long, and M. Wang. GaussianGrasper: 3d language gaussian splatting for open-vocabulary robotic grasping.IEEE Robotics and Automation Letters, 9(9): 7827–7834, 2024

    2024

  67. [67]

    Zhang, Y

    Z. Zhang, Y . Jiang, Q. Cheng, S. Liu, D. Zhao, P. Luo, W. Zhou, and H. Yu. Gaussiandream: A feed-forward 3d gaussian world model for robotic manipulation, 2026. URLhttps:// arxiv.org/abs/2605.20752

    Pith/arXiv arXiv 2026

  68. [68]

    J. Yu, X. Ren, Y . Gu, H. Lin, T. Wang, Y . Zhu, H. Xu, Y .-G. Jiang, X. Xue, and Y . Fu. Sparsegrasp: Robotic grasping via 3d semantic gaussian splatting from sparse multi-view rgb images, 2024. URLhttps://arxiv.org/abs/2412.02140

    Pith/arXiv arXiv 2024

  69. [69]

    Ji, R.-Z

    M. Ji, R.-Z. Qiu, X. Zou, and X. Wang. GraspSplats: Efficient manipulation with 3d feature splatting. InConference on Robot Learning, 2024

    2024

  70. [70]

    G. Lu, B. Jia, P. Li, Y . Chen, Z. Wang, Y . Tang, and S. Huang. GWM: Towards scalable gaus- sian world models for robotic manipulation. InProceedings of the IEEE/CVF International Conference on Computer Vision, 2025

    2025

  71. [71]

    G. Lu, S. Zhang, Z. Wang, C. Liu, J. Lu, and Y . Tang. ManiGaussian: Dynamic gaussian splatting for multi-task robotic manipulation. InEuropean Conference on Computer Vision, 2024

    2024

  72. [72]

    C. Yan, D. Qu, D. Xu, B. Zhao, Z. Wang, D. Wang, and X. Li. GS-SLAM: Dense visual slam with 3d gaussian splatting. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19595–19604, June 2024

    2024

  73. [73]

    Y . Chai, L. Deng, R. Shao, J. Zhang, K. Lv, L. Xing, X. Li, H. Zhang, and Y . Liu. GAF: Gaus- sian action field as a 4d representation for dynamic world modeling in robotic manipulation. arXiv preprint arXiv:2506.14135, 2025. doi:10.48550/arXiv.2506.14135

    Pith/arXiv arXiv doi:10.48550/arxiv.2506.14135 2025

  74. [74]

    Matsuki, R

    H. Matsuki, R. Murai, P. H. Kelly, and A. J. Davison. Gaussian splatting slam. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18039–18048, June 2024

    2024

  75. [75]

    Keetha, J

    N. Keetha, J. Karhade, K. M. Jatavallabhula, G. Yang, S. Scherer, D. Ramanan, and J. Luiten. SplaTAM: Splat, track & map 3d gaussians for dense rgb-d slam. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 21357– 21366, June 2024. 13

    2024

  76. [76]

    Y . Rao, W. Zhao, B. Liu, J. Lu, J. Zhou, and C.-J. Hsieh. Dynamicvit: Efficient vision trans- formers with dynamic token sparsification. InAdvances in Neural Information Processing Systems (NeurIPS), 2021

    2021

  77. [77]

    M. S. Ryoo, A. Piergiovanni, A. Arnab, M. Dehghani, and A. Angelova. Tokenlearner: Adap- tive space-time tokenization for videos. InAdvances in Neural Information Processing Systems (NeurIPS), 2021

    2021

  78. [78]

    Alayrac, J

    J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y . Hasson, K. Lenc, A. Mensch, K. Milli- can, M. Reynolds, R. Ring, E. Rutherford, S. Cabi, T. Han, Z. Gong, S. Samangooei, M. Mon- teiro, J. Menick, S. Borgeaud, A. Brock, A. Nematzadeh, S. Sharifzadeh, M. Bi ´nkowski, R. Barreira, O. Vinyals, A. Zisserman, and K. Simonyan. Flamingo: A visual languag...

    2022

  79. [79]

    Bolya, C.-Y

    D. Bolya, C.-Y . Fu, X. Dai, P. Zhang, C. Feichtenhofer, and J. Hoffman. Token merging: Your vit but faster. InInternational Conference on Learning Representations (ICLR), 2023

    2023

  80. [80]

    Z. Fan, K. Wang, K. Wen, Z. Zhu, D. Xu, and Z. Wang. Lightgaussian: Unbounded 3d gaussian compression with 15x reduction and 200+ FPS. InAdvances in Neural Information Processing Systems, 2024

    2024

Showing first 80 references.