Pith. sign in

REVIEW 4 major objections 5 minor 59 references

StellaVLA: In-Context Structured Demonstration for Generalizable Vision-Language-Action Models

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A robot policy can generalize to new tasks by reading one retrieved demonstration rewritten as the expert's reasoning, the paper reports — first on VLA-Arena, 98.8% on LIBERO, no retraining.

desk verdict A promising structured-demonstration method for in-context VLA adaptation, but the headline SOTA claims rest on comparisons that give StellaVLA a test-time demonstration the baselines don't get. read the letter →

arxiv 2608.11671 v1 pith:NF2C2HTA submitted 2026-08-12 cs.RO

Siyu Xu , Yunke Wang , Zijian Wang , Dihao Zhu , Chenghao Xia , Chengbin Du , Daochang Liu , Tao Huang
show 1 more author
Chang Xu
This is my paper · ORCID
classification cs.RO
keywords vision-language-actionmodelsin-contextimitationlearningstructureddemonstrationsspatiallanguagesupervisionroboticmanipulationtest-timeadaptationwithoutfine-tuningcross-embodimenttransferretrieval-augmentedpolicy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a robot policy can adapt out of distribution at test time — no fine-tuning, no weight updates — if it is given one retrieved demonstration rewritten as a structured explanation of the expert's reasoning: a task plan broken into sub-goals, each paired with a verbalized 3D movement and its 2D image-plane trace. The authors argue that prior in-context imitation fails because the policy sees what the expert did without the why behind it, and lacking that structure it falls back on its pretrained habits when the scene, viewpoint, or object changes. StellaVLA conditions the policy on that reasoning, extracted automatically by a vision-language model and reinforced by a training objective that makes the policy state the same rationale while it predicts actions. On this basis the paper reports ranking first on the VLA-Arena leaderboard (0.63 overall versus 0.44 for the strongest baseline), 98.8% average success on LIBERO, 85.1% zero-shot success on the perturbed LIBERO-Plus suite, and real-robot success using human, robot, and XR demonstrations as context. If true, the contribution is a parameter-free route to generalization across both tasks and embodiments: any demonstration source, reduced to the same structured language, can steer a fixed policy.

What carries the argument

The load-bearing object is the structured demonstration: a rationale-augmented trajectory in which every time step carries its current sub-goal label and its segment's structured rationale. Each rationale has two tiers — a semantic rationale (a sub-goal description such as "reach for the handle of the blue mug") and a kinematic rationale produced by the deterministic verbalizer $\Phi$, which maps any contiguous span of actions to a 3D displacement in the workspace and the 2D projection of that displacement onto the camera plane. Because the same $\Phi$ is reused on the policy's own predicted action chunks during training, retrieved demonstrations and the policy's predictions share one text vocabulary. Training couples two parallel experts on a single backbone representation $h_t$: an MLP action expert that regresses continuous action chunks under an L1 loss, and the native autoregressive language head that predicts the subtask plus the verbalized movement of the identical chunk under cross-entropy with weight $\lambda = 0.3$. Since $\Phi$ is deterministic, both heads supervise the same motion, which is what lets the language objective shape the shared representation without being needed at run time. At inference the language head is removed and the immutable demonstration prefix's key–value attention cache is computed once per episode, so the paper's measured model-side latency stays at 88–91 ms per action chunk instead of the 3177 ms it would cost to decode the language.

What would settle it

Take a held-out sample of the same demonstrations, have human annotators segment them into sub-goals, and measure agreement against the automatic annotations; if agreement is low on a substantial fraction of trajectories, the structured-demonstration premise fails. A sharper test that removes the annotation question: feed a trained StellaVLA checkpoint the correct demonstration but with its rationale text randomly permuted across segments — if LIBERO average success stays near the 98.8% baseline, the rationales are not the component carrying the effect.

Watch

Extended reading notes

Core claim

The central claim is that in-context imitation of raw trajectories teaches a VLA policy what an expert did but not why, and that missing why is what blocks out-of-distribution generalization. StellaVLA replaces the raw trajectory with a structured demonstration: an off-the-shelf vision-language model decomposes each demonstration into discrete sub-goals (semantic rationales), and a deterministic verbalizer $\Phi$ converts each segment's actions into a textual 3D workspace movement and its 2D image-plane trace (kinematic rationales). A parallel dual-training design then sends one shared representation through two heads — a continuous action expert that regresses the next action chunk, and a spatial-language expert that predicts the current sub-goal and the verbalized movement of that same chunk — so the backbone internalizes the reasoning while the language head can be discarded at inference. With one retrieved demonstration supplied as a cached prefix, the paper reports first place on VLA-Arena (0.63 overall versus 0.44 for the strongest baseline), 98.8% average success on LIBERO, and 85.1% zero-shot success on LIBERO-Plus. Ablations support the mechanism: removing the demonstration drops LIBERO average success from 98.8% to 62.4%, and a wrong-task demonstration drops it further to 44.9%, indicating that the policy treats the demonstration as an active task specification rather than inert context.

Load-bearing premise

Everything rests on the offline vision-language model correctly reading each demonstration — segmenting it into the right sub-goals and describing the right movements — and the paper never measures how often that reading is right; the wrong-demonstration ablation (98.8% down to 44.9%) shows how directly performance depends on prompt quality.

Editorial extensions

If this is right

  • A fixed checkpoint can be adapted to an unseen task, scene, or object by retrieving one structured demonstration with no parameter updates; the largest LIBERO-Plus gains appear exactly where the paper says OOD failure usually strikes — camera viewpoint (+23.5 points), sensor noise (+19.7), and robot initial state (+14.7).
  • Demonstrations from other embodiments can steer the policy: under matched observations, switching the demonstration source between real-robot, human-hand, and XR-retargeted versions changes predicted actions by only $0.0014$–$0.0016\sigma$, which the paper interprets as source-invariant structured context.
  • The transferable information lives in the structured language, not the demonstration images: text-only demonstrations nearly match the full image+text input at evaluation (98.8/84.4 versus 98.8/85.1 on LIBERO average and LIBERO-Plus), while image-only demonstrations drop to 92.9/75.7.
  • The demonstration acts as an explicit task specification: supplying the correct demonstration yields 98.8% average LIBERO success, removing it drops to 62.4%, and supplying a wrong-task demonstration drops further to 44.9%, with Goal-suite performance collapsing to 0.0% under the wrong demo.
  • Structured context does not by itself solve long-horizon error accumulation: on the VLA-Arena Long Horizon suite every method, including StellaVLA, stays near zero at L1 and L2 because a fixed prefix cannot re-plan after execution drifts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the near-flat sub-goal granularity curve (98.1% at three keyframes versus 98.8% at ten) suggests the useful content is the plan skeleton, so the retrieved prefix could probably be compressed to a handful of plan-level text tokens, cutting per-episode memory and latency further.
  • Beyond the paper: retrieval is purely linguistic, and inside the closed instruction sets of the simulation benchmarks it reduces to exact task matching, so behavior under paraphrased or novel instructions is untested; that regime is the natural next stress test for the framework.
  • Beyond the paper: the cross-source consistency result covers single-step predictions under matched observations, and the paper itself cautions that small disagreements may accumulate in closed loop; pinning one demonstration source per full rollout would test whether the embodiment-agnostic claim survives the control loop.
  • Beyond the paper: the dual-training recipe is demonstrated only with an MLP regression action head, so the same language-supervision-forges-the-representation effect remains untested for flow-matching or tokenized action heads; if it transfers, the structured-demonstration idea would port to most VLA families.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes StellaVLA, an in-context imitation learning framework in which a raw demonstration is converted offline into a structured demonstration comprising high-level sub-goal descriptions and verbalized 2D/3D kinematic rationales. At test time, one retrieved structured demonstration is prepended to the policy input, with the prefix KV-cached for efficiency. During training, a parallel spatial-language expert supervises the same representation with the sub-task label and a deterministic verbalization of the action chunk; during inference this expert is removed and only the action expert is used. The authors report state-of-the-art results on LIBERO (98.8% average success), VLA-Arena (overall 0.63), LIBERO-Plus (85.1%), and a real-robot benchmark, and argue that in-context structured demonstrations shift imitation from surface-level action mimicry to reasoning-based adaptation.

Significance. If the central claims hold, the paper makes a useful contribution: it offers a practical way to inject structured, language-mediated rationales into VLA policies without retraining at deployment, preserves high-frequency control by avoiding autoregressive decoding, and shows encouraging robustness gains on several benchmarks. The paper is also commendable for including a matched control (StarVLA-OFT), detailed hyperparameters, latency measurements, an ablation of demonstration content, and an explicit limitations section. However, the evidence as presented does not yet establish the headline claims. The main benchmark comparisons do not control for the extra test-time demonstration provided to StellaVLA, the matched control changes two variables at once, and the VLM-generated semantic annotations that are load-bearing for the method are never validated. These are correctness risks rather than presentation issues, so the paper needs substantial revision before the claims can be accepted.

major comments (4)
  1. [Sec. 4.2.2, Table 2; Sec. 4.2.3, Table 3] The leaderboard comparisons are not matched on test-time demonstration availability. In Sec. 4.2.2, StellaVLA "receives one structured demonstration of the target task without any parameter update," while the baselines in Table 2 are reported as being from the VLA-Arena leaderboard without any indication that they receive a target-task demonstration. The same asymmetry appears in Sec. 4.2.3, where LIBERO-Plus baselines are zero-shot results from [10] but StellaVLA uses a retrieved same-task demonstration. Since the paper's central claim is that conditioning on one retrieved structured demonstration yields state-of-the-art results, the margin could reflect additional test-time information rather than the method's intrinsic generalization. Please either report leaderboard protocols that give all methods the same demonstration budget, or add a demonstration-free version of StellaVLA under identical conditions for each comparison.
  2. [Sec. 4.2, 'Protocol and Matched Control'; Sec. 4.3] The matched control StarVLA-OFT differs from StellaVLA in two variables at once: it receives neither the retrieved demonstration nor the spatial-language supervision. Consequently, the gains reported against StarVLA-OFT (e.g., LIBERO 96.6 vs. 98.8, LIBERO-Plus 75.0 vs. 85.1, and the real-robot OOD-L1 paired degradation) cannot be attributed to the structured demonstration alone or to the language loss alone. To isolate the contribution of the in-context demonstration, the paper needs a control that adds only the retrieved demonstration to StarVLA-OFT, or a StellaVLA variant trained without the spatial-language loss while still receiving the demonstration.
  3. [Sec. 3.1, Eq. (2), Sec. 4.4, Table 5] The correctness of the entire method rests on the offline VLM extraction of sub-goal segmentations and semantic rationale descriptions, but this extraction is never validated. Sec. 3.1 states that Qwen3-VL "deduces the expert's underlying thought process" from raw trajectories, yet there is no human agreement study, no quantitative measure of segmentation accuracy, and no failure analysis. The sensitivity demonstrated in Table 5 (a wrong demonstration drops average success from 98.8% to 44.9%) shows how directly the policy depends on the content of the retrieved structured prompt; if the extracted rationales are noisy or systematically wrong, the language supervision in Eq. (2) and the in-context prompt could both be misleading. Please provide an evaluation of annotation quality, e.g., agreement with human segmentations on a sample of trajectories, and an analysis of how annotation errors propagate to downstream success.
  4. [Table 5 and Appendix A] The no-demonstration ablation in Table 5 is not a clean control. According to Appendix A, context-demonstration dropout is 0.0 on LIBERO and LIBERO-Plus, meaning the model was never trained with the demonstration prefix removed. Removing the demonstration at evaluation therefore feeds the model an input distribution it never encountered during training, which explains the large drop from 98.8% to 62.4% without implying that the demonstration contributes 36 points of task specification. This interpretation is supported by the demonstration-free StarVLA-OFT, which reaches 96.6% on the same benchmark and suggests the marginal effect of the demonstration is much smaller. Please report a no-demonstration evaluation using a checkpoint trained with context-demonstration dropout, or explicitly reinterpret Table 5 as an out-of-distribution sensitivity test rather than a causal decomposition.
minor comments (5)
  1. [Sec. 4.3, Table 4] The cross-source disagreement values in Table 4 are extremely small, but the table does not report confidence intervals or significance tests. Given that the conclusion is "source consistency," please add error bars or a statistical test over the 1,780 and 1,719 frames.
  2. [Sec. 4.4, Table 7] The terms "text-only" and "image-only" demonstrations are not defined precisely enough. It should be stated explicitly that the current observation and instruction remain multimodal and that only the demonstration prefix is restricted, so that a reader can judge what information is actually ablated.
  3. [Appendix B] The real-robot evaluation uses only 10 rollouts per cell, and the authors themselves note that single cells carry roughly 10-15 points of noise. Please include confidence intervals in Figure 4 or aggregate results with a bootstrap interval, since several cross-method differences appear close to this noise level.
  4. [Sec. 4.2.1, Table 1] The claim that the gain on Goal (+3.4) and Long (+3.0) is "larger" than on Object (+0.4) is supported, but the table would benefit from confidence intervals or at least the number of rollouts per suite to allow the reader to judge whether the per-suite differences are meaningful.
  5. [Sec. 3.3, Table 8] The latency comparison between action-only (88 ms) and language decoding (3177 ms) is presented as a paired measurement, but it is not stated whether the language-decoding condition also uses the cached demonstration prefix. Please clarify the exact setup so the 36x slowdown can be interpreted correctly.

Circularity Check

1 steps flagged · score 1.0 of 10

No significant circularity: the auxiliary language target is a transparent verbalization of the action target, not a hidden input-output inversion.

  1. self definitional [Sec. 3.2, Eq. (2) and dual rationale objective (Eq. (3))]
    "ct = (st, Φ(At)), (2) where st is the subtask label for ot, At = (at,...,at+H−1) is the ground-truth action chunk, and Φ(At) verbalises its 3D and 2D movement. ... Since Φ is deterministic, At and Φ(At) describe the same motion in continuous and linguistic forms."

    The kinematic component of the spatial-language target is defined as a deterministic function Φ of the exact action chunk At used in the regression loss Lact. Consequently, the language loss on the movement description carries no information beyond the action regression target: it is a re-encoding of the same quantity rather than an independent rationale. This is a construction-level redundancy. It is not load-bearing for the central claims because the language expert operates in parallel and is removed at inference, and the semantic subtask label st comes from an external VLM, so the benchmark results do not reduce to this equation.

full rationale

Score 1 reflects one minor construction-level redundancy: the kinematic component of the spatial-language target in Eq. (2) is a deterministic verbalization of the same action chunk used in the action regression loss, so the language loss's movement part adds no external information. This is a self-definitional relationship, but it is not load-bearing: the language head is parallel and removed at inference, and the semantic subtask labels come from an external VLM (Qwen3-VL). The central claims rest on external benchmarks (LIBERO, VLA-Arena, LIBERO-Plus) and a matched control (StarVLA-OFT), and self-citations are used only for motivation or implementation details. The unmatched test-time demonstration availability is an experimental-design concern, not a circular derivation.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on four unvalidated premises: the offline VLM produces correct sub-goal annotations; a correct target-task demonstration is retrievable at test time; the discretized verbalization of 3D motion preserves the control-relevant information; and the auxiliary language loss improves the shared representation. None is machine-checked; the paper provides ablations for the third and fourth but no validation for the first and no no-demo control on LIBERO-Plus for the second.

free parameters (5)
  • Lambda (language loss weight) = 0.3
    Chosen as best in-distribution setting on LIBERO (Table 6). LIBERO-Plus robustness is highest at lambda=0 (86.9) and lowest at 1.0 (81.9), so the final value is a trade-off, not a unique optimum.
  • Context-demonstration dropout = 0.0 (LIBERO), 0.5 (VLA-Arena/real)
    Set according to demonstration availability at deployment; no sensitivity analysis is reported.
  • 2D gripper-path dropout = 0.5
    Regularizer to keep 3D steering usable when the pixel trace is missing; fixed without ablation.
  • Action chunk length H = 8 (sim), 16 (real)
    Design choice for control frequency; no ablation reported.
  • Number of subgoal keyframes = 10 (sim), 8 (real)
    Table 9 shows 3 keyframes already reach 98.1 average vs 98.8 with 10, so the exact count is not critical.
assumptions (4)
  • domain assumption The offline Qwen3-VL produces correct sub-goal segmentations and semantic rationale descriptions of expert demonstrations.
    Invoked in Sec 3.1 ('a powerful off-the-shelf VLM... to explicitly decompose the continuous trajectory'), with no quantitative validation or human agreement study.
  • domain assumption A relevant demonstration is retrievable at test time; in closed-set simulation this reduces to exact task match.
    Appendix A states the instruction set is closed; Table 5 shows removing the demonstration drops LIBERO average from 98.8 to 62.4, so the claim depends on this availability.
  • domain assumption The deterministic verbalizer Phi preserves control-relevant information after discretizing 3D motion into text.
    Phi is defined in Sec 3.1; Sec D acknowledges 'coarse kinematic discretization' may limit fine-grained precision.
  • ad hoc to paper Language supervision on the deterministic transform Phi(A_t) improves the shared representation for action prediction.
    Proposed in Sec 3.2; the lambda sweep (Table 6) shows the benefit is non-monotonic and the mechanism is not directly measured.

how reviews work

0 comments
Cite this review

Pith. "Pith review of StellaVLA: In-Context Structured Demonstration for Generalizable Vision-Language-Action Models." pith.science (2026). https://pith.science/paper/NF2C2HTA

@misc{pith2026260811671,
  author       = {Pith},
  title        = {Pith review of: StellaVLA: In-Context Structured Demonstration for Generalizable Vision-Language-Action Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NF2C2HTA}},
  note         = {Machine review of arXiv:2608.11671}
}
abstract

Vision-Language-Action (VLA) models can follow instructions and manipulate objects, but their performance often collapses out of distribution (OOD), when the scene, viewpoint, or object differs from training. Adapting to each new situation typically requires collecting more data and fine-tuning. We present StellaVLA, a framework that instead adapts at test time by conditioning on a single retrieved demonstration. The key idea is to move beyond imitating what an expert did and instead convey why: an automated offline pipeline converts each raw trajectory into a structured demonstration, e.g., a task plan, sub-goal descriptions, and verbalized 3D motion, at zero human-annotation cost. Provided as in-context guidance, this structured demonstration lets the policy reason about the task rather than mimic a pixel trajectory, which also makes it transferable across embodiments (real-robot, human-hand, or XR demonstrations). A parallel dual-training design internalizes this reasoning during training through a joint action-and-language objective, while inference uses the action expert alone, preserving real-time, high-frequency control with no added latency. On the VLA-Arena leaderboard(Aug 1, 2026), StellaVLA ranks first with an overall score of 0.63, versus 0.44 and 0.22 for the strong prior models ($\pi_{0.5}$ and LingBot-VLA), and it further leads on LIBERO with 98.8% average success rate and LIBERO-Plus with 85.1% success rate. Our real-robot benchmark demonstrates that StellaVLA can use both human/robot demos and human-to-robot (XR) demos as in-context structured demonstration to help VLA model adapt to OOD tasks.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 19 canonical work pages

  1. [10]

    LIBERO-Plus: In-depth Robustness Analysis of Vision-Language- Action Models.arXiv preprint arXiv:2510.13626, 2025

    Senyu Fei, Siyin Wang, Junhao Shi, Zihao Dai, Jikun Cai, Pengfang Qian, Li Ji, Xinzhe He, Shiduo Zhang, Zhaoye Fei, Jinlan Fu, Jingjing Gong, and Xipeng Qiu. LIBERO-Plus: In-depth Robustness Analysis of Vision-Language- Action Models.arXiv preprint arXiv:2510.13626, 2025

    arXiv

  2. [1]

    Qwen3-VL Technical Report.arXiv preprint arXiv:2511.21631, 2025

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, et al. Qwen3-VL Technical Report.arXiv preprint arXiv:2511.21631, 2025

    arXiv 2025

  3. [2]

    Rt-h: Action hierarchies using language

    Suneel Belkhale, Tianli Ding, Ted Xiao, Pierre Sermanet, Quon Vuong, Jonathan Tompson, Yevgen Chebotar, De- bidatta Dwibedi, and Dorsa Sadigh. Rt-h: Action hierarchies using language. Inhttps://arxiv.org/abs/2403.01823, 2024

    arXiv 2024

  4. [3]

    Motus: A Unified Latent Action World Model.arXiv preprint arXiv:2512.13030, 2025

    Hongzhe Bi, Hengkai Tan, Shenghao Xie, Zeyuan Wang, Shuhe Huang, Haitian Liu, Ruowen Zhao, Yao Feng, Chendong Xiang, Yinze Rong, Hongyan Zhao, Hanyu Liu, Zhizhong Su, Lei Ma, Hang Su, and Jun Zhu. Motus: A Unified Latent Action World Model.arXiv preprint arXiv:2512.13030, 2025

    arXiv 2025

  5. [4]

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, James Tanner, Quan Vuong, Anna Walling, Haohuan Wang, and Ury Zhilinsky.π0: A Visio...

    arXiv

  6. [6]

    UniVLA: Learning to Act Anywhere with Task-Centric Latent Actions.arXiv preprint arXiv:2505.06111, 2025

    Qingwen Bu, Yanting Yang, Jisong Cai, Shenyuan Gao, Guanghui Ren, Maoqing Yao, Ping Luo, and Hongyang Li. UniVLA: Learning to Act Anywhere with Task-Centric Latent Actions.arXiv preprint arXiv:2505.06111, 2025

    arXiv 2025

  7. [7]

    See Once, Then Act: Vision-Language-Action Model with Task Learning from One-Shot Video Demonstrations.arXiv preprint arXiv:2512.07582, 2025

    Guangyan Chen, Meiling Wang, Qi Shao, Zichen Zhou, Weixin Mao, Te Cui, Minzhao Zhu, Yinan Deng, Luojie Yang, Zhanqi Zhang, Yi Yang, Hua Chen, and Yufeng Yue. See Once, Then Act: Vision-Language-Action Model with Task Learning from One-Shot Video Demonstrations.arXiv preprint arXiv:2512.07582, 2025

    arXiv 2025

  8. [8]

    Steerable Vision-Language-Action Policies for Embodied Reasoning and Hierarchical Control.arXiv preprint arXiv:2602.13193, 2026

    William Chen, Jagdeep Singh Bhatia, Catherine Glossop, Nikhil Mathihalli, Ria Doshi, Andy Tang, Danny Driess, Karl Pertsch, and Sergey Levine. Steerable Vision-Language-Action Policies for Embodied Reasoning and Hierarchical Control.arXiv preprint arXiv:2602.13193, 2026

    arXiv 2026

Show all 59 references
  1. [9]

    From intention to execution: Probing the generalization boundaries of vision-language-action models.arXiv preprint arXiv:2506.09930, 2025

    Irving Fang, Juexiao Zhang, Shengbang Tong, and Chen Feng. From intention to execution: Probing the generalization boundaries of vision-language-action models.arXiv preprint arXiv:2506.09930, 2025

    2025 arXiv

  2. [11]

    See what matters: Differentiable grid sample pruning for generalizable vision-language-action model

    Yixu Feng, Zinan Zhao, Yanxiang Ma, Chenghao Xia, Chengbin Du, Yunke Wang, and Chang Xu. See what matters: Differentiable grid sample pruning for generalizable vision-language-action model. InForty-third International Conference on Machine Learning, 2026

    2026

  3. [12]

    W AM-TTT: Steering World-Action Models by Watching Human Play at Test Time.arXiv preprint arXiv:2607.06988, 2026

    Yusen Feng, Bingchen Han, Jiangran Lyu, Kai Liu, Yixin Zheng, Yuxuan Wan, Weiheng Liu, Sun Han, Ruiqin Li, Yulong Zhang, Fangfu Liu, Xuesong Shi, Libin Liu, Yizhou Wang, Zhizheng Zhang, and He Wang. W AM-TTT: Steering World-Action Models by Watching Human Play at Test Time.arX...

    2026 arXiv

  4. [13]

    Icrt: In-context imitation learning via next-token prediction

    Letian Fu, Huang Huang, Gaurav Datta, Lawrence Yunliang Chen, William Chung-Ho Panitch, Fangchen Liu, Hui Li, and Ken Goldberg. Icrt: In-context imitation learning via next-token prediction. InIEEE International Conference on Robotics and Automation (ICRA), 2025

    2025

  5. [14]

    DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos

    Shenyuan Gao, William Liang, Kaiyuan Zheng, Seonghyeon Ye, Qianli Ma, Ruijie Zheng, Pieter Abbeel, Yuke Zhu, Joel Jang, Linxi Fan, et al. DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos. arXiv preprint arXiv:2602.06949, 2026

    2026 arXiv

  6. [15]

    Hancock, Xindi Wu, Lihan Zha, Olga Russakovsky, and Anirudha Majumdar

    Asher J. Hancock, Xindi Wu, Lihan Zha, Olga Russakovsky, and Anirudha Majumdar. Actions as Language: Fine-Tuning VLMs into VLAs Without Catastrophic Forgetting.arXiv preprint arXiv:2509.22195, 2025. 15

    2025

  7. [16]

    Motion dynamics learning for few-shot embodied adaptation

    Sibo He, Weiying Xie, Daixun Li, Junhao Zhong, Jiayun Tian, Yunke Wang, Leyuan Fang, Gang He, and Yunsong Li. Motion dynamics learning for few-shot embodied adaptation. InForty-third International Conference on Machine Learning

  8. [17]

    Thinkact: Vision-language- action reasoning via reinforced visual latent planning.Advances in Neural Information Processing Systems, 38: 82782–82802, 2026

    Chi-Pin Huang, Yueh-Hua Wu, Min-Hung Chen, Frank Wang, and Fu-En Yang. Thinkact: Vision-language- action reasoning via reinforced visual latent planning.Advances in Neural Information Processing Systems, 38: 82782–82802, 2026

    2026

  9. [18]

    Seeing realism from simulation: Efficient video transfer for vision-language-action data augmentation.arXiv preprint arXiv:2605.02757, 2026

    Chenyu Hui, Xiaodi Huang, Siyu Xu, Yunke Wang, Shan You, Fei Wang, Tao Huang, and Chang Xu. Seeing realism from simulation: Efficient video transfer for vision-language-action data augmentation.arXiv preprint arXiv:2605.02757, 2026

    2026 arXiv

  10. [19]

    NORA: A Small Open-Sourced Generalist Vision Language Action Model for Embodied Tasks.arXiv preprint arXiv:2504.19854, 2025

    Chia-Yu Hung, Qi Sun, Pengfei Hong, Amir Zadeh, Chuan Li, U.-Xuan Tan, Navonil Majumder, and Soujanya Poria. NORA: A Small Open-Sourced Generalist Vision Language Action Model for Embodied Tasks.arXiv preprint arXiv:2504.19854, 2025

    2025 arXiv

  11. [20]

    Ra-vla: Retrieval- augmented vla for test-time adaptation

    Sanghwan Jang, Minjin Jeon, Minsoo Kim, Seong Jin Choi, Dongha Kim, and Hwanjo Yu. Ra-vla: Retrieval- augmented vla for test-time adaptation. InProceedings of the 43rd International Conference on Machine Learning. PMLR, 2026. URLhttps://openreview.net/forum?id=ut6HebnnQe

    2026

  12. [21]

    Ra-vla: Retrieval- augmented vla for test-time adaptation

    Sanghwan Jang, Minjin Jeon, Minsoo Kim, Seong Jin Choi, Dongha Kim, and Hwanjo Yu. Ra-vla: Retrieval- augmented vla for test-time adaptation. InForty-third International Conference on Machine Learning, 2026

    2026

  13. [22]

    RoboTTT: Context Scaling for Robot Policies.arXiv preprint, 2026

    Yunfan Jiang, Yevgen Chebotar, Ruijie Zheng, Fengyuan Hu, Yunhao Ge, Jimmy Wu, Tianyuan Dai, Scott Reed, Li Fei-Fei, and Yuke Zhu. RoboTTT: Context Scaling for Robot Policies.arXiv preprint, 2026

    2026

  14. [23]

    OpenVLA: An Open-Source Vision-Language-Action Model.arXiv preprint arXiv:2406.09246, 2024

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. OpenVLA...

    2024 arXiv

  15. [24]

    Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success.arXiv preprint arXiv:2502.19645, 2025

    Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success.arXiv preprint arXiv:2502.19645, 2025

    2025 arXiv

  16. [25]

    Molmoact: Action reasoning models that can reason in space.arXiv preprint arXiv:2508.07917, 2025

    Jason Lee, Jiafei Duan, Haoquan Fang, Yuquan Deng, Shuo Liu, Boyang Li, Bohan Fang, Jieyu Zhang, Yi Ru Wang, Sangho Lee, et al. Molmoact: Action reasoning models that can reason in space.arXiv preprint arXiv:2508.07917, 2025

    2025 arXiv

  17. [26]

    CogVLA: Cognition-Aligned Vision-Language-Action Model via Instruction-Driven Routing & Sparsification.arXiv preprint arXiv:2508.21046, 2025

    Wei Li, Renshan Zhang, Rui Shao, Jie He, and Liqiang Nie. CogVLA: Cognition-Aligned Vision-Language-Action Model via Instruction-Driven Routing & Sparsification.arXiv preprint arXiv:2508.21046, 2025

    2025 arXiv

  18. [27]

    Evo-Depth: A Lightweight Depth-Enhanced Vision-Language-Action Model.arXiv preprint arXiv:2605.14950, 2026

    Tao Lin, Yuxin Du, Jiting Liu, Nuobei Zhu, Yunhe Li, Yuqian Fu, Yinxinyu Chen, Hongyi Cai, Zewei Ye, Bing Cheng, Kai Ye, Yiran Mao, Yilei Zhong, MingKang Dong, Junchi Yan, Gen Li, and Bo Zhao. Evo-Depth: A Lightweight Depth-Enhanced Vision-Language-Action Model.arXiv preprint ...

    2026 arXiv

  19. [29]

    La4vla: Learning to act without seeing via language-action pretraining.arXiv preprint arXiv:2606.27295, 2026

    Tao Lin, Yuxin Du, Yiran Mao, Zewei Ye, Yilei Zhong, Bing Cheng, Yiming Wang, Jiting Liu, Yang Tian, Junchi Yan, et al. La4vla: Learning to act without seeing via language-action pretraining.arXiv preprint arXiv:2606.27295, 2026

    2026 arXiv

  20. [30]

    LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning.arXiv preprint arXiv:2306.03310, 2023

    Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning.arXiv preprint arXiv:2306.03310, 2023

    2023 arXiv

  21. [31]

    LocoFormer: Generalist Locomotion via Long-Context Adaptation

    LocoFormer. LocoFormer: Generalist Locomotion via Long-Context Adaptation. OpenReview, 2025

    2025

  22. [32]

    GR00T N1: An Open Foundation Model for Generalist Humanoid Robots.arXiv preprint arXiv:2503.14734, 2025

    NVIDIA, Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Jim Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, Joel Jang, Zhenyu Jiang, Jan Kautz, Kaushil Kundalia, Lawrence Lao, Zhiqi Li, Zongyu Lin, Kevin Lin, Guilin Liu, Edith Llontop, L...

    2025 arXiv

  23. [33]

    Behavior Prompting Policy: Demonstrations as Prompts for Manipulation.arXiv preprint arXiv:2606.30457, 2026

    Austin Patel, Ben Pekarek, Joel Enrique Castro Hernandez, and Shuran Song. Behavior Prompting Policy: Demonstrations as Prompts for Manipulation.arXiv preprint arXiv:2606.30457, 2026

    2026 arXiv

  24. [34]

    Action-aware dynamic pruning for efficient vision-language-action manipulation

    Xiaohuan Pei, Yuxing Chen, Siyu Xu, Yunke Wang, Yuheng Shi, and Chang Xu. Action-aware dynamic pruning for efficient vision-language-action manipulation. InInternational Conference on Learning Representations, volume 2026, pages 10832–10851, 2026

    2026

  25. [35]

    FAST: Efficient Action Tokenization for Vision-Language-Action Models.arXiv preprint arXiv:2501.09747, 2025

    Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. FAST: Efficient Action Tokenization for Vision-Language-Action Models.arXiv preprint arXiv:2501.09747, 2025

    2025 arXiv

  26. [36]

    π0.7: A Steerable Generalist Robotic Foundation Model with Emergent Capabilities.arXiv preprint arXiv:2604.15483, 2026

    Physical Intelligence. π0.7: A Steerable Generalist Robotic Foundation Model with Emergent Capabilities.arXiv preprint arXiv:2604.15483, 2026

    2026 arXiv

  27. [37]

    Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Manuel Y . Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Devin L...

    2025 arXiv

  28. [38]

    Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Models.arXiv preprint arXiv:2606.17846, 2026

    Qwen Team. Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Models.arXiv preprint arXiv:2606.17846, 2026

    2026 arXiv

  29. [39]

    MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic Manipulation.arXiv preprint arXiv:2508.19236, 2026

    Hao Shi, Bin Xie, Yingfei Liu, Lin Sun, Fengrong Liu, Tiancai Wang, Erjin Zhou, Haoqiang Fan, Xiangyu Zhang, and Gao Huang. MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic Manipulation.arXiv preprint arXiv:2508.19236, 2026

    2026 arXiv

  30. [40]

    SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics.arXiv preprint arXiv:2506.01844, 2025

    Mustafa Shukor, Dana Aubakirova, Francesco Capuano, Pepijn Kooijmans, Steven Palma, Adil Zouitine, Michel Aractingi, Caroline Pascal, Martino Russi, Andres Marafioti, Simon Alibert, Matthieu Cord, Thomas Wolf, and Remi Cadene. SmolVLA: A Vision-Language-Action Model for Afford...

    2025 arXiv

  31. [41]

    Ricl: Adding in-context adaptability to pre-trained vision-language-action models.arXiv preprint arXiv:2508.02062, 2025

    Kaustubh Sridhar, Souradeep Dutta, Dinesh Jayaraman, and Insup Lee. Ricl: Adding in-context adaptability to pre-trained vision-language-action models.arXiv preprint arXiv:2508.02062, 2025

    2025 arXiv

  32. [42]

    StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing.arXiv preprint arXiv:2604.05014, 2026

    StarVLA Community. StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing.arXiv preprint arXiv:2604.05014, 2026

    2026 arXiv

  33. [43]

    Efros, and Moritz Hardt

    Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei A. Efros, and Moritz Hardt. Test-Time Training with Self-Supervision for Generalization under Distribution Shifts.arXiv preprint arXiv:1909.13231, 2020

    1909 arXiv

  34. [44]

    X-op: Cross-morphology whole-body teleoperation via mpc retargeting.arXiv preprint arXiv:2606.07934, 2026

    Jen-Wei Wang, Sarthak Kaingade, Andrea Tagliabue, and Nicholas Morozovsky. X-op: Cross-morphology whole-body teleoperation via mpc retargeting.arXiv preprint arXiv:2606.07934, 2026

    2026 arXiv

  35. [45]

    From Foundation to Application: Improving VLA Models in Practice.arXiv preprint arXiv:2607.06403, 2026

    Wei Wu, Fangjing Wang, Fan Lu, He Sun, Shi Liu, Yunnan Wang, Yibin Yan, Yong Wang, Shuailei Ma, Xinyang Wang, Yibin Liu, Shuai Yang, Tianxiang Zhou, Kejia Zhang, Lei Zhou, Cheng Su, Nan Xue, Bin Tan, Han Zhang, Youchao Zhang, Fei Liao, Xing Zhu, Yujun Shen, and Kecheng Zheng. ...

    2026 arXiv

  36. [46]

    A V A-VLA: Improving Vision-Language-Action Models with Active Visual Attention

    Lei Xiao, Jifeng Li, Juntao Gao, Feiyang Ye, Yan Jin, Jingjing Qian, Jing Zhang, Yong Wu, and Xiaoyuan Yu. A V A-VLA: Improving Vision-Language-Action Models with Active Visual Attention. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR...

    2026

  37. [47]

    Towards efficient embodied reasoning: Mixture-of-depth compute allocation for vision- 17 language-action model

    Weiying Xie, Qingchen Zeng, Zihan Meng, Jiayun Tian, Sibo He, Danian Yang, Jie Du, Yunke Wang, Daixun Li, Hengyi Wang, et al. Towards efficient embodied reasoning: Mixture-of-depth compute allocation for vision- 17 language-action model. InProceedings of the 32nd ACM SIGKDD Co...

    2026

  38. [48]

    Vla-cache: Efficient vision- language-action manipulation via adaptive token caching.Advances in Neural Information Processing Systems, 38:164448–164473, 2026

    Siyu Xu, Yunke Wang, Chenghao Xia, Dihao Zhu, Tao Huang, and Chang Xu. Vla-cache: Efficient vision- language-action manipulation via adaptive token caching.Advances in Neural Information Processing Systems, 38:164448–164473, 2026

    2026

  39. [49]

    Affordance field intervention: Enabling vlas to escape memory traps in robotic manipulation

    Siyu Xu, Zijian Wang, Yunke Wang, Chenghao Xia, Tao Huang, and Chang Xu. Affordance field intervention: Enabling vlas to escape memory traps in robotic manipulation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 37206–37215, 2026

    2026

  40. [50]

    Latent Action Pretraining from Videos.arXiv preprint arXiv:2410.11758, 2024

    Seonghyeon Ye, Joel Jang, Byeongguk Jeon, Sejune Joo, Jianwei Yang, Baolin Peng, Ajay Mandlekar, Reuben Tan, Yu-Wei Chao, Bill Yuchen Lin, Lars Liden, Kimin Lee, Jianfeng Gao, Luke Zettlemoyer, Dieter Fox, and Minjoon Seo. Latent Action Pretraining from Videos.arXiv preprint a...

    arXiv

  41. [51]

    Robotic control via embodied chain-of-thought reasoning.arXiv preprint arXiv:2407.08693, 2024

    Michał Zawalski, William Chen, Karl Pertsch, Oier Mees, Chelsea Finn, and Sergey Levine. Robotic control via embodied chain-of-thought reasoning.arXiv preprint arXiv:2407.08693, 2024

    2024 arXiv

  42. [52]

    Hancock, Mingtong Zhang, Tenny Yin, Yixuan Huang, Dhruv Shah, Allen Z

    Lihan Zha, Asher J. Hancock, Mingtong Zhang, Tenny Yin, Yixuan Huang, Dhruv Shah, Allen Z. Ren, and Anirudha Majumdar. LAP: Language-Action Pre-Training Enables Zero-Shot Cross-Embodiment Transfer.arXiv preprint arXiv:2602.10556, 2026

    2026

  43. [53]

    VLA-Arena: An Open-Source Framework for Benchmarking Vision-Language-Action Models

    Borong Zhang, Jiahao Li, Jiachen Shen, Yishuai Cai, Yuhao Zhang, Yuanpei Chen, Juntao Dai, Jiaming Ji, and Yaodong Yang. VLA-Arena: An Open-Source Framework for Benchmarking Vision-Language-Action Models. arXiv preprint arXiv:2512.22539, 2025

    2025 arXiv

  44. [54]

    Revisiting parameter redundancy in vision- language-action models: Insights from vlm-to-vla adaptation.arXiv preprint arXiv:2606.31382, 2026

    Fengnian Zhang, Tao Huang, Siyu Xu, Zhong Jin, and Chang Xu. Revisiting parameter redundancy in vision- language-action models: Insights from vlm-to-vla adaptation.arXiv preprint arXiv:2606.31382, 2026

    2026 arXiv

  45. [55]

    TTT-VLA: Test-Time Latent Prompt Optimization for Vision-Language-Action Models.arXiv preprint arXiv:2606.03127, 2026

    Wenbo Zhang, Jianxiong Li, Shuai Yang, Sijin Chen, Jiajun Liu, Lingqiao Liu, and Xiao Ma. TTT-VLA: Test-Time Latent Prompt Optimization for Vision-Language-Action Models.arXiv preprint arXiv:2606.03127, 2026

    2026 arXiv

  46. [56]

    DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledge.arXiv preprint arXiv:2507.04447, 2025

    Wenyao Zhang, Hongsi Liu, Zekun Qi, Yunnan Wang, Xinqiang Yu, Jiazhao Zhang, Runpei Dong, Jiawei He, Fan Lu, He Wang, Zhizheng Zhang, Li Yi, Wenjun Zeng, and Xin Jin. DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledge.arXiv preprint arXiv:2507....

    2025 arXiv

  47. [57]

    Retrieval-VLA: Training-Free In-Context Adaptation for Vision-Language-Action Models

    Yue Zhang, Rui Wang, Jiehong Lin, Zhongrui Wang, and Xiaojuan Qi. Retrieval-VLA: Training-Free In-Context Adaptation for Vision-Language-Action Models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1358–1367, 2026

    2026

  48. [58]

    Retrieval-vla: Training-free in-context adaptation for vision-language-action models

    Yue Zhang, Rui Wang, Jiehong Lin, Zhongrui Wang, and Xiaojuan Qi. Retrieval-vla: Training-free in-context adaptation for vision-language-action models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1358–1367, 2026

    2026

  49. [59]

    CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models.arXiv preprint arXiv:2503.22020, 2025

    Qingqing Zhao, Yao Lu, Moo Jin Kim, Zipeng Fu, Zhuoyang Zhang, Yecheng Wu, Zhaoshuo Li, Qianli Ma, Song Han, Chelsea Finn, Ankur Handa, Ming-Yu Liu, Donglai Xiang, Gordon Wetzstein, and Tsung-Yi Lin. CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models....

    2025 arXiv

  50. [60]

    ACoT-VLA: Action Chain-of-Thought for Vision-Language-Action Models

    Linqing Zhong, Yi Liu, Yifei Wei, Ziyu Xiong, Si Liu, and Guanghui Ren. ACoT-VLA: Action Chain-of-Thought for Vision-Language-Action Models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8152–8162, 2026

    2026

  51. [61]

    subtask + 3D/2D steering command

    Xueyang Zhou, Yangming Xu, Guiyao Tie, Yongchao Chen, Guowen Zhang, Duanfeng Chu, Pan Zhou, and Lichao Sun. Libero-pro: Towards robust and fair evaluation of vision-language-action models beyond memorization. arXiv preprint arXiv:2510.03827, 2025. 18 A Implementation Details O...

    arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.