Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

ThinkProprio matches full-token VLA performance using 15% of visual tokens, at 58% lower latency.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-03 03:50 UTC pith:DOABYQXN

load-bearing objection Useful and honest efficiency study; the latency win is real, the performance win is not yet statistically supported. the 3 major comments →

arxiv 2602.06575 v2 pith:DOABYQXN submitted 2026-02-06 cs.RO cs.CV

Think Proprioceptively: State-Grounded Visual Token Selection for VLA Policies

Fangyuan Wang , Peng Zhou , Jiaming Qi , Shipeng Lyu , Chengyang He , David Navarro-Alarcon , Guodong Guo This is my paper
classification cs.RO cs.CV
keywords vision-language-action modelsproprioceptionvisual token selectiontoken reductionlong-horizon manipulationCALVINLIBEROinference latency
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that a robot's proprioceptive state—joint angles, gripper width, end-effector pose—should be treated as an active participant in visual reasoning, not a late conditioning signal. The proposed method, ThinkProprio, discretizes state values into text tokens from the vision-language model's own vocabulary and uses them, together with the language instruction, to score and select which visual patches enter the model. This cuts the visual token count to about 15% of the original, reduces end-to-end inference latency from 52 ms to 22 ms on CALVIN ABC→D, and slightly improves task success (average chain length 4.44 to 4.55; LIBERO average 96.9% to 97.3%). A careful reader would care because it identifies where embodied state creates value—in shaping attention before compute—rather than merely adding information at the action head, and because the efficiency gains are large enough to matter for real-time control.

Core claim

ThinkProprio's core discovery is that proprioception helps a vision-language-action policy most when it is turned into VLM-vocabulary text tokens and used as part of the query that selects which visual patches the VLM processes. Joint angles and gripper state are discretized into 256 uniform bins mapped to the last token IDs of the VLM's embedding table; these proprio tokens, concatenated with the instruction, generate per-patch attention scores, and a vote-based selection with annealed Gumbel noise and a straight-through estimator keeps patches receiving at least one vote. With about 15% of visual tokens plus one global context token, the policy reaches average completed chain length 4.55 o

What carries the argument

The central mechanism is the instruction-proprio guided visual token selector. Proprioception is encoded by uniform binning (256 bins, range [-3,3]) mapped to the last vocabulary token IDs of the pretrained VLM, yielding embeddings identical in kind to language tokens. The selector concatenates these proprio tokens with instruction tokens to form query tokens; each visual patch embedding attends to the queries via RMSNorm and scaled dot product, producing a per-query score matrix; after adding annealed Gumbel noise, each query votes for one patch, and all patches receiving at least one vote are retained. A straight-through estimator lets gradients flow through the discrete mask, and a global

Load-bearing premise

The central claim depends on the reported performance gains (0.11 in average chain length, 0.4 percentage points in average success) being real rather than run-to-run noise, since the paper gives no variance or seeds; if they evaporate under repeated runs, only the latency reduction survives.

What would settle it

Run at least 10 independent training runs of ThinkProprio and the FLOWER baseline on CALVIN ABC→D (and LIBERO-Long) with identical seeds and report the full distribution of Avg. Len. and success; if the claimed 0.11 and 0.4-point deltas fall within one combined standard error, the performance-improvement claim is not established, leaving only the latency reduction.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • VLA policies can run on roughly 15% of visual tokens with end-to-end latency cut by more than half (52 ms to 22 ms on CALVIN ABC→D), enabling faster control loops on commodity GPUs.
  • Text-tokenized proprioception in the VLM's native embedding space is a more effective interface than learned MLP projections: it improves CALVIN Avg. Len. from 4.44 to 4.48 versus a drop to 4.15 for the MLP-to-VLM entry.
  • Joint instruction-and-proprioception querying outperforms either signal alone for token retention (4.55 vs 3.40 for instruction-only and 3.12 for proprio-only), so state and task context are complementary filters.
  • The largest relative gains appear at long horizons: on CALVIN LH-5 ThinkProprio reaches 82.1% vs 77.8% for FLOWER†, and on LIBERO-Long it reaches 95.2% vs 94.9%, suggesting state-grounded selection reduces compounding errors.
  • Real-world pick-place and push trials (80 per category) show ThinkProprio at 91.3% overall vs 87.5% for FLOWER, providing a preliminary sanity check that the mechanism transfers to physical hardware.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper reports no standard deviations or seeds for its main results, so the small performance gains (0.11 Avg. Len.; 0.4 points success) could lie within run-to-run noise; our inference is that the secure contribution is the large and consistent latency reduction, while the performance improvement needs multi-seed confirmation.
  • The tokenization recipe generalizes: any low-dimensional embodied signal (e.g., force-torque, tactile, IMU) can be discretized into the VLM vocabulary and folded into the same query-selection mechanism, offering a unified way to ground perception in embodiment.
  • The selector's scoring cost grows quadratically in the number of patches (O(Nv²D)); at higher image resolutions this will dominate, so scaling the idea likely requires a cheaper score function (e.g., a learned score head or low-rank attention) rather than the current pairwise dot-product votes.
  • The observed phase-dependent attention shifts (object-focused during approach, gripper-focused during contact) suggest a testable extension: a policy that explicitly modulates the selection query by contact phase could push long-horizon robustness further.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes ThinkProprio, a vision-language-action (VLA) policy that discretizes proprioceptive state into VLM-vocabulary text tokens and uses these tokens jointly with the instruction to select a small subset of visual patches before VLM computation. The authors claim that this state-grounded token selection improves long-horizon task performance (CALVIN ABC→D Avg. Len. 4.44→4.55; LIBERO average 96.9%→97.3%) while reducing end-to-end inference latency by 58% (52 ms→22 ms). The paper includes controlled ablations on a fixed FLOWER backbone varying proprioception encoding, entry point, and token-selection query, as well as a small real-world evaluation.

Significance. If the reported results are statistically robust, the paper makes a useful contribution by systematically isolating the role of proprioception in VLA design and by demonstrating that tokenizing state as an active query for visual selection can enable aggressive token reduction with no loss of task performance. The latency and VRAM gains are large and credible. The controlled ablation study on a single backbone is a strength, as is the authors' transparency about the limited scope of the real-world experiments. However, the central 'improves performance' claim rests on small margins that are not accompanied by any measure of variance, so the significance of the empirical contribution is currently uncertain.

major comments (3)
  1. [§4.1, Tables 2, 3, 6, 7, 8, 15] The paper states results are 'averaged over 5 independent evaluation runs' but reports no standard deviations, confidence intervals, or per-run values. The headline improvements are small: CALVIN ABC→D Avg. Len. 4.55 vs. 4.44 (Δ=0.11), LIBERO average 97.3% vs. 96.9% (Δ=0.4 points), LIBERO-Long 95.2% vs. 94.9% (Δ=0.3). Without uncertainty estimates, these deltas cannot be distinguished from run-to-run noise. This is load-bearing because the abstract and §4.3 claim 'improves' and 'increases.' Please report mean±std across the 5 runs (and the real-world Table 15 stats), state the number of evaluation episodes per run, and, if the differences are not significant, rephrase the claim as 'no degradation with large latency reduction' rather than 'improves.'
  2. [Appendix Table 12, Abstract] The abstract and §4.3 claim ThinkProprio 'matches or improves' on CALVIN, but the D→D split in Table 12 shows ThinkProprio (Avg. Len. 4.23) below FLOWER† (4.35). The authors attribute this to FLOWER†'s additional pretraining, yet no training-matched FLOWER baseline is provided for D→D. Since D→D is a CALVIN split, the 'matches or improves' claim is not empirically supported for this split. Please either add a training-matched comparison or explicitly limit the performance claim to ABC→D/ABCD→D and to LIBERO.
  3. [§5.1, Table 6] The component ablation shows that adding physically grounded selection without the global context token (H_ctx) degrades Avg. Len. from 4.44 to 4.35; only with H_ctx appended does performance rise to 4.55. Thus the reported improvement is not due to token selection alone but to the combination of selection and a global summary token. The paper's narrative emphasizes proprioception-guided selection as the key mechanism. Please clarify in the text how much of the gain is attributable to the selection mechanism versus the global summary, and consider an ablation that adds H_ctx to the no-selection baseline to isolate the selection effect.
minor comments (4)
  1. [Abstract vs. §3.3 and Figure 1] The abstract says 'retaining only about 12%' of visual tokens, while §3.3, Figure 1, and Table 4 state 15%. Please make these numbers consistent, or explain the discrepancy (e.g., different benchmark averages).
  2. [§3.3, Eq. (4)-(7)] Please specify the Gumbel temperature at inference. The text says the temperature is cosine-annealed from α_start to α_end; it should state whether the selector is run with α=0 (deterministic argmax) during evaluation and how this affects the reported token counts.
  3. [§4.2, Table 3] In §4.2 you mark π0 and π0.5 results on CALVIN as 'finetuned by us,' but Table 3 lists π0 and π0.5 with no star. Please add stars or clarify whether the LIBERO numbers are from your own runs or from the original papers.
  4. [Appendix D, Table 15] The real-world evaluation reports aggregate success rates over 20 trials per task, but no per-task breakdown or error bars. Given the small sample (160 trials total), a per-task table (or at least per-category standard errors) would help assess the claimed 91.3% vs. 87.5% margin.

Circularity Check

0 steps flagged

No significant circularity: the central claims are evaluated against external benchmarks and do not reduce to fitted inputs or self-citation.

full rationale

The paper's derivation chain is self-contained and empirically grounded rather than circular. The proprioceptive encoding (Eq. 1), query-based token selection (Eqs. 2-8), and flow-matching objective (Eq. 10) do not embed the reported Avg. Len. or success-rate targets; those numbers are experimental outputs measured on external benchmarks (CALVIN, LIBERO) against external and self-finetuned baselines. Hyperparameters such as 256 bins, the [-3,3] clipping range, and the Gumbel temperature schedule are design choices, not fitted to the headline deltas. The 12% vs 15% token-fraction discrepancy and the absence of variance reporting are reproducibility and statistical-evidence concerns, not circularity. The acknowledged limitations (e.g., Appendix D: 'Our real-world evaluation is intentionally limited in scope', and the unmatched D→D comparison) are honestly stated and do not hide an input-output equivalence. The only self-citations (He et al., 2025a,b) appear in related-work background on diffusion policies and are not load-bearing for ThinkProprio's contribution. No step reduces by construction to its own inputs, so the appropriate finding is no significant circularity.

Axiom & Free-Parameter Ledger

2 free parameters · 5 axioms · 0 invented entities

The method introduces no new physical entities. It relies on domain assumptions about quantization, token-space compatibility, the global context token, benchmark signal reliability, and the standard flow-matching objective.

free parameters (2)
  • proprio discretization bins and range = B=256, qmin/qmax=-3/3
    Eq. 1/§4.1: chosen by hand; the tokenization claim and final performance depend on this quantization grid.
  • Gumbel temperature schedule = α: 1.0 → 0.01 cosine-annealed
    §4.1: controls exploration-to-determinism of token selection; not derived, likely tuned.
axioms (5)
  • domain assumption Uniform binning of proprioception with B=256 over [-3,3] retains action-relevant precision
    Eq. 1 discretizes each scalar; this is the key encoding premise. If precision is insufficient, the tokenization advantage over continuous MLP could reverse. Range and bins are chosen, not derived.
  • domain assumption VLM token embeddings are an appropriate shared space for proprioceptive state
    §3.2 maps bin indices to the last 256 vocabulary token IDs and reuses the embedding table; assumes no distribution mismatch and that the VLM interprets these tokens as body state.
  • domain assumption A mean-pooled global context token preserves enough scene information to compensate for aggressive pruning
    §3.3 Eq. 8; without H_ctx Avg. Len. drops from 4.55 to 4.35 (Table 6), so the central performance result depends on this token.
  • domain assumption The small CALVIN/LIBERO success differences are real signals rather than evaluation noise
    §4.1 reports 'averaged over 5 independent evaluation runs' but no error bars; the 'improves performance' claim depends on this assumption.
  • standard math Flow-matching action-head objective is a valid action-generation training framework
    §3.4 Eq. 10; standard rectified-flow/flow-matching objective from Lipman et al. treated as background.

pith-pipeline@v1.3.0-alltime-deepseek · 20461 in / 12143 out tokens · 120914 ms · 2026-08-03T03:50:48.285212+00:00 · methodology

0 comments
read the original abstract

Vision-language-action (VLA) models typically inject proprioception only as a late conditioning signal, preventing robot state from grounding instruction understanding or directing visual attention. We introduce ThinkProprio, which discretizes proprioception into VLM-vocabulary tokens and uses them jointly with the instruction to gate visual patches before VLM computation, steering the model toward action-relevant evidence while discarding redundant tokens early. We find that proprioception added as a passive conditioning signal leaves performance essentially unchanged; its value emerges when token-form state acts as an active query that, with the instruction, selects which visual patches the VLM processes. Systematic ablations show that VLM-vocabulary tokens outperform learned projectors as the state encoding, and that retaining only about \SI{12}{\percent} of the visual tokens surpasses on CALVIN ABC$\to$D. Across CALVIN, LIBERO, and real-world manipulation, ThinkProprio reduces end-to-end inference latency while improving the matched full-token baseline.

Figures

Figures reproduced from arXiv: 2602.06575 by Chengyang He, David Navarro-Alarcon, Fangyuan Wang, Guodong Guo, Jiaming Qi, Peng Zhou, Shipeng Lyu.

Figure 1
Figure 1. Figure 1: ThinkProprio tokenizes proprioception into the VLM space to guide early visual reasoning. This yields strong CALVIN/LIBERO performance with 15 % of visual tokens and 58 % lower latency than prior VLA policies. Abstract Vision-language-action (VLA) models typically inject proprioception only as a late conditioning signal, which prevents robot state from shaping instruction understanding and from influencing… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of ThinkProprio. Proprioception is text-tokenized and combined with the instruction to guide visual token selection via cross-attention, retaining only task-relevant patches alongside a global context token. The compact token set is processed by the VLM, and the action head attends to the resulting features through cross-attention. where Q˜ = RMSNorm(Q). Each row of S scores all visual tokens for … view at source ↗
Figure 3
Figure 3. Figure 3: Token retention across four timesteps for two tasks, shown with paired static and wrist-mounted gripper views. Heatmaps visualize token retention scores, and labels indicate whether the retained tokens primarily focus on objects, proprioception, or both. The overlay Sel in each frame reports the number of retained tokens out of available visual tokens at that timestep. pink block red block [PITH_FULL_IMAG… view at source ↗
Figure 4
Figure 4. Figure 4: Recovery behavior on a challenging stacking task. The policy requires more than 300 steps to stack the pink block on the smaller red support, and the heatmaps show persistent attention on the object and gripper as it repeatedly corrects the placement. 6. Analysis 6.1. Visualizing Physically Grounded Token Retention To examine what visual evidence the physically grounded conditioning mechanism preserves, we… view at source ↗
Figure 5
Figure 5. Figure 5: Per-subtask success rates for the 34 CALVIN long-horizon subtasks on task ABC→D. The horizontal lollipop plot reports success rate on a zoomed 0.75–1.00 axis, with colors indicating the subtask category. • π0 and π0.5 (Black et al., 2024a; Intelligence et al., 2025) are dual-system VLAs that combine a PaliGemma VLM backbone with a flow-based action generator. In the full π0.5 system, hierarchical inference… view at source ↗
Figure 6
Figure 6. Figure 6: Failure case analysis on Push Pink Block Right. We show sequential timesteps from a representative failed rollout, with paired views from the static camera and the wrist-mounted gripper camera. Heatmaps overlay the selector’s token-retention scores (higher intensity indicates higher retention priority), illustrating where the model attends when deciding which visual evidence to preserve for action predicti… view at source ↗
Figure 7
Figure 7. Figure 7: Per-task success rates (%) on LIBERO. For each task, we report the mean success over 20 evaluation rollouts for each method, with tasks grouped by suite (Spatial, Object, Goal, Long). This task-level view complements suite averages by revealing which individual tasks drive method differences, and it shows that LIBERO-Long exhibits the widest spread, making it the most diagnostic suite for compounding error… view at source ↗
Figure 8
Figure 8. Figure 8: Real-world evaluation setup with an xArm arm and a parallel-jaw gripper. Two Intel RealSense D455 RGB cameras provide a third-person static view of the workspace and a wrist view rigidly mounted on the gripper [PITH_FULL_IMAGE:figures/full_fig_p018_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Representative real-world rollouts for ThinkProprio on Pick-Place and Push tasks. We show frames from the third-person static camera and the wrist-mounted gripper camera to illustrate approach, grasp or contact, and placement or object displacement under partial occlusions. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. When Video Misreads: Closed-Loop Distillation of Reading Heuristics for Exploratory Manipulation Trace QA

    cs.RO 2026-06 unverdicted novelty 6.0

    Closed-Loop Trace Distillation distills one-line natural-language prompts from labeled training traces to improve VLM accuracy on predicting minimal-success action chains in Exploratory Manipulation Trace QA by 0.38-0...

Reference graph

Works this paper leans on

32 extracted references · 16 linked inside Pith · cited by 1 Pith paper

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Gr00t n1: An open foundation model for generalist humanoid robots

    Bjorck, J., Casta \ n eda, F., Cherniadev, N., Da, X., Ding, R., Fan, L., Fang, Y., Fox, D., Hu, F., Huang, S., et al. Gr00t n1: An open foundation model for generalist humanoid robots. arXiv preprint arXiv:2503.14734, 2025

    Pith/arXiv arXiv 2025

  3. [3]

    X., Tanner, J., Vuong, Q., Walling, A., Wang, H., and Zhilinsky, U

    Black, K., Brown, N., Driess, D., Esmail, A., Equi, M., Finn, C., Fusai, N., Groom, L., Hausman, K., Ichter, B., Jakubczak, S., Jones, T., Ke, L., Levine, S., Li-Bell, A., Mothukuri, M., Nair, S., Pertsch, K., Shi, L. X., Tanner, J., Vuong, Q., Walling, A., Wang, H., and Zhilinsky, U. _0 : A vision-language-action flow model for general robot control, 202...

    Pith/arXiv arXiv 2024

  4. [4]

    R., Finn, C., Kumar, A., and Levine, S

    Black, K., Nakamoto, M., Atreya, P., Walke, H. R., Finn, C., Kumar, A., and Levine, S. Zero-shot robotic manipulation with pre-trained image-editing diffusion models. In The Twelfth International Conference on Learning Representations, 2024 b . URL https://openreview.net/forum?id=c0chJTSbci

    2024

  5. [5]

    Diffusion policy: Visuomotor policy learning via action diffusion

    Chi, C., Xu, Z., Feng, S., Cousineau, E., Du, Y., Burchfiel, B., Tedrake, R., and Song, S. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, 44 0 (10-11): 0 1684--1704, 2025

    2025

  6. [6]

    K., and Levine, S

    Dasari, S., Mees, O., Zhao, S., Srirama, M. K., and Levine, S. The ingredients for robotic diffusion transformers. arXiv preprint arXiv:2410.10088, 2024

    Pith/arXiv arXiv 2024

  7. [7]

    U., Akram, W., Saoud, L

    Din, M. U., Akram, W., Saoud, L. S., Rosell, J., and Hussain, I. Vision language action models in robotic manipulation: A systematic review. arXiv preprint arXiv:2507.10672, 2025

    arXiv 2025

  8. [8]

    S., Liu, X., Schwager, M., and Sartoretti, G

    He, C., Camps, G. S., Liu, X., Schwager, M., and Sartoretti, G. Latent theory of mind: A decentralized diffusion architecture for cooperative manipulation. arXiv preprint arXiv:2505.09144, 2025 a

    Pith/arXiv arXiv 2025

  9. [9]

    S., Sartoretti, G., and Schwager, M

    He, C., Liu, X., Camps, G. S., Sartoretti, G., and Schwager, M. Demystifying diffusion policies: Action memorization and simple lookup table alternatives, 2025 b . URL https://arxiv.org/abs/2505.05787

    Pith/arXiv arXiv 2025

  10. [10]

    Dita: Scaling diffusion transformer for generalist vision-language-action policy

    Hou, Z., Zhang, T., Xiong, Y., Duan, H., Pu, H., Tong, R., Zhao, C., Zhu, X., Qiao, Y., Dai, J., and Chen, Y. Dita: Scaling diffusion transformer for generalist vision-language-action policy. arXiv preprint arXiv:2503.19757, 2025

    Pith/arXiv arXiv 2025

  11. [11]

    Video prediction policy: A generalist robot policy with predictive visual representations

    Hu, Y., Guo, Y., Wang, P., Chen, X., Wang, Y.-J., Zhang, J., Sreenath, K., Lu, C., and Chen, J. Video prediction policy: A generalist robot policy with predictive visual representations. In Forty-second International Conference on Machine Learning

  12. [12]

    Otter: A vision-language-action model with text-aware visual feature extraction

    Huang, H., Liu, F., Fu, L., Wu, T., Mukadam, M., Malik, J., Goldberg, K., and Abbeel, P. Otter: A vision-language-action model with text-aware visual feature extraction. arXiv preprint arXiv:2503.03734, 2025

    arXiv 2025

  13. [13]

    Y., Ghosh, D., Groom, L., Hausman, K., Ichter, B., Jakubczak, S., Jones, T., Ke, L., LeBlanc, D., Levine, S., Li-Bell, A., Mothukuri, M., Nair, S., Pertsch, K., Ren, A

    Intelligence, P., Black, K., Brown, N., Darpinian, J., Dhabalia, K., Driess, D., Esmail, A., Equi, M., Finn, C., Fusai, N., Galliker, M. Y., Ghosh, D., Groom, L., Hausman, K., Ichter, B., Jakubczak, S., Jones, T., Ke, L., LeBlanc, D., Levine, S., Li-Bell, A., Mothukuri, M., Nair, S., Pertsch, K., Ren, A. Z., Shi, L. X., Smith, L., Springenberg, J. T., Sta...

    Pith/arXiv arXiv 2025

  14. [14]

    The better you learn, the smarter you prune: Towards efficient vision-language-action models via differentiable token pruning

    Jiang, T., Jiang, X., Ma, Y., Wen, X., Li, B., Zhan, K., Jia, P., Liu, Y., Sun, S., and Lang, X. The better you learn, the smarter you prune: Towards efficient vision-language-action models via differentiable token pruning. arXiv preprint arXiv:2509.12594, 2025

    arXiv 2025

  15. [15]

    J., Finn, C., and Liang, P

    Kim, M. J., Finn, C., and Liang, P. Fine-tuning vision-language-action models: Optimizing speed and success, 2025 a . URL https://arxiv.org/abs/2502.19645

    Pith/arXiv arXiv 2025

  16. [16]

    J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., Nair, S., Rafailov, R., Foster, E

    Kim, M. J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., Nair, S., Rafailov, R., Foster, E. P., Sanketi, P. R., Vuong, Q., et al. Openvla: An open-source vision-language-action model. In Conference on Robot Learning, pp.\ 2679--2713. PMLR, 2025 b

    2025

  17. [17]

    Coa-vla: Improving vision-language-action models via visual-textual chain-of-affordance, 2025

    Li, J., Zhu, Y., Tang, Z., Wen, J., Zhu, M., Liu, X., Li, C., Cheng, R., Peng, Y., Peng, Y., and Feng, F. Coa-vla: Improving vision-language-action models via visual-textual chain-of-affordance, 2025. URL https://arxiv.org/abs/2412.20451

    Pith/arXiv arXiv 2025

  18. [18]

    Cogact: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation

    Li, Q., Liang, Y., Wang, Z., Luo, L., Chen, X., Liao, M., Wei, F., Deng, Y., Xu, S., Zhang, Y., et al. Cogact: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation. arXiv preprint arXiv:2411.19650, 2024

    Pith/arXiv arXiv 2024

  19. [19]

    Vision-language foundation models as effective robot imitators

    Li, X., Liu, M., Zhang, H., Yu, C., Xu, J., Wu, H., Cheang, C., Jing, Y., Zhang, W., Liu, H., Li, H., and Kong, T. Vision-language foundation models as effective robot imitators. arXiv preprint arXiv:2311.01378, 2023

    Pith/arXiv arXiv 2023

  20. [20]

    Lipman, Y., Chen, R. T. Q., Ben-Hamu, H., Nickel, M., and Le, M. Flow matching for generative modeling, 2023. URL https://arxiv.org/abs/2210.02747

    Pith/arXiv arXiv 2023

  21. [21]

    Libero: Benchmarking knowledge transfer for lifelong robot learning

    Liu, B., Zhu, Y., Gao, C., Feng, Y., Liu, Q., Zhu, Y., and Stone, P. Libero: Benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems, 36: 0 44776--44791, 2023

    2023

  22. [22]

    Calvin: A benchmark for language-conditioned policy learning for long-horizon robot manipulation tasks

    Mees, O., Hermann, L., Rosete-Beas, E., and Burgard, W. Calvin: A benchmark for language-conditioned policy learning for long-horizon robot manipulation tasks. IEEE Robotics and Automation Letters, 7 0 (3): 0 7327--7334, 2022

    2022

  23. [23]

    and Xie, S

    Peebles, W. and Xie, S. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp.\ 4195--4205, 2023

    2023

  24. [24]

    Dynamicvit: Efficient vision transformers with dynamic token sparsification

    Rao, Y., Zhao, W., Liu, B., Lu, J., Zhou, J., and Hsieh, C.-J. Dynamicvit: Efficient vision transformers with dynamic token sparsification. Advances in neural information processing systems, 34: 0 13937--13949, 2021

    2021

  25. [25]

    E., Wenzel, F., and Lioutikov, R

    Reuss, M., Ya g murlu, \"O . E., Wenzel, F., and Lioutikov, R. Multimodal diffusion transformer: Learning versatile behavior from multimodal goals. arXiv preprint arXiv:2407.05996, 2024

    Pith/arXiv arXiv 2024

  26. [26]

    E., Otto, F., and Lioutikov, R

    Reuss, M., Zhou, H., R\" u hle, M., Ya g murlu, O. E., Otto, F., and Lioutikov, R. Flower: Democratizing generalist robot policies with efficient vision-language-flow models. In Lim, J., Song, S., and Park, H.-W. (eds.), Proceedings of The 9th Conference on Robot Learning, volume 305 of Proceedings of Machine Learning Research, pp.\ 3736--3761. PMLR, 27--...

    2025

  27. [27]

    S., Piergiovanni, A., Arnab, A., Dehghani, M., and Angelova, A

    Ryoo, M. S., Piergiovanni, A., Arnab, A., Dehghani, M., and Angelova, A. Tokenlearner: What can 8 learned tokens do for images and videos? arXiv preprint arXiv:2106.11297, 2021

    Pith/arXiv arXiv 2021

  28. [28]

    Smolvla: A vision-language-action model for affordable and efficient robotics

    Shukor, M., Aubakirova, D., Capuano, F., Kooijmans, P., Palma, S., Zouitine, A., Aractingi, M., Pascal, C., Russi, M., Marafioti, A., et al. Smolvla: A vision-language-action model for affordable and efficient robotics. arXiv preprint arXiv:2506.01844, 2025

    Pith/arXiv arXiv 2025

  29. [29]

    Predictive inverse dynamics models are scalable learners for robotic manipulation

    Tian, Y., Yang, S., Zeng, J., Wang, P., Lin, D., Dong, H., and Pang, J. Predictive inverse dynamics models are scalable learners for robotic manipulation. arXiv preprint arXiv:2412.15109, 2024

    Pith/arXiv arXiv 2024

  30. [30]

    Unleashing large-scale video generative pre-training for visual robot manipulation

    Wu, H., Jing, Y., Cheang, C., Chen, G., Xu, J., Li, X., Liu, M., Li, H., and Kong, T. Unleashing large-scale video generative pre-training for visual robot manipulation. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=NxoFmGgWC9

    2024

  31. [31]

    Deer-vla: Dynamic inference of multimodal large language models for efficient robot execution

    Yue, Y., Wang, Y., Kang, B., Han, Y., Wang, S., Song, S., Feng, J., and Huang, G. Deer-vla: Dynamic inference of multimodal large language models for efficient robot execution. Advances in Neural Information Processing Systems, 37: 0 56619--56643, 2024

    2024

  32. [32]

    Rt-2: Vision-language-action models transfer web knowledge to robotic control

    Zitkovich, B., Yu, T., Xu, S., Xu, P., Xiao, T., Xia, F., Wu, J., Wohlhart, P., Welker, S., Wahid, A., et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pp.\ 2165--2183. PMLR, 2023

    2023

This paper was first reviewed by deepseek-v4-flash on August 3, 2026.