Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Two coupled claims carry this paper: embodied reasoning can be measured independently of execution, and a flow-matching action tokenizer lets one model reason and act precisely in a unified autoregressive space.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 13:25 UTC pith:W25MSGKI

load-bearing objection FACT is a real, checkable contribution; the ERIQ-to-generalization correlation is asserted, not shown, and the benchmark's train/test separation is the first thing to ask the authors for. the 4 major comments →

arxiv 2512.24125 v3 pith:W25MSGKI submitted 2025-12-30 cs.RO cs.AI

Unified Embodied VLM Reasoning with Robotic Action via Autoregressive Discretized Pre-training

Yi Liu , Sukai Wang , Dafeng Wei , Xiaowei Cai , Linqing Zhong , Jiange Yang , Guanghui Ren , Jinyu Zhang
show 5 more authors
Maoqing Yao Chuankang Li Xindong He Liliang Chen Jianlan Luo
This is my paper
classification cs.RO cs.AI
keywords embodied reasoningvision-language-action modelaction tokenizationflow matchingrobotic manipulation benchmarkdiscrete action tokensautoregressive policygeneralization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

ERIQ, a new 6,052-question benchmark built from real robot trials, is offered as a way to measure a robot model's embodied reasoning — spatial perception, planning, error recovery, and human intent — without running any physical policy. The paper's central evidence is a strong positive correlation between scores on ERIQ and end-to-end task success, which it presents as proof that reasoning capability drives generalization in vision-language-action models. To close the gap from reasoning to action, it introduces FACT, a tokenizer that compresses continuous robot trajectories into short discrete codes using a sign-quantization encoder and a flow-matching decoder that reconstructs high-fidelity motion. The resulting model, GenieReasoner, jointly trains on general question-answering, embodied question-answering, and tokenized actions, and is claimed to beat both continuous-action and discrete-action baselines on real-world manipulation. If correct, this gives the field a cheap, decoupled diagnostic for reasoning and a way to remove the precision-versus-reasoning trade-off.

Core claim

The paper's central claim is double. First, ERIQ can quantify embodied reasoning separately from motor control: it is a 6,052-item multiple-choice question set spanning spatial perception and grounding, planning and monitoring, error detection and recovery, and human intent understanding, drawn entirely from real, first-person robotic trials. On this benchmark the base vision-language model scores 58.64%; after co-training on general and embodied question-answering data plus tokenized actions, the same backbone reaches 82.72%, a 41% relative improvement. Second, the paper claims this reasoning gain is not cosmetic: ERIQ scores correlate strongly with end-to-end manipulation success, and the

What carries the argument

Two instruments carry the argument. ERIQ is the measuring stick: 6,052 multiple-choice questions across four reasoning dimensions, with deterministic scoring, meant to isolate cognition from control. FACT is the bridge: a vector-quantized encoder (a lookup-free sign quantizer that turns continuous action chunks into compact bit-like codes) paired with a flow-matching decoder that learns a velocity field and integrates an ordinary differential equation to reconstruct smooth trajectories from those codes. The mechanism that makes the system work is the alignment of the action space with the VLM's discrete token space: action prediction becomes next-token prediction, so cross-entropy reasoning

Load-bearing premise

The load-bearing premise is that ERIQ truly measures embodied reasoning without contamination: the benchmark questions and the model's embodied question-answering training data come from the same real-world robotic trials, and the paper never states that ERIQ items were excluded from training, so if the model has memorized those scenes the reported 41% reasoning gain and its correlation with task success are partly artifacts.

What would settle it

Build an ERIQ-style benchmark whose questions come from a completely different robot dataset or from held-out trials never seen in any training mixture, retrain the same model with the same recipe, and check two things: does the ERIQ score stay near 82.72 instead of dropping toward the base 58.64, and does the rank order of multiple VLMs on this new benchmark still match their rank order on real-robot success? A large drop or a broken ranking would refute the paper's claim that ERIQ measures a generalizable reasoning capability that drives generalization.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • ERIQ gives developers a way to measure embodied reasoning in minutes without expensive physical rollouts, enabling early screening of VLM backbones before committing to action training.
  • FACT's compact discrete codes let a single autoregressive model carry both reasoning and control, removing the need for a separate continuous action head and the gradient conflicts it introduces.
  • Because the flow-matching decoder preserves trajectory fidelity, discrete-action policies built this way should avoid the precision failures that plagued earlier tokenizers like uniform binning or variable-length encoding.
  • The claimed correlation implies that improving a model's ERIQ score—via co-training on embodied reasoning data—should directly improve end-to-end manipulation success, not just instruction following.
  • The joint post-training recipe (keeping embodied VQA in the final stage) is presented as necessary to preserve reasoning while aligning actions, so the paper's success hinges on the optimizer not un-learning reasoning when action tokens are added.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If ERIQ is a valid decoupled diagnostic, then the standard development loop could invert: iterate on the VLM's reasoning score first, and reserve real-robot rollouts for final verification—a workflow the paper describes only implicitly.
  • An independent replication could take a fixed policy-training pipeline and swap only the pretrained VLM backbone; if the real-world success ranking follows the ERIQ ranking, the correlation claim would be robust; the paper's own evidence comes from a single pipeline.
  • A natural stress test is to measure FACT's reconstruction fidelity on sub-millimeter precision tasks or high-frequency dynamic motions (fast reaching, cloth manipulation); the benchmark tasks here are mostly pick-and-place style, and the paper does not report how the tokenizer behaves at the edge of the action distribution.
  • Because the reasoning training data and the ERIQ benchmark share the same real-robot trials, an external, independently-sourced version of ERIQ would be needed to confirm that the measured reasoning gain reflects general embodied intelligence rather than memorized scene statistics.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes two coupled contributions for generalist robot manipulation. First, it introduces ERIQ, a 6,052-question multiple-choice benchmark for embodied reasoning, covering Spatial Perception & Grounding, Planning & Monitoring, Error Detection & Recovery, and Human Intent Understanding. The authors use ERIQ to claim a strong positive correlation between embodied reasoning ability and end-to-end VLA generalization. Second, it proposes FACT, a flow-matching action tokenizer that maps continuous action trajectories to discrete sign-quantized tokens and reconstructs them with a rectified-flow ODE decoder. This tokenizer is embedded in GenieReasoner, a Qwen2.5-VL-based VLA that is jointly pre-trained on general VQA, embodied VQA, and tokenized action data, then post-trained with a mixed VQA+action objective. Experiments report large ERIQ gains over the base VLM (58.64 to 82.72), lower reconstruction MSE than FAST+ at matched code lengths, ablations in the GenieSim simulator, and real-world comparisons against GR00T, π0, π0.5, and π0-FAST.

Significance. If both central claims are valid, the paper would give the community a cheap, decoupled diagnostic for embodied reasoning and an action tokenizer that avoids the precision-reasoning trade-off in discrete VLAs. The paper has genuine strengths: the task-level 40:1 split for FACT evaluation is appropriate; the ablation in Table III cleanly separates the contributions of embodied VQA and action alignment; and the representation-level MSE comparison with FAST+ is informative. However, the headline correlation claim is currently not established: the ERIQ benchmark and the embodied VQA training set may share the same AgiBot World source with no reported train/test separation, and Table II mixes model sizes without reporting a correlation statistic. The real-world superiority claim likewise lacks essential experimental detail. These issues are fixable but are load-bearing for the paper's main contributions.

major comments (4)
  1. [Section III-A, V-A, V-B] The central correlation claim depends on ERIQ being a clean, independent measure of embodied reasoning. ERIQ is 'derived entirely from authentic, real-world robotic trials' (Section III-A), while the embodied VQA training data is 'curated based on AgiBot World' and includes 2D trajectory data, grounding annotations, sub-task planning, and scene understanding (Section V-A) — precisely the content dimensions ERIQ tests. The manuscript never states a task/scene/instance split between ERIQ and the training data. If ERIQ items or their underlying episodes overlap with the embodied VQA pre-training data, the reported improvement from 58.64 to 82.72 and the claimed reasoning-to-generalization correlation are confounded by distributional leakage. Please specify the exact split, report whether any ERIQ items or scenes appear in training, and provide an analysis (e.g., per-dimension performance on
  2. [Section V-B, Table II] The assertion of a 'strong positive correlation between embodied reasoning capability and end-to-end VLA generalization' is not supported by any correlation coefficient, confidence interval, or significance test. Table II mixes models of different sizes (3B, 7B, 8B) and different training corpora, so the apparent ordering could reflect scale or data rather than embodied reasoning per se. Please report a quantitative correlation (e.g., Pearson/Spearman) between ERIQ-Avg and the downstream success/language-following metrics, ideally with a partial correlation controlling for parameter count or within a single model family, and state the number of models and p-value. The current text overstates what the data show.
  3. [Section V-E, Figures 8-10] The real-world superiority claim over π0.5, GR00T, π0, and π0-FAST is not fully assessable from the manuscript. Figures 8–10 present aggregate scores without error bars, trial counts, or standard deviations. It is also unclear whether the baselines were adapted to the AgiBot G1 embodiment (fine-tuning data, number of steps, action horizon, inference settings) or were used with their default configurations. The 'Total Score' is defined as a fixed weighted average, Score=(1.0×Success+0.5×Following)/1.5, with no sensitivity analysis. Please report per-trial results, the number of rollouts per setting, baseline adaptation details, and the raw success/following scores behind the aggregate.
  4. [Section IV-C, Eq. (3)-(8), Section V-C] The FACT tokenizer description is mostly clear, but the relation between the sign quantizer and the reported vocabulary size needs clarification. With c=sign(e), the effective codebook size is 2^D, where D is the latent dimension; the text later mentions a '212-entry vocabulary' (Section V-C), which appears to be a typo for 2^12 or another power of two. Please state D explicitly and correct the notation. This is not a fundamental flaw, but it is necessary for reproducing the tokenizer and interpreting Figure 7.
minor comments (5)
  1. [Figure 1] '41% Accuracy Improvement' is ambiguous: the ERIQ score increases by 24.08 percentage points (58.64 to 82.72), which is a 41.1% relative improvement. Please label which convention is used.
  2. [Table II caption] The caption refers to 'ER-6K Benchmark'; this should likely be 'ERIQ Benchmark'.
  3. [Section V-B] The sentence 'Dualview Matching (an absolute increase of 31.01% over the base model and Relative Position Grounding (+24.9%)' is missing a closing parenthesis and a conjunction; please rephrase.
  4. [Section V-C] The task-level 40:1 split is good, but please also report the number of tasks/episodes in the split and whether the evaluation tasks are disjoint from those used for the VLM's embodied VQA training. This matters for the contamination concern raised above.
  5. [General] No data or code release is mentioned. For ERIQ to serve as a community benchmark and for FACT to be compared, the benchmark, tokenizer code, and training/inference recipes should be made available.

Circularity Check

0 steps flagged

No circularity demonstrated; ERIQ provenance overlap is a benchmark-validity risk, not a by-construction reduction.

full rationale

The paper's derivation chain does not reduce any claimed result to its inputs by construction. The strongest candidate for circularity is the ERIQ benchmark and the GenieReasoner training data sharing the AgiBot World source: Section III-A states that all 6,052 ERIQ QA pairs are 'derived entirely from authentic, real-world robotic trials,' while Section V-A says the embodied reasoning training set is 'curated on AgiBot World' and includes 2D trajectory, grounding, sub-task planning, and scene-understanding data. These passages show shared provenance, but the paper never states that the exact ERIQ items, scenes, or question templates appear in the training mixture. Without that identity, the concern is a contamination/leakage risk for ERIQ's validity as an independent measure, not a demonstrated reduction such as 'the test set equals the training set by construction.' The claimed correlation between ERIQ and end-to-end success is also weakly supported: no correlation statistic is reported, and Table III includes Exp #1 (ERIQ 82.72, success 0.05) versus Exp #2 (ERIQ 0.00, success 0.20), which is an empirical-support problem, not a circularity. FACT's reconstruction fidelity is evaluated on a held-out task-level split (40:1) against FAST+, and the flow-matching/quantization components are drawn from external prior work (Rectified Flow, MM-DiT, lookup-free quantization). Real-world comparisons against pi0, pi0.5, GR00T, and pi0-FAST are external baselines. Self-citations to AgiBot World and GenieSim are data/simulator provenance, not load-bearing self-citations that substitute for argument. Therefore no circular step meets the required evidentiary bar.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 3 invented entities

The ledger shows where the paper spends its credibility. FACT rests on standard, externally checkable machinery (flow matching, VQ losses) plus two hand-tuned hyperparameters. ERIQ's diagnostic claim rests on the unstated assumption that its benchmark items are not leaked from the same AgiBot World data used for training, and its correlation claim is supported only by the authors' own ablations. The invented entities are a benchmark and a model rather than physical postulates, but the audit function is the same: neither ERIQ nor GenieReasoner ships with artifacts that let outsiders test them.

free parameters (4)
  • FACT code length = 20
    Chosen by ablation in Section V-C as the best trade-off between reconstruction MSE and autoregressive prediction difficulty; a hand-selected design parameter.
  • FACT vocabulary size = 2^12 = 4096
    Selected via ablation over 2^10, 2^12, 2^14 in Section V-C / Figure 7.
  • Total score weighting = (1.0 x Success + 0.5 x Following)/1.5
    Ad hoc aggregate introduced without justification in the Figure 10 caption; favors success over instruction-following by 2:1.
  • Training data mixture proportions
    Weights across Cambrian-10M, LLaVA-OneVision, Cosmos-Reason1, AgiBot World VQA, and action data in each of the three stages are unspecified, yet Exp #0-#4 differ only in these mixtures.
axioms (6)
  • standard math Rectified-flow objective (Eqs. 4-8) learns the data distribution by regressing the target velocity field a-z
    Invokes Liu et al. 2022; standard generative-model result, not derived in this paper.
  • standard math Lookup-free sign quantization with entropy and commitment losses yields usable discrete action codes
    Invokes Yu et al. 2023 (TiTok); taken as given in Section IV-C.
  • domain assumption Autoregressive next-token training on VQA plus action tokens preserves and improves VLM reasoning while adding control
    Core premise of the unified-space design (Section IV-B); tested empirically in Table III but not derived from any principle.
  • domain assumption Reconstruction MSE on task-level 40:1 splits predicts real-world control precision
    Section V-C links FACT fidelity to deployment precision without a closed-loop experiment connecting MSE to task success.
  • ad hoc to paper Multiple-choice accuracy on ERIQ measures embodied reasoning rather than benchmark-specific shortcuts or leaked training distribution
    ERIQ's validity as a diagnostic is assumed; no train/test separation is stated (Section III-A vs V-A), and MC format invites shortcut-solving.
  • domain assumption GenieSim 50-rollout evaluations are representative of open-world generalization
    Training-recipe conclusions in Table III rest on one simulated pick task; real-world transfer is shown only qualitatively.
invented entities (3)
  • ERIQ benchmark (6,052 QA pairs) no independent evidence
    purpose: Decoupled measurement of embodied reasoning across four dimensions; supports the reasoning-to-generalization correlation claim.
    Not released; built entirely from the authors' AgiBot World data, so it cannot currently be externally probed or falsified.
  • FACT action code space (sign-quantized VQ latents + flow decoder) independent evidence
    purpose: Maps continuous action chunks to discrete tokens and back, enabling unified autoregressive reasoning + control.
    Reconstruction fidelity is compared externally against FAST+ on held-out task splits (Figure 7), providing a falsifiable handle.
  • GenieReasoner 3B (Ours-3B) no independent evidence
    purpose: Concrete instantiation of the unified pipeline; carries the ERIQ score (82.72) and real-robot superiority claims.
    No weights or deployment artifacts released; claims rest on project-page demonstrations and in-paper tables.

pith-pipeline@v1.3.0-alltime-deepseek · 19755 in / 15071 out tokens · 150625 ms · 2026-08-03T13:25:31.809770+00:00 · methodology

0 comments
read the original abstract

General-purpose robotic systems operating in open-world environments must achieve both broad generalization and high-precision action execution, a combination that remains challenging for existing Vision-Language-Action (VLA) models. While large Vision-Language Models (VLMs) improve semantic generalization, insufficient embodied reasoning leads to brittle behavior, and conversely, strong reasoning alone is inadequate without precise control. To provide a decoupled and quantitative assessment of this bottleneck, we introduce Embodied Reasoning Intelligence Quotient (ERIQ), a large-scale embodied reasoning benchmark in robotic manipulation, comprising 6K+ question-answer pairs across four reasoning dimensions. By decoupling reasoning from execution, ERIQ enables systematic evaluation and reveals a strong positive correlation between embodied reasoning capability and end-to-end VLA generalization. To bridge the gap from reasoning to precise execution, we propose FACT, a flow-matching-based action tokenizer that converts continuous control into discrete sequences while preserving high-fidelity trajectory reconstruction. The resulting GenieReasoner jointly optimizes reasoning and action in a unified space, outperforming both continuous-action and prior discrete-action baselines in real-world tasks. Together, ERIQ and FACT provide a principled framework for diagnosing and overcoming the reasoning-precision trade-off, advancing robust, general-purpose robotic manipulation. Project page: https://geniereasoner.github.io/GenieReasoner/

Figures

Figures reproduced from arXiv: 2512.24125 by Chuankang Li, Dafeng Wei, Guanghui Ren, Jiange Yang, Jianlan Luo, Jinyu Zhang, Liliang Chen, Linqing Zhong, Maoqing Yao, Sukai Wang, Xiaowei Cai, Xindong He, Yi Liu.

Figure 2
Figure 2. Figure 2: Illustration of the ERIQ benchmark. Example samples from the four major categories of embodied reasoning. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: The GenieReasoner system architecture. (a) Training: Our unified pipeline jointly optimizes the VLM [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: The FACT Action Tokenizer. We discretize [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Qualitative visualization of the VLM’s multi-task reasoning capabilities. We display spatial reasoning, [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 8
Figure 8. Figure 8: Real-robot language following evaluation. Ge [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗
Figure 11
Figure 11. Figure 11: Visualization of different manipulation tasks in real world. [PITH_FULL_IMAGE:figures/full_fig_p012_11.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Probing Collision Grounding in Vision-Language Models for Safe Human-Robot Collaboration

    cs.CV 2026-05 unverdicted novelty 6.0

    TouchSafeBench evaluates VLMs on collision grounding, finding best Macro-F1 below 50% and that explicit depth does not yield reliable robot-body contact inference.

Reference graph

Works this paper leans on

66 extracted references · 43 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Rt- 1: Robotics transformer for real-world control at scale,

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, J. Hsuet al., “Rt- 1: Robotics transformer for real-world control at scale,”arXiv preprint arXiv:2212.06817, 2022

    Pith/arXiv arXiv 2022

  2. [2]

    Rt-2: Vision-language- action models transfer web knowledge to robotic control,

    B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahidet al., “Rt-2: Vision-language- action models transfer web knowledge to robotic control,” in Conference on Robot Learning. PMLR, 2023, pp. 2165–2183

    2023

  3. [3]

    Openvla: An open-source vision-language-action model,

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketiet al., “Openvla: An open-source vision-language-action model,”arXiv preprint arXiv:2406.09246, 2024

    Pith/arXiv arXiv 2024

  4. [4]

    π 0: A vision-language-action flow model for general robot control. corr, abs/2410.24164, 2024. doi: 10.48550,

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichteret al., “π 0: A vision-language-action flow model for general robot control. corr, abs/2410.24164, 2024. doi: 10.48550,”arXiv preprint ARXIV .2410.24164

    Pith/arXiv arXiv 2024

  5. [5]

    Agibot world colosseo: A large- scale manipulation platform for scalable and intelligent embodied systems,

    Q. Bu, J. Cai, L. Chen, X. Cui, Y . Ding, S. Feng, S. Gao, X. He, X. Hu, X. Huanget al., “Agibot world colosseo: A large- scale manipulation platform for scalable and intelligent embodied systems,”arXiv preprint arXiv:2503.06669, 2025

    Pith/arXiv arXiv 2025

  6. [6]

    π 0.5: a vision- language-action model with open-world generalization,

    K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusaiet al., “π 0.5: a vision- language-action model with open-world generalization,”arXiv preprint arXiv:2504.16054, 2025

    Pith/arXiv arXiv 2025

  7. [7]

    Gemini robotics: Bringing ai into the physical world,

    G. R. Team, S. Abeyruwan, J. Ainslie, J.-B. Alayrac, M. G. Arenas, T. Armstrong, A. Balakrishna, R. Baruch, M. Bauza, M. Blokzijlet al., “Gemini robotics: Bringing ai into the physical world,”arXiv preprint arXiv:2503.20020, 2025

    Pith/arXiv arXiv 2025

  8. [8]

    Octo: An open- source generalist robot policy,

    O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xuet al., “Octo: An open- source generalist robot policy,”arXiv preprint arXiv:2405.12213, 2024

    Pith/arXiv arXiv 2024

  9. [9]

    Diffusion policy: Visuomotor policy learning via action diffusion,

    C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,”The International Journal of Robotics Research, vol. 44, no. 10-11, pp. 1684–1704, 2025

    2025

  10. [10]

    Learning fine- grained bimanual manipulation with low-cost hardware,

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn, “Learning fine- grained bimanual manipulation with low-cost hardware,”arXiv preprint arXiv:2304.13705, 2023

    Pith/arXiv arXiv 2023

  11. [11]

    Onetwovla: A unified vision-language-action model with adaptive reasoning,

    F. Lin, R. Nai, Y . Hu, J. You, J. Zhao, and Y . Gao, “Onetwovla: A unified vision-language-action model with adaptive reasoning,” arXiv preprint arXiv:2505.11917, 2025

    arXiv 2025

  12. [12]

    Fast: Efficient action tokenization for vision-language-action models,

    K. Pertsch, K. Stachowicz, B. Ichter, D. Driess, S. Nair, Q. Vuong, O. Mees, C. Finn, and S. Levine, “Fast: Efficient action tokenization for vision-language-action models,”arXiv preprint arXiv:2501.09747, 2025

    Pith/arXiv arXiv 2025

  13. [13]

    F1: A vision-language-action model bridging understanding and generation to actions,

    Q. Lv, W. Kong, H. Li, J. Zeng, Z. Qiu, D. Qu, H. Song, Q. Chen, X. Deng, and J. Pang, “F1: A vision-language-action model bridging understanding and generation to actions,”arXiv preprint arXiv:2509.06951, 2025

    Pith/arXiv arXiv 2025

  14. [14]

    Chatvla: Unified multimodal understanding and robot control with vision-language-action model,

    Z. Zhou, Y . Zhu, M. Zhu, J. Wen, N. Liu, Z. Xu, W. Meng, R. Cheng, Y . Peng, C. Shenet al., “Chatvla: Unified multimodal understanding and robot control with vision-language-action model,”arXiv preprint arXiv:2502.14420, 2025

    Pith/arXiv arXiv 2025

  15. [15]

    Spatialvla: Exploring spatial representations for visual-language-action model,

    D. Qu, H. Song, Q. Chen, Y . Yao, X. Ye, Y . Ding, Z. Wang, J. Gu, B. Zhao, D. Wanget al., “Spatialvla: Exploring spatial representations for visual-language-action model,”arXiv preprint arXiv:2501.15830, 2025

    Pith/arXiv arXiv 2025

  16. [16]

    Hybridvla: Collaborative diffusion and autoregression in a unified vision-language-action model,

    J. Liu, H. Chen, P. An, Z. Liu, R. Zhang, C. Gu, X. Li, Z. Guo, S. Chen, M. Liuet al., “Hybridvla: Collaborative diffusion and autoregression in a unified vision-language-action model,”arXiv preprint arXiv:2503.10631, 2025

    Pith/arXiv arXiv 2025

  17. [17]

    Smolvla: A vision-language-action model for affordable and efficient robotics,

    M. Shukor, D. Aubakirova, F. Capuano, P. Kooijmans, S. Palma, A. Zouitine, M. Aractingi, C. Pascal, M. Russi, A. Marafioti et al., “Smolvla: A vision-language-action model for affordable and efficient robotics,”arXiv preprint arXiv:2506.01844, 2025

    Pith/arXiv arXiv 2025

  18. [18]

    Tracevla: Visual trace prompting en- hances spatial-temporal awareness for generalist robotic poli- cies,

    R. Zheng, Y . Liang, S. Huang, J. Gao, H. Daum´e III, A. Kolobov, F. Huang, and J. Yang, “Tracevla: Visual trace prompting en- hances spatial-temporal awareness for generalist robotic poli- cies,”arXiv preprint arXiv:2412.10345, 2024

    Pith/arXiv arXiv 2024

  19. [19]

    Robotic control via embodied chain-of-thought rea- soning,

    M. Zawalski, W. Chen, K. Pertsch, O. Mees, C. Finn, and S. Levine, “Robotic control via embodied chain-of-thought rea- soning,”arXiv preprint arXiv:2407.08693, 2024

    Pith/arXiv arXiv 2024

  20. [20]

    Knowledge insulating vision-language-action mod- els: Train fast, run fast, generalize better,

    D. Driess, J. T. Springenberg, B. Ichter, L. Yu, A. Li- Bell, K. Pertsch, A. Z. Ren, H. Walke, Q. Vuong, L. X. Shiet al., “Knowledge insulating vision-language-action mod- els: Train fast, run fast, generalize better,”arXiv preprint arXiv:2505.23705, 2025

    Pith/arXiv arXiv 2025

  21. [21]

    Vq- vla: Improving vision-language-action models via scaling vector- quantized action tokenizers,

    Y . Wang, H. Zhu, M. Liu, J. Yang, H.-S. Fang, and T. He, “Vq- vla: Improving vision-language-action models via scaling vector- quantized action tokenizers,”arXiv preprint arXiv:2507.01016, 2025

    Pith/arXiv arXiv 2025

  22. [22]

    Grounding multimodal large language models in actions,

    A. Szot, B. Mazoure, H. Agrawal, R. D. Hjelm, Z. Kira, and A. Toshev, “Grounding multimodal large language models in actions,”Advances in Neural Information Processing Systems, vol. 37, pp. 20 198–20 224, 2024

    2024

  23. [23]

    Stable-fast: Stabilizing inference of autoregressive vision-language-action models

    X. Luo, X. Yin, H. Wu, L. Gao, and J. Song, “Stable-fast: Stabilizing inference of autoregressive vision-language-action models.”

  24. [24]

    Do as i can, not as i say: Grounding language in robotic affordances,

    M. Ahn, A. Brohan, N. Brown, Y . Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakrishnan, K. Hausmanet al., “Do as i can, not as i say: Grounding language in robotic affordances,” arXiv preprint arXiv:2204.01691, 2022

    Pith/arXiv arXiv 2022

  25. [25]

    Language models as zero-shot planners: Extracting actionable knowledge for embodied agents,

    W. Huang, P. Abbeel, D. Pathak, and I. Mordatch, “Language models as zero-shot planners: Extracting actionable knowledge for embodied agents,” inInternational conference on machine learning. PMLR, 2022, pp. 9118–9147

    2022

  26. [26]

    Palm- e: An embodied multimodal language model,

    D. Driess, F. Xia, M. S. Sajjadi, C. Lynch, A. Chowdhery, Draft A. Wahid, J. Tompson, Q. Vuong, T. Yu, W. Huanget al., “Palm- e: An embodied multimodal language model,” 2023

    2023

  27. [27]

    Inner monologue: Embodied reasoning through planning with language models,

    W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Florence, A. Zeng, J. Tompson, I. Mordatch, Y . Chebotaret al., “Inner monologue: Embodied reasoning through planning with language models,”arXiv preprint arXiv:2207.05608, 2022

    Pith/arXiv arXiv 2022

  28. [28]

    Rt-h: Action hierarchies using language,

    S. Belkhale, T. Ding, T. Xiao, P. Sermanet, Q. Vuong, J. Tomp- son, Y . Chebotar, D. Dwibedi, and D. Sadigh, “Rt-h: Action hierarchies using language,”arXiv preprint arXiv:2403.01823, 2024

    Pith/arXiv arXiv 2024

  29. [29]

    Code as policies: Language model pro- grams for embodied control,

    J. Liang, W. Huang, F. Xia, P. Xu, K. Hausman, B. Ichter, P. Flo- rence, and A. Zeng, “Code as policies: Language model pro- grams for embodied control,”arXiv preprint arXiv:2209.07753, 2022

    Pith/arXiv arXiv 2022

  30. [30]

    Socratic models: Composing zero-shot multimodal reasoning with language,

    A. Zeng, M. Attarian, B. Ichter, K. Choromanski, A. Wong, S. Welker, F. Tombari, A. Purohit, M. Ryoo, V . Sindhwaniet al., “Socratic models: Composing zero-shot multimodal reasoning with language,”arXiv preprint arXiv:2204.00598, 2022

    Pith/arXiv arXiv 2022

  31. [31]

    Grounding language with visual affordances over unstructured data,

    O. Mees, J. Borja-Diaz, and W. Burgard, “Grounding language with visual affordances over unstructured data,”arXiv preprint arXiv:2210.01911, 2022

    Pith/arXiv arXiv 2022

  32. [32]

    Skill induction and planning with latent language,

    P. Sharma, A. Torralba, and J. Andreas, “Skill induction and planning with latent language,” inProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022, pp. 1713–1726

    2022

  33. [33]

    Hi robot: Open- ended instruction following with hierarchical vision-language- action models,

    L. X. Shi, B. Ichter, M. Equi, L. Ke, K. Pertsch, Q. Vuong, J. Tanner, A. Walling, H. Wang, N. Fusaiet al., “Hi robot: Open- ended instruction following with hierarchical vision-language- action models,”arXiv preprint arXiv:2502.19417, 2025

    Pith/arXiv arXiv 2025

  34. [34]

    Robobrain: A unified brain model for robotic manipulation from abstract to concrete,

    Y . Ji, H. Tan, J. Shi, X. Hao, Y . Zhang, H. Zhang, P. Wang, M. Zhao, Y . Mu, P. Anet al., “Robobrain: A unified brain model for robotic manipulation from abstract to concrete,” in Proceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 1724–1734

    2025

  35. [35]

    Eo-1: Interleaved vision- text-action pretraining for general robot control,

    D. Qu, H. Song, Q. Chen, Z. Chen, X. Gao, X. Ye, Q. Lv, M. Shi, G. Ren, C. Ruanet al., “Eo-1: Interleaved vision- text-action pretraining for general robot control,”arXiv preprint arXiv:2508.21112, 2025

    arXiv 2025

  36. [36]

    Training strategies for efficient embodied reasoning,

    W. Chen, S. Belkhale, S. Mirchandani, O. Mees, D. Driess, K. Pertsch, and S. Levine, “Training strategies for efficient embodied reasoning,”arXiv preprint arXiv:2505.08243, 2025

    Pith/arXiv arXiv 2025

  37. [37]

    Instructvla: Vision-language- action instruction tuning from understanding to manipulation,

    S. Yang, H. Li, Y . Chen, B. Wang, Y . Tian, T. Wang, H. Wang, F. Zhao, Y . Liao, and J. Pang, “Instructvla: Vision-language- action instruction tuning from understanding to manipulation,” arXiv preprint arXiv:2507.17520, 2025

    arXiv 2025

  38. [38]

    Robovqa: Multimodal long-horizon reasoning for robotics,

    P. Sermanet, T. Ding, J. Zhao, F. Xia, D. Dwibedi, K. Gopalakr- ishnan, C. Chan, G. Dulac-Arnold, S. Maddineni, N. J. Joshiet al., “Robovqa: Multimodal long-horizon reasoning for robotics,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 645–652

    2024

  39. [39]

    Egoplan-bench: Benchmarking multimodal large language models for human-level planning,

    Y . Chen, Y . Ge, Y . Ge, M. Ding, B. Li, R. Wang, R. Xu, Y . Shan, and X. Liu, “Egoplan-bench: Benchmarking multimodal large language models for human-level planning,”arXiv preprint arXiv:2312.06722, 2023

    Pith/arXiv arXiv 2023

  40. [40]

    Mmro: Are multimodal llms eligible as the brain for in-home robotics?

    J. Li, Y . Zhu, Z. Xu, J. Gu, M. Zhu, X. Liu, N. Liu, Y . Peng, F. Feng, and J. Tang, “Mmro: Are multimodal llms eligible as the brain for in-home robotics?”arXiv preprint arXiv:2406.19693, 2024

    Pith/arXiv arXiv 2024

  41. [41]

    Egothink: Evaluating first-person perspective thinking capabil- ity of vision-language models,

    S. Cheng, Z. Guo, J. Wu, K. Fang, P. Li, H. Liu, and Y . Liu, “Egothink: Evaluating first-person perspective thinking capabil- ity of vision-language models,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 14 291–14 302

    2024

  42. [42]

    Embod- iedbench: Comprehensive benchmarking multi-modal large lan- guage models for vision-driven embodied agents,

    R. Yang, H. Chen, J. Zhang, M. Zhao, C. Qian, K. Wang, Q. Wang, T. V . Koripella, M. Movahedi, M. Liet al., “Embod- iedbench: Comprehensive benchmarking multi-modal large lan- guage models for vision-driven embodied agents,”arXiv preprint arXiv:2502.09560, 2025

    Pith/arXiv arXiv 2025

  43. [43]

    Robospatial: Teaching spatial understanding to 2d and 3d vision-language models for robotics,

    C. H. Song, V . Blukis, J. Tremblay, S. Tyree, Y . Su, and S. Birchfield, “Robospatial: Teaching spatial understanding to 2d and 3d vision-language models for robotics,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 15 768–15 780

    2025

  44. [44]

    Openeqa: Embodied question answering in the era of foundation models,

    A. Majumdar, A. Ajay, X. Zhang, P. Putta, S. Yenamandra, M. Henaff, S. Silwal, P. Mcvay, O. Maksymets, S. Arnaud et al., “Openeqa: Embodied question answering in the era of foundation models,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 16 488– 16 498

    2024

  45. [45]

    Robopoint: A vision-language model for spatial affordance prediction for robotics,

    W. Yuan, J. Duan, V . Blukis, W. Pumacay, R. Krishna, A. Murali, A. Mousavian, and D. Fox, “Robopoint: A vision-language model for spatial affordance prediction for robotics,”arXiv preprint arXiv:2406.10721, 2024

    Pith/arXiv arXiv 2024

  46. [46]

    Manipbench: Benchmarking vision-language models for low-level robot manipulation,

    E. Zhao, V . Raval, H. Zhang, J. Mao, Z. Shangguan, S. Niko- laidis, Y . Wang, and D. Seita, “Manipbench: Benchmarking vision-language models for low-level robot manipulation,”arXiv preprint arXiv:2505.09698, 2025

    Pith/arXiv arXiv 2025

  47. [47]

    Robobench: A com- prehensive evaluation benchmark for multimodal large language models as embodied brain,

    Y . Luo, C.-K. Fan, M. Dong, J. Shi, M. Zhao, B.-W. Zhang, C. Chi, J. Liu, G. Dai, R. Zhanget al., “Robobench: A com- prehensive evaluation benchmark for multimodal large language models as embodied brain,”arXiv preprint arXiv:2510.17801, 2025

    Pith/arXiv arXiv 2025

  48. [48]

    Discrete diffusion vla: Bringing discrete diffusion to action decoding in vision-language-action policies,

    Z. Liang, Y . Li, T. Yang, C. Wu, S. Mao, L. Pei, X. Yang, J. Pang, Y . Mu, and P. Luo, “Discrete diffusion vla: Bringing discrete diffusion to action decoding in vision-language-action policies,” arXiv preprint arXiv:2508.20072, 2025

    Pith/arXiv arXiv 2025

  49. [49]

    Vla- 0: Building state-of-the-art vlas with zero modification,

    A. Goyal, H. Hadfield, X. Yang, V . Blukis, and F. Ramos, “Vla- 0: Building state-of-the-art vlas with zero modification,”arXiv preprint arXiv:2510.13054, 2025

    arXiv 2025

  50. [50]

    Octo: An open-source generalist robot policy,

    D. G. Octo, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, C. Xu, J. Luo, T. Kreimanet al., “Octo: An open-source generalist robot policy,” inProceedings of Robotics: Science and Systems (RSS), 2024

    2024

  51. [51]

    Gr00t n1: An open foundation model for generalist humanoid robots,

    J. Bjorck, F. Casta ˜neda, N. Cherniadev, X. Da, R. Ding, L. Fan, Y . Fang, D. Fox, F. Hu, S. Huanget al., “Gr00t n1: An open foundation model for generalist humanoid robots,”arXiv preprint arXiv:2503.14734, 2025

    Pith/arXiv arXiv 2025

  52. [52]

    Memoryvla: Perceptual- cognitive memory in vision-language-action models for robotic manipulation,

    H. Shi, B. Xie, Y . Liu, L. Sun, F. Liu, T. Wang, E. Zhou, H. Fan, X. Zhang, and G. Huang, “Memoryvla: Perceptual- cognitive memory in vision-language-action models for robotic manipulation,”arXiv preprint arXiv:2508.19236, 2025

    Pith/arXiv arXiv 2025

  53. [53]

    Embodiedeval: Evaluate multimodal llms as embodied agents,

    Z. Cheng, Y . Tu, R. Li, S. Dai, J. Hu, S. Hu, J. Li, Y . Shi, T. Yu, W. Chen, L. Shi, and M. Sun, “Embodiedeval: Evaluate multimodal llms as embodied agents,” 2025. [Online]. Available: https://arxiv.org/abs/2501.11858

    Pith/arXiv arXiv 2025

  54. [54]

    Cosmos- reason1: From physical common sense to embodied reasoning,

    A. Azzolini, J. Bai, H. Brandon, J. Cao, P. Chattopadhyay, H. Chen, J. Chu, Y . Cui, J. Diamond, Y . Dinget al., “Cosmos- reason1: From physical common sense to embodied reasoning,” arXiv preprint arXiv:2503.15558, 2025

    Pith/arXiv arXiv 2025

  55. [55]

    Videgothink: Assessing egocentric video understanding capabilities for embodied ai,

    S. Cheng, K. Fang, Y . Yu, S. Zhou, B. Li, Y . Tian, T. Li, L. Han, and Y . Liu, “Videgothink: Assessing egocentric video understanding capabilities for embodied ai,”arXiv preprint arXiv:2410.11623, 2024

    Pith/arXiv arXiv 2024

  56. [56]

    Flow straight and fast: Learning to generate and transfer data with rectified flow,

    X. Liu, C. Gong, and Q. Liu, “Flow straight and fast: Learning to generate and transfer data with rectified flow,”arXiv preprint arXiv:2209.03003, 2022

    Pith/arXiv arXiv 2022

  57. [57]

    Scalable diffusion models with trans- formers,

    W. Peebles and S. Xie, “Scalable diffusion models with trans- formers,” inProceedings of the IEEE/CVF international confer- ence on computer vision, 2023, pp. 4195–4205

    2023

  58. [58]

    Language model beats diffusion–tokenizer is key to visual generation,

    L. Yu, J. Lezama, N. B. Gundavarapu, L. Versari, K. Sohn, D. Minnen, Y . Cheng, V . Birodkar, A. Gupta, X. Guet al., “Language model beats diffusion–tokenizer is key to visual generation,”arXiv preprint arXiv:2310.05737, 2023

    Pith/arXiv arXiv 2023

  59. [59]

    Cambrian- 1: A fully open, vision-centric exploration of multimodal llms,

    P. Tong, E. Brown, P. Wu, S. Woo, A. J. V . IYER, S. C. Akula, S. Yang, J. Yang, M. Middepogu, Z. Wanget al., “Cambrian- 1: A fully open, vision-centric exploration of multimodal llms,” Advances in Neural Information Processing Systems, vol. 37, pp. 87 310–87 356, 2024

    2024

  60. [60]

    Llava-onevision: Easy visual task transfer,

    B. Li, Y . Zhang, D. Guo, R. Zhang, F. Li, H. Zhang, K. Zhang, P. Zhang, Y . Li, Z. Liuet al., “Llava-onevision: Easy visual task transfer,”arXiv preprint arXiv:2408.03326, 2024

    Pith/arXiv arXiv 2024

  61. [61]

    Describe anything: Detailed localized image and video captioning,

    L. Lian, Y . Ding, Y . Ge, S. Liu, H. Mao, B. Li, M. Pavone, Draft M.-Y . Liu, T. Darrell, A. Yalaet al., “Describe anything: Detailed localized image and video captioning,”arXiv preprint arXiv:2504.16072, 2025

    Pith/arXiv arXiv 2025

  62. [62]

    Cogvlm: Visual expert for pretrained language models,

    W. Wang, Q. Lv, W. Yu, W. Hong, J. Qi, Y . Wang, J. Ji, Z. Yang, L. Zhao, S. XiXuanet al., “Cogvlm: Visual expert for pretrained language models,”Advances in Neural Information Processing Systems, vol. 37, pp. 121 475–121 499, 2024

    2024

  63. [63]

    xgen- mm (blip-3): A family of open large multimodal models,

    L. Xue, M. Shu, A. Awadalla, J. Wang, A. Yan, S. Purush- walkam, H. Zhou, V . Prabhu, Y . Dai, M. S. Ryooet al., “xgen- mm (blip-3): A family of open large multimodal models,”arXiv preprint arXiv:2408.08872, 2024

    arXiv 2024

  64. [64]

    Robo2vlm: Visual question answering from large- scale in-the-wild robot manipulation datasets,

    K. Chen, S. Xie, Z. Ma, P. R. Sanketi, and K. Gold- berg, “Robo2vlm: Visual question answering from large- scale in-the-wild robot manipulation datasets,”arXiv preprint arXiv:2505.15517, 2025

    Pith/arXiv arXiv 2025

  65. [65]

    Embspatial- bench: Benchmarking spatial understanding for embodied tasks with large vision-language models,

    M. Du, B. Wu, Z. Li, X.-J. Huang, and Z. Wei, “Embspatial- bench: Benchmarking spatial understanding for embodied tasks with large vision-language models,” inProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2024, pp. 346–355

    2024

  66. [66]

    Geniesim,

    G. Team, “Geniesim,” 2025. [Online]. Available: https: //github.com/AgibotTech/genie sim

    2025