Pith. sign in

REVIEW 4 major objections 6 minor 169 references

One 8B robot model can run perception, planning, action, and progress monitoring as a single closed loop.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 19:15 UTC pith:EHOD6VFY

load-bearing objection Solid four-function closed-loop 8B robot brain with a practical merge recipe; the fifth function (self-improvement) is mostly harness and oracle flywheel, not backbone self-evolution. the 4 major comments →

arxiv 2607.04426 v1 pith:EHOD6VFY submitted 2026-07-05 cs.RO

ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AI

ACE-Brain Team: Ziyang Gong , Haoming Gu , Zehang Luo , Tianyi Zhang , Tao Tao , Yixiao Chi , Zhe Liu , Lingsi Zhu
show 22 more authors
Jingyuan Liu Anke Tang Songze Li Yilun Kong Ningjing Liu Tianyu Zhu Yunpeng Qing Shuang Luo Xiang Liu Shi Fu Dawei Nie Sixiang Liu Zhexi Wen Feng Pan Xiaofeng Wang Zhi Hou Chunxiao Liu Xue Yang Junchi Yan Hengshuang Zhao Dacheng Tao Xiaogang Wang
This is my paper
classification cs.RO
keywords embodied foundation modelphysical agentic AIspatial intelligencevision-language-actiontask-vector mergingprogress estimationself-improving robotsSSR+
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that general physical agentic AI needs one shared robot brain, not separate perception, planning, action, and evaluation modules. ACE-Brain-0.5 builds on a prior spatial-intelligence model and puts four of five core functions—spatial perception, decision making, embodied interaction, and self-monitoring—into a single 8B backbone that grounds objects, plans subgoals, navigates and manipulates, and scores execution progress. A fifth function, self-improvement, updates external task memory and recovery cases from rollouts rather than retraining the whole network. The authors introduce SSR+, a train-specialize-merge-then-reactivate recipe that fuses heterogeneous task interfaces without them cancelling each other out. Across more than fifteen benchmarks, the unified model beats its predecessor on most spatial and grounding tests, stays competitive on navigation and manipulation, and estimates progress well in both in-distribution and out-of-distribution settings.

Core claim

Spatial perception, decision making, embodied interaction, self-monitoring, and self-improvement can be unified inside one robot foundation model: a single 8B backbone realizes the first four as a Perception–Planning–Action–Evaluation loop, while self-improvement is supplied by a companion framework that revises external execution state from experience, and joint training under SSR+ yields gains over the prior understanding-only model and competitive results against specialist systems.

What carries the argument

SSR+ (Scaffold–Specialize–Reconcile–Reactivate): start from a spatial scaffold, train specialist checkpoints for QA, grounding, navigation, and progress, merge them by task-vector fusion that preserves intermediate representations, then run a short mixed fine-tune to re-align output formats so one model can switch interfaces without cross-task collapse.

Load-bearing premise

That after merging specialists, the model still holds the real task knowledge and only needs a brief fine-tune to fix output formats, and that updating external memory and recovery cases from rollouts counts as self-improvement without changing the model’s own weights.

What would settle it

Train the same data with plain mixed supervised fine-tuning versus SSR+; if the merged-then-reactivated model loses navigation success, manipulation success, or progress correlation relative to specialists or mixed SFT, or if oracle-free closed-loop rollouts show no gain after external-state updates, the unification claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Robot stacks can drop multi-model orchestration for grounding, planning, control, and progress scoring and run them from one backbone.
  • Spatial pretraining becomes the shared initialization for navigation, manipulation, and reward-style progress estimation.
  • Progress scores from the same model can serve as rewards or recovery triggers in closed-loop training.
  • Self-improvement can start as lightweight updates to task schemas and failure cases rather than full weight retraining.
  • Cross-task interference in multi-interface robot models can be managed by merge-then-reactivate instead of only data mixing.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If Reactivate only restores formats, similar merge-and-calibrate pipelines may scale to more action interfaces (mobile base, dual-arm, whole-body) without full multi-task retraining.
  • Strong OOD progress estimation suggests the same backbone could grade other agents’ trajectories as a shared critic, not only its own rollouts.
  • Keeping self-improvement outside the weights may trade long-term policy evolution for deployability; a natural next test is whether external H updates eventually demand model-level fine-tuning to keep improving.
  • Driving-benchmark dips while spatial and interaction gains rise imply a capacity trade-off: unified brains may need explicit load balancing across domains as more interfaces are added.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. ACE-Brain-0.5 claims a Unified Embodied Foundation Model that organizes robot intelligence into five functions—spatial perception, decision making, embodied interaction, self-monitoring, and self-improvement—on a single 8B backbone built from ACE-Brain-0. The first four are realized as a Perception–Planning–Action–Evaluation loop via an omni-vision encoder, LLM decoder, Fast Vision pathway, and flow-matching Action Expert; heterogeneous interfaces are unified by SSR+ (Scaffold–Specialize–Reconcile plus Reactivate after task-vector merging). Self-improvement is a companion framework that updates external execution state H and a navigation-evolving flywheel that uses oracle-recovered corrective trajectories. Across 15+ benchmarks the model improves ACE-Brain-0 on most spatial/grounding tasks, is competitive on VLN-CE and LIBERO/SimplerEnv, and leads on RBM-EVAL progress estimation including reverse-trajectory refined splits.

Significance. If the results hold, the paper is a substantial systems contribution: it shows that spatial QA, 3D grounding, navigation, continuous manipulation, and progress estimation can share one backbone without catastrophic interference, with SSR+ as a practical multi-interface recipe and RBM-EVAL-Refined as a useful temporal-direction control. Strengths include broad external benchmarks (Tables 2–6), competitive or SOTA manipulation (Tables 4–5), strong progress VOC including OOD and refined reverse trajectories (Table 6, Fig. 3), and an explicit architecture for low-latency control (Fast Vision + Action Expert). The five-function framing and Table 1 Self-Improving checkmark overstate what the single model learns relative to the external/oracle self-improvement path, but the four-function closed loop is still a clear step beyond fragmented VLA or agent stacks.

major comments (4)
  1. Abstract, §3.3 Eq. (10), and Table 1: Self-improvement is defined as updates to external H (schemas, spatial memory, failure cases) without weight updates, and §4.5 Navigation Evolving relies on an oracle teacher that detects deviations and produces Devo recovery trajectories before re-imitation (Table 7). That is a closed-loop data flywheel plus harness adaptation, not model-level self-evolution of the backbone that implements the other four functions. Table 1’s full ✓ for Self-Improving and the five-function “unified in one model” claim therefore rest on a broader definition than the four-function loop. Either restrict Self-Improving to partial support / companion framework, or add ablations that improve without oracle recovery and with model-weight adaptation from self-generated experience.
  2. §3.2 Stages 3–4 and Appendix A.5: The load-bearing claim that task-vector merging preserves specialist semantic knowledge so that only output-format conventions need short Reactivate SFT is asserted empirically (“central empirical finding”) and via transfer bounds adapted from ACE-Brain-0, but the manuscript does not report pre- vs post-merge vs post-Reactivate metrics per interface, nor Reactivate step budget / Dmix composition. Without those ablations, it is hard to separate true multi-task unification from residual specialist capacity plus light mixed SFT. A compact table of θ_i / θ_merge / θ_0.5 on grounding, nav, progress, and spatial QA would make the SSR+ claim falsifiable.
  3. Table 2 driving block and §4.2: ACE-Brain-0.5 drops relative to ACE-Brain-0 on several decision-centric driving metrics (e.g., MAPLM 71.3 vs 77.8, NuScenesQA 44.7 vs 58.8, LingoQA 55.6 vs 65.8, MME-RealWorld 66.6 vs 71.2). The text acknowledges moderate drops but still markets unified decision making. Either quantify the multi-task trade-off (SSR+ vs single-task specialists on the same backbone) or narrow the decision-making claim to planning/subgoal decomposition rather than driving-specialized QA.
  4. Tables 3–5: Navigation and manipulation results mix the unified model with ACE-Brain-0.5-Specialist (VLN) and ACE-Brain-0.5-VLA (SimplerEnv; backbone fully fine-tuned, no FastVision, and without ACE-Brain-0.5 manipulation pretraining weights). The central “single unified architecture without task-specific architectural modifications” claim (§4 intro) is therefore only partially supported for embodied interaction. Report the frozen-backbone unified model on SimplerEnv and the non-specialist unified model as the primary numbers; treat specialist/VLA variants as ablations.
minor comments (6)
  1. Table 1 legend and Self-Improving row: align the ✓ definition with the external-H / oracle description in §3.3–§4.5 so the comparison table does not over-claim relative to peers.
  2. Eq. (4)–(5): clarify the relationship between the layer-wise output-discrepancy objective and the Frobenius task-vector surrogate; a one-line statement of what is optimized in FusionBench would help reproducibility.
  3. Progress supervision (§5.1): scaling labels by 1000 and T=8 sampling are free design choices; a short sensitivity note would strengthen the reward-model claim.
  4. Appendix A.4 / Table 8: RBM-EVAL-Refined is a useful control; state selection criteria and whether reverse trajectories are used only at eval or also in training.
  5. Presentation: long nested pie charts in Fig. 5 and dense multi-model Table 2 would benefit from clearer ranking highlights and consistent asterisks for numbers taken from other papers.
  6. Date line “July 7, 2026” / arXiv 2607.04426: ensure consistency with camera-ready metadata.

Circularity Check

0 steps flagged

No significant circularity: empirical multi-task systems paper evaluated on external benchmarks; self-improvement is explicitly external/oracle-aided rather than a definitional self-prediction.

full rationale

ACE-Brain-0.5 is an empirical foundation-model paper. The central claims (single 8B backbone for spatial perception, decision making, embodied interaction, and self-monitoring via SSR+; competitive results on VSI, MMSI, MindCube, Scan*, VLN-CE, LIBERO, SimplerEnv-Bridge, RBM-EVAL, etc.) are supported by training on public and in-house corpora and evaluation against external or held-out benchmarks and baselines. SSR+ is a training recipe (scaffold from ACE-Brain-0, specialize, task-vector merge, short Reactivate SFT); the Reactivate recovery claim is an empirical observation about merged weights, not a quantity fitted then re-predicted. Self-improvement is stated as a companion framework that updates external execution state H (Eq. 10) and a navigation flywheel that uses an oracle teacher to build corrective Devo trajectories (Table 7)—transparent closed-loop data collection, not a self-definitional prediction of the model’s own success. RBM-EVAL-Refined adds reversed trajectories as negative controls; that is a controlled test design, not fitting the VOC metric to the model. Self-citation of ACE-Brain-0 (scaffold, SSR, Appendix A.5 theorems) and Robometer-style data is normal prior-work scaffolding and does not force the multi-benchmark results by construction. No step reduces a claimed first-principles prediction to its own fitted input or to an unverified uniqueness theorem. Score 1 reflects only routine self-citation of the prior scaffold, not load-bearing circularity.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 3 invented entities

Empirical robotics/ML systems paper. Load-bearing content is architectural choices, multi-task merge recipe, and large heterogeneous datasets rather than free physical constants. Domain assumptions (spatial scaffold transfers to action; flow-matching action expert; external H as self-improvement) are stated or implied; free parameters are standard training/merge knobs.

free parameters (4)
  • SSR+ merge iterations and layer-wise objective weights = 1000 iterations
    1,000 data-free Adam iterations via FusionBench; layer-wise residual interference objective is a design choice that shapes the merged checkpoint.
  • Reactivate mixed-SFT step budget and Dmix composition
    Lightweight post-merge fine-tune size and per-interface sampling ratios are chosen to restore formats without re-specializing.
  • Action chunk length / flow-matching schedule and FastVision injection = 10-step action sequence (2B variant detail)
    10-step chunks, DINOv3 FastVision path, frozen VLM during action training are engineering hyperparameters controlling control latency and performance.
  • Progress token scaling (×1000) and T=8 frame sampling = T=8; scores scaled to [0,1000]
    Autoregressive progress representation and frame sampling strategy for RBM-1M are design choices affecting supervision density.
axioms (4)
  • domain assumption Spatial intelligence is a transferable shared scaffold across heterogeneous robot platforms and tasks.
    Inherited from ACE-Brain-0 and used as Stage-1 initialization for all specialists (§1, §3.2).
  • ad hoc to paper Task-vector merging approximately preserves specialist intermediate representations; residual loss is mainly output-format desynchronization.
    Central empirical premise of Reactivate (§3.2 Stage 4); supported by recovery observations but not proved for all interfaces.
  • domain assumption Flow-matching action experts conditioned on cached LLM state plus fast vision features suffice for closed-loop manipulation/navigation control.
    Follows π0-style design (§3.1); standard in recent VLA literature.
  • ad hoc to paper External execution-state updates (schemas, spatial memory, failure cases) plus oracle-recovered trajectories count as Self Improvement for Physical Agentic AI.
    Defines the fifth function without requiring continual weight updates (§3.3, §4.5).
invented entities (3)
  • SSR+ (Scaffold–Specialize–Reconcile–Reactivate) no independent evidence
    purpose: Unify heterogeneous robot interfaces in one parameter space without cross-task interference.
    Extends authors’ SSR with a post-merge Reactivate stage; independent evidence is the reported multi-benchmark recovery, not external theory.
  • Five-function Unified Embodied Foundation Model paradigm no independent evidence
    purpose: Organize robot intelligence as spatial perception, decision making, embodied interaction, self-monitoring, self-improvement.
    Framing device for architecture and evaluation; not a physical entity with external measurement.
  • RBM-EVAL-Refined reverse-trajectory control set no independent evidence
    purpose: Test temporal/action-direction understanding beyond monotonic progress shortcuts.
    Author-constructed complement to Robometer splits; useful control but paper-defined.

pith-pipeline@v1.1.0-grok45 · 56308 in / 3260 out tokens · 41987 ms · 2026-07-11T19:15:42.263060+00:00 · methodology

0 comments
read the original abstract

Embodied AI is moving from isolated perception or action modules toward physical agents that understand, plan under goals, act through robot bodies, monitor progress, and improve from experience. Existing systems address this loop only in parts: end-to-end policies generate actions but often lack spatial reasoning, planning, and execution assessment, while robot-agent systems orchestrate tools or specialists but do not learn a shared representation. This fragmentation limits general Physical Agentic AI. We present ACE-Brain-0.5, a unified embodied foundation model that organizes robot intelligence into five coupled functions: spatial perception, decision making, embodied interaction, self-monitoring, and self-improvement. Built on ACE-Brain-0, which established spatial intelligence as a shared scaffold across robot platforms, ACE-Brain-0.5 extends an understanding-centric model into a closed-loop foundation model. A single 8B backbone instantiates the first four functions: grounding objects and affordances, reasoning over 3D and egocentric spatial relations, decomposing instructions into subgoals, generating navigation and manipulation actions, and estimating progress for verification and recovery. To unify these capabilities without cross-task interference, we introduce SSR+, which extends Scaffold-Specialize-Reconcile with a Reactivate stage after task-vector merging. The fifth function, self-improvement, is realized by a companion framework that updates external execution state, including task schemas, spatial memory, and failure-recovery cases, from rollouts. Across fifteen benchmarks, ACE-Brain-0.5 improves over ACE-Brain-0 on 14 of 18 spatial perception and grounding benchmarks, achieves competitive navigation and manipulation performance, and provides strong progress estimation in ID and OOD settings. Together, these results mark an early step toward general Physical Agentic AI.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

169 extracted references · 78 linked inside Pith

  1. [1]

    A survey of embodied ai: From simulators to research tasks

    Jiafei Duan, Samson Yu, Hui Li Tan, Hongyuan Zhu, and Cheston Tan. A survey of embodied ai: From simulators to research tasks. IEEE Transactions on Emerging Topics in Computational Intelligence, 6(2):230–244, 2022

    2022

  2. [2]

    A survey on robotics with foundation models: Toward embodied ai

    Zhiyuan Xu, Kun Wu, Junjie Wen, Jinming Li, Ning Liu, Zhengping Che, and Jian Tang. A survey on robotics with foundation models: Toward embodied ai. arXiv preprint arXiv:2402.02385, 2024

    Pith/arXiv arXiv 2024

  3. [3]

    Robotwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation

    Tianxing Chen, Zanxin Chen, Baijun Chen, Zijian Cai, Yibin Liu, Zixuan Li, Qiwei Liang, Xianliang Lin, Yiheng Ge, Zhenyu Gu, et al. Robotwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation. arXiv preprint arXiv:2506.18088, 2025

    Pith/arXiv arXiv 2025

  4. [4]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479, 2025

    Pith/arXiv arXiv 2025

  5. [5]

    Qwen3-vl technical report

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan ...

    Pith/arXiv arXiv 2025

  6. [6]

    Qwen3 technical report

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025

    Pith/arXiv arXiv 2025

  7. [7]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023

    Pith/arXiv arXiv 2023

  8. [8]

    Qwen-robotnav technical report: A scalable navigation model designed for an agentic navigation system

    Jiazhao Zhang, Gengze Zhou, Hale Yin, Yiyang Huang, Zixing Lei, Qihang Peng, Haoqi Yuan, Jie Zhang, Xudong Guo, Xiaoyue Chen, et al. Qwen-robotnav technical report: A scalable navigation model designed for an agentic navigation system. arXiv preprint arXiv:2606.18112, 2026

    Pith/arXiv arXiv 2026

  9. [9]

    Embodied navigation foundation model

    Jiazhao Zhang, Anqi Li, Yunpeng Qi, Minghan Li, Jiahang Liu, Shaoan Wang, Haoran Liu, Gengze Zhou, Yuze Wu, Xingxing Li, et al. Embodied navigation foundation model. arXiv preprint arXiv:2509.12129, 2025

    arXiv 2025

  10. [10]

    π0: A vision-language-action flow model for general robot control

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. π0: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164, 2024

    Pith/arXiv arXiv 2024

  11. [11]

    World action models are zero-shot policies

    Seonghyeon Ye, Yunhao Ge, Kaiyuan Zheng, Shenyuan Gao, Sihyun Yu, George Kurian, Suneel Indupuru, You Liang Tan, Chuning Zhu, Jiannan Xiang, et al. World action models are zero-shot policies. arXiv preprint arXiv:2602.15922, 2026

    Pith/arXiv arXiv 2026

  12. [12]

    RoboReward: General-purpose vision-language reward models for robotics

    Tony Lee et al. RoboReward: General-purpose vision-language reward models for robotics. arXiv preprint arXiv:2601.00675, 2026

    arXiv 2026

  13. [13]

    RoboMeter: Scaling general-purpose robotic reward models via trajectory comparisons

    Anthony Liang et al. RoboMeter: Scaling general-purpose robotic reward models via trajectory comparisons. arXiv preprint arXiv:2603.02115, 2026

    Pith/arXiv arXiv 2026

  14. [14]

    Nils J. Nilsson. Shakey the robot. In SRI International Technical Note 323. SRI International, 1984

    1984

  15. [15]

    Rodney A. Brooks. A robust layered control system for a mobile robot. IEEE Journal on Robotics and Automation, 2(1):14–23, 1986

    1986

  16. [16]

    Robin R. Murphy. Introduction to AI Robotics. MIT Press, 2000

    2000

  17. [17]

    π0.5: A vision-language-action model with open-world generalization

    Physical Intelligence, Kevin Black, et al. π0.5: A vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054, 2025. 24

    Pith/arXiv arXiv 2025

  18. [18]

    Interleave-vla: Enhancing robot manipulation with interleaved image-text instructions, 2025

    Cunxin Fan, Xiaosong Jia, Yihang Sun, Yixiao Wang, Jianglan Wei, Ziyang Gong, Xiangyu Zhao, Masayoshi Tomizuka, Xue Yang, Junchi Yan, and Mingyu Ding. Interleave-vla: Enhancing robot manipulation with interleaved image-text instructions, 2025. URLhttps://arxiv.org/abs/2505.02152

    arXiv 2025

  19. [19]

    Do as i can, not as i say: Grounding language in robotic affordances

    Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Karol Gopalakrishnan, Karol Hausman, et al. Do as i can, not as i say: Grounding language in robotic affordances. In Conference on Robot Learning (CoRL), 2022

    2022

  20. [20]

    PaLM-E: An embodied multimodal language model

    Danny Driess et al. PaLM-E: An embodied multimodal language model. In International Conference on Machine Learning (ICML), 2023

    2023

  21. [21]

    Code as policies: Language model programs for embodied control

    Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as policies: Language model programs for embodied control. In IEEE International Conference on Robotics and Automation (ICRA), pages 9493–9500, 2023

    2023

  22. [22]

    Voxposer: Composable 3d value maps for robotic manipulation with language models

    Wenlong Huang, Chen Wang, Ruohan Zhang, Yunzhu Li, Jiajun Wu, and Li Fei-Fei. Voxposer: Composable 3d value maps for robotic manipulation with language models. In Conference on Robot Learning (CoRL), 2023

    2023

  23. [23]

    RoboCodeX: Multimodal code generation for robotic behavior synthesis

    Yao Mu et al. RoboCodeX: Multimodal code generation for robotic behavior synthesis. In International Conference on Machine Learning (ICML), 2024

    2024

  24. [24]

    RoboAgent: Chaining basic capabilities for embodied task planning

    Peiran Xu, Jiaqi Zheng, and Yadong Mu. RoboAgent: Chaining basic capabilities for embodied task planning. arXiv preprint arXiv:2604.07774, 2026

    Pith/arXiv arXiv 2026

  25. [25]

    Gr00t n1: An open foundation model for generalist humanoid robots

    NVIDIA, Johan Bjorck, Fernando Casta˜ neda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, Joel Jang, Zhenyu Jiang, Jan Kautz, Kaushil Kundalia, Lawrence Lao, Zhiqi Li, Zongyu Lin, Kevin Lin, Guilin Liu, Edith Llontop, Loic Magne, Ajay Mandlekar, Avnish Narayan, Soroush Nasiriany, Scott Reed, You Lian...

    Pith/arXiv arXiv doi:10.48550/arxiv.2503.14734 2025

  26. [26]

    Gr00t n1.5: An improved open foundation model for generalist humanoid robots

    NVIDIA. Gr00t n1.5: An improved open foundation model for generalist humanoid robots. https://research. nvidia.com/labs/gear/gr00t-n1_5/, 2025. NVIDIA GEAR Lab technical blog

    2025

  27. [27]

    Qwen-vla: Unifying vision-language-action modeling across tasks, environments, and robot embodiments

    Qiuyue Wang, Mingsheng Li, Jian Guan, Jinhui Ye, Sicheng Xie, Yitao Liu, Junhao Chen, Zhixuan Liang, Jie Zhang, Xintong Hu, et al. Qwen-vla: Unifying vision-language-action modeling across tasks, environments, and robot embodiments. arXiv preprint arXiv:2605.30280, 2026

    Pith/arXiv arXiv 2026

  28. [28]

    Qwen-robotworld technical report: Unifying embodied world modeling through language- conditioned video generation

    Jie Zhang, Xiaoyue Chen, Anzhe Chen, Chenxu Lv, Deqing Li, Gengze Zhou, Hang Yin, Haoqi Yuan, Haoyang Li, Jiahao Li, et al. Qwen-robotworld technical report: Unifying embodied world modeling through language- conditioned video generation. arXiv preprint arXiv:2606.17030, 2026

    Pith/arXiv arXiv 2026

  29. [29]

    Abot-m0: Vla foundation model for robotic manipulation with action manifold learning

    Yandan Yang, Shuang Zeng, Tong Lin, Xinyuan Chang, Dekang Qi, Junjin Xiao, Haoyun Liu, Ronghan Chen, Yuzhi Chen, Dongjie Huo, et al. Abot-m0: Vla foundation model for robotic manipulation with action manifold learning. arXiv preprint arXiv:2602.11236, 2026

    Pith/arXiv arXiv 2026

  30. [30]

    Rynnbrain: Open embodied foundation models

    Ronghao Dang, Jiayan Guo, Bohan Hou, Sicong Leng, Kehan Li, Xin Li, Jiangpin Liu, Yunxuan Mao, Zhikai Wang, Yuqian Yuan, et al. Rynnbrain: Open embodied foundation models. arXiv preprint arXiv:2602.14979, 2026

    arXiv 2026

  31. [31]

    Cosmos 3: Omnimodal world models for physical ai

    Niket Agarwal, Arslan Ali, Jon Allen, Martin Antolini, Adeline Aubame, Alisson Azzolini, Junjie Bai, Maciej Bala, Yogesh Balaji, Josh Bapst, et al. Cosmos 3: Omnimodal world models for physical ai. arXiv preprint arXiv:2606.02800, 2026

    Pith/arXiv arXiv 2026

  32. [32]

    Ace-brain-0: Spatial intelligence as a shared scaffold for universal embodiments

    Ziyang Gong, Zehang Luo, Anke Tang, Zhe Liu, Shi Fu, Zhi Hou, Ganlin Yang, Weiyun Wang, Xiaofeng Wang, Jianbo Liu, Gen Luo, Haolan Kang, Shuang Luo, Yue Zhou, Yong Luo, Li Shen, Xiaosong Jia, Yao Mu, Xue Yang, Chunxiao Liu, Junchi Yan, Hengshuang Zhao, Dacheng Tao, and Xiaogang Wang. Ace-brain-0: Spatial intelligence as a shared scaffold for universal emb...

    arXiv 2026

  33. [33]

    Pelican-unify 1.0: A unified embodied intelligence model for understanding, reasoning, imagination and action

    Yi Zhang, Yinda Chen, Che Liu, Zeyuan Ding, Jin Xu, Shilong Zou, Junwei Liao, Jiayu Hu, Xiancong Ren, Xiaopeng Zhang, et al. Pelican-unify 1.0: A unified embodied intelligence model for understanding, reasoning, imagination and action. arXiv preprint arXiv:2605.15153, 2026. 25

    Pith/arXiv arXiv 2026

  34. [34]

    Hy-embodied-0.5: Embodied foundation models for real-world agents

    HY Team, Xumin Yu, Zuyan Liu, Ziyi Wang, He Zhang, Yongming Rao, Fangfu Liu, Yani Zhang, Ruowen Zhao, Oran Wang, et al. Hy-embodied-0.5: Embodied foundation models for real-world agents. arXiv preprint arXiv:2604.07430, 2026

    Pith/arXiv arXiv 2026

  35. [35]

    Physical Intelligence, Bo Ai, Ali Amin, R Aniceto, A Balakrishna, G Balke, K Black, G Bokinsky, S Cao, T Charbonnier, et al. π0. 7: a steerable generalist robotic foundation model with emergent capabilities, 2026. URL https://arxiv. org/abs/2604.15483

    Pith/arXiv arXiv 2026

  36. [36]

    π∗ 0.6: a vla that learns from experience

    Physical Intelligence, Ali Amin, Raichelle Aniceto, Ashwin Balakrishna, Kevin Black, Ken Conley, Grace Connors, James Darpinian, Karan Dhabalia, Jared DiCarlo, et al. π∗ 0.6: a vla that learns from experience. arXiv preprint arXiv:2511.14759, 2025

    Pith/arXiv arXiv 2025

  37. [37]

    Embodied-r1

    Yifu Yuan, Yaoting Huang, Xianze Yao, Yutong Li, Shuoheng Zhang, Linqi Han, Pengyi Li, Jiangeng Sun, Wenting Jia, Zhao Zhang, et al. Embodied-r1. 5: Evolving physical intelligence via embodied foundation models. arXiv preprint arXiv:2606.11324, 2026

    Pith/arXiv arXiv 2026

  38. [38]

    Molmoact2: Action reasoning models for real-world deployment

    Haoquan Fang, Jiafei Duan, Donovan Clay, Sam Wang, Shuo Liu, Weikai Huang, Xiang Fan, Wei-Chuan Tsai, Shirui Chen, Yi Ru Wang, et al. Molmoact2: Action reasoning models for real-world deployment. arXiv preprint arXiv:2605.02881, 2026

    Pith/arXiv arXiv 2026

  39. [39]

    Qwen-robotmanip technical report: Alignment unlocks scale for robotic manipulation foundation models

    Haoqi Yuan, Zhixuan Liang, Anzhe Chen, Ye Wang, Haoyang Li, Pei Lin, Yiyang Huang, Zixing Lei, Tong Zhang, Jiazhao Zhang, et al. Qwen-robotmanip technical report: Alignment unlocks scale for robotic manipulation foundation models. arXiv preprint arXiv:2606.17846, 2026

    Pith/arXiv arXiv 2026

  40. [40]

    Abot-n0: Technical report on the vla foundation model for versatile embodied navigation

    Zedong Chu, Shichao Xie, Xiaolong Wu, Yanfen Shen, Minghua Luo, Zhengbo Wang, Fei Liu, Xiaoxu Leng, Junjun Hu, Mingyang Yin, et al. Abot-n0: Technical report on the vla foundation model for versatile embodied navigation. arXiv preprint arXiv:2602.11598, 2026

    arXiv 2026

  41. [41]

    Gemini Robotics-ER 1.6: Powering real-world robotics tasks through enhanced em- bodied reasoning

    Laura Graesser and Peng Xu. Gemini Robotics-ER 1.6: Powering real-world robotics tasks through enhanced em- bodied reasoning. https://deepmind.google/blog/gemini-robotics-er-1-6/ , April 2026. Google DeepMind Blog, accessed June 26, 2026

    2026

  42. [42]

    Introducing helix 02: Full-body autonomy, 2026

    AI Figure. Introducing helix 02: Full-body autonomy, 2026

    2026

  43. [43]

    Model merging in llms, mllms, and beyond: Methods, theories, applications, and opportunities

    Enneng Yang, Li Shen, Guibing Guo, Xingwei Wang, Xiaochun Cao, Jie Zhang, and Dacheng Tao. Model merging in llms, mllms, and beyond: Methods, theories, applications, and opportunities. ACM Computing Surveys, 58(8):1–41, 2026

    2026

  44. [44]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025

    Pith/arXiv arXiv 2025

  45. [45]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 26296–26306, 2024

    2024

  46. [46]

    Gpt-4o system card.https://openai.com/index/gpt-4o-system-card/, 2025

    OpenAI. Gpt-4o system card.https://openai.com/index/gpt-4o-system-card/, 2025

    2025

  47. [47]

    Claude sonnet 4

    Anthropic. Claude sonnet 4. 2025

    2025

  48. [48]

    SpatialVLM: Endowing vision-language models with spatial reasoning capabilities

    Boyuan Chen et al. SpatialVLM: Endowing vision-language models with spatial reasoning capabilities. In CVPR, 2024

    2024

  49. [49]

    RoboPoint: A vision-language model for spatial affordance prediction for robotics

    Wentao Yuan et al. RoboPoint: A vision-language model for spatial affordance prediction for robotics. In Conference on Robot Learning (CoRL), 2024

    2024

  50. [50]

    RoboRefer: Towards spatial referring with reasoning in vision-language models for robotics

    Enshen Zhou et al. RoboRefer: Towards spatial referring with reasoning in vision-language models for robotics. arXiv preprint arXiv:2506.04308, 2025

    arXiv 2025

  51. [51]

    Robobrain: A unified brain model for robotic manipulation from abstract to concrete

    Yuheng Ji, Huajie Tan, Jiayu Shi, Xiaoshuai Hao, Yuan Zhang, Hengyuan Zhang, Pengwei Wang, Mengdi Zhao, Yao Mu, Pengju An, et al. Robobrain: A unified brain model for robotic manipulation from abstract to concrete. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 1724–1734, 2025

    2025

  52. [52]

    Robobrain 2.5: Depth in sight, time in mind

    Huajie Tan, Enshen Zhou, Zhiyu Li, Yijie Xu, Yuheng Ji, Xiansheng Chen, Cheng Chi, Pengwei Wang, Huizhu Jia, Yulong Ao, et al. Robobrain 2.5: Depth in sight, time in mind. arXiv preprint arXiv:2601.14352, 2026. 26

    arXiv 2026

  53. [53]

    Mimo-embodied: X-embodied foundation model technical report

    Xiaoshuai Hao, Lei Zhou, Zhijian Huang, Zhiwen Hou, Yingbo Tang, Lingfeng Zhang, Guang Li, Zheng Lu, Shuhuai Ren, Xianhui Meng, et al. Mimo-embodied: X-embodied foundation model technical report. arXiv preprint arXiv:2511.16518, 2025

    Pith/arXiv arXiv 2025

  54. [54]

    Visual embodied brain: Let multimodal large language models see, think, and control in spaces

    Gen Luo, Ganlin Yang, Ziyang Gong, Guanzhou Chen, Haonan Duan, Erfei Cui, Ronglei Tong, Zhi Hou, Tianyi Zhang, Zhe Chen, et al. Visual embodied brain: Let multimodal large language models see, think, and control in spaces. arXiv preprint arXiv:2506.00123, 2025

    Pith/arXiv arXiv 2025

  55. [55]

    Robobrain 2.0 technical report

    BAAI RoboBrain Team, Mingyu Cao, Huajie Tan, Yuheng Ji, Xiansheng Chen, Minglan Lin, Zhiyu Li, Zhou Cao, Pengwei Wang, Enshen Zhou, et al. Robobrain 2.0 technical report. arXiv preprint arXiv:2507.02029, 2025

    Pith/arXiv arXiv 2025

  56. [56]

    Vlaser: Vision-language-action model with synergistic embodied reasoning

    Ganlin Yang, Tianyi Zhang, Haoran Hao, Weiyun Wang, Yibin Liu, Dehui Wang, Guanzhou Chen, Zijian Cai, Junting Chen, Weijie Su, et al. Vlaser: Vision-language-action model with synergistic embodied reasoning. arXiv preprint arXiv:2510.11027, 2025

    arXiv 2025

  57. [57]

    Pelican-vl 1.0: A foundation brain model for embodied intelligence

    Yi Zhang, Che Liu, Xiancong Ren, Hanchu Ni, Shuai Zhang, Zeyuan Ding, Jiayu Hu, Hanzhe Shan, Zhenwei Niu, Zhaoyang Liu, et al. Pelican-vl 1.0: A foundation brain model for embodied intelligence. arXiv preprint arXiv:2511.00108, 2025

    arXiv 2025

  58. [58]

    RT-1: Robotics transformer for real-world control at scale

    Anthony Brohan et al. RT-1: Robotics transformer for real-world control at scale. In Robotics: Science and Systems (RSS), 2023

    2023

  59. [59]

    RT-2: Vision-language-action models transfer web knowledge to robotic control

    Brianna Zitkovich et al. RT-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning (CoRL), 2023

    2023

  60. [60]

    Octo: An open-source generalist robot policy

    Octo Model Team. Octo: An open-source generalist robot policy. In Robotics: Science and Systems (RSS), 2024

    2024

  61. [61]

    OpenVLA: An open-source vision-language-action model

    Moo Jin Kim et al. OpenVLA: An open-source vision-language-action model. In Conference on Robot Learning (CoRL), 2024

    2024

  62. [62]

    Open X-Embodiment: Robotic learning datasets and RT-X models

    Open X-Embodiment Collaboration, Abby O’Neill, Abdul Rehman, Abhinav Gupta, ..., Zipeng Lin, and Zubair Irshad. Open X-Embodiment: Robotic learning datasets and RT-X models. In 2024 IEEE International Conference on Robotics and Automation (ICRA), 2024

    2024

  63. [63]

    RT-H: Action hierarchies using language

    Suneel Belkhale, Tianli Ding, Ted Xiao, Pierre Sermanet, Quan Vuong, Jonathan Tompson, Yevgen Chebotar, Debidatta Dwibedi, and Dorsa Sadigh. RT-H: Action hierarchies using language. In Robotics: Science and Systems (RSS), 2024. arXiv:2403.01823

    Pith/arXiv arXiv 2024

  64. [64]

    Eo-1: Interleaved vision-text- action pretraining for general robot control

    Delin Qu, Haoming Song, Qizhi Chen, Zhaoqing Chen, Xianqiang Gao, Xinyi Ye, Qi Lv, Modi Shi, Guanghui Ren, Cheng Ruan, Maoqing Yao, Haoran Yang, Jiacheng Bao, Bin Zhao, and Dong Wang. Eo-1: Interleaved vision-text- action pretraining for general robot control. arXiv preprint, 2025. URLhttps://arxiv.org/abs/2508.21112

    arXiv 2025

  65. [65]

    CogACT: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation

    Qixiu Li et al. CogACT: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation. arXiv preprint arXiv:2411.19650, 2024

    Pith/arXiv arXiv 2024

  66. [66]

    Gr-2: A generative video-language-action model with web-scale knowledge for robot manipulation

    Chi-Lam Cheang, Guangzeng Chen, Ya Jing, Tao Kong, Hang Li, Yifeng Li, Yuxiao Liu, Hongtao Wu, Jiafeng Xu, Yichu Yang, et al. Gr-2: A generative video-language-action model with web-scale knowledge for robot manipulation. arXiv preprint arXiv:2410.06158, 2024

    Pith/arXiv arXiv 2024

  67. [67]

    GR-3 technical report

    Chilam Cheang et al. GR-3 technical report. arXiv preprint arXiv:2507.15493, 2025

    Pith/arXiv arXiv 2025

  68. [68]

    Gaze-regularized vision-language-action models for robotic manipulation

    Anupam Pani et al. Gaze-regularized vision-language-action models for robotic manipulation. arXiv preprint arXiv:2603.23202, 2026

    Pith/arXiv arXiv 2026

  69. [69]

    Vla-jepa: Enhancing vision-language-action model with latent world model, 2026

    Jingwen Sun, Wenyao Zhang, Zekun Qi, Shaojie Ren, Zezhi Liu, Hanxin Zhu, Guangzhong Sun, Xin Jin, and Zhibo Chen. Vla-jepa: Enhancing vision-language-action model with latent world model, 2026. URL https://arxiv.org/abs/2602.10098

    arXiv 2026

  70. [70]

    Cosmos policy: Fine-tuning video models for visuomotor control and planning, 2026

    Moo Jin Kim, Yihuai Gao, Tsung-Yi Lin, Yen-Chen Lin, Yunhao Ge, Grace Lam, Percy Liang, Shuran Song, Ming-Yu Liu, Chelsea Finn, and Jinwei Gu. Cosmos policy: Fine-tuning video models for visuomotor control and planning, 2026. URLhttps://arxiv.org/abs/2601.16163

    Pith/arXiv arXiv 2026

  71. [71]

    Gigaworld-policy: An efficient action-centered world–action model

    Angen Ye, Boyuan Wang, Chaojun Ni, Guan Huang, Guosheng Zhao, Hao Li, Hengtao Li, Jie Li, Jindi Lv, Jingyu Liu, et al. Gigaworld-policy: An efficient action-centered world–action model. arXiv preprint arXiv:2603.17240, 2026. 27

    arXiv 2026

  72. [72]

    Unified 4d world action modeling from video priors with asynchronous denoising

    Jun Guo, Qiwei Li, Peiyan Li, Zilong Chen, Nan Sun, Yifei Su, Heyun Wang, Yuan Zhang, Xinghang Li, and Huaping Liu. Unified 4d world action modeling from video priors with asynchronous denoising. arXiv preprint arXiv:2604.26694, 2026

    Pith/arXiv arXiv 2026

  73. [73]

    Fast-wam: Do world action models need test-time future imagination?, 2026

    Tianyuan Yuan, Zibin Dong, Yicheng Liu, and Hang Zhao. Fast-wam: Do world action models need test-time future imagination?, 2026. URLhttps://arxiv.org/abs/2603.16666

    Pith/arXiv arXiv 2026

  74. [74]

    Dreamvla: a vision-language-action model dreamed with comprehensive world knowledge

    Wenyao Zhang, Hongsi Liu, Zekun Qi, Yunnan Wang, Xinqiang Yu, Jiazhao Zhang, Runpei Dong, Jiawei He, He Wang, Zhizheng Zhang, et al. Dreamvla: a vision-language-action model dreamed with comprehensive world knowledge. Advances in Neural Information Processing Systems, 38:24195–24228, 2026

    2026

  75. [75]

    Being-h0

    Hao Luo, Wanpeng Zhang, Yicheng Feng, Sipeng Zheng, Haiweng Xu, Chaoyi Xu, Ziheng Xi, Yuhui Fu, and Zongqing Lu. Being-h0. 7: A latent world-action model from egocentric videos. arXiv preprint arXiv:2605.00078, 2026

    Pith/arXiv arXiv 2026

  76. [76]

    Rynnvla-002: A unified vision-language-action and world model

    Jun Cen, Siteng Huang, Yuqian Yuan, Kehan Li, Hangjie Yuan, Chaohui Yu, Yuming Jiang, Jiayan Guo, Xin Li, Hao Luo, et al. Rynnvla-002: A unified vision-language-action and world model. arXiv preprint arXiv:2511.17502, 2025

    Pith/arXiv arXiv 2025

  77. [77]

    Abot-m0.5: Unified mobility-and-manipulation world action model, 2026

    Ronghan Chen, Yandan Yang, Zuojin Tang, Dongjie Huo, Tong Lin, Haoning Wu, Haoyun Liu, Yuzhi Chen, Lulu Zheng, Botai Yuan, Tianlun Li, Mingxin Wang, Dekang Qi, Bin Hu, Wei Mei, Yuze Xuan, Haolong Yang, Yanqing Zhu, Mu Xu, Zhiheng Ma, and Xinyuan Chang. Abot-m0.5: Unified mobility-and-manipulation world action model, 2026. URLhttps://arxiv.org/abs/2607.00678

    Pith/arXiv arXiv 2026

  78. [78]

    Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments

    Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sunderhauf, Ian Reid, Stephen Gould, and Anton van den Hengel. Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018

    2018

  79. [79]

    Think global, act local: Dual-scale graph transformer for vision-and-language navigation

    Shizhe Chen, Pierre-Louis Guhur, Makarand Tapaswi, Cordelia Schmid, and Ivan Laptev. Think global, act local: Dual-scale graph transformer for vision-and-language navigation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16537–16547, 2022

    2022

  80. [80]

    Beyond the nav-graph: Vision- and-language navigation in continuous environments

    Jacob Krantz, Erik Wijmans, Arjun Majumdar, Dhruv Batra, and Stefan Lee. Beyond the nav-graph: Vision- and-language navigation in continuous environments. In European Conference on Computer Vision, 2020

    2020

Showing first 80 references.