Pith. sign in

REVIEW 3 major objections 6 minor 96 references

Embodied Operators and Benchmarking: Toward Reusable and Deployable Embodied Intelligence Systems

T0 review · 3 major / 6 minor · reviewed 2026-07-12 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Embodied intelligence needs reusable, deployable functional modules—not only end-to-end policies—and those modules must be optimized and judged as whole system components, not isolated neural nets.

desk verdict Useful packaging of modular robotics for the VLA era; the benchmark is a real proposal, not a demonstrated foundation. read the letter →

arxiv 2607.03283 v1 pith:CGIS7NRK submitted 2026-07-03 cs.AI

Junwu Xiong , Jiaxuan Gao , Wei Chai , Renxing Chen , Yuzhen Li , Yu Guo , Yucheng Guo , Mingxi Luo
show 5 more authors
Wenyang Ma Yiyun Mou Yifei Zhang Chen Zhou Yongjian Guo
This is my paper
classification cs.AI
keywords embodiedoperatorsintelligenceoperatortaxonomymulti-dimensionalbenchmarkvision-language-actionhandmotionrecoveryrobotdeploymentworkflowacceleration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that practical embodied intelligence cannot rest on end-to-end policy models alone. Real pipelines for data collection, demonstration understanding, scene reconstruction, learning, planning, and robot execution depend on many reusable functional modules that turn multimodal observations, robot states, human demos, and task context into structured representations, decisions, trajectories, control references, and system services. The authors name these modules embodied operators, define their boundary by task semantics, standardized input-output contracts, deployability, reusability, and multi-layer optimizability, and organize them into five categories spanning perception, 3D understanding, hand motion recovery, foundation-model decision, and planning/control/system support. They further propose a multi-dimensional benchmark that scores correctness together with end-to-end efficiency, resource use, temporal stability, portability, interface compatibility, deployment reliability, and downstream task utility. A sympathetic reader cares because without this modular, system-level view, accuracy gains on single models will not reliably produce reusable, scalable, or verifiable robot systems in the real world.

What carries the argument

Embodied operators: independent yet composable functional units that transform sensory, spatial, human, task, and system inputs into representations, decisions, trajectories, control references, or services, governed by five properties (functional independence, explicit I/O contract, reusability, deployability, multi-layer optimizability) and assessed by a multi-dimensional benchmark across correctness, efficiency, resources, stability, portability, and task utility.

What would settle it

Build a shared operator library with the proposed contracts, run the three-track benchmark on at least two full manipulation pipelines across different robots and hardware, and check whether operator-level gains in the listed dimensions reliably raise end-to-end task success, lower intervention rate, and transfer across platforms; if they do not, the central claim fails.

Watch

Extended reading notes

Core claim

The central claim is that embodied operators—reusable computational modules with explicit task semantics and standardized contracts—should be optimized and evaluated as holistic deployable components rather than as isolated neural networks, and that doing so provides a foundation for reusable, scalable, and verifiable embodied intelligence systems.

Load-bearing premise

The paper assumes that defining operator contracts, a five-category taxonomy, and a multi-dimensional evaluation checklist is enough to ground reusable operator libraries, without yet showing a working pilot of that benchmark on real pipelines.

Editorial extensions

If this is right

  • Operator libraries would standardize interfaces for masks, poses, hand trajectories, actions, plans, and ROS-style messages so modules can be swapped without rewriting pipelines.
  • Benchmarks would stop treating FPS or single-model accuracy as sufficient and would require joint reporting of latency tails, memory, temporal stability, failure recovery, and downstream utility.
  • Acceleration work would target workflow bottlenecks (data movement, serialization, scheduling, safety fallback) rather than decoder speed alone.
  • VLA and world-model outputs would sit above deterministic planners and controllers that enforce feasibility, collision, and safety contracts.
  • Near-term industrial, warehouse, and inspection deployments would prioritize composable operator stacks with measurable ROI and intervention rates.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If contracts and multi-track benchmarks become the default, vendor claims of open-vocabulary or VLA speed will face harder cross-platform reproducibility tests than leaderboard scores alone.
  • The same operator lens could force sim-to-real and synthetic-data generators to ship failure modes and coordinate/timestamp contracts, not only pretty rollouts.
  • Hierarchical control (slow semantic layer, mid-rate tracking/pose, high-rate safety control) is the practical deployment pattern the taxonomy already implies for edge robots.
  • Without shared failure codes and fallback behaviors, operator composition will remain brittle even when individual modules improve.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This white paper defines embodied operators as reusable, deployable functional modules with task semantics, standardized I/O contracts, reusability, deployability, and multi-layer optimizability. It organizes them into five categories (detection/segmentation; spatial localization and 3D understanding; hand motion recovery; foundation models and task decision; planning, control, and system support), surveys representative methods and limitations in each, and proposes a multi-dimensional operator benchmark spanning correctness, end-to-end efficiency, resource use, temporal stability, portability, interface compatibility, deployment reliability, and downstream task utility, together with Operator Cards, Run Manifests, and three evaluation tracks. The central claim is that operators should be optimized and evaluated as holistic deployable components rather than isolated neural networks, thereby providing a foundation for reusable, scalable, and verifiable embodied systems.

Significance. If adopted, the framing would help the field move from model-centric leaderboards toward system-level evaluation of perception–decision–execution pipelines, which is a genuine gap in embodied AI. The taxonomy usefully connects visual perception, 3D geometry, human demonstration recovery, VLA/world models, and ROS 2-style runtime support under one contract-oriented vocabulary, and the proposed dimensions (especially temporal stability, portability, and downstream utility) correctly target failure modes that isolated accuracy/FPS metrics miss. The contribution is primarily definitional and architectural rather than empirical: its value depends on whether the contracts and benchmark protocol become usable community infrastructure, not on a new algorithm or theorem.

major comments (3)
  1. [§8.1–8.5, Tables 2–3] §8.1–8.5 and Tables 2–3 propose Operator Cards, Run Manifests, three tracks (Correctness-Preserving / Approximate / Deployment), and multi-dimensional metrics, but the manuscript never instantiates the protocol on any real operator. No filled Operator Card, no track comparison, and no multi-operator pipeline result (e.g., detection → pose → grasp → trajectory) are reported. For a paper whose central claim is that this framework provides a foundation for reusable and verifiable systems, at least one pilot evaluation is load-bearing; without it the sufficiency of the dimensions remains an untested design assumption.
  2. [Abstract; §1; §9] Abstract, §1, and §9 assert that the taxonomy plus multi-dimensional benchmark “provide a foundation” for reusable, scalable, and verifiable systems. That is stronger than what the evidence supports. The literature summaries in §§3–7 establish motivation and known limitations, but they do not show that optimizing under Table 3 metrics improves composition, portability, or downstream task success relative to isolated model metrics. Either add a minimal pilot that demonstrates such improvement, or temper the claim to a well-motivated evaluation agenda rather than a demonstrated foundation.
  3. [§2.1; Table 3; §7.5] §2.1 defines five characterizing properties (independence, I/O contract including frames/timestamps/confidence/failure, reusability, deployability, multi-layer optimizability), yet §8’s metrics only partially operationalize them. Interface compatibility and failure/fallback contracts are emphasized in the definition and in §7.5, but Table 3 does not give explicit, measurable criteria for contract compliance, failure-code completeness, or cross-module schema compatibility. Without those, the benchmark cannot verify the very contracts that distinguish “embodied operators” from ordinary models.
minor comments (6)
  1. [Table 1] Table 1 has a broken citation string: “MANO [ 14 COPSMPL re-construction”. Clean the entry and ensure consistent citation formatting.
  2. [Abstract; Table 3] Abstract lists “interface compatibility” among evaluation dimensions, but Table 3’s named dimensions do not isolate it clearly from portability/task utility. Align the abstract list with Table 3.
  3. [§3.2] §3.2 cites SAM3 and related very recent works; ensure all arXiv-only items are consistently cited and that claims about their embodied readiness are hedged where only general vision results exist.
  4. [§6.2] §6.2 discusses OpenVLA vs π0 action heads clearly, but the transition to JoyAI-RA 0.1 and Qwen-RobotManip is denser and more self-referential than surrounding survey text; a short neutral comparison table would improve balance.
  5. [§8.5] Several long paragraphs in §8.5 restate acceleration principles already covered in §7.3–7.4; modest compression would improve readability without loss of content.
  6. [Throughout] Typographical spacing around citations is inconsistent throughout (e.g., “SAM2 [ 26]”). Normalize citation spacing in production.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: position/survey paper defines a taxonomy and benchmark; no fitted inputs renamed as predictions and no load-bearing self-citation chain.

full rationale

This manuscript is a white-paper survey and design proposal, not a derivation of quantitative predictions. It defines “embodied operators,” organizes existing methods into five categories, reviews external technical paradigms (SAM2, FoundationPose, OpenVLA, MoveIt 2, etc.), and proposes multi-dimensional evaluation dimensions and registration rules (Operator Card, Run Manifest, three tracks). There is no equation chain in which a fitted parameter is re-labeled as a prediction, no uniqueness theorem imported from the authors to forbid alternatives, and no ansatz smuggled in via self-citation. Overlapping-author citations (e.g., JoyAI-RA 0.1, SWORD, Pre-VLA, thousand-GPU infrastructure) appear only as illustrative examples of VLA or systems work; the central claim—that operators should be optimized and evaluated as holistic deployable components—is a design stance, not a result forced by those citations or by redefining its own inputs. Renaming modules as “embodied operators” is explicit conceptual framing, not a claimed first-principles discovery of a known empirical law. Circularity score is therefore 0; empty steps.

Assumptions & free parameters 0 free parameters · 4 assumptions · 2 invented entities

The paper is conceptual; it introduces almost no free numeric parameters. Its load-bearing content rests on domain assumptions about the necessity of intermediate modules and on the invented entity 'embodied operator' together with the proposed benchmark dimensions. No physical constants or fitted scales appear.

assumptions (4)
  • domain assumption High-quality embodied systems cannot rely solely on end-to-end policy models; reusable intermediate modules are required for data collection, demonstration understanding, reconstruction, decision, and execution.
    Stated in §1 and the Abstract as the motivation for defining operators; treated as given rather than demonstrated by controlled ablation against pure end-to-end stacks.
  • ad hoc to paper An operator is adequately characterized by five properties: functional independence, explicit I/O contract (including frames/timestamps/confidence/failure), reusability, deployability (service/SDK/ROS node/plugin), and multi-layer optimizability.
    Definition boundary in §2.1; the five properties are stipulated by the authors as the boundary of the concept.
  • ad hoc to paper Operator value must be measured jointly by correctness, end-to-end efficiency, resource use, temporal stability, portability, interface compatibility, deployment reliability, and downstream task utility rather than by isolated model accuracy or FPS.
    Core of the benchmark proposal in §8.1 and Table 3; presented as necessary without empirical ranking of which dimensions dominate failure modes.
  • domain assumption Foundation-model / task-decision operators determine 'what' while planning/control/system operators determine 'how' under kinematic, dynamic, collision, and runtime constraints.
    Stated in §2.2; standard hierarchical robotics assumption used to keep the fourth and fifth categories distinct.
invented entities (2)
  • embodied operator
    purpose: Name and bound reusable, deployable functional modules that carry both computational and task semantics inside embodied pipelines.
    Introduced in Abstract and formalized in §2.1 with five properties and Table 1; the exact term and contract-centric boundary are paper-specific even though related ideas exist in ROS and modular robotics.
  • multi-dimensional embodied-operator benchmark (Operator Card + Run Manifest + three tracks)
    purpose: Provide a registration and scoring protocol that evaluates operators as deployable system components rather than isolated networks.
    Defined in §8.2–8.5 and Tables 2–3; no external prior suite matches the full set of dimensions and tracks as specified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Embodied Operators and Benchmarking: Toward Reusable and Deployable Embodied Intelligence Systems." pith.science (2026). https://pith.science/paper/CGIS7NRK

@misc{pith2026260703283,
  author       = {Pith},
  title        = {Pith review of: Embodied Operators and Benchmarking: Toward Reusable and Deployable Embodied Intelligence Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CGIS7NRK}},
  note         = {Machine review of arXiv:2607.03283}
}
read the original abstract

Embodied intelligence systems require not only end-to-end policy models, but also reusable functional modules that transform multimodal observations, robot states, human demonstrations, and task contexts into structured representations, decisions, trajectories, control references, and system services. This work defines these modules as embodied operators and studies them as independent yet composable units in embodied intelligence pipelines. We clarify their definition boundary, emphasizing task semantics, standardized input-output contracts, deployability, reusability, and multi-layer optimizability. We further construct a taxonomy covering five categories: detection and segmentation, spatial localization and 3D understanding, hand motion recovery, embodied foundation models and task-decision operators, and planning, control, and system support operators. For each category, we summarize representative functions, technical paradigms, application roles, and practical limitations. Beyond taxonomy, we propose a multi-dimensional benchmark framework that evaluates embodied operators in terms of correctness, end-to-end efficiency, resource usage, temporal stability, portability, interface compatibility, deployment reliability, and downstream task utility. We also discuss workflow-level operator acceleration and open challenges in operator composition, data standardization, world models, VLA safety, edge deployment, and real-world application value. Overall, this work argues that embodied operators should be optimized and evaluated as holistic deployable components, providing a foundation for reusable, scalable, and verifiable embodied intelligence systems.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

96 extracted references · 22 linked inside Pith

  1. [1]

    Aligning cyber space with physical world: A comprehensive survey on embodied ai

    Yang Liu, Weixing Chen, Yongjie Bai, Xiaodan Liang, Guanbin Li, Wen Gao, and Liang Lin. Aligning cyber space with physical world: A comprehensive survey on embodied ai. IEEE/ASME Transactions on Mechatronics, 2025

    2025

  2. [2]

    Openvla: An open-source vision-language-action model

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. Openvla: An open-source vision-language-action model. arXiv preprint arXiv:2406.09246, 2024

    arXiv 2024

  3. [3]

    Octo: An open-source generalist robot policy

    Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: An open-source generalist robot policy. arXiv preprint arXiv:2405.12213, 2024

    arXiv 2024

  4. [4]

    π0: A vision-language-action flow model for general robot control

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. π0: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164, 2024

    arXiv 2024

  5. [5]

    Gr00t n1: An open foundation model for generalist humanoid robots

    Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, et al. Gr00t n1: An open foundation model for generalist humanoid robots. arXiv preprint arXiv:2503.14734, 2025

    arXiv 2025

  6. [6]

    Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0

    Abby O’Neill, Abdul Rehman, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, et al. Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 6892–6903. IEEE, 2024

    2024

  7. [7]

    Droid: A large-scale in-the-wild robot manipulation dataset

    Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, et al. Droid: A large-scale in-the-wild robot manipulation dataset. arXiv preprint arXiv:2403.12945, 2024

    arXiv 2024

  8. [8]

    Lerobot: An open-source library for end-to-end robot learning

    Remi Cadene, Simon Aliberts, Francesco Capuano, Michel Aractingi, Adil Zouitine, Pepijn Kooijmans, Jade Choghari, Martino Russi, Caroline Pascal, Steven Palma, et al. Lerobot: An open-source library for end-to-end robot learning. arXiv preprint arXiv:2602.22818, 2026

    arXiv 2026

Show all 96 references
  1. [9]

    Neuman, Brian Plancher, and Vijay Janapa Reddi

    V’ictor Mayoral-Vilches, Sabrina M. Neuman, Brian Plancher, and Vijay Janapa Reddi. RobotCore: An open architecture for hardware acceleration in ROS 2. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 9692–9699, 2022

    2022

  2. [10]

    Gibbons, Sabrina M

    V’ictor Mayoral-Vilches, Jason Jabbour, Yu-Shun Hsiao, Zishen Wan, Marti no Crespo-’Alvarez, Matthew Stew- art, Juan Manuel Reina-Mu noz, Prateek Nagras, Gaurav Vikhe, Mohammad Bakhshalipour, Martin Pinzger, Stefan Rass, Smruti Panigrahi, Giulio Corradi, Niladri Roy, Phillip B...

    2024

  3. [11]

    Robot Operating System 2: Design, architecture, and uses in the wild

    Steven Macenski, Tully Foote, Brian Gerkey, Chris Lalancette, and William Woodall. Robot Operating System 2: Design, architecture, and uses in the wild. Science Robotics, 7(66):eabm6074, 2022

    2022

  4. [12]

    Wilor: End-to-end 3d hand localization and reconstruction in-the-wild, 2025

    Rolandos Alexandros Potamias, Jinglei Zhang, Jiankang Deng, and Stefanos Zafeiriou. Wilor: End-to-end 3d hand localization and reconstruction in-the-wild, 2025

    2025

  5. [13]

    Hawor: World-space hand motion reconstruction from egocentric videos, 2025

    Jinglei Zhang, Jiankang Deng, Chao Ma, and Rolandos Alexandros Potamias. Hawor: World-space hand motion reconstruction from egocentric videos, 2025

    2025

  6. [14]

    Javier Romero, Dimitrios Tzionas, and Michael J. Black. Embodied hands: Modeling and capturing hands and bodies together. ACM Transactions on Graphics, (Proc. SIGGRAPH Asia), 36(6), November 2017

    2017

  7. [15]

    Hand-object interaction: From human demonstrations to robot manipulation

    Alessandro Carfì, Timothy Patten, Yingyi Kuang, Ali Hammoud, Mohamad Alameh, Elisa Maiettini, Abra- ham Itzhak Weinberg, Diego Faria, Fulvio Mastrogiovanni, Guillem Alenyà, et al. Hand-object interaction: From human demonstrations to robot manipulation. Frontiers in Robotics a...

    2021

  8. [16]

    Mediapipe: A framework for building perception pipelines

    Camillo Lugaresi, Jiuqiang Tang, Hadon Nash, Chris McClanahan, Esha Uboweja, Michael Hays, Fan Zhang, Chuo-Ling Chang, Ming Guang Yong, Juhyun Lee, et al. Mediapipe: A framework for building perception pipelines. arXiv preprint arXiv:1906.08172, 2019

    1906 arXiv

  9. [17]

    Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors

    Chien-Yao Wang, Alexey Bochkovskiy, and Hong-Yuan Mark Liao. Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7464–7475, 2023

    2023

  10. [18]

    Dino: Detr with improved denoising anchor boxes for end-to-end object detection

    Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung-Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605, 2022

    2022 arXiv

  11. [19]

    Ormnet: Object-centric relationship modeling for egocentric hand- object segmentation

    Yuejiao Su, Yi Wang, and Lap-Pui Chau. Ormnet: Object-centric relationship modeling for egocentric hand- object segmentation. arXiv preprint arXiv:2407.05576, 2(3), 2024

    2024 arXiv

  12. [20]

    Care-ego: Contact-aware relationship modeling for egocentric interac- tive hand-object segmentation

    Yuejiao Su, Yi Wang, and Lap-Pui Chau. Care-ego: Contact-aware relationship modeling for egocentric interac- tive hand-object segmentation. Expert Systems with Applications, page 129148, 2025

    2025

  13. [21]

    Interaction-aware representation modeling with co-occurrence consistency for egocentric hand-object parsing

    Yuejiao Su, Yi Wang, Lei Yao, Yawen Cui, and Lap-Pui Chau. Interaction-aware representation modeling with co-occurrence consistency for egocentric hand-object parsing. arXiv preprint arXiv:2602.20597, 2026

    2026

  14. [22]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European conference on computer vision, pages 38–55. Springer, 2024

    2024

  15. [23]

    Grounding dino 1.5: Advance the” edge” of open-set object detection

    Tianhe Ren, Qing Jiang, Shilong Liu, Zhaoyang Zeng, Wenlong Liu, Han Gao, Hongjie Huang, Zhengyu Ma, Xiaoke Jiang, Yihao Chen, et al. Grounding dino 1.5: Advance the” edge” of open-set object detection. arXiv preprint arXiv:2405.10300, 2024

    2024 arXiv

  16. [24]

    Yolo-world: Real-time open-vocabulary object detection

    Tianheng Cheng, Lin Song, Yixiao Ge, Wenyu Liu, Xinggang Wang, and Ying Shan. Yolo-world: Real-time open-vocabulary object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16901–16911, 2024

    2024

  17. [25]

    Segment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4015–4026, 2023

    2023

  18. [26]

    Sam 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao-Yuan Wu, Ross Girshick, Piotr Dollár, and Christoph Feichtenh...

    2025

  19. [27]

    Sam 3: Segment anything with concepts

    Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoubhik Debnath, Ronghang Hu, Didac Suris, Chaitanya Ryali, Kalyan Vasudev Alwala, Haitham Khedr, Andrew Huang, et al. Sam 3: Segment anything with concepts. arXiv preprint arXiv:2511.16719, 2025

    2025 arXiv

  20. [28]

    Gómez Rodríguez, José M

    Carlos Campos, Richard Elvira, Juan J. Gómez Rodríguez, José M. M. Montiel, and Juan D. Tardós. ORB- SLAM3: An accurate open-source library for visual, visual-inertial, and multimap SLAM. IEEE Transactions on Robotics, 37(6):1874–1890, 2021. 25

    2021

  21. [29]

    Splatam: Splat, track & map 3d gaussians for dense rgb-d slam, 2024

    Nikhil Keetha, Jay Karhade, Krishna Murthy Jatavallabhula, Gengshan Yang, Sebastian Scherer, Deva Ramanan, and Jonathon Luiten. Splatam: Splat, track & map 3d gaussians for dense rgb-d slam, 2024

    2024

  22. [30]

    Riku Murai, Eric Dexheimer, and Andrew J. Davison. Mast3r-slam: Real-time dense slam with 3d reconstruction priors, 2025

    2025

  23. [31]

    Foundationpose: Unified 6d pose estimation and tracking of novel objects

    Bowen Wen, Wei Yang, Jan Kautz, and Stan Birchfield. Foundationpose: Unified 6d pose estimation and tracking of novel objects. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 17868–17879, 2024

    2024

  24. [32]

    Bundlesdf: Neural 6-dof tracking and 3d reconstruction of unknown objects, 2023

    Bowen Wen, Jonathan Tremblay, Valts Blukis, Stephen Tyree, Thomas Muller, Alex Evans, Dieter Fox, Jan Kautz, and Stan Birchfield. Bundlesdf: Neural 6-dof tracking and 3d reconstruction of unknown objects, 2023

    2023

  25. [33]

    Schönberger and Jan-Michael Frahm

    Johannes L. Schönberger and Jan-Michael Frahm. Structure-from-motion revisited. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4104–4113, 2016

    2016

  26. [34]

    Chen, Zhenyu Li, Guang Shi, Jiashi Feng, and Bingyi Kang

    Haotong Lin, Sili Chen, Junhao Liew, Donny Y. Chen, Zhenyu Li, Guang Shi, Jiashi Feng, and Bingyi Kang. Depth anything 3: Recovering the visual space from any views, 2025

    2025

  27. [35]

    Metric3d: Towards zero-shot metric 3d prediction from a single image

    Wei Yin, Chi Zhang, Hao Chen, Zhipeng Cai, Gang Yu, Kaixuan Wang, Xiaozhi Chen, and Chunhua Shen. Metric3d: Towards zero-shot metric 3d prediction from a single image. 2023

    2023

  28. [36]

    UniDepth: Universal monocular metric depth estimation

    Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. UniDepth: Universal monocular metric depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  29. [37]

    UniDepthV2: Universal monocular metric depth estimation made simpler, 2025

    Luigi Piccinelli, Christos Sakaridis, Yung-Hsu Yang, Mattia Segu, Siyuan Li, Wim Abbeloos, and Luc Van Gool. UniDepthV2: Universal monocular metric depth estimation made simpler, 2025

    2025

  30. [38]

    Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision

    Ruicheng Wang, Sicheng Xu, Cassie Dai, Jianfeng Xiang, Yu Deng, Xin Tong, and Jiaolong Yang. Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision. In Proceedings of the Computer Vision and Pattern Recognition Conferenc...

    2025

  31. [39]

    Moge-2: Accurate monocular geometry with metric scale and sharp details, 2025

    Ruicheng Wang, Sicheng Xu, Yue Dong, Yu Deng, Jianfeng Xiang, Zelong Lv, Guangzhong Sun, Xin Tong, and Jiaolong Yang. Moge-2: Accurate monocular geometry with metric scale and sharp details, 2025

    2025

  32. [40]

    Vggt: Visual geometry grounded transformer, 2025

    Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer, 2025

    2025

  33. [41]

    Dust3r: Geometric 3d vision made easy, 2024

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made easy, 2024

    2024

  34. [42]

    Grounding image matching in 3d with mast3r, 2024

    Vincent Leroy, Yohann Cabon, and Jérôme Revaud. Grounding image matching in 3d with mast3r, 2024

    2024

  35. [43]

    NeRFPrior: Learning neural radiance field as a prior for indoor scene reconstruction

    Wenyuan Zhang, Emily Yue-ting Jia, Junsheng Zhou, Baorui Ma, Kanle Shi, Yu-Shen Liu, and Zhizhong Han. NeRFPrior: Learning neural radiance field as a prior for indoor scene reconstruction. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 11317–11...

    2025

  36. [44]

    Objectgs: Object-aware scene reconstruction and scene understanding via gaussian splatting

    Ruijie Zhu, Mulin Yu, Linning Xu, Lihan Jiang, Yixuan Li, Tianzhu Zhang, Jiangmiao Pang, and Bo Dai. Objectgs: Object-aware scene reconstruction and scene understanding via gaussian splatting. arXiv preprint arXiv:2507.15454, 2025

    2025 arXiv

  37. [45]

    Sam 3d: 3dfy anything in images

    Xingyu Chen, Fu-Jen Chu, Pierre Gleize, Kevin J Liang, Alexander Sax, Hao Tang, Weiyao Wang, Michelle Guo, Thibaut Hardin, Xiang Li, et al. Sam 3d: 3dfy anything in images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7220–7232, 2026

    2026

  38. [46]

    Handoccnet: Occlusion- robust 3d hand mesh estimation network, 2022

    JoonKyu Park, Yeonguk Oh, Gyeongsik Moon, Hongsuk Choi, and Kyoung Mu Lee. Handoccnet: Occlusion- robust 3d hand mesh estimation network, 2022

    2022

  39. [47]

    Williams

    Zheheng Jiang, Hossein Rahmani, Sue Black, and Bryan M. Williams. A probabilistic attention model with occlusion-aware texture regression for 3d hand reconstruction from a single rgb image, 2023

    2023

  40. [48]

    Reconstructing hands in 3d with transformers, 2023

    Georgios Pavlakos, Dandan Shan, Ilija Radosavovic, Angjoo Kanazawa, David Fouhey, and Jitendra Malik. Reconstructing hands in 3d with transformers, 2023. 26

    2023

  41. [49]

    Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966, 2023

    2023 arXiv

  42. [50]

    Rt-2: Vision-language-action models transfer web knowledge to robotic control

    Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pages 2165–2183. PMLR, 2023

    2023

  43. [51]

    Joyai-ra 0.1: A foundation model for robotic autonomy, 2026

    Tianle Zhang, Zhihao Yuan, Dafeng Chi, Peidong Liu, Dongwei Li, Kejun Hu, Likui Zhang, Junnan Nie, Ziming Wei, Zengjue Chen, Yili Tang, Jiayi Li, Zhiyuan Xiang, Mingyang Li, Tianci Luo, Hanwen Wan, Ao Li, Linbo Zhai, Zhihao Zhan, Xiaodong Bai, Jiakun Cai, Peng Cao, Kangliang C...

    2026

  44. [52]

    Qwen-robotmanip technical report: Alignment unlocks scale for robotic manipulation foundation models

    Qwen Team. Qwen-robotmanip technical report: Alignment unlocks scale for robotic manipulation foundation models. 2026

    2026

  45. [53]

    Causal world modeling for robot control

    Lin Li, Qihang Zhang, Yiming Luo, Shuai Yang, Ruilin Wang, Fei Han, Mingrui Yu, Zelin Gao, Nan Xue, Xing Zhu, Yujun Shen, and Yinghao Xu. Causal world modeling for robot control. arXiv preprint arXiv:2601.21998, 2026

    2026 arXiv

  46. [54]

    NVIDIA, :, Aditi, Niket Agarwal, Arslan Ali, Jon Allen, Martin Antolini, Adeline Aubame, Alisson Azzolini, Junjie Bai, Maciej Bala, Yogesh Balaji, Josh Bapst, Aarti Basant, Mukesh Beladiya, Mohammad Qazim Bhat, Zaid Pervaiz Bhat, Dan Blick, Vanni Brighella, Han Cai, Tiffany Ca...

    2026

  47. [55]

    Wmpo: World model-based policy optimization for vision-language-action models, 2025

    Fangqi Zhu, Zhengyang Yan, Zicong Hong, Quanxin Shou, Xiao Ma, and Song Guo. Wmpo: World model-based policy optimization for vision-language-action models, 2025

    2025

  48. [56]

    Wovr: World models as reliable simulators for post-training vla policies with rl, 2026

    Zhennan Jiang, Shangqing Zhou, Yutong Jiang, Zefang Huang, Mingjie Wei, Yuhui Chen, Tianxing Zhou, Zhen Guo, Hao Lin, Quanlu Zhang, Yu Wang, Haoran Li, Chao Yu, and Dongbin Zhao. Wovr: World models as reliable simulators for post-training vla policies with rl, 2026

    2026

  49. [57]

    Sword: Style-robust world models as simulators via dynamic latent bootstrapping for vla policy post-training, 2026

    Jiaxuan Gao, Yongjian Guo, Zhong Guan, Wen Huang, Wanlun Ma, Xi Xiao, Junwu Xiong, and Sheng Wen. Sword: Style-robust world models as simulators via dynamic latent bootstrapping for vla policy post-training, 2026

    2026

  50. [58]

    Contact-GraspNet: Efficient 6-DoF grasp generation in cluttered scenes

    Martin Sundermeyer, Arsalan Mousavian, Rudolph Triebel, and Dieter Fox. Contact-GraspNet: Efficient 6-DoF grasp generation in cluttered scenes. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 13438–13444, 2021

    2021

  51. [59]

    AnyGrasp: Robust and efficient grasp perception in spatial and temporal domains

    Hao-Shu Fang, Chenxi Wang, Hongjie Fang, Minghao Gou, Jirong Liu, Hengxu Yan, Wenhai Liu, Yichen Xie, and Cewu Lu. AnyGrasp: Robust and efficient grasp perception in spatial and temporal domains. IEEE Transactions on Robotics, 39(5):3929–3945, 2023

    2023

  52. [60]

    MoveIt 2 Documentation

    MoveIt Project. MoveIt 2 Documentation. https://moveit.picknik.ai/. Accessed: 2026-07-03

    2026

  53. [61]

    CuRobo: Parallelized collision-free robot motion generation

    Balakumar Sundaralingam, Siva Kumar Sastry Hari, Adam Fishman, Caelan Garrett, Karl Van Wyk, Valts Blukis, Alexander Millane, Helen Oleynikova, Ankur Handa, Fabio Ramos, Nathan Ratliff, and Dieter Fox. CuRobo: Parallelized collision-free robot motion generation. In 2023 IEEE I...

    2023

  54. [62]

    Impact of ROS 2 node composition in robotic systems

    Steven Macenski, Alberto Soragna, Michael Carroll, and Zhenpeng Ge. Impact of ROS 2 node composition in robotic systems. IEEE Robotics and Automation Letters, 8(7):3996–4003, 2023

    2023

  55. [63]

    Nvidia isaac ros nitros documentation

    NVIDIA. Nvidia isaac ros nitros documentation. https://nvidia-isaac-ros.github.io/concepts/nitros/ index.html, 2024. Accessed: 2026-06-16

    2024

  56. [64]

    PAAM: A framework for coordinated and priority-driven accelerator management in ROS 2

    Daniel Enright, Yecheng Xiang, Hyunjong Choi, and Hyoseung Kim. PAAM: A framework for coordinated and priority-driven accelerator management in ROS 2. In 2024 IEEE 30th Real-Time and Embedded Technology and Applications Symposium (RTAS), pages 81–94, 2024

    2024

  57. [65]

    Dexycb: A benchmark for capturing hand grasping of objects

    Yu-Wei Chao, Wei Yang, Yu Xiang, Pavlo Molchanov, Ankur Handa, Jonathan Tremblay, Yashraj S Narang, Karl Van Wyk, Umar Iqbal, Stan Birchfield, et al. Dexycb: A benchmark for capturing hand grasping of objects. In Proceedings of the IEEE/CVF conference on computer vision and pa...

    2021

  58. [66]

    Hot3d: Hand and object tracking in 3d from egocentric multi-view videos

    Prithviraj Banerjee, Sindi Shkodrani, Pierre Moulon, Shreyas Hampali, Shangchen Han, Fan Zhang, Linguang Zhang, Jade Fountain, Edward Miller, Selen Basol, et al. Hot3d: Hand and object tracking in 3d from egocentric multi-view videos. In Proceedings of the IEEE/CVF Conference ...

    2025

  59. [67]

    Nvidia tensorrt documentation

    NVIDIA. Nvidia tensorrt documentation. https://docs.nvidia.com/deeplearning/tensorrt/latest/index. html, 2026. Accessed: 2026-06-16

    2026

  60. [68]

    Nvidia deepstream sdk documentation

    NVIDIA. Nvidia deepstream sdk documentation. https://docs.nvidia.com/metropolis/deepstream/ dev-guide/, 2026. Accessed: 2026-06-16

    2026

  61. [69]

    Nvidia isaac sim documentation

    NVIDIA. Nvidia isaac sim documentation. https://docs.isaacsim.omniverse.nvidia.com/, 2026. Accessed: 2026-06-16

    2026

  62. [70]

    Isaac lab: A gpu-accelerated simulation framework for multi- modal robot learning

    Mayank Mittal, Pascal Roth, James Tigue, Antoine Richard, Octi Zhang, Peter Du, Antonio Serrano-Munoz, Xinjie Yao, René Zurbrügg, Nikita Rudin, et al. Isaac lab: A gpu-accelerated simulation framework for multi- modal robot learning. arXiv preprint arXiv:2511.04831, 2025

    2025 arXiv

  63. [71]

    Dyn-hamr: Recovering 4d interacting hand motion from a dynamic camera

    Zhengdi Yu, Stefanos Zafeiriou, and Tolga Birdal. Dyn-hamr: Recovering 4d interacting hand motion from a dynamic camera. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 27716–27726, 2025. 28

    2025

  64. [72]

    Fast: Efficient action tokenization for vision-language-action models

    Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. Fast: Efficient action tokenization for vision-language-action models. arXiv preprint arXiv:2501.09747, 2025

    2025 arXiv

  65. [73]

    Fine-tuning vision-language-action models: Optimizing speed and success

    Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision-language-action models: Optimizing speed and success. arXiv preprint arXiv:2502.19645, 2025

    2025 arXiv

  66. [74]

    Embodied foundation models at the edge: A survey of deployment constraints and mitigation strategies

    Utkarsh Grover, Ravi Ranjan, Mingyang Mao, Trung Tien Dong, Satvik Praveen, Zhenqi Wu, J Morris Chang, Tinoosh Mohsenin, Yi Sheng, Agoritsa Polyzou, et al. Embodied foundation models at the edge: A survey of deployment constraints and mitigation strategies. arXiv preprint arXi...

    2026

  67. [75]

    Gemini robotics: Bringing ai into the physical world

    Gemini Robotics Team. Gemini robotics: Bringing ai into the physical world. arXiv preprint arXiv:2503.20020, 2025

    2025 arXiv

  68. [76]

    Nvidia cosmos: A world foundation model platform for physical ai

    NVIDIA. Nvidia cosmos: A world foundation model platform for physical ai. https://www.nvidia.com/en-us/ ai/cosmos/, 2025. Accessed: 2026-06-26

    2025

  69. [77]

    All robots in one: A new standard and unified dataset for versatile, general-purpose embodied agents

    Zhiqiang Wang, Hao Zheng, Yunshuang Nie, Wenjun Xu, Qingwei Wang, Hua Ye, Zhe Li, Kaidong Zhang, Xuewen Cheng, Wanxi Dong, et al. All robots in one: A new standard and unified dataset for versatile, general-purpose embodied agents. arXiv preprint arXiv:2408.10899, 2024

    2024 arXiv

  70. [78]

    World robotics 2025: Industrial robots

    International Federation of Robotics. World robotics 2025: Industrial robots. https://ifr.org/worldrobotics/,

    2025

  71. [80]

    World robotics 2025: Service robots

    International Federation of Robotics. World robotics 2025: Service robots. https://ifr.org/worldrobotics/,

    2025

  72. [81]

    Accessed: 2026-06-27

    2026

  73. [82]

    ISO 10218-1:2025 robotics – safety requirements – part 1: Indus- trial robots

    International Organization for Standardization. ISO 10218-1:2025 robotics – safety requirements – part 1: Indus- trial robots. https://www.iso.org/standard/73933.html, 2025. Accessed: 2026-06-27

    2025

  74. [83]

    ISO 10218-2:2025 robotics – safety requirements – part 2: Indus- trial robot applications and robot cells

    International Organization for Standardization. ISO 10218-2:2025 robotics – safety requirements – part 2: Indus- trial robot applications and robot cells. https://www.iso.org/standard/73934.html, 2025. Accessed: 2026-06- 27

    2025

  75. [84]

    Vision-language-action safety: Threats, challenges, evaluations, and mechanisms

    Qi Li, Bo Yin, Weiqi Huang, Ruhao Liu, Bojun Zou, Runpeng Yu, Jingwen Ye, Weihao Yu, and Xinchao Wang. Vision-language-action safety: Threats, challenges, evaluations, and mechanisms. arXiv preprint arXiv:2604.23775, 2026

    2026 arXiv

  76. [85]

    Robocasa: Large-scale simulation of everyday tasks for generalist robots

    Soroush Nasiriany, Abhiram Maddukuri, Lance Zhang, Adeet Parikh, Aaron Lo, Abhishek Joshi, Ajay Man- dlekar, and Yuke Zhu. Robocasa: Large-scale simulation of everyday tasks for generalist robots. arXiv preprint arXiv:2406.02523, 2024

    2024 arXiv

  77. [86]

    Robocasa365: A large-scale simulation framework for training and benchmarking generalist robots

    Soroush Nasiriany, Sepehr Nasiriany, Abhiram Maddukuri, and Yuke Zhu. Robocasa365: A large-scale simulation framework for training and benchmarking generalist robots. arXiv preprint arXiv:2603.04356, 2026

    2026

  78. [87]

    Safevla: Towards safety alignment of vision-language-action model via constrained learning

    Borong Zhang, Yuhao Zhang, Jiaming Ji, Yingshan Lei, Josef Dai, Yuanpei Chen, and Yaodong Yang. Safevla: Towards safety alignment of vision-language-action model via constrained learning. In Thirty-Ninth Conference on Neural Information Processing Systems, 2025

    2025

  79. [88]

    Embodiedgovbench: A benchmark for governance, recovery, and upgrade safety in embodied agent systems

    Xue Qin, Simin Luan, John See, Cong Yang, and Zhijun Li. Embodiedgovbench: A benchmark for governance, recovery, and upgrade safety in embodied agent systems. arXiv preprint arXiv:2604.11174, 2026

    2026 arXiv

  80. [89]

    Diffusion policy: Visuomotor policy learning via action diffusion

    Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. arXiv preprint arXiv:2303.04137, 2023

    2023 arXiv

  81. [90]

    Sucan, and Steve Cousins

    Sachin Chitta, Ioan A. Sucan, and Steve Cousins. Moveit![ros topics]. IEEE Robotics & Automation Magazine, 19(1):18–19, 2012

    2012

  82. [91]

    The marathon 2: A navigation system

    Steve Macenski, Francisco Martin, Ruffin White, and Jonatan Ginés Clavero. The marathon 2: A navigation system. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2718–2725. IEEE, 2020

    2020

  83. [92]

    Mayne, James B

    David Q. Mayne, James B. Rawlings, Christopher V. Rao, and Pierre O. M. Scokaert. Constrained model predictive control: Stability and optimality. Automatica, 36(6):789–814, 2000

    2000

  84. [93]

    Goodrich and Alan C

    Michael A. Goodrich and Alan C. Schultz. Human-robot interaction: A survey. Foundations and Trends in Human-Computer Interaction, 1(3):203–275, 2007. 29

    2007

  85. [94]

    ISO/TS 15066:2016 robots and robotic devices – collaborative robots

    International Organization for Standardization. ISO/TS 15066:2016 robots and robotic devices – collaborative robots. https://www.iso.org/standard/62996.html, 2016. Accessed: 2026-06-27

    2016

  86. [95]

    Foundationstereo: Zero-shot stereo matching

    Bowen Wen, Matthew Trepte, Joseph Aribido, Jan Kautz, Orazio Gallo, and Stan Birchfield. Foundationstereo: Zero-shot stereo matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5249–5260, 2025

    2025

  87. [96]

    Pre-vla: Preemptive runtime verification for reliable vision-language-action and world-model rollouts, 2026

    Zhen Sun, Yongjian Guo, Haoran Sun, Luqiao Wang, Wei Lu, Jiachi Ji, Shengzhe Ji, Junwu Xiong, and Zhijun Meng. Pre-vla: Preemptive runtime verification for reliable vision-language-action and world-model rollouts, 2026

    2026

  88. [97]

    Thousand-gpu large-scale training and optimization recipe for ai-native cloud embodied intelligence infrastructure, 2026

    Yongjian Guo, Yunxuan Ma, Haoran Sun, Zhong Guan, Shuai Di, Jing Long, Wanting Xu, Xiaodong Bai, Wen Huang, Yucheng Guo, Chen Zhou, Qiming Yang, Mingxi Luo, Tianyun Zhao, Hedan Yang, Song Wang, Xiaomeng Tian, Xiaolong Xiang, Zhen Sun, Yu Wei, Luqiao Wang, Yuzhen Li, Chenfeng G...

    2026

Pith tools

Reviewed July 12, 2026 · model on record in the stance chip above.