Pith. sign in

REVIEW 4 major objections 4 minor 55 references

Exploring the Link Between Bayesian Inference and Embodied Intelligence: Toward Open Physical-World Embodied AI Systems

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper argues that open-world embodied AI is best understood and built as a hierarchical Bayesian inference engine.

desk verdict A clearly written position piece whose Bitter Lesson analysis is valuable but whose central claim about Bayesian inference in open worlds is asserted, not supported. read the letter →

arxiv 2507.21589 v1 pith:4YDG5QQE submitted 2025-07-29 cs.AI

Bin Liu This is my paper
classification cs.AI
keywords embodiedintelligenceBayesianinferenceopenphysicalworldclosedtheBitterLessonbeliefupdatinghierarchicalenginefoundationmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the two fields are not just metaphorically aligned: the core operations of an embodied agent—perceiving, choosing actions, learning, and higher-level cognition—can be modeled as Bayesian inference, and an agent that must operate in a genuinely open physical world should be designed as a hierarchical Bayesian inference engine. The author reaches this conclusion by reading modern embodied AI through Sutton's 'Bitter Lesson' distinction between scalable search and learning versus hand-crafted knowledge. That lens explains why Bayesian methods have stayed peripheral: they typically require explicit priors, likelihoods, and structured models, which do not scale as cleanly as deep learning. The payoff of the argument is a concrete design orientation: treat everything a robot learned in its training environments as prior knowledge, and let continuous evidence-based belief updating carry it beyond those closed worlds.

What carries the argument

The load-bearing object is the hierarchical Bayesian inference engine: a system in which an embodied agent represents its world knowledge as probability distributions and revises them level by level—perception, action, learning, cognition—using Bayes' rule as evidence arrives, $P(\theta \mid \mathcal{D}) \propto P(\mathcal{D} \mid \theta) P(\theta)$ in the simplest case. The analytical machinery that carries the argument is Sutton's pair of search and learning: Bayesian inference is read as internal belief-guided search plus incremental learning, while modern embodied AI is read as external search (e.g., Monte Carlo tree search) plus batch data-driven learning. The engine then explains both why Bayesianism has been sidelined—structured priors and inference are not as scalable as data-driven learning—and why it is needed for open worlds, where continuous inference under uncertainty is unavoidable.

What would settle it

Run a controlled deployment-shift experiment: train a foundation-model robot in a set of simulated homes, then drop it into a novel home with new objects, and compare a version that updates explicit posterior beliefs online against one that only uses its frozen pretrained policy; the paper's central claim weakens if the Bayesian version does not adapt faster or more reliably.

Watch

Extended reading notes

Core claim

The central claim is that an embodied intelligence system designed for an open physical world can be understood as a hierarchical Bayesian inference engine. At each level—perception, action selection, learning, and higher-level cognition—the system maintains probabilistic beliefs and updates them as sensorimotor evidence arrives. Knowledge and skills acquired in closed training environments play the role of prior distributions; entering the open world is a process of sequential posterior updating rather than a switch to a new static dataset. The paper also offers a diagnosis, not just a proposal: current foundation-model and end-to-end systems dominate because they align with the expensive, assumption-light learning and search that Sutton's Bitter Lesson endorses, whereas Bayesian methods' reliance on explicit structure and inference has kept them out of the mainstream.

Load-bearing premise

The argument depends on a sharp, usable distinction between closed and open physical worlds, but the paper never defines either; without that boundary, the claim that Bayesian methods are the key to open-world embodied intelligence cannot be tested.

Editorial extensions

If this is right

  • Bayesian principles should move from a peripheral tool to a central design layer in embodied AI systems aimed at open-world operation.
  • Closed-world training data should be treated as prior knowledge, and deployment as online belief updating, rather than expecting a fixed policy to cover all situations.
  • The analysis predicts that systems with explicit Bayesian components—such as posterior estimation, sequential Monte Carlo, or Bayesian optimization—will become more common in robotics as the field moves beyond bounded environments.
  • The same search-and-learning lens can be applied to other computational paradigms to explain why they did or did not dominate AI, as the paper's concluding remarks state.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If open-world competence is fundamentally online belief updating, the practical bottleneck shifts to the prior: robots trained on diverse closed worlds are only as adaptable as the prior distributions they carry, so designing those priors becomes the central engineering problem.
  • A testable consequence the paper leaves implicit: a Bayesian-updating agent should outperform an equal-sized batch-trained agent specifically in the low-data regime of a deployment shift, and the performance gap should widen with the novelty of the target environment.
  • The paper deliberately remains conceptual and does not instantiate its proposed engine; the natural next step is to build a small-scale embodiment (e.g., a manipulation or navigation agent) that maintains explicit posteriors and compare its open-world robustness against a foundation-model baseline.
  • Formalizing 'open physical world' as, say, a non-stationary distribution or an unbounded novelty rate would turn the thesis into a measurable claim: the value of Bayesian updating should increase exactly when the deployment distribution shifts away from the training distribution.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This position paper argues that Bayesian inference is conceptually central to embodied intelligence and should be a key design principle for future embodied AI systems operating in open physical worlds. It reviews Sutton's "Bitter Lesson" and interprets search and learning as the two scalable forces in modern AI. It then categorizes current embodied AI approaches into foundation-model-powered systems (FMS) and end-to-end modeling (E2E), and uses the search/learning lens to explain why Bayesian methods have remained peripheral in these systems. The paper's forward-looking claim is that an open-world embodied intelligence system can be framed as a hierarchical Bayesian inference engine, and that Bayesian tools such as Sequential Monte Carlo and Bayesian optimization can play a foundational role. No formal model, theorem, algorithm, or experimental validation is provided; the contribution is conceptual and agenda-setting.

Significance. If the thesis held, the paper would reframe Bayesian methods from a niche toolset to a core architectural principle for embodied AI, a consequential claim given the current dominance of large-scale batch-trained foundation models. The paper's descriptive contribution is real: the FMS/E2E taxonomy is clear and accurate, and the application of Sutton's search/learning dichotomy to explain the marginalization of Bayesian methods is a useful organizing insight that names a genuine tension between continual Bayesian updating and scalable batch learning. The author also cites concrete ready-to-use Bayesian techniques, including SMC for co-design and Bayesian optimization for AutoML, which illustrate partial connections. However, the central forward-looking claim is currently asserted rather than derived or tested; the manuscript's own Section 4.2 documents scalability problems that Section 5 does not resolve. As a research agenda the paper is suggestive, but as a demonstrated thesis it needs substantial additional work.

major comments (4)
  1. [Section 5] The distinction between 'closed physical worlds' and 'truly open physical worlds' is never formally defined. The paper claims current systems operate in closed worlds and that open worlds require continuous Bayesian adaptation, but no criterion (e.g., distribution shift, novelty rate, non-stationarity, unbounded state space) is given. Please provide an operational definition, or at least a precise formal criterion, so that the claim that Bayesian inference is necessary for open worlds can be evaluated and potentially falsified.
  2. [Section 4.2, Table 2, and Section 5] The conclusion that Bayesian methods can be 'a promising foundation' for open-world embodied intelligence is not supported by the paper's own analysis. Section 4.2 and Table 2 state that Bayesian learning is 'limited by computation/inference complexity' and depends on structured priors that 'hinder scalability.' The only response in Section 5, that structured assumptions 'can be relaxed' via ensembles, does not address the inference-complexity burden; ensembles generally multiply the posterior-inference cost. No tractable inference algorithm for a hierarchical Bayesian engine in high-dimensional, partially observable, open environments is specified. Please provide a mechanism or concrete instantiation, or revise the claim to a hypothesis whose scope and scalability are explicitly assessed.
  3. [Abstract and Section 4.1] The claim that perception, action selection, learning, and higher-level cognition 'can be effectively understood and modeled as forms of Bayesian inference' is asserted without a formal mapping, a theorem, or empirical support. Section 6 acknowledges that the paper proceeds 'without delving into specific models or algorithmic details,' which leaves the central thesis unfalsifiable. Please either provide a formal framework (e.g., a POMDP or hierarchical generative model specification) or explicitly reframe the contribution as a research agenda with falsifiable predictions. The paper's own limitation statement should be reconciled with the strong wording used in the abstract.
  4. [Biography, page 16] The appended biography states that the author came to realize that 'Bayesian approaches alone are far from sufficient to tackle the complexities of real-world problems' and consequently embraced deep learning and large pre-trained models. This statement appears to contradict, or at least substantially qualify, the strong Section 5 claim that Bayesian methods should serve as the foundation for open-world embodied intelligence. Please address this tension explicitly in the main text, clarifying whether Bayesian inference is claimed to be the sole engine or a component within a hybrid system.
minor comments (4)
  1. [Section 5] The inline citation 'sim-to-real gap Muratore et al. (2021); Antonova et al. (2020); ?' contains an unresolved placeholder reference; replace '?' with the intended citation.
  2. [Section 3] The phrase 'guiding and controling a robot' contains a typo; 'controling' should be 'controlling.'
  3. [Section 6] The phrase 'truly open physical-worlds' should be 'truly open physical worlds' for grammatical consistency.
  4. [Table 2] The 'Learning Scalability' row conflates computational complexity with model-dependence; consider separating 'inference complexity' from 'dependence on structured priors,' since these are distinct obstacles with different remedies.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is a conceptual/interpretive essay whose self-citations are illustrative background, not load-bearing inputs that predetermine its conclusions.

full rationale

This paper is a conceptual position piece, not an empirical or formal derivation. It contains no fitted parameters, no quantitative predictions, and no equations that could reduce an output to an input. The central thesis is that embodied intelligence processes can be 'effectively understood and modeled as forms of Bayesian inference' (Abstract and Section 4.1). This is presented as an interpretive mapping based on the shared theme of continuous belief updating under uncertainty, not as a result derived from a definition that already assumes the conclusion. The self-citations (Liu 2016, 2017, 2018, 2020, 2023, 2025b) appear as background references illustrating that Bayesian optimization, sequential Monte Carlo methods, and embodied-intelligence surveys exist; none of these citations is invoked as the sole justification for the claim that Bayesian inference is central to open-world embodied intelligence. The 'closed physical world' vs. 'open physical world' distinction is informal and under-specified, which is a clarity weakness rather than a circularity weakness, because the paper does not use the distinction to construct a prediction that secretly presupposes its own conclusion. Section 5's proposal that an open-world system 'can be framed as a hierarchical Bayesian inference engine' is an assertion of plausibility and conceptual framing, explicitly deferred as future work ('without delving into specific models or algorithmic details', Section 6). Consequently, there is no step where the paper's claimed result is equivalent to its inputs by construction, no fitted quantity renamed as a prediction, and no self-citation chain that forces the conclusion. The correct finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 5 assumptions · 1 invented entities

The paper is a conceptual position piece. Its claims rest on several untested premises: that Sutton's Bitter Lesson is the valid lens for explaining AI progress, that embodied intelligence requires Bayesian-style uncertainty processing, that current systems are confined to closed physical worlds, and that Bayesian methods are inherently less scalable. None of these are derived or quantified in the paper. One proposed construct, the hierarchical Bayesian inference engine, is introduced without independent evidence.

assumptions (5)
  • domain assumption Sutton's 'Bitter Lesson' correctly identifies search and learning as the primary drivers of AI progress and the appropriate lens for evaluating methods.
    Used throughout Sections 2, 4.2, and 5 to explain why Bayesian methods have not dominated; if this lens is wrong or incomplete, the paper's explanation loses its foundation.
  • domain assumption Embodied intelligence inherently requires continuous inference under uncertainty.
    Stated in the Abstract and Section 4.1 as the bridge between embodied intelligence and Bayesian inference; no evidence is given that this requirement is what actually shapes embodied cognitive architectures.
  • domain assumption Current embodied AI systems operate in closed physical worlds, which are categorically different from open physical worlds.
    Section 5 asserts this distinction but provides no formal definition; the entire forward-looking argument depends on the boundary being meaningful.
  • domain assumption Bayesian learning methods are limited by their reliance on structured priors and inference complexity, making them less scalable.
    Section 4.2 and Table 2 contrast Bayesian methods with Sutton's scalability preference; this characterization is asserted rather than measured.
  • domain assumption All cognitive processes from perception to higher-level cognition can be modeled as Bayesian inference.
    Abstract and Section 4.1 make this philosophical claim without specific evidence; it is a foundational premise for proposing hierarchical Bayesian inference engines.
invented entities (1)
  • hierarchical Bayesian inference engine
    purpose: Proposed abstract architecture for an embodied agent operating in an open physical world, where perception, action, and learning are layered Bayesian belief updates.
    Introduced in Section 5 ('can be framed as a hierarchical Bayesian inference engine') with no design details, equations, or falsifiable predictions, so there is no external handle to test it independently.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Exploring the Link Between Bayesian Inference and Embodied Intelligence: Toward Open Physical-World Embodied AI Systems." pith.science (2026). https://pith.science/paper/4YDG5QQE

@misc{pith2026250721589,
  author       = {Pith},
  title        = {Pith review of: Exploring the Link Between Bayesian Inference and Embodied Intelligence: Toward Open Physical-World Embodied AI Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4YDG5QQE}},
  note         = {Machine review of arXiv:2507.21589}
}
read the original abstract

Embodied intelligence posits that cognitive capabilities fundamentally emerge from - and are shaped by - an agent's real-time sensorimotor interactions with its environment. Such adaptive behavior inherently requires continuous inference under uncertainty. Bayesian statistics offers a principled probabilistic framework to address this challenge by representing knowledge as probability distributions and updating beliefs in response to new evidence. The core computational processes underlying embodied intelligence - including perception, action selection, learning, and even higher-level cognition - can be effectively understood and modeled as forms of Bayesian inference. Despite the deep conceptual connection between Bayesian statistics and embodied intelligence, Bayesian principles have not been widely or explicitly applied in today's embodied intelligence systems. In this work, we examine both Bayesian and contemporary embodied intelligence approaches through two fundamental lenses: search and learning - the two central themes in modern AI, as highlighted in Rich Sutton's influential essay "The Bitter Lesson". This analysis sheds light on why Bayesian inference has not played a central role in the development of modern embodied intelligence. At the same time, it reveals that current embodied intelligence systems remain largely confined to closed-physical-world environments, and highlights the potential for Bayesian methods to play a key role in extending these systems toward truly open physical-world embodied intelligence.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

55 extracted references · 11 canonical work pages

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    arXiv

  2. [6]

    Rt-h: Action hierarchies using language

    Suneel Belkhale, Tianli Ding, Ted Xiao, Pierre Sermanet, Quon Vuong, Jonathan Tompson, Yevgen Cheb- otar, Debidatta Dwibedi, and Dorsa Sadigh. Rt-h: Action hierarchies using language. arXiv preprint arXiv:2403.01823,

    arXiv

  3. [12]

    Gr-2: A generative video-language-action model with web-scale knowledge for robot manipulation

    Chi-Lam Cheang, Guangzeng Chen, Ya Jing, Tao Kong, Hang Li, Yifeng Li, Yuxiao Liu, Hongtao Wu, Jiafeng Xu, Yichu Yang, et al. Gr-2: A generative video-language-action model with web-scale knowledge for robot manipulation. arXiv preprint arXiv:2410.06158,

    arXiv

  4. [13]

    Collaborating with language models for embodied reasoning

    Ishita Dasgupta, Christine Kaeser-Chen, Kenneth Marino, Arun Ahuja, Sheila Babayan, Felix Hill, and Rob Fergus. Collaborating with language models for embodied reasoning. arXiv preprint arXiv:2302.00763,

    arXiv

  5. [14]

    Automated co-design of soft hand mor- phology and control strategy for grasping

    Raphael Deimel, Patrick Irmisch, Vincent Wall, and Oliver Brock. Automated co-design of soft hand mor- phology and control strategy for grasping. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1213–1218. IEEE,

    work page 2017

  6. [15]

    Bayesian active learning for sim- to-real robotic perception

    Jianxiang Feng, Jongseok Lee, Maximilian Durner, and Rudolph Triebel. Bayesian active learning for sim- to-real robotic perception. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 10820–10827. IEEE,

    work page 2022

  7. [17]

    Embodied ai agents: Modeling the world

    Pascale Fung, Yoram Bachrach, Asli Celikyilmaz, Kamalika Chaudhuri, Delong Chen, Willy Chung, Em- manuel Dupoux, Hervé Jégou, Alessandro Lazaric, Arjun Majumdar, et al. Embodied ai agents: Modeling the world. arXiv preprint arXiv:2506.22355,

    arXiv

  8. [18]

    Towards generalizable vision-language robotic manipu- lation: A benchmark and llm-guided 3d policy

    Ricardo Garcia, Shizhe Chen, and Cordelia Schmid. Towards generalizable vision-language robotic manipu- lation: A benchmark and llm-guided 3d policy. arXiv preprint arXiv:2410.01345,

    arXiv

Show all 55 references
  1. [20]

    Rt-trajectory: Robotic task generalization via hindsight trajectory sketches

    Jiayuan Gu, Sean Kirmani, Paul Wohlhart, Yao Lu, Montserrat Gonzalez Arenas, Kanishka Rao, Wenhao Yu, Chuyuan Fu, Keerthana Gopalakrishnan, Zhuo Xu, et al. Rt-trajectory: Robotic task generalization via hindsight trajectory sketches. arXiv preprint arXiv:2311.01977,

    arXiv

  2. [21]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948,

    arXiv

  3. [22]

    The essential role of causality in foundation world models for embodied ai

    Tarun Gupta, Wenbo Gong, Chao Ma, Nick Pawlowski, Agrin Hilmkil, Meyer Scetbon, Marc Rigter, Ade Famoti, Ashley Juan Llorens, Jianfeng Gao, et al. The essential role of causality in foundation world models for embodied ai. arXiv preprint arXiv:2402.06665,

    arXiv

  4. [23]

    An embodied generalist agent in 3d world

    Jiangyong Huang, Silong Yong, Xiaojian Ma, Xiongkun Linghu, Puhao Li, Yan Wang, Qing Li, Song- Chun Zhu, Baoxiong Jia, and Siyuan Huang. An embodied generalist agent in 3d world. arXiv preprint arXiv:2311.12871,

    arXiv

  5. [24]

    Rekep: Spatio-temporal reasoning of relational keypoint constraints for robotic manipulation

    Wenlong Huang, Chen Wang, Yunzhu Li, Ruohan Zhang, and Li Fei-Fei. Rekep: Spatio-temporal reasoning of relational keypoint constraints for robotic manipulation. arXiv preprint arXiv:2409.01652,

    arXiv

  6. [25]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276,

    arXiv

  7. [26]

    π0.5: a vision-language-action model with open-world generalization

    Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, et al. π0.5: a vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054,

    arXiv

  8. [28]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980,

    arXiv

  9. [30]

    Making sense of vision and touch: Self-supervised learning of multimodal representations for contact-rich tasks

    Michelle A Lee, Yuke Zhu, Krishnan Srinivasan, Parth Shah, Silvio Savarese, Li Fei-Fei, Animesh Garg, and Jeannette Bohg. Making sense of vision and touch: Self-supervised learning of multimodal representations for contact-rich tasks. In 2019 International conference on roboti...

    2019

  10. [31]

    Cogact: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation

    Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, et al. Cogact: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation. arXiv preprint arXiv:2411.19650,

    arXiv

  11. [32]

    Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024a

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024a. Bang Liu, Xinfeng Li, Jiayi Zhang, Jinlin Wang, Tanjin He, Sirui Hong, Hongz...

    arXiv

  12. [34]

    Harnessing low-fidelity data to accelerate bayesian optimization via posterior regularization

    Bin Liu. Harnessing low-fidelity data to accelerate bayesian optimization via posterior regularization. In 2020 IEEE International Conference on Big Data and Smart Computing (BigComp) , pp. 140–146. IEEE,

    2020

  13. [35]

    Embodied intelligence: A synergy of morphology, action, perception and learning

    Huaping Liu, Di Guo, and Angelo Cangelosi. Embodied intelligence: A synergy of morphology, action, perception and learning. ACM Computing Surveys, 57(7):1–36, 2025b. Jiaming Liu, Mengzhen Liu, Zhenyu Wang, Pengju An, Xiaoqi Li, Kaichen Zhou, Senqiao Yang, Renrui Zhang, Yandong...

    arXiv

  14. [37]

    Closed loop interactive embodied reasoning for robot manipulation

    Michal Nazarczuk, Jan Kristof Behrens, Karla Stepanova, Matej Hoffmann, and Krystian Mikolajczyk. Closed loop interactive embodied reasoning for robot manipulation. arXiv preprint arXiv:2404.15194 ,

    arXiv

  15. [38]

    On-robot bayesian reinforcement learning for pomdps

    Hai Nguyen, Sammie Katt, Yuchen Xiao, and Christopher Amato. On-robot bayesian reinforcement learning for pomdps. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pp. 9480–9487. IEEE,

    2023

  16. [39]

    Vision-language-action model and diffusion policy switching enables dexterous control of an anthropomorphic hand

    Cheng Pan, Kai Junge, and Josie Hughes. Vision-language-action model and diffusion policy switching enables dexterous control of an anthropomorphic hand. arXiv preprint arXiv:2410.14022,

    arXiv

  17. [40]

    Sayplan: Grounding large language models using 3d scene graphs for scalable robot task planning

    Krishan Rana, Jesse Haviland, Sourav Garg, Jad Abou-Chakra, Ian Reid, and Niko Suenderhauf. Sayplan: Grounding large language models using 3d scene graphs for scalable robot task planning. arXiv preprint arXiv:2307.06135,

    arXiv

  18. [41]

    A generalist agent

    Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander Novikov, Gabriel Barth- Maron, Mai Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, et al. A generalist agent. arXiv preprint arXiv:2205.06175,

    arXiv

  19. [42]

    An embodied incremental bayesian model of cross- situational word learning

    12 Sepideh Sadeghi, Matthias Scheutz, and Evan Krause. An embodied incremental bayesian model of cross- situational word learning. In 2017 joint IEEE international conference on development and learning and epigenetic robotics (ICDL-EpiRob), pp. 172–177. IEEE,

    2017

  20. [45]

    V oyager: An open-ended embodied agent with large language models

    13 Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. V oyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291, 2023a. Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, J...

    arXiv

  21. [46]

    Visual transformers: Token-based image representation and processing for computer vision

    Bichen Wu, Chenfeng Xu, Xiaoliang Dai, Alvin Wan, Peizhao Zhang, Zhicheng Yan, Masayoshi Tomizuka, Joseph Gonzalez, Kurt Keutzer, and Peter Vajda. Visual transformers: Token-based image representation and processing for computer vision. arXiv preprint arXiv:2006.03677,

    2006 arXiv

  22. [47]

    Embodied-rag: General non-parametric embodied memory for retrieval and generation

    Quanting Xie, So Yeon Min, Pengliang Ji, Yue Yang, Tianyi Zhang, Kedi Xu, Aarav Bajaj, Ruslan Salakhut- dinov, Matthew Johnson-Roberson, and Yonatan Bisk. Embodied-rag: General non-parametric embodied memory for retrieval and generation. arXiv preprint arXiv:2409.18313,

    arXiv

  23. [49]

    Building cooperative embodied agents modularly with large language models

    Hongxin Zhang, Weihua Du, Jiaming Shan, Qinhong Zhou, Yilun Du, Joshua B Tenenbaum, Tianmin Shu, and Chuang Gan. Building cooperative embodied agents modularly with large language models. arXiv preprint arXiv:2307.02485, 2023a. Jianke Zhang, Yanjiang Guo, Xiaoyu Chen, Yen-Jen ...

    arXiv

  24. [50]

    Exploring collaboration mechanisms for llm agents: A social psychology view

    Jintian Zhang, Xin Xu, Ningyu Zhang, Ruibo Liu, Bryan Hooi, and Shumin Deng. Exploring collaboration mechanisms for llm agents: A social psychology view. arXiv preprint arXiv:2310.02124, 2023b. Qiang Zhang, Jinyuan Fang, Zaiqiao Meng, Shangsong Liang, and Emine Yilmaz. Variati...

    arXiv

  25. [51]

    Prompt, generate, then cache: Cascade of foundation models makes strong few-shot learners

    Renrui Zhang, Xiangfei Hu, Bohao Li, Siyuan Huang, Hanqiu Deng, Yu Qiao, Peng Gao, and Hongsheng Li. Prompt, generate, then cache: Cascade of foundation models makes strong few-shot learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition ...

    arXiv

  26. [52]

    3d-vla: A 3d vision-language-action generative world model

    Haoyu Zhen, Xiaowen Qiu, Peihao Chen, Jincheng Yang, Xin Yan, Yilun Du, Yining Hong, and Chuang Gan. 3d-vla: A 3d vision-language-action generative world model. arXiv preprint arXiv:2403.09631,

    arXiv

  27. [53]

    Agentstudio: A toolkit for building general virtual agents

    Longtao Zheng, Zhiyuan Huang, Zhenghai Xue, Xinrun Wang, Bo An, and Shuicheng Yan. Agentstudio: A toolkit for building general virtual agents. arXiv preprint arXiv:2403.17918,

    arXiv

  28. [54]

    Dexgraspvla: A vision-language-action framework towards general dexterous grasping

    Yifan Zhong, Xuchuan Huang, Ruochong Li, Ceyao Zhang, Yitao Liang, Yaodong Yang, and Yuanpei Chen. Dexgraspvla: A vision-language-action framework towards general dexterous grasping. arXiv preprint arXiv:2502.20900,

  29. [55]

    Webarena: A realistic web environment for building autonomous agents

    Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, et al. Webarena: A realistic web environment for building autonomous agents. arXiv preprint arXiv:2307.13854, 2023a. Siyuan Zhou, Yilun Du, Jiaben Ch...

    arXiv

  30. [56]

    Large language model as a policy teacher for training reinforcement learning agents

    15 Zihao Zhou, Bin Hu, Chenyang Zhao, Pu Zhang, and Bin Liu. Large language model as a policy teacher for training reinforcement learning agents. In The 33rd Inter. Joint Conf. on Artificial Intelligence (IJCAI 2024), 2023b. Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Te...

    2024

  31. [1995]

    Openvla: An open-source vision-language- action model

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision-language- action model. arXiv preprint arXiv:2406.09246,

    arXiv

  32. [2003]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language...

    arXiv

  33. [2007]

    Chatglm: A family of large language models from glm-130b to glm-4 all tools

    Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Dan Zhang, Diego Rojas, Guanyu Feng, Hanlin Zhao, et al. Chatglm: A family of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406.12793,

    arXiv

  34. [2011]

    Rt-1: Robotics transformer for real- world control at scale

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. Rt-1: Robotics transformer for real- world control at scale. arXiv preprint arXiv:2212.06817,

    arXiv

  35. [2012]

    Agibot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems

    Qingwen Bu, Jisong Cai, Li Chen, Xiuqi Cui, Yan Ding, Siyuan Feng, Shenyuan Gao, Xindong He, Xuan Hu, Xu Huang, et al. Agibot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems. arXiv preprint arXiv:2503.06669,

    arXiv

  36. [2013]

    Gr00t n1: An open foundation model for generalist humanoid robots

    Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, et al. Gr00t n1: An open foundation model for generalist humanoid robots. arXiv preprint arXiv:2503.14734,

    arXiv

  37. [2015]

    Qwen2 technical report

    Qwen Team. Qwen2 technical report. arXiv preprint arXiv:2407.10671,

    arXiv

  38. [2016]

    Spa- tiallm: Training large language models for structured indoor modeling

    11 Yongsen Mao, Junhao Zhong, Chuan Fang, Jia Zheng, Rui Tang, Hao Zhu, Ping Tan, and Zihan Zhou. Spa- tiallm: Training large language models for structured indoor modeling. arXiv preprint arXiv:2506.07491,

  39. [2017]

    A very brief and critical discussion on automl

    Bin Liu. A very brief and critical discussion on automl. arXiv preprint arXiv:1811.03822,

    arXiv

  40. [2018]

    Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation

    Zipeng Fu, Tony Z Zhao, and Chelsea Finn. Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation. arXiv preprint arXiv:2401.02117,

    arXiv

  41. [2019]

    Embodiedsam: Online segment any 3d thing in real time

    Xiuwei Xu, Huangxing Chen, Linqing Zhao, Ziwei Wang, Jie Zhou, and Jiwen Lu. Embodiedsam: Online segment any 3d thing in real time. arXiv preprint arXiv:2408.11811,

    arXiv

  42. [2020]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609,

    arXiv

  43. [2021]

    Sara-rt: Scaling up robotics transformers with self- adaptive robust attention

    Isabel Leal, Krzysztof Choromanski, Deepali Jain, Avinava Dubey, Jake Varley, Michael Ryoo, Yao Lu, Frederick Liu, Vikas Sindhwani, Quan Vuong, et al. Sara-rt: Scaling up robotics transformers with self- adaptive robust attention. In 2024 IEEE International Conference on Robot...

    2024

  44. [2022]

    Autort: Embodied foundation models for large scale orchestration of robotic agents

    Michael Ahn, Debidatta Dwibedi, Chelsea Finn, Montse Gonzalez Arenas, Keerthana Gopalakrishnan, Karol Hausman, Brian Ichter, Alex Irpan, Nikhil Joshi, Ryan Julian, et al. Autort: Embodied foundation models for large scale orchestration of robotic agents. arXiv preprint arXiv:2...

  45. [2023]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923,

    arXiv

  46. [2024]

    arXiv preprint arXiv:2410.24164,

    doi: 10.48550. arXiv preprint arXiv:2410.24164,

    arXiv

  47. [2025]

    How to sim2real with gaussian processes: Prior mean versus kernels as priors

    Rika Antonova, Akshara Rai, and Danica Kragic. How to sim2real with gaussian processes: Prior mean versus kernels as priors. In 2nd Workshop on Closing the Reality Gap in Sim2Real Transfer for Robotics. RSS, 2020.,

    2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.