Pith. sign in

REVIEW 3 major objections 5 minor 102 references

Hypothesis-driven Model Expansion under Uncertainty for Open-World Robot Planning

T0 review · 3 major / 5 minor · reviewed 2026-07-11 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Robots can expand incomplete world models by treating missing facts as uncertain hypotheses and verifying them while planning toward goals.

desk verdict Solid systems paper: treating LLM model expansion as uncertain hypotheses and interleaving verification with classical planning is a clean, useful engineering move, with real-robot evidence that the ablations matter. read the letter →

arxiv 2607.06501 v1 pith:BQPE5SNB submitted 2026-07-07 cs.RO

Anxing Xiao , Hanbo Zhang , Tianrun Hu , David Hsu This is my paper · ORCID
classification cs.RO
keywords open-worldplanninghypothesis-drivenmodelexpansionuncertainty-awareservicerobotsfoundationmodelsPDDLmobilemanipulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Service robots in homes face incomplete knowledge: objects may be hidden, attributes unknown, and action effects underspecified. Closed-world planners fail when the model is incomplete, and language models alone can invent facts that look plausible but are wrong. This paper claims that the right move is not to replace planning with a language model, but to use the language model to propose object-centric hypotheses about missing locations, attributes, and action effects, keep those hypotheses uncertain, and plan so that verification and task progress happen together. The resulting system, HUME, iteratively generates hypotheses, plans under their uncertainty, executes verification actions, and updates the model from foundation-model feedback. Experiments in block-processing worlds, household mobile manipulation, real Fetch-robot kitchens, and microwave operation show large gains in success and path efficiency when uncertainty is made explicit, for both classical and language-model planners. A sympathetic reader cares because this is a concrete path from static pre-programmed knowledge to autonomous knowledge expansion in real open environments.

What carries the argument

HUME (Hypothesis-driven Uncertainty-aware Model Expansion): foundation models generate factorized object-centric hypotheses; all-outcomes determinization plus cost penalties produce plans that interleave verification with task actions; verification feedback updates the symbolic model and triggers re-generation.

What would settle it

Run the same open-world household tasks with a language model that systematically invents wrong locations or attributes and mislabels verification images: if success collapses to the no-expansion baseline, the central claim fails.

Watch

Extended reading notes

Core claim

Explicitly representing model expansion as a set of uncertain object-centric hypotheses, and integrating hypothesis verification into goal-reaching planning, restores solvability and substantially raises success rates under incomplete knowledge, for both formal PDDL planners and language-model planners, in simulation and on real robots.

Load-bearing premise

The language model must generate correct hypotheses in a bounded number of tries, propose workable verification conditions, and judge them correctly from sensory observations; if it cannot, the expansion loop cannot close the model gap.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes HUME, an open-world planning framework in which incomplete symbolic models are expanded by object-centric hypotheses (existence, attributes, action effects) generated by foundation models. Hypotheses are treated as uncertain latent variables; classical or LLM planners produce plans that interleave task actions with verification actions via all-outcomes determinization and cost bias, then update the model from VLM/perception feedback and replan (Alg. 1, §IV). Experiments in Block Processing World, AI2-THOR mobile manipulation, real Fetch household tasks, and a microwave appliance demo show that uncertainty-aware expansion substantially improves success rate and SPL over no-expansion and deterministic-expansion ablations for both PDDL and LLM planners.

Significance. If the empirical claims hold under the stated assumptions, the work offers a practical bridge between structured automated planning and unstructured foundation-model knowledge for household robots: model expansion is made explicit, uncertainty-aware, and actively verifiable inside goal-directed planning rather than treated as passive domain generation. Strengths include a clean six-way ablation (representation × inference), consistent gains across simulation and real hardware, dual support for formal and LLM planners, and an honest limitations discussion (§VI, App. A.IV). The hypothesis structure and verification-in-planning design are reusable engineering contributions for open-world service robotics.

major comments (3)
  1. [§IV.F, App. A.II-B/C, Figs. 6/8/10] §IV.F Assumption (3) and App. A.II-B: The iterative loop (Alg. 1) closes the model gap only if the LM generates correct hypotheses within I_max attempts, proposes sufficient verification_conditions, and the VLM correctly evaluates them. Residual real-world failures are already attributed to attribute misclassification and grasp-dependent visibility (App. A.II-C). The main results (Figs. 6, 8, 10) do not report hypothesis-generation success rates, verification accuracy, or how often regeneration was required. Without these metrics, the claimed advantage of uncertainty-aware expansion over deterministic expansion cannot be fully separated from the reliability of the particular gpt-4.1/VLM stack. Please add quantitative failure breakdowns for generation and verification, or qualify the autonomy claims accordingly.
  2. [§V.D, Abstract, §VII] §V.D Real-world protocol: Execution noise is mitigated by retrying skills until success and manually resetting invalid states (e.g., drops). This is reasonable for isolating planning, but the abstract and conclusion claim “autonomous knowledge expansion” and “effective operation” without reporting how often retries/resets occurred or how many trials would have failed without intervention. Please report intervention rates per task/planner or explicitly scope the real-world claims to planning performance under idealized low-level execution.
  3. [§IV.D, §III.B, §VI.a] §IV.D Determinization: Verification actions are all-outcomes-determinized and a_h−_verify is excluded so the planner is optimistic; negative outcomes are handled only by rejection and replanning. The paper frames the setting as Bayes-adaptive (§I, §III.B), yet ternary beliefs and branch-cut optimism do not reason about graded risk or irreversible verification side-effects (acknowledged in §VI.a and App. A.IV-D). This is acceptable as a design choice, but the Bayes-adaptive framing should be softened or the optimistic bias stated as a first-class limitation of the planning objective, not only of future work.
minor comments (5)
  1. [Fig. 3, §IV.B, App. A.III-A] Fig. 3 and hypothesis JSON examples: “object_existance” / “existance” should be “existence”; “Trigged” in domain listings should be “Triggered” for consistency with prose.
  2. [Fig. 4, §V.A] Fig. 4’s six-category taxonomy is useful but the caption and body (§V.A) could more explicitly map each bar group in Figs. 6/8/10 to the six named approaches to avoid reader cross-referencing.
  3. [§IV.D, App. A.III] Free parameters (verification penalty c ≫ 0, I_max, LLM sampling) are listed only implicitly; a short sensitivity note or fixed values in the appendix would aid reproducibility.
  4. [§II.B] Related work §II.B: Tru-POMDP [58] and Seeing-is-Believing [75] are close; a one-sentence contrast on whether verification actions are planned vs. manually specified would sharpen novelty.
  5. [§VI, Fig. 3] Typo: “preconditiaons” in §VI; “deks1” in Fig. 3 plan snippet; “wallmountedcontrolpanel1” style names are fine but ensure PDDL listings match figure labels.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: empirical systems paper whose claims rest on external task success metrics, not on predictions forced by construction or self-citation chains.

full rationale

HUME is a method/systems paper that generates object-centric hypotheses via foundation models, augments a PDDL (or LLM) planning problem with verification actions under all-outcomes determinization, executes, and updates from sensory/VLM feedback (Alg. 1, §IV). The central claims are comparative success-rate and SPL gains of uncertainty-aware expansion over static and deterministic-expansion baselines (Figs. 6–8, 10; real-world T1–T5). These metrics are external environment outcomes, not quantities fitted then re-predicted. There is no self-definitional loop (hypotheses are not defined in terms of the success metric), no parameter fit re-labeled as prediction, no uniqueness theorem or ansatz imported from overlapping-author prior work that forces the result, and no renaming of a known empirical pattern. Stated assumptions (IV.F) about LM/VLM reliability are soft spots for correctness risk, not circularity. Self-citations, if any, are background; the load-bearing evidence is the new closed-loop experiments. Score 0 is therefore the honest finding.

Assumptions & free parameters 3 free parameters · 5 assumptions · 1 invented entities

The central claim rests on standard planning machinery plus several domain assumptions about foundation-model reliability and task scoping that are necessary for the loop to terminate successfully. Free parameters are engineering choices (costs, iteration bounds) rather than fitted scientific constants. No new physical entities are postulated.

free parameters (3)
  • verification penalty cost c ≫ 0
    Hand-chosen high cost on actions that depend on unverified hypotheses to bias the planner toward early verification; ratio 1:10 used in implementation.
  • I_max (max hypothesis-generation / replan iterations)
    Termination bound; value not critical to the claim but required for practical runs.
  • LLM temperature / sampling settings (gpt-4.1)
    Implicit free choices that affect hypothesis quality; not ablated.
assumptions (5)
  • domain assumption Atomic skills execute reliably under full observability once the symbolic state is correct (Task Scoping).
    Stated in IV.F (1); isolates planning from low-level control noise.
  • domain assumption Task goals are expressible in structured logic and encode the abstraction of missing knowledge (Goal Requirements).
    IV.F (2); missing concepts/predicates assumed present in the instruction.
  • domain assumption Foundation model can generate correct hypotheses in bounded attempts, propose verification conditions, and evaluate them from sensory data.
    IV.F (3); load-bearing for the expansion loop.
  • domain assumption All-outcomes determinization plus optimistic branch-cut (exclude false outcomes) yields useful plans; negative outcomes handled by replan.
    Section IV.D; standard technique but optimistic bias is acknowledged as a limitation.
  • standard math Classical PDDL / numeric planning (Fast Downward, ENHSP) correctly solves the augmented deterministic problems.
    Standard automated-planning assumption.
invented entities (1)
  • Object-centric hypothesis structure (id, type, content API, condition, verification_condition)
    purpose: Factorizes model incompleteness into verifiable latent facts that can be injected into PDDL and tracked by belief variables.
    Methodological construct; no independent physical existence claimed. Evidence is empirical success of the loop.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hypothesis-driven Model Expansion under Uncertainty for Open-World Robot Planning." pith.science (2026). https://pith.science/paper/BQPE5SNB

@misc{pith2026260706501,
  author       = {Pith},
  title        = {Pith review of: Hypothesis-driven Model Expansion under Uncertainty for Open-World Robot Planning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BQPE5SNB}},
  note         = {Machine review of arXiv:2607.06501}
}
read the original abstract

We consider an open-world planning setting in which service robots must operate in unknown environments with incomplete knowledge of objects and actions. Traditional closed-world approaches with pre-programmed knowledge bases fail when robots encounter unexpected situations and tasks, posing a fundamental challenge for autonomous knowledge expansion in human environments. In this work, we propose an open-world planning framework that enables robots to automatically generate, verify, and update hypotheses about their abstract world models. Our key insight is to explicitly maintain uncertainty-aware knowledge expansion and integrate hypothesis verification into goal-reaching planning. The framework leverages foundation models to generate initial hypotheses over states and transitions, and applies automated planning to produce action sequences that jointly address hypothesis verification and task execution. Through iterative execution and refinement, the robot expands its knowledge by incorporating verification feedback from the foundation models when hypotheses prove incorrect. Extensive experiments in simulated and real-world environments demonstrate that our framework enables autonomous knowledge expansion and effective operation in open-world settings. These results indicate that integrating uncertainty-aware model expansion from robot foundation models with planning advances the practical deployment of household service robots.

Figures

Figures reproduced from arXiv: 2607.06501 by the authors.

Figure 1
Figure 1. Illustration of a service robot operating in an open-world scenario. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. High-level idea. It combines foundation-model priors as hypotheses [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Overview of the HUME. The robot iteratively generates hypotheses to expand its model, plans with a model augmented by uncertain hypotheses, [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (25 more)
Figure 4
Figure 4. Figure 4: Overview of the analysis of six planning categories, organized by [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 6
Figure 6. Figure 6: Simulation Results on Block Processing World: Success Rate and [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 9
Figure 9. Figure 9: The real-world experimental setup comprising a kitchen, living room, [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 8
Figure 8. Figure 8: Simulation Results on Mobile Manipulation in Unknown Environ [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 10
Figure 10. Figure 10: Real-World Experiment Results: Success Rate and Success weighted [PITH_FULL_IMAGE:figures/full_fig_p007_10.png]
Figure 11
Figure 11. Figure 11: Snapshot of completing the task T1: Deliver a zero-sugar drink to the table. The robot is required to infer the drink’s location, identify the zero-sugar option, and deliver it to the table [PITH_FULL_IMAGE:figures/full_fig_p008_11.png]
Figure 12
Figure 12. Figure 12: Snapshot of completing the task T5: Throw away a blue-floral bowl and make the living room light-off. The robot is required to locate the bowl, identify the blue-floral pattern, and determine how to turn off the living room light. verifies object existence. After pick…
Figure 13
Figure 13. Figure 13: Demonstration of a robot discovering microwave button functions [PITH_FULL_IMAGE:figures/full_fig_p008_13.png]
Figure 14
Figure 14. Figure 14: Real-world experimental setup 1) Environments and Tasks: Our real-robot experiments were conducted in a real household environment comprising three rooms: a kitchen, a living room, and a meeting room [PITH_FULL_IMAGE:figures/full_fig_p015_14.png]
Figure 15
Figure 15. Figure 15: The behavior tree for picking up an object [PITH_FULL_IMAGE:figures/full_fig_p016_15.png]
Figure 16
Figure 16. Figure 16: The four different views of the in-hand object [PITH_FULL_IMAGE:figures/full_fig_p017_16.png]
Figure 17
Figure 17. Figure 17: The visualization of the Block Processing World, triggering a processor will change the state of the block on the processor, and the robot can only [PITH_FULL_IMAGE:figures/full_fig_p018_17.png]
Figure 18
Figure 18. Figure 18: Visualizations of all ProcTHOR house environments used in our experiments. [PITH_FULL_IMAGE:figures/full_fig_p018_18.png]
Figure 19
Figure 19. Figure 19: The visualization of the objects with the specified attributes in our experiments. [PITH_FULL_IMAGE:figures/full_fig_p019_19.png]
Figure 20
Figure 20. Figure 20: Visualization of the Appliance Operation Demo setup. [PITH_FULL_IMAGE:figures/full_fig_p019_20.png]
Figure 21
Figure 21. Figure 21: Behavioral Analysis of Task 1: Deliver a zero-sugar drink to the table. [PITH_FULL_IMAGE:figures/full_fig_p021_21.png]
Figure 22
Figure 22. Figure 22: Behavioral Analysis of Task 2: Place the remote with red button into the cabinet [PITH_FULL_IMAGE:figures/full_fig_p021_22.png]
Figure 23
Figure 23. Figure 23: Behavioral Analysis of Task 3: Move the smiley-face mug to the fridge [PITH_FULL_IMAGE:figures/full_fig_p022_23.png]
Figure 24
Figure 24. Figure 24: Behavioral Analysis of Task 4: Serve a heated chicken burger on the coffee table. [PITH_FULL_IMAGE:figures/full_fig_p022_24.png]
Figure 25
Figure 25. Figure 25: Behavioral Analysis of Task 5: Throw away the blue-floral bowl and turn off the living room light. [PITH_FULL_IMAGE:figures/full_fig_p022_25.png]
Figure 26
Figure 26. Figure 26: Failure mode: Uncertain Model Expansion failure due to attribute misclassification. [PITH_FULL_IMAGE:figures/full_fig_p023_26.png]
Figure 27
Figure 27. Figure 27: Failure mode: Missing proactive search/verification (Baseline). [PITH_FULL_IMAGE:figures/full_fig_p023_27.png]
Figure 28
Figure 28. Figure 28: Analysis of grasping strategy impact on attribute verification. [PITH_FULL_IMAGE:figures/full_fig_p024_28.png]
Figure 29
Figure 29. Figure 29: Detailed quantitative results for Block Processing World. [PITH_FULL_IMAGE:figures/full_fig_p025_29.png]
Figure 30
Figure 30. Figure 30: Detailed quantitative results for AI2THOR experiments. [PITH_FULL_IMAGE:figures/full_fig_p025_30.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

102 extracted references · 102 canonical work pages

  1. [1]

    From pixels to predicates: Learn- ing symbolic world models via pretrained vlms.IEEE Robotics and Automation Letters, 2026

    Ashay Athalye, Nishanth Kumar, Tom Silver, Yichao Liang, Jiuguang Wang, Tom ´as Lozano-P ´erez, and Leslie Pack Kaelbling. From pixels to predicates: Learn- ing symbolic world models via pretrained vlms.IEEE Robotics and Automation Letters, 2026

    work page 2026

  2. [2]

    Active visual object search in unknown environments using uncertain semantics.IEEE Transactions on Robotics, 29(4):986–1002, 2013

    Alper Aydemir, Andrzej Pronobis, Moritz G ¨obelbecker, and Patric Jensfelt. Active visual object search in unknown environments using uncertain semantics.IEEE Transactions on Robotics, 29(4):986–1002, 2013

    work page 2013

  3. [3]

    Qwen3-vl technical report,

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shu- tong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixua...

    work page

  4. [4]

    URL https://arxiv.org/abs/2511.21631

    work page Pith review arXiv

  5. [5]

    Autogpt+ p: Affordance-based task plan- ning with large language models

    Timo Birr, Christoph Pohl, Abdelrahman Younes, and Tamim Asfour. Autogpt+ p: Affordance-based task plan- ning with large language models. InRobotics: Science and Systems (RSS), 2024

    work page 2024

  6. [6]

    Answer set programming at a glance

    Gerhard Brewka, Thomas Eiter, and Mirosław Truszczy´nski. Answer set programming at a glance. Communications of the ACM, 54(12):92–103, 2011

    work page 2011

  7. [7]

    Do as i can, not as i say: Grounding language in robotic affordances

    Anthony Brohan, Yevgen Chebotar, Chelsea Finn, Karol Hausman, Alexander Herzog, Daniel Ho, Julian Ibarz, Alex Irpan, Eric Jang, Ryan Julian, et al. Do as i can, not as i say: Grounding language in robotic affordances. InConference on robot learning, pages 287–318. PMLR, 2023

    work page 2023

  8. [8]

    Language-augmented symbolic planner for open-world task planning

    Guanqi Chen, Lei Yang, Ruixing Jia, Zhe Hu, Yizhou Chen, Wei Zhang, Wenping Wang, and Jia Pan. Language-augmented symbolic planner for open-world task planning. InRobotics: Science and Systems (RSS), 2024

    work page 2024

Show all 102 references
  1. [9]

    Situated bayesian reasoning framework for robots operating in diverse everyday environments

    Sonia Chernova, Vivian Chu, Angel Daruna, Haley Gar- rison, Meera Hahn, Priyanka Khante, Weiyu Liu, and Andrea Thomaz. Situated bayesian reasoning framework for robots operating in diverse everyday environments. InRobotics Research: The 18th International Symposium ISRR, pages...

    2020

  2. [10]

    Generalised do- main model acquisition from action traces

    Stephen Cresswell and Peter Gregory. Generalised do- main model acquisition from action traces. InPro- ceedings of the international conference on automated planning and scheduling, volume 21, pages 42–49, 2011

    2011

  3. [11]

    Trust the proc3s: Solving long-horizon robotics problems with llms and constraint satisfaction

    Aidan Curtis, Nishanth Kumar, Jing Cao, Tom ´as Lozano- P´erez, and Leslie Pack Kaelbling. Trust the proc3s: Solving long-horizon robotics problems with llms and constraint satisfaction. InConference on Robot Learning, 2024

    2024

  4. [12]

    Partially observable task and motion planning with uncertainty and risk awareness

    Aidan Curtis, George Matheos, Nishad Gothoskar, Vikash Mansinghka, Joshua Tenenbaum, Tom´as Lozano- P´erez, and Leslie Pack Kaelbling. Partially observable task and motion planning with uncertainty and risk awareness. InRobotics: Science and Systems (RSS), 2024

    2024

  5. [13]

    Llm-guided probabilistic program induction for pomdp model estimation

    Aidan Curtis, Hao Tang, Thiago Veloso, Kevin El- lis, Joshua B Tenenbaum, Tom ´as Lozano-P ´erez, and Leslie Pack Kaelbling. Llm-guided probabilistic program induction for pomdp model estimation. InConference on Robot Learning, pages 3137–3184. PMLR, 2025

    2025

  6. [14]

    Embodied question answering

    Abhishek Das, Samyak Datta, Georgia Gkioxari, Stefan Lee, Devi Parikh, and Dhruv Batra. Embodied question answering. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 1–10, 2018

    2018

  7. [15]

    Procthor: large-scale embodied ai using procedural generation

    Matt Deitke, Eli Vander Bilt, Alvaro Herrasti, Luca Weihs, Jordi Salvador, Kiana Ehsani, Winson Han, Eric Kolve, Ali Farhadi, Aniruddha Kembhavi, et al. Procthor: large-scale embodied ai using procedural generation. In Proceedings of the 36th International Conference on Neural...

    2022

  8. [16]

    Procthor: Large-scale embodied ai using procedural gen- eration.Advances in Neural Information Processing Systems, 35:5982–5994, 2022

    Matt Deitke, Eli VanderBilt, Alvaro Herrasti, Luca Weihs, Kiana Ehsani, Jordi Salvador, Winson Han, Eric Kolve, Aniruddha Kembhavi, and Roozbeh Mottaghi. Procthor: Large-scale embodied ai using procedural gen- eration.Advances in Neural Information Processing Systems, 35:5982–...

    2022

  9. [17]

    Automated generation of robotic planning do- mains from observations

    Maximilian Diehl, Chris Paxton, and Karinne Ramirez- Amaro. Automated generation of robotic planning do- mains from observations. In2021 IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems (IROS), pages 6732–6738. IEEE, 2021

    2021

  10. [18]

    Integrating action knowledge and llms for task planning and situation handling in open worlds.Au- tonomous Robots, 47(8):981–997, 2023

    Yan Ding, Xiaohan Zhang, Saeid Amiri, Nieqing Cao, Hao Yang, Andy Kaminski, Chad Esselink, and Shiqi Zhang. Integrating action knowledge and llms for task planning and situation handling in open worlds.Au- tonomous Robots, 47(8):981–997, 2023

    2023

  11. [19]

    Palm- e: An embodied multimodal language model.arXiv preprint arXiv:2303.03378, 2023

    Danny Driess, Fei Xia, Mehdi SM Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, et al. Palm- e: An embodied multimodal language model.arXiv preprint arXiv:2303.03378, 2023

    2023 arXiv

  12. [20]

    University of Massachusetts Amherst, 2002

    Michael O’Gordon Duff.Optimal Learning: Computa- tional procedures for Bayes-adaptive Markov decision processes. University of Massachusetts Amherst, 2002

    2002

  13. [21]

    Maria Fox and Derek Long. Pddl2. 1: An extension to pddl for expressing temporal planning domains.Journal of artificial intelligence research, 20:61–124, 2003

    2003

  14. [22]

    Pddlstream: Integrating symbolic planners and blackbox samplers via optimistic adaptive planning

    Caelan Reed Garrett, Tom ´as Lozano-P ´erez, and Leslie Pack Kaelbling. Pddlstream: Integrating symbolic planners and blackbox samplers via optimistic adaptive planning. InProceedings of the international conference on automated planning and scheduling, volume 30, pages 440–448, 2020

    2020

  15. [23]

    Online replanning in belief space for partially observable task and motion problems

    Caelan Reed Garrett, Chris Paxton, Tom ´as Lozano-P´erez, Leslie Pack Kaelbling, and Dieter Fox. Online replanning in belief space for partially observable task and motion problems. In2020 IEEE International Conference on Robotics and Automation (ICRA), pages 5678–5684. IEEE, 2020

    2020

  16. [24]

    Integrated task and motion plan- ning.Annual review of control, robotics, and autonomous systems, 4(1):265–293, 2021

    Caelan Reed Garrett, Rohan Chitnis, Rachel Holladay, Beomjoon Kim, Tom Silver, Leslie Pack Kaelbling, and Tom´as Lozano-P ´erez. Integrated task and motion plan- ning.Annual review of control, robotics, and autonomous systems, 4(1):265–293, 2021

    2021

  17. [25]

    Improved techniques for grid mapping with rao- blackwellized particle filters.IEEE transactions on Robotics, 2007

    Giorgio Grisetti, Cyrill Stachniss, and Wolfram Bur- gard. Improved techniques for grid mapping with rao- blackwellized particle filters.IEEE transactions on Robotics, 2007

    2007

  18. [26]

    Ros navigation: Concepts and tutorial

    Rodrigo Longhi Guimar ˜aes, Andr ´e Schneider de Oliveira, Jo ˜ao Alberto Fabro, Thiago Becker, and Vin ´ıcius Amilgar Brenner. Ros navigation: Concepts and tutorial. InRobot Operating System (ROS) The Complete Reference (Volume 1), pages 121–160. Springer, 2016

    2016

  19. [27]

    The fast downward planning system

    Malte Helmert. The fast downward planning system. Journal of Artificial Intelligence Research, 26:191–246, 2006

    2006

  20. [28]

    Vectorized online pomdp planning.arXiv preprint arXiv:2510.27191, 2025

    Marcus Hoerger, Muhammad Sudrajat, and Hanna Kurni- awati. Vectorized online pomdp planning.arXiv preprint arXiv:2510.27191, 2025

    2025 arXiv

  21. [29]

    Language models as zero-shot planners: Extracting actionable knowledge for embodied agents

    Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. InInternational conference on machine learning, pages 9118–9147. PMLR, 2022

    2022

  22. [30]

    Inner monologue: Embodied reasoning through planning with language models

    Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, et al. Inner monologue: Embodied reasoning through planning with language models. InConference on Robot Learning, pages 1769–

  23. [31]

    5: a vision-language-action model with open-world generalization, 2025.URL https://arxiv

    Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Es- mail, Michael Equi, Chelsea Finn, Niccolo Fusai, et al.π0. 5: a vision-language-action model with open-world generalization, 2025.URL https://arxiv. org/abs/2504.16054, 1(2):3

    2025 arXiv

  24. [32]

    Open-world reasoning for service robots

    Yuqian Jiang, Nick Walker, Justin Hart, and Peter Stone. Open-world reasoning for service robots. InProceedings of the international conference on automated planning and scheduling, volume 29, pages 725–733, 2019

    2019

  25. [33]

    Vec-qmdp: Vectorized pomdp planning on cpus for real-time au- tonomous driving.arXiv preprint arXiv:2602.08334, 2026

    Xuanjin Jin, Yanxin Dong, Bin Sun, Huan Xu, Zhihui Hao, XianPeng Lang, and Panpan Cai. Vec-qmdp: Vectorized pomdp planning on cpus for real-time au- tonomous driving.arXiv preprint arXiv:2602.08334, 2026

    2026 arXiv

  26. [34]

    Inte- grated task and motion planning in belief space.The International Journal of Robotics Research, 32(9-10): 1194–1227, 2013

    Leslie Pack Kaelbling and Tom ´as Lozano-P ´erez. Inte- grated task and motion planning in belief space.The International Journal of Robotics Research, 32(9-10): 1194–1227, 2013

    2013

  27. [35]

    Segment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. InProceedings of the IEEE/CVF International Conference on Computer Vision, 2023

    2023

  28. [36]

    Ai2-thor: An interactive 3d environment for visual ai.arXiv preprint arXiv:1712.05474, 2017

    Eric Kolve, Roozbeh Mottaghi, Winson Han, Eli Vander- Bilt, Luca Weihs, Alvaro Herrasti, Matt Deitke, Kiana Ehsani, Daniel Gordon, Yuke Zhu, et al. Ai2-thor: An interactive 3d environment for visual ai.arXiv preprint arXiv:1712.05474, 2017

    2017 arXiv

  29. [37]

    Open-world task and motion plan- ning via vision-language model generated constraints

    Nishanth Kumar, William Shen, Fabio Ramos, Dieter Fox, Tom ´as Lozano-P ´erez, Leslie Pack Kaelbling, and Caelan Reed Garrett. Open-world task and motion plan- ning via vision-language model generated constraints. IEEE Robotics and Automation Letters, 2026

    2026

  30. [38]

    Kinodynamic task and motion planning using vlm-guided and interleaved sampling

    Minseo Kwon and Young J Kim. Kinodynamic task and motion planning using vlm-guided and interleaved sampling. InIEEE International Conference on Robotics and Automation (ICRA), 2026

    2026

  31. [39]

    Code as policies: Language model programs for em- bodied control

    Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as policies: Language model programs for em- bodied control. In2023 IEEE International Conference on Robotics and Automation (ICRA), pages 9493–9500. IEEE, 2023

    2023

  32. [40]

    Visualpredicator: Learning abstract world models with neuro-symbolic predicates for robot planning

    Yichao Liang, Nishanth Kumar, Hao Tang, Adrian Weller, Joshua B Tenenbaum, Tom Silver, Jo ˜ao F Hen- riques, and Kevin Ellis. Visualpredicator: Learning abstract world models with neuro-symbolic predicates for robot planning. InInternational Conference on Learning Representati...

    2025

  33. [41]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll ´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. InEuropean conference on computer vision, pages 740–755. Springer, 2014

    2014

  34. [42]

    Llm+ p: Em- powering large language models with optimal planning proficiency.arXiv preprint arXiv:2304.11477, 2023

    Bo Liu, Yuqian Jiang, Xiaohan Zhang, Qiang Liu, Shiqi Zhang, Joydeep Biswas, and Peter Stone. Llm+ p: Em- powering large language models with optimal planning proficiency.arXiv preprint arXiv:2304.11477, 2023

    2023 arXiv

  35. [43]

    Learning compositional behaviors from demonstration and language

    Weiyu Liu, Neil Nie, Ruohan Zhang, Jiayuan Mao, and Jiajun Wu. Learning compositional behaviors from demonstration and language. In8th Annual Conference on Robot Learning, 2024

    2024

  36. [44]

    Uncertainty quantification and confidence calibration in large language models: A survey

    Xiaoou Liu, Tiejin Chen, Longchao Da, Chacha Chen, Zhen Lin, and Hua Wei. Uncertainty quantification and confidence calibration in large language models: A survey. InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V . 2, pages 6107–6117, 2025

    2025

  37. [45]

    Reflect: Summarizing robot experiences for failure explanation and correction

    Zeyi Liu, Arpit Bahety, and Shuran Song. Reflect: Summarizing robot experiences for failure explanation and correction. InConference on Robot Learning, pages 3468–3484. PMLR, 2023

    2023

  38. [46]

    Unified planning: Mod- eling, manipulating and solving ai planning problems in python.SoftwareX, 29:102012, 2025

    Andrea Micheli, Arthur Bit-Monnot, Gabriele R ¨oger, Enrico Scala, Alessandro Valentini, Luca Framba, Al- berto Rovetta, Alessandro Trapasso, Luigi Bonassi, Al- fonso Emilio Gerevini, Luca Iocchi, Felix Ingrand, Uwe K¨ockemann, Fabio Patrizi, Alessandro Saetti, Ivan Se- rina, ...

    2025 doi

  39. [47]

    Scaling open-vocabulary object detection.Advances in Neural Information Processing Systems, 36, 2024

    Matthias Minderer, Alexey Gritsenko, and Neil Houlsby. Scaling open-vocabulary object detection.Advances in Neural Information Processing Systems, 36, 2024

    2024

  40. [48]

    Behavior trees in robot control systems.Annual Review of Control, Robotics, and Autonomous Systems, 5(1):81–107, 2022

    Petter ¨Ogren and Christopher I Sprague. Behavior trees in robot control systems.Annual Review of Control, Robotics, and Autonomous Systems, 5(1):81–107, 2022

    2022

  41. [49]

    Say- plan: Grounding large language models using 3d scene graphs for scalable task planning.arXiv preprint arXiv:2307.06135, 2023

    Krishan Rana, Jesse Haviland, Sourav Garg, Jad Abou- Chakra, Ian Reid, and Niko Suenderhauf. Say- plan: Grounding large language models using 3d scene graphs for scalable task planning.arXiv preprint arXiv:2307.06135, 2023

    2023 arXiv

  42. [50]

    Explore until confident: Efficient exploration for embodied question answering

    Allen Z Ren, Jaden Clark, Anushri Dixit, Masha Itkina, Anirudha Majumdar, and Dorsa Sadigh. Explore until confident: Efficient exploration for embodied question answering. InRobotics: Science and Systems (RSS), 2024

    2024

  43. [51]

    Heuristics for numeric planning via subgoaling

    Enrico Scala, Patrik Haslum, and Sylvie Thi ´ebaux. Heuristics for numeric planning via subgoaling. 2016

    2016

  44. [52]

    Tiptop: A modular open-vocabulary plan- ning system for robotic manipulation.arXiv preprint arXiv:2603.09971, 2026

    William Shen, Nishanth Kumar, Sahit Chintalapudi, Jie Wang, Christopher Watson, Edward Hu, Jing Cao, Di- nesh Jayaraman, Leslie Pack Kaelbling, and Tom ´as Lozano-P´erez. Tiptop: A modular open-vocabulary plan- ning system for robotic manipulation.arXiv preprint arXiv:2603.09971, 2026

    2026

  45. [53]

    Re- flexion: an autonomous agent with dynamic memory and self-reflection.arXiv preprint arXiv:2303.11366, 2023

    Noah Shinn, Beck Labash, and Ashwin Gopinath. Re- flexion: an autonomous agent with dynamic memory and self-reflection.arXiv preprint arXiv:2303.11366, 2023

    2023 arXiv

  46. [54]

    Pddl planning with pretrained large language models

    Tom Silver, Varun Hariprasad, Reece S Shuttleworth, Nishanth Kumar, Tom ´as Lozano-P ´erez, and Leslie Pack Kaelbling. Pddl planning with pretrained large language models. InNeurIPS 2022 Foundation Models for Deci- sion Making Workshop, 2022

    2022

  47. [55]

    Generalized planning in pddl domains with pretrained large language models.arXiv preprint arXiv:2305.11014, 2023

    Tom Silver, Soham Dan, Kavitha Srinivas, Joshua B Tenenbaum, Leslie Pack Kaelbling, and Michael Katz. Generalized planning in pddl domains with pretrained large language models.arXiv preprint arXiv:2305.11014, 2023

    2023 arXiv

  48. [56]

    Progprompt: Generating situated robot task plans using large language models

    Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, and Animesh Garg. Progprompt: Generating situated robot task plans using large language models. In2023 IEEE International Conference on Robotics and Automation...

    2023

  49. [57]

    Sadler, Wei-Lun Chao, and Yu Su

    Chan Hee Song, Jiaman Wu, Clayton Washington, Brian M. Sadler, Wei-Lun Chao, and Yu Su. Llm- planner: Few-shot grounded planning for embodied agents with large language models. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2023

    2023

  50. [58]

    Interactive planning using large language models for partially observable robotic tasks

    Lingfeng Sun, Devesh K Jha, Chiori Hori, Siddarth Jain, Radu Corcodel, Xinghao Zhu, Masayoshi Tomizuka, and Diego Romeres. Interactive planning using large language models for partially observable robotic tasks. In2024 IEEE International Conference on Robotics and Automation (...

    2024

  51. [59]

    Tru-pomdp: Task planning under uncertainty via tree of hypotheses and open-ended pomdps

    Wenjing Tang, Xinyu He, Yongxi Huang, Yunxiao Xiao, Cewu Lu, and Panpan Cai. Tru-pomdp: Task planning under uncertainty via tree of hypotheses and open-ended pomdps. InAdvances in Neural Information Processing Systems, 2025

    2025

  52. [60]

    Wil Thomason, Zachary Kingston, and Lydia E. Kavraki. Motions in microseconds via vectorized sampling-based planning. InIEEE International Conference on Robotics and Automation, pages 8749–8756. doi: 10.1109/ ICRA57147.2024.10611190. URL http://arxiv.org/abs/ 2309.14545

    2024

  53. [61]

    Robot placement based on reachability inversion

    Nikolaus Vahrenkamp, Tamim Asfour, and R ¨udiger Dill- mann. Robot placement based on reachability inversion. In2013 IEEE International Conference on Robotics and Automation, pages 1970–1975. IEEE, 2013

    1970

  54. [62]

    Chatgpt for robotics: Design principles and model abilities.Microsoft Auton

    Sai Vemprala, Rogerio Bonatti, Arthur Bucker, and Ashish Kapoor. Chatgpt for robotics: Design principles and model abilities.Microsoft Auton. Syst. Robot. Res, 2:20, 2023

    2023

  55. [63]

    Multi-object search using object-oriented pomdps

    Arthur Wandzel, Yoonseon Oh, Michael Fishman, Nis- hanth Kumar, Lawson LS Wong, and Stefanie Tellex. Multi-object search using object-oriented pomdps. In 2019 International Conference on Robotics and Automa- tion (ICRA), pages 7194–7200. IEEE, 2019

    2019

  56. [64]

    Describe, explain, plan and select: interactive planning with large language models enables open-world multi-task agents

    Zihao Wang, Shaofei Cai, Guanzhou Chen, Anji Liu, Xi- aojian Ma, Yitao Liang, and Team CraftJarvis. Describe, explain, plan and select: interactive planning with large language models enables open-world multi-task agents. InProceedings of the 37th International Conference on N...

    2023

  57. [65]

    Jarvis-1: Open-world multi-task agents with memory-augmented multimodal language models.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    Zihao Wang, Shaofei Cai, Anji Liu, Yonggang Jin, Jin- bing Hou, Bowei Zhang, Haowei Lin, Zhaofeng He, Zilong Zheng, Yaodong Yang, et al. Jarvis-1: Open-world multi-task agents with memory-augmented multimodal language models.IEEE Transactions on Pattern Analysis and Machine In...

    2024

  58. [66]

    Fetch and freight: Standard platforms for service robot applications

    Melonee Wise, Michael Ferguson, Derek King, Eric Diehr, and David Dymesich. Fetch and freight: Standard platforms for service robot applications. InWorkshop on autonomous mobile service robots, volume 5, 2016

    2016

  59. [67]

    Manipulation-based active search for occluded objects

    Lawson LS Wong, Leslie Pack Kaelbling, and Tom ´as Lozano-P´erez. Manipulation-based active search for occluded objects. In2013 IEEE International Conference on Robotics and Automation, pages 2814–2819. IEEE, 2013

    2013

  60. [68]

    Learning adaptive planning representations with natural language guidance

    Lionel Wong, Jiayuan Mao, Pratyusha Sharma, Zachary S Siegel, Jiahai Feng, Noa Korneev, Joshua B Tenenbaum, and Jacob Andreas. Learning adaptive planning representations with natural language guidance. arXiv preprint arXiv:2312.08566, 2023

    2023 arXiv

  61. [69]

    Translating natural language to plan- ning goals with large-language models.arXiv preprint arXiv:2302.05128, 2023

    Yaqi Xie, Chen Yu, Tongyao Zhu, Jinbin Bai, Ze Gong, and Harold Soh. Translating natural language to plan- ning goals with large-language models.arXiv preprint arXiv:2302.05128, 2023

    2023 arXiv

  62. [70]

    Guiding long- horizon task and motion planning with vision language models

    Zhutian Yang, Caelan Garrett, Dieter Fox, Tom ´as Lozano-P´erez, and Leslie Pack Kaelbling. Guiding long- horizon task and motion planning with vision language models. In2025 IEEE International Conference on Robotics and Automation (ICRA), pages 16847–16853. IEEE, 2025

    2025

  63. [71]

    Skillwrapper: Generative predicate invention for skill abstraction.arXiv preprint arXiv:2511.18203, 2025

    Ziyi Yang, Benned Hedegaard, Ahmed Jaafar, Yichen Wei, Skye Thompson, Shreyas S Raman, Haotian Fu, Stefanie Tellex, George Konidaris, David Paulius, et al. Skillwrapper: Generative predicate invention for skill abstraction.arXiv preprint arXiv:2511.18203, 2025

    2025 arXiv

  64. [72]

    Unidomain: Pretraining a unified pddl domain from real-world demonstrations for generalizable robot task planning

    Haoming Ye, Yunxiao Xiao, Cewu Lu, and Panpan Cai. Unidomain: Pretraining a unified pddl domain from real-world demonstrations for generalizable robot task planning. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems

  65. [73]

    Uniplan: Vision-language task planning for mobile ma- nipulation with unified pddl formulation.arXiv preprint arXiv:2602.08537, 2026

    Haoming Ye, Yunxiao Xiao, Cewu Lu, and Panpan Cai. Uniplan: Vision-language task planning for mobile ma- nipulation with unified pddl formulation.arXiv preprint arXiv:2602.08537, 2026

    2026

  66. [74]

    Ff- replan: A baseline for probabilistic planning

    Sung Wook Yoon, Alan Fern, and Robert Givan. Ff- replan: A baseline for probabilistic planning. InICAPS, volume 7, pages 352–359, 2007

    2007

  67. [75]

    Rational inverse reasoning.arXiv preprint arXiv:2508.08983, 2025

    Ben Zandonati, Tom ´as Lozano-P ´erez, and Leslie Pack Kaelbling. Rational inverse reasoning.arXiv preprint arXiv:2508.08983, 2025

    2025 arXiv

  68. [76]

    Seeing is believing: Belief-space planning with foundation models as uncertainty estimators.arXiv preprint arXiv:2504.03245, 2025

    Linfeng Zhao, Willie McClinton, Aidan Curtis, Nishanth Kumar, Tom Silver, Leslie Pack Kaelbling, and Law- son LS Wong. Seeing is believing: Belief-space planning with foundation models as uncertainty estimators.arXiv preprint arXiv:2504.03245, 2025

    2025 arXiv

  69. [77]

    Large language models as commonsense knowledge for large- scale task planning.Advances in Neural Information Processing Systems, 36, 2024

    Zirui Zhao, Wee Sun Lee, and David Hsu. Large language models as commonsense knowledge for large- scale task planning.Advances in Neural Information Processing Systems, 36, 2024

    2024

  70. [78]

    Language models can infer action semantics for symbolic planners from environment feedback

    Wang Bill Zhu, Ishika Singh, Robin Jia, and Jesse Thomason. Language models can infer action semantics for symbolic planners from environment feedback. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics...

    2025

  71. [79]

    zero-sugar

    Environments and Tasks:Our real-robot experiments were conducted in a real household environment comprising three rooms: a kitchen, a living room, and a meeting room. Figure 14 illustrates a top-down view of the experimental setup. The kitchen, located at the top right and ser...

  72. [80]

    Its base is a differential-drive mobile platform, and it is equipped with a 1-DOF torso, a 7-DOF arm with parallel-jaw gripper

    Real-World Robot System: a) Hardware Setup:We adopt the commercially available mobile manipulator Fetch [65]. Its base is a differential-drive mobile platform, and it is equipped with a 1-DOF torso, a 7-DOF arm with parallel-jaw gripper. The robot also includes an RGB-D camera...

  73. [81]

    To simulate imperfect real-world recognition, labels are removed from a randomly selected half of the processors

    Block Processing World:The task is to achieve a target configuration in which certain objects exhibit specified effects, corresponding to preparing ingredients in the correct states for cooking. To simulate imperfect real-world recognition, labels are removed from a randomly s...

  74. [82]

    Mobile Manipulation in Unknown Environment (AI2THOR):In this setting, we consider the senario of a mobile manipulator operating in a household-like environment with objects of unknown locations and attributes. Fig. 18. Visualizations of all ProcTHOR house environments used in ...

  75. [83]

    In this demo, we use the Autolife S2 humanoid robot, which features an omnidirectional base, a 4-DOF torso, and two 7-DOF arms

    Robot System Integration:We consider a scenario in which the robot is required to operate an unknown appliance. In this demo, we use the Autolife S2 humanoid robot, which features an omnidirectional base, a 4-DOF torso, and two 7-DOF arms. The robot is equipped with one RGB-D ...

  76. [84]

    Defrost the food for 40 seconds

    Tasks Setup:The demonstration involves a multi-stage task:“Defrost the food for 40 seconds. ”Although the instruction appears simple, it entails substantial complexity for a robot. In our setting, the robot can only reliably perceive the microwave display and buttons when the ...

  77. [85]

    The domain includes the following predicates and actions: Listing 1

    Block World Domain:The Block World domain models a tabletop manipulation scenario where blocks can be stacked on each other or placed on designated regions that may trigger state changes. The domain includes the following predicates and actions: Listing 1. Block World PDDL Dom...

  78. [86]

    This domain is used for both AI2THOR simulations and real-world experiments

    Mobile Manipulation Domain:The mobile manipulation domain extends the planning capabilities to household environments where a mobile robot navigates between locations, manipulates objects, and interacts with appliances. This domain is used for both AI2THOR simulations and real...

  79. [87]

    Different buttons switch between operating modes, and numeric adjustments are mode-dependent

    Microwave Panel Domain:The microwave panel domain models appliance control through button presses. Different buttons switch between operating modes, and numeric adjustments are mode-dependent. Listing 3. Microwave Panel PDDL Domain (define (domain microwave_panel) (:requiremen...

  80. [88]

    Hypothesis-Related Predicates:We extend the PDDL domain with additional predicates to track hypothesis status: Listing 4. Hypothesis Tracking Predicates (related ?h ?x); hypothesis h is related to object/location x (true ?h); hypothesis h has been verified as true (hypothesis_...

  81. [89]

    The hypothesis-related version incurs a higher cost, biasing the planner to operate on verified objects when possible while still allowing exploration of uncertain hypotheses

    Cost-Based Action Splitting:Since FastDownward does not support conditional action costs, we implement a workaround by splitting each action into two variants: a normal version for verified objects and a hypothesis-related version for objects with unverified hypotheses. The hy...

  82. [90]

    Verify hypotheses early when they lie on the critical path

  83. [91]

    Avoid unnecessary manipulation of unverified objects

  84. [92]

    Dynamic Verification Action Generation:For each hypothesis, we generate a corresponding verification action that can only be executed when the appropriate verification conditions are met (e.g., holding the object for attribute verification, or being at the location for existen...

  85. [93]

    Augmented Problem Construction:When hypotheses are generated, the problem file is augmented with: •Hypothesis constants declared in the domain •Initial state facts for hypothesized objects and their attributes •relatedpredicates linking objects to their hypotheses •Goal augmen...

  86. [94]

    Predicate Classification Prompt SYSTEM_PROMPT_FOR_CLASSIFICATION = ’’’ Classify the type of predicate into attribute predicates (fixed attribute of an object, e.g

    Classification Prompt:Before hypothesis generation, predicates are classified into attribute predicates (fixed object properties likeplastic,glass) and effect predicates (states that can change through actions liketoasted,washed): Listing 7. Predicate Classification Prompt SYS...

  87. [95]

    id": "h1

    Hypothesis Generation Prompt:The hypothesis generation prompt instructs the LLM to propose hypotheses about object locations, attributes, and action effects based on commonsense reasoning: Listing 8. Hypothesis Generation Prompt (Mobile Manipulation) SYSTEM_PROMPT_FOR_HYPOTHES...

  88. [96]

    We provide a simple example prompt for the mobile manipulation setting.: Listing 10

    Verification Prompt:The verification prompt enables the LLM to reason about whether a hypothesis is confirmed or refuted based on robot observations. We provide a simple example prompt for the mobile manipulation setting.: Listing 10. Verification Prompt SYSTEM_PROMPT_FOR_VERI...

  89. [97]

    The prompts for the block world follow a similar structure but contain reduced content

    Language Model Planning Prompts:We provide the prompts for the most complex mobile manipulation domain. The prompts for the block world follow a similar structure but contain reduced content. Three variants of planning prompts are used depending on the ablation condition: a) H...

  90. [98]

    Appliance Hypothesis Generation SYSTEM_PROMPT_FOR_APPLIANCE_HYPOTHESIS = ’’’ You are given a planning problem for controlling an appliance and need to extend button press effects

    Appliance Control Prompts:For the microwave panel experiments, specialized prompts handle button effect hypotheses: Listing 14. Appliance Hypothesis Generation SYSTEM_PROMPT_FOR_APPLIANCE_HYPOTHESIS = ’’’ You are given a planning problem for controlling an appliance and need t...

  91. [99]

    Missing FUNCTIONS are adjusted by press-increase-button

  92. [100]

    Effect should be CONDITIONAL on current mode

  93. [101]

    id": "h1

    Use: (when (MODE ?m) (increase (FUNCTION ?m) STEP)) Example: { "id": "h1", "type": "action_effect", "description": "Level button in set-waiting-time mode increases waiting time by 10 seconds", "content": {"press-level-button": ["(when (set-waiting-time-mode ?m) (increase (wait...

  94. [102]

    Goal Parsing Prompt SYSTEM_PROMPT_FOR_GOAL = ’’’ Convert natural language goal to PDDL goal format

    Goal Parsing Prompt:Natural language goals are converted to PDDL goal specifications: Listing 15. Goal Parsing Prompt SYSTEM_PROMPT_FOR_GOAL = ’’’ Convert natural language goal to PDDL goal format. Input: Bring an ivory-white towel to the countertop Output: (exists (?o ?l) (an...

Pith tools

Reviewed July 11, 2026 · model on record in the stance chip above.