{"id":"823f446e-2549-4f04-9826-aaa7cf950a5d","arxiv_id":"2505.04493","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":1.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A structured survey of model-based AI planning and execution systems for robotics, comparing ROSPlan, CLIPS Executive, PlanSys2, SkiROS2, the Skill-Based Architecture, and AOS, and proposing a future research agenda.","lead":"This paper reviews six software systems that combine AI planning with robot control, comparing how they model skills, make decisions, and integrate with ROS. It argues that model-based planning is a transparent alternative to end-to-end learning and outlines future research directions for the field.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central feasibility claim—that LLMs can cheaply generate reliable formal skill models—is asserted, not supported, and the cited evidence is weak.","rationale":"The reader identified the same weakest assumption: formal skill models must be cheap and accurate enough, with replanning covering the rest. My stress-test sharpens that assumption by focusing on the semantic correctness of LLM-generated models and the inadequacy of the cited evidence. This is not an internal inconsistency in the survey; it is an unsupported empirical prediction in Section II, reinforced by the future-work section's reliance on 'Model Learning/Adaptation' and 'LLM Integration' as research directions rather than demonstrated capabilities. The survey itself is competent and useful as a comparative review; Table I and the per-system descriptions provide real value. The concern does not justify rejection, but it does justify keeping the verdict conditional and adding a specific condition that the central feasibility claim be either supported by evidence or explicitly labeled as an open hypothesis.","tokens_in":23756,"tokens_out":3929,"duration_ms":41337,"concrete_test":"Run a benchmark study: take 50 natural-language skill descriptions drawn from the tutorials and examples of ROSPlan, SkiROS2, Plansys2, and AOS; prompt a modern LLM to generate the corresponding PDDL/SDL skill models; and measure (i) syntactic/type correctness, (ii) semantic correctness against a ground-truth simulator, and (iii) correctness after one round of human correction. If the LLM-generated models are correct and usable in fewer than, say, 90% of cases after a small number of corrections, then Section II's claim that LLMs 'show a clear and increasing ability' to generate formal models is unsupported, and the survey should present model authoring as an open problem rather than a solved non-issue.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central argument in Section II requires that formal skill models can be produced at acceptable cost and remain accurate enough for planning, with LLMs as the proposed remedy. This is the load-bearing assumption: if model authoring remains expensive or error-prone, MPERs lose their practical edge over hand-written controllers and end-to-end learning, and the claimed 'MPER as master' advantage weakens substantially. The paper explicitly concedes that 'models are difficult to provide,' then dismisses the subjective half as a 'non-issue' because 'open-model communities ... are likely to arise' and because LLMs 'show a clear and increasing ability to generate such formal models' citing [6]–[11]. That evidence does not support the claim. [6] is a single paper on constructing world models for LLM-based task planning, not a verified pipeline for generating robot skill models; [7] is titled 'LLMs can't plan, but can help planning in LLM-modulo frameworks'; [8] evaluates embodied planning and reports substantial failures; and several cited references are anonymous preprints. None demonstrate reliable, correct, verifiable generation of formal skill models for real robot skills. Moreover, replanning cannot compensate for a semantically wrong model when the error concerns unobserved preconditions or effects: the planner may confidently select actions that are unsafe or ineffective in the real world, and the claimed 'certificate' property of skill models is then illusory. The paper's central promise therefore rests on an empirical bet about LLM model-generation accuracy and community model-sharing that is neither supported nor experimentally tested.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper is a survey of model-based planning and execution systems for robotics (MPERs). It argues that MPERs offer a transparent, controllable, and verifiable middle path between end-to-end learning and hand-written controllers, and it develops a five-dimensional taxonomy: modeling, decision making, state representation and update, integration, and impact. Six systems are reviewed in detail: ROSPlan, the CLIPS Executive, PlanSys2, SkiROS2, the ONERA skill-based architecture, and the authors' own AOS. The paper closes with future research directions, including multi-level hybrid architectures, sensing and monitoring, automated model simplification, human-control interfaces, and LLM-based model generation.","tokens_in":24105,"tokens_out":7538,"duration_ms":71686,"significance":"If the comparative analysis is accepted, the survey provides a useful organizing reference for practitioners choosing among MPERs and for researchers working on integrated planning and execution. The taxonomy is reasonable, and the system descriptions appear consistent with the cited primary sources. Table I condenses the design tradeoffs effectively, and the future-directions section identifies concrete open problems such as automated model learning, anomaly detection, and safety monitoring. The paper does not provide new systems or experiments, which is appropriate for a survey; its value is in the synthesis and the explicit treatment of sensing, monitoring, and integration effort. The main weaknesses are an unsupported feasibility claim about LLM-generated models in Section II and the undisclosed inclusion of the authors' own AOS as a surveyed system with favorable entries in Table I.","major_comments":[{"comment":"The paragraph beginning 'One major objection to the use of model-based systems in robotics is that models are difficult to provide' bases its dismissal of the subjective difficulty on the claim that 'LLMs are showing a clear and increasing ability to generate such formal models from natural language descriptions' and cites [6]-[11]. The cited evidence does not support this strong claim: [6] is a single research paper on world models constructed for LLM-based task planning; [7] is an anonymous preprint whose own title says LLMs cannot plan but can help in LLM-modulo frameworks; [8] is an anonymous preprint that evaluates pretrained LLMs in embodied planning and reports substantial failures; and [11] is also anonymous. None of these demonstrates a reliable, verified pipeline for generating correct robot skill models. Because Section II's overall argument that MPERs are practically viable because model authoring is cheap rests on this assertion, the passage is load-bearing. The manuscript should either replace these citations with systematic evidence or weaken the claim and mark model-authoring cost as an open challenge. The same paragraph should also qualify the statement that replanning can 'compensate for such inaccuracies': replanning cannot repair a model that is semantically wrong about unobserved preconditions or effects, since the planner may confidently select actions that are unsafe or ineffective.","section":"II (model-authoring feasibility)"},{"comment":"The survey includes AOS as one of the six reviewed systems and gives it the only positive entries in Table I for 'Plug'n Play,' 'Auto Mapping,' 'Probabilistic Effects,' 'Partial Observability,' 'Noisy Sensing,' and 'Supported Platforms' (ROS extendable), without disclosing that AOS is the authors' own system: reference [57] is by Wertheim, Suissa, and Brafman, and reference [25] is by Brafman, Tolpin, and Wertheim. In a survey intended to help readers choose among MPERs, this is an undisclosed conflict of interest that affects the credibility of the comparison. The manuscript should add an explicit statement of the authors' involvement with AOS and SDL, and it should either apply the same level of critical distance to AOS as to the other systems or explain why that is not possible.","section":"IV.F and Table I (self-citation disclosure)"},{"comment":"The text claims that AOS provides 'true plug-and-play capability' and that 'AOS users need only supply SDL documentation files.' This is still a model-authoring burden: users must write an Environment File, a Skill Documentation file, and an Abstraction Mapping file, which require expertise in SDL and in formalizing skill effects. Table I's 'Plug'n Play' entry for AOS should therefore be qualified so that readers do not infer that the model-authoring burden is eliminated; otherwise the comparison with other systems is misleading.","section":"IV.F.4 (plug-and-play claim)"}],"minor_comments":[{"comment":"The sentence 'The AOS does not support a Goal Reasoning (GR) mechanism' is immediately followed by a description of how users can implement GR by loading EF files with new utility functions; please clarify the difference between a built-in mechanism and a user-extensible capability.","section":"IV.F.2 (Goal Reasoning)"},{"comment":"The table caption should state explicitly that '✓' means supported and 'X' means not supported or not applicable; currently the reader has to infer this from the text.","section":"Table I caption"},{"comment":"References [7], [8], and [11] are anonymous preprints; if they remain in the bibliography, the entries should be marked as preprints rather than appearing as plain arXiv references, since the survey relies on them for a load-bearing claim.","section":"References [7], [8], [11]"},{"comment":"Reference [12] appears to have an incorrect title: the cited PLDI 2019 paper is 'Gen: A General-Purpose Probabilistic Programming System with Programmable Inference,' not 'Gentl: The design and implementation of probabilistic programming languages'; please correct the entry and the URL.","section":"Reference [12]"},{"comment":"The sentence 'open-model communities and contributors are likely to arise' is an unsupported prediction; if the central claim is softened, this sentence should be hedged or removed.","section":"II (open-model prediction)"}],"recommendation":"major_revision","confidential_remarks":"The main concern is not the technical soundness of the survey but the undisclosed self-citation of AOS and the overstrong LLM-feasibility claim in Section II. I recommend requiring an explicit disclosure of the authors' involvement with AOS and SDL, and either a systematic review of evidence for LLM-generated formal models or a softened claim. The survey is otherwise competent and should be reconsidered after these revisions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a genuinely useful survey of model-based planning and execution systems for robotics. The design-dimension taxonomy (modeling, decision making, state representation, integration, impact) is sensible, the system descriptions line up with what I know of the cited papers, and the comparison table alone is worth the read. The paper deserves a serious referee.\n\nWhat's actually new is the synthesis. The field is young enough that a structured comparison of ROSPlan, CLIPS Executive, PlanSys2, SkiROS2, the ONERA skill-based architecture, and AOS helps people choose a framework and see what design choices exist. The future-directions section also has a couple of genuinely good ideas, especially the multi-level hybrid design where solvers and predefined policies are composed as skills.\n\nSoft spots: the undisclosed AOS authorship is the main issue. Section F presents AOS as if it were just another surveyed system, and Table I credits it with capabilities none of the others have (probabilistic effects, partial observability, noisy sensing, plug-and-play, auto mapping). Those capabilities may be real, but the only citation is the authors' own RA-L paper, and the survey never says 'this is our system.' A one-sentence conflict-of-interest statement would fix most of this. Second, Section II claims LLMs 'show a clear and increasing ability' to generate formal skill models, citing [6]-[11]. Those references do not support that. [6] is about constructing world models for LLM-based planning, [7] literally says LLMs can't plan, [8] reports substantial failures, and several are anonymous preprints. The paper also dismisses model-authoring cost because 'open-model communities are likely to arise.' That is a hope, not evidence. I'd treat this as a minor flaw, not a fatal one, because the survey's value does not depend on that bet, but the language should be softened.\n\nThe selection of surveyed systems is not justified by explicit criteria, though for a six-system subfield that is a minor issue. The descriptions are accurate enough, and the paper is clearly written.\n\nBottom line: I'd send this to peer review with a request for a COI statement, a more measured LLM claim, and a note on how systems were selected. After that, it is worth publishing as a reference survey.","headline":"A useful, well-organized survey of robot planning-execution systems, undercut by an undisclosed conflict of interest around the authors' own AOS system.","tokens_in":24516,"tokens_out":3341,"would_cite":true,"duration_ms":29210,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Model-based planning and execution systems, with the planner as master, offer the most promising route to flexible, transparent robot autonomy.","keywords":["model-based planning","robotics","task-level control","planning and execution","PDDL","POMDP","robot skills","LLM-generated models"],"falsifier":"Count published, reusable skill models per MPER and measure the time a new user needs to integrate a new skill; if most real deployments still require expert hand-written PDDL or ontology authoring and model quality degrades in novel contexts, the paper's claim that model difficulty is a 'non-issue' fails.","tokens_in":23523,"feed_emoji":"🤖","tokens_out":6557,"duration_ms":58568,"temperature":0.7,"pith_summary":"Model-based planning and execution systems for robotics (MPERs) work by giving the robot a formal model of its skills, environment, and task, then using automated planning to decide which skill to run when. This review argues that this architecture is the most practical engineering route to flexible autonomous robots, because the decision-making core stays transparent, verifiable, and controllable. The paper's key thesis is that learning and large language models should be used where they excel—acquiring repetitive skills and generating formal models from natural language—while the MPER remains the master that chooses actions through well-understood optimization. If that thesis holds, robots can be reconfigured by editing or generating models rather than rewriting hand-coded controllers, which would lower the barrier to new applications and support regulatory oversight.","feed_headline":"Planning models should stay the master of robot autonomy","feed_subtitle":"A survey argues they let robots combine learned skills and LLM-generated models with a verifiable decision core.","key_machinery":"The central object is the MPER architecture itself: a software layer that maintains a world state (the abstract facts the planner reasons about) separate from an execution state (the low-level data skills need), represents each skill as an action in a formal description language such as PDDL or the probabilistic Skill Documentation Language, and uses a planning algorithm plus sensing, monitoring, and replanning to drive the robot online. The argument runs through this machinery: because every decision traces back to an explicit model and an explicit optimization criterion, the system is legible to users and regulators, and because the model is the only thing that changes when the task changes, the same skill base can be reused across tasks. The paper also introduces the notion of a multi-level hybrid design where solvers and predefined policies are skills that can call other skills, which is what would give MPERs both flexibility and hierarchical structure.","core_discovery":"The central claim is that MPERs offer the best available compromise for autonomous robot control: skills can be learned by deep learning or imported from code libraries, LLMs can help build the formal skill models and translate natural-language tasks, but a model-based planner should remain the final decision maker because it optimizes against explicit criteria and produces readable, auditable decisions. The paper supports this by surveying the main design dimensions—modeling language, online versus offline decision making, state representation and update, integration effort, and impact—across the current generation of systems. It shows that today's systems already cover a spectrum from deterministic classical/temporal planning with replanning, to goal-tree executives with goal reasoning, to POMDP-based systems that handle probabilistic effects, partial observability, and noisy sensing, and it identifies a future hybrid architecture in which solvers and predefined policies are both treated as skills.","pith_inferences":["Beyond the paper: if LLM-generated skill models become reliable enough, the bottleneck in MPER adoption shifts from model authoring to verification; a natural next step the paper does not develop is automated checking of generated models against logged executions of the real skill.","Beyond the paper: the prediction that open-model communities will arise is testable—track whether shared, reusable skill-model repositories grow at a rate comparable to open-source code; if they do not, MPERs may remain confined to well-engineered industrial settings.","Beyond the paper: the multi-level hybrid design suggests a direct empirical benchmark—reimplement the paper's building-cleaning example on a standard mobile manipulator and measure development time, replanning frequency, and task success against a hand-written behavior tree."],"forward_implications":["If MPERs mature as described, changing a robot's behavior becomes a matter of revising or regenerating its model rather than rewriting state machines or scripts.","Task-level control can be made accessible to non-experts: natural-language task specification and LLM-generated domains could let casual users issue new tasks to a robot on the fly.","Skill models can double as certificates, giving regulators and safety auditors a concrete artifact to inspect, which addresses concerns like those in the EU AI Act.","A POMDP-based MPER can handle realistic uncertainty—probabilistic skill outcomes, partial observability, noisy sensing—at the price of solving harder models, and the paper's design choices show how to mitigate that cost with generative models and online solvers.","The proposed multi-level hybrid architecture, where solvers and predefined policies are both skills, would let engineers build robot autonomy incrementally from either the bottom up or the top down."],"supporting_citations":[{"why":"Introduces the first modern MPER for the robot operating system, establishing the reference architecture that later systems compare against.","marker":"[2]"},{"why":"Provides the definition of robot skills used throughout the paper and models skills with descriptive and operational components.","marker":"[3]"},{"why":"Shows large language models can construct and use world models for model-based task planning, supporting the paper's claim that model authoring can be automated.","marker":"[6]"},{"why":"Defines goal reasoning, the mechanism the paper identifies for autonomous goal selection in dynamic environments.","marker":"[13]"},{"why":"Argues probabilistic programs can serve as an action description language, underpinning the generative skill models in one of the surveyed systems.","marker":"[25]"},{"why":"Presents a next-generation MPER for the robot operating system that the paper uses to contrast integration effort and behavior-tree plan execution.","marker":"[34]"},{"why":"Describes a POMDP-based plug-and-play MPER, the paper's main evidence that probabilistic effects, partial observability, and noisy sensing can be handled natively.","marker":"[57]"}],"fun_headline_variants":["Let model-based planning be the robot's decision core","Planners, not LLMs, should make the final robot call","Model-based planning: the auditable brain for robot skills","Survey: model-based planners should stay in command for robots","Model-based planning remains the robot's final decision maker"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that formal skill models can be written or generated at acceptable cost and remain accurate enough, with replanning covering the rest; the paper itself concedes models are difficult to provide, and its confidence that open-model communities will arise is a prediction rather than evidence.","fun_headline_variants_meta":{"raw":{"variants":["Let model-based planning be the robot's decision core","Planners, not LLMs, should make the final robot call","Model-based planning: the auditable brain for robot skills","Survey: model-based planners should stay in command for robots","Model-based planning remains the robot's final decision maker"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001174,"raw_usage":{"total_tokens":4787,"prompt_tokens":809,"completion_tokens":3978,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":425,"completion_tokens_details":{"reasoning_tokens":3897}},"tokens_in":425,"tokens_out":3978,"duration_ms":30139,"temperature":1.0,"reasoning_tokens":3897,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:26:38.005301+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Count published, reusable skill models per MPER and measure the time a new user needs to integrate a new skill; if most real deployments still require expert hand-written PDDL or ontology authoring and model quality degrades in novel contexts, the paper's claim that model difficulty is a 'non-issue' fails.","supporting_citations":[{"cited_title":"Rosplan: Planning in the robot operating system,","cited_arxiv_id":null,"evidence_quote":"Introduces the first modern MPER for the robot operating system, establishing the reference architecture that later systems compare against."},{"cited_title":"Formalization of robot skills with descriptive and operational models,","cited_arxiv_id":null,"evidence_quote":"Provides the definition of robot skills used throughout the paper and models skills with descriptive and operational components."},{"cited_title":"Lever- aging pre-trained large language models to construct and utilize world models for model-based task planning,","cited_arxiv_id":null,"evidence_quote":"Shows large language models can construct and use world models for model-based task planning, supporting the paper's claim that model authoring can be automated."},{"cited_title":"Goal reasoning: Foundations, emerging applications, and prospects,","cited_arxiv_id":null,"evidence_quote":"Defines goal reasoning, the mechanism the paper identifies for autonomous goal selection in dynamic environments."},{"cited_title":"Probabilistic programs as an action description language,","cited_arxiv_id":null,"evidence_quote":"Argues probabilistic programs can serve as an action description language, underpinning the generative skill models in one of the surveyed systems."},{"cited_title":"FF: The fast-forward planning system,","cited_arxiv_id":null,"evidence_quote":"Presents a next-generation MPER for the robot operating system that the paper uses to contrast integration effort and behavior-tree plan execution."},{"cited_title":"Plug’n play task-level autonomy for robotics using pomdps and probabilistic programs,","cited_arxiv_id":null,"evidence_quote":"Describes a POMDP-based plug-and-play MPER, the paper's main evidence that probabilistic effects, partial observability, and noisy sensing can be handled natively."}],"review_version":1}