Acting and Planning with Hierarchical Operational Models on a Mobile Robot: A Study with RAE+UPOM

Dana Nau; Joachim Hertzberg; Malik Ghallab; Marc Vinci; Martin Atzmueller; Oscar Lima; Paolo Traverso; Sebastian Stock; Sunandita Patra

REVIEW 4 major objections 6 minor 13 references

Acting and Planning with Hierarchical Operational Models on a Mobile Robot: A Study with RAE+UPOM

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read RAE+UPOM is the first integrated actor-planner system sharing hierarchical operational models to be deployed on a physical robot, and it completes object collection under real failures.

desk verdict A solid but narrowly evidenced systems paper: first real-robot RAE+UPOM deployment works for action failures, but the abstract oversells sensor-noise robustness and the perception-failure gap needs explicit handling. read the letter →

arxiv 2507.11345 v1 pith:6SC64UQC submitted 2025-07-15 cs.RO cs.AI

Oscar Lima , Marc Vinci , Sunandita Patra , Sebastian Stock , Joachim Hertzberg , Martin Atzmueller , Malik Ghallab , Dana Nau

show 1 more author

Paolo Traverso

This is my paper · ORCID

classification cs.ROcs.AI

keywords hierarchicaloperationalmodelsrefinementactingMonteCarloplanningRAE+UPOMmobilemanipulationobjectcollectionplanning-executionintegrationactionfailurerecovery

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that a robot can use one and the same hierarchical operational model for both acting and planning, and that the integrated system RAE+UPOM works on real hardware, not only in toy simulations. It presents the first physical deployment on the Mobipick mobile manipulator, where the robot collects objects from tables despite navigation, perception, and arm failures. If the claim is right, roboticists no longer need to maintain a separate abstract planner model alongside the concrete control code, because the planner can simulate the actual code the actor will run. The experiments also provide a look inside the planner's rollouts, showing how method choices and the emergent use of a transport box arise from expected-utility calculations.

What carries the argument

The central object is a hierarchical operational model: a set of refinement methods, which are computer programs giving alternative procedures for accomplishing tasks, with parameters that can be instantiated in many ways. RAE executes the body of a chosen method instance step by step, while UPOM runs UCT-like Monte Carlo rollouts using a lightweight simulated counterpart of every low-level robot routine, sampling probabilistic outcomes and cost estimates provided by human experts. The coupling that carries the argument is that the actor and planner share the exact same method code, so planning is a simulated execution of the code that will later run on the robot, and the utility function uses an exponentially decaying reward to favor early collection of high-value objects.

What would settle it

Run repeated object-collection trials under an identical layout while setting one object's simulated collection probability much higher than its measured physical success rate; if the planner consistently selects that object first and the realized total utility is lower than an alternative order, the rollouts' relative comparisons are not reliable enough to support the deployment claim.

Watch

Extended reading notes

Core claim

The paper claims that RAE+UPOM, a system that combines the Reactive Acting Engine with the anytime UCT-like Monte Carlo planner UPOM, can be deployed on a physical robot and can complete a hierarchical object collection task under partial observability, sensor noise, and action failures. The actor and planner share the same set of refinement methods, which are Python-coded procedures for tasks such as collecting objects from a table, perceiving an object, and moving objects with or without a box. During execution, UPOM performs Monte Carlo rollouts over alternative method instances using simulated counterparts of the robot's low-level commands, then suggests the method instance with the highest expected utility. The paper reports successful trials with all objects collected, as well as trials where cable entanglement, repeated navigation failure, or missed perception caused partial collection, and it argues that these results demonstrate that planning by simulated execution of operational code closes the gap between descriptive planner models and the real control structures on the robot.

Load-bearing premise

The central load-bearing premise is that the human-supplied simulated models of robot actions are realistic enough for relative comparison, so the method instance with the highest simulated expected utility is also the best choice on the physical robot.

Editorial extensions

If this is right

Robots can plan by simulating the same Python-coded refinement methods they will execute, removing the need for a separate PDDL-style descriptive model.
Online planning is fast enough for real interleaving: UPOM used about one second for 100 rollouts per subtask, allowing decisions during execution rather than only before it.
Failures need not abort the mission: with configurable retry counts, the robot returned to a table after three navigation failures and recovered from an emergency stop caused by cable entanglement.
Optional equipment such as a zero-utility transport box is adopted spontaneously when planning discovers that grouping objects raises expected utility.
Simulated camera-frustum checks can select pan and lift joint angles before grasping, and those choices transfer to real perception through a learned 6D pose estimator.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the shared-model approach generalizes, a practical design rule follows: every low-level robot routine should ship with a lightweight simulated counterpart and a success detector, so planning and acting cannot drift apart; the paper flags this as an upfront effort cost, which suggests it could become a standard engineering practice.
Because exploration and perception tasks receive zero reward in the rollouts, the planner is blind to information value, and early planner calls often produce no utility; adding an explicit information-gain term to the utility function could reduce this waste and is a testable extension the paper does not explore.
The two modifications made for real hardware, partial-utility accounting after command failure and bounded retries of the same method, are likely portable to other refinement acting engines even without UPOM, and could be evaluated in simulated factory or fleet tasks.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The useful thing here is the concrete integration work: RAE+UPOM, previously only in toy domains and simulation, actually runs on the Mobipick and completes object collection in six trials. The authors also added two sensible engineering updates (utility on partial success, retry counts) and a heatmap visualization of planner rollouts that gives a real look inside the anytime UCT search. That is genuine empirical content, and the paper is honest about the architecture and its limitations.

The soft spots are real but not fatal. The evaluation is thin: one trial per condition, no error bars, no baseline comparison, and the simulation parameters are hand-populated by human experts. The reader's circularity concern is fair—the planner picks methods based on rollouts from a model tuned for this exact object set—but the real-robot counts do provide some external grounding. The bigger issue, and the one that should bother you most, is the perception gap. The simulator's 'read current pose' is a geometric frustum check; it does not model DOPE confidence falling below threshold. Trials 2.1 and 3.3 both show objects left uncollected because the robot never perceives them, and the paper admits 'such cases remain unhandled.' That is not a minor edge case for an abstract that promises robust execution under sensor noise. The planner is overconfident about perception because the simulation cannot represent low-confidence detections, so the claimed adaptation to perception failures is unsupported and in fact contradicted for this failure class. The authors should either handle that failure mode or explicitly scope the claim to action failures.

Even with that, the paper deserves a serious referee. It is a reproducible system paper with honest reporting, and the underlying shared-model idea is worth debating. For a revision, I would ask for more trials or at least repeated runs, a simple baseline (e.g., greedy or fixed-order collection), and an explicit discussion of where the hand-crafted model diverges from real perception. The citation pattern looks fine—the prior RAE+UPOM work is cited, and the new content is clearly the deployment.

Who is this for? Researchers working on integrated acting and planning, especially in the ROS/refinement-actor community. It is not a breakthrough theory paper, but it is a legitimate engineering study that exposes real mismatches between simulation and reality. I would bring it to the reading group and, with the stated caveats, would accept it for peer review.

Referee Report

4 major / 6 minor

Summary. The paper describes the first physical deployment of RAE+UPOM, an integrated actor-planner system that shares hierarchical operational models between acting and planning. The system interleaves the Reactive Acting Engine (RAE) with a UCT-like Monte Carlo planner (UPOM) on the Mobipick mobile manipulator for an object-collection task. The authors model perception, manipulation, and navigation through refinement methods, pair each low-level command with a simulated counterpart for rollouts, and report six single-run trials under nominal conditions, spatial perturbation, and induced failures. The paper claims robust execution under action failures and sensor noise and offers observations about planner rollouts via heatmaps. The main contributions are the deployment, the hierarchical operational model, and an empirical study of the interleaved decision-making.

Significance. If the deployment is taken at face value, the paper demonstrates a useful architectural proof-of-concept: a single hierarchical operational model can drive both a reactive actor and a Monte Carlo planner on a physical robot, and the system can recover from certain navigation and arm failures. This is a positive step toward closing the model gap between planning and execution. The authors are candid about some limitations, and the accompanying website may help reproducibility. However, the empirical evidence is too thin to support the paper's broader claims: trials are not repeated, there is no baseline, and the simulator's perception model demonstrably does not capture the failures that occur. The significance of the work is therefore conditional on substantially strengthened evaluation and more careful claims.

major comments (4)

[Abstract; Section V, Trials 2.1 and 3.3] The abstract claims 'robust task execution under action failures and sensor noise,' but two of the six trials (2.1 and 3.3) end with objects uncollected because DOPE detection confidence fell below 0.5 and the robot never became aware of the objects. Section V, Trial 3.3 explicitly states 'At present, such cases remain unhandled.' The claim is therefore overstated: the system is robust to observable command failures (navigation, arm) but not to perception failures, which are a form of sensor noise. The conclusion and abstract should be qualified to say that the system handles selected action failures, with perception-failure robustness left as future work.
[Section III-A; Section IV-B] The simulator used by UPOM models the read current pose command as a geometric visibility check: it succeeds when all bounding-box vertices lie inside the camera frustum. It does not model the learned DOPE pose estimator's confidence threshold. Because the planner's rollouts assume geometric visibility implies successful detection, UPOM is systematically overconfident about perception. This is not a minor implementation detail: it is the mechanism behind the unhandled failures in Trials 2.1 and 3.3, and it contradicts the paper's claim that the hand-crafted simulation outcomes are 'sufficiently realistic for relative comparison.' A concrete test is to add a probabilistic detection model conditioned on viewing distance and angle to the simulator and check whether the planner changes its selected camera poses in the affected trials.
[Section V, Table II] Each trial condition was executed once. There are no repeated runs, no variance measures, and no statistical tests, yet the paper draws quantitative comparisons, e.g., 'Using the box enabled the robot to complete the task ∼49 seconds faster' (Section V, Study 1). Single-run results cannot support such comparative claims, and the absence of any baseline or ablation means the contribution of UPOM's planning, as opposed to the hand-coded methods, is not demonstrated. At minimum, multiple runs per condition and a comparison against RAE without UPOM (or random method selection) are needed before claiming empirical insights.
[Section IV-B; Section V] The simulation model is hand-crafted: Section IV-B says costs and outcome probabilities are 'provided by human experts,' and Section V says collection probabilities were 'empirically determined' for the specific object set in the test scenario. Because UPOM's choices are evaluated using this same model, the favorable behavior in the trials may be due to the model being tuned to the scenario rather than to the planner's own merits. This circularity risk is load-bearing for the planner's decision quality. The paper should include a sensitivity analysis over the model parameters, or test a held-out object or table configuration not used in calibration, to show that the planner's choices transfer.

minor comments (6)

[Section V, Study 2] The sentence 'The results aligned with the expectations' is immediately followed by the statement that the robot failed to perceive objects on Table 3 and collected only 2/4 objects. This is contradictory as written and should be rephrased to distinguish the successful adaptation to spacing from the perception failure.
[Section V, Trial 3.1] The paper states in the introduction that the robot 'operates in a fully autonomous manner,' but Trial 3.1 required operator intervention to recover from cable entanglement. The role of operator intervention should be described explicitly in the definition of 'fully autonomous.'
[Equation (1)] The values of the decay constant k and the scaling constants c1 and c2 are not reported, and no sensitivity analysis is given. Since c1+c2=1 leaves one free scale, the reader cannot assess how sensitive the planner's choices are to this function.
[Section III-A] The discrete ranges for SHOULDER_PAN_JOINT and ELBOW_LIFT_JOINT and the value of d_MAX are not specified, so the size of the method-instance space is not reproducible. Please provide these ranges or a reference to the simulator configuration.
[Fig. 5] The heatmap description mentions rollout containers, red dots, and bar height, but a reader of the text alone cannot map colors and heights to axes or understand how 'rollout cluster' is defined. Please add axis labels and a legend, or defer details to the caption.
[Section V, Study 1] The text reports '32.1' for trial 1.2 while Table II lists '32.09'; please make the numbers consistent.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: real-robot trials provide external grounding; hand-crafted simulator models are acknowledged inputs, not fitted predictions.

full rationale

No load-bearing circular step was found. UPOM's rollouts use a hand-crafted simulation model (Section IV-B: 'probabilistic action outcomes given by a simulated hand-crafted model', with costs and probabilities 'provided by human experts'); these are explicit modeling inputs, not outputs claimed as predictions, and the paper caveats that the estimates 'may be inaccurate' and are intended only for 'relative comparison.' The planner's choices are evaluated against real physical executions in Table II, which is an independent external benchmark, so the evaluation does not reduce to the simulator. Equation (1) defines the utility objective used both for planning and for reporting collected utility; that is the intended optimization criterion for the task, not a fitted parameter renamed as a prediction. The paper self-cites [2] for RAE+UPOM and convergence guarantees, but the deployment claim and all experimental trials are new empirical content; the citation is background information and is not the load-bearing justification for the paper's central result. Trials 2.1 and 3.3 openly report perception failures and state 'At present, such cases remain unhandled', which weakens the abstract's claim of robustness under sensor noise as a correctness or overclaiming issue, but this is not circular reasoning. The score of 1 reflects only the presence of a minor, non-load-bearing self-citation to prior work by overlapping authors.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The planning and acting behavior depends on several hand-set or empirically fitted numbers (success probabilities, utility constants, simulation costs, retry counts, perception ranges) and on domain assumptions about unique object types, simulation fidelity, the frustum-based success condition, and manually verified holding state. No new physical entities are introduced.

free parameters (5)

Per-object collection success probability p(o) = Not provided
Section V says these probabilities were "empirically determined" and combined into a single metric for perception and grasp reliability; they directly shape UPOM's rollouts and method selection.
Utility function decay constant k and scaling constants c1, c2 = Not provided; c1 + c2 = 1 stated
Eq. 1 defines utility with an exponentially decaying reward factor; the constants are chosen by the authors and affect the prioritization of early collection.
Simulated command cost estimates and outcome probabilities = Not provided
Section IV-B: the rollouts use a "simulated hand-crafted model" with costs and probabilities "provided by human experts"; these determine which method instances appear best in rollouts.
Retry count per method = 2 for navigation in Trial 3.2
Section IV-A introduces a retry count parameter; in Trial 3.2 the navigation retry count was set to 2, affecting failure handling behavior.
Perception parameter ranges (SHOULDER_PAN_JOINT, ELBOW_LIFT_JOINT) and d_MAX = Not provided
Section III-A: pan and lift take values from discrete sets defined by a human expert, and d_MAX is the maximum distance for pose estimation; these bound the perception model.

assumptions (4)

domain assumption Each object type appears only once in the environment.
Section III: "Each object type appears only once in this environment, eliminating the need for persistent object anchoring across perception steps." This simplifies perception and state tracking.
domain assumption The hand-crafted simulation models are sufficiently realistic for relative comparison.
Section IV-B: "While these estimates may be inaccurate... they should at least be sufficiently realistic for relative comparison." The planner relies on this for selecting good methods.
domain assumption If an object's bounding box is fully inside the camera frustum, the vision system (DOPE) will produce a usable pose estimate.
Section III-A: during planning the read current pose command succeeds only if all bounding box vertices are inside the frustum; at execution time DOPE is called. The equivalence between the frustum check and DOPE success is assumed.
domain assumption The holding state is verified manually and may be incorrect.
Table I lists the holding state as "Verified manually as it may be incorrect (e.g., if the object is dropped)." The actor assumes this state is accurate enough for decision-making.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Acting and Planning with Hierarchical Operational Models on a Mobile Robot: A Study with RAE+UPOM." pith.science (2026). https://pith.science/paper/6SC64UQC

@misc{pith2026250711345,
  author       = {Pith},
  title        = {Pith review of: Acting and Planning with Hierarchical Operational Models on a Mobile Robot: A Study with RAE+UPOM},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6SC64UQC}},
  note         = {Machine review of arXiv:2507.11345}
}

read the original abstract

Robotic task execution faces challenges due to the inconsistency between symbolic planner models and the rich control structures actually running on the robot. In this paper, we present the first physical deployment of an integrated actor-planner system that shares hierarchical operational models for both acting and planning, interleaving the Reactive Acting Engine (RAE) with an anytime UCT-like Monte Carlo planner (UPOM). We implement RAE+UPOM on a mobile manipulator in a real-world deployment for an object collection task. Our experiments demonstrate robust task execution under action failures and sensor noise, and provide empirical insights into the interleaved acting-and-planning decision making process.

Figures

Figures reproduced from arXiv: 2507.11345 by the authors.

**Figure 2.** A dynamically UPOM-generated refinement tree for a specific collection task collect objs from table(r1) for the current state. In [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Visualization of the object visibility check function used for the [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Left: schematic diagram of our acting and planning architecture [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

**Figure 5.** Figure 5: Heatmap of rollout utility across trials 1.1 and 1.2 (with and [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 12 canonical work pages

[1]

Deliberation for autonomous robots: A survey,

F. Ingrand and M. Ghallab, “Deliberation for autonomous robots: A survey,” Artificial Intelligence, vol. 247, pp. 10–44, 2017

work page 2017
[2]

Deliberative acting, planning and learning with hierarchical operational models,

S. Patra, J. Mason, M. Ghallab, D. Nau, and P. Traverso, “Deliberative acting, planning and learning with hierarchical operational models,” Artificial Intelligence, vol. 299, p. 103523, 2021

work page 2021
[3]

Ghallab, D

M. Ghallab, D. Nau, and P. Traverso, Automated planning and acting . Cambridge University Press, 2016

work page 2016
[4]

A physics-based simulated robotics testbed for planning and acting research,

O. Lima, M. G ¨unther, A. Sung, S. Stock, M. Vinci, J. Krause, J. Hertzberg, et al. , “A physics-based simulated robotics testbed for planning and acting research,” in ICAPS PlanRob Workshop, 2023

work page 2023
[5]

ROSPlan: Planning in the robot operating system,

M. Cashmore, M. Fox, D. Long, et al. , “ROSPlan: Planning in the robot operating system,” in ICAPS, vol. 25, 2015, pp. 333–341

work page 2015
[6]

PlanSys2: A Planning System Framework for ROS2,

F. Mart ´ın, J. G. Clavero, et al. , “PlanSys2: A Planning System Framework for ROS2,” in IROS, 2021, pp. 9742–9749

work page 2021
[7]

A closed-loop framework-independent bridge from aiplan4eu’s unified planning platform to embedded sys- tems,

S. H. S. S. Sadanandam, S. Stock, A. Sung, F. Ingrand, O. Lima, M. Vinci, and J. Hertzberg, “A closed-loop framework-independent bridge from aiplan4eu’s unified planning platform to embedded sys- tems,” in ICAPS PlanRob Workshop, 2023

work page 2023
[8]

Unified planning: Modeling, manipulating and solving ai planning problems in python,

A. Micheli, A. Bit-Monnot, G. R ¨oger, et al. , “Unified planning: Modeling, manipulating and solving ai planning problems in python,” SoftwareX, vol. 29, p. 102012, 2025

work page 2025

Show all 13 references

[9]

Extending a refinement acting engine for fleet management: Concurrency and resources,

J. Turi and A. Bit-Monnot, “Extending a refinement acting engine for fleet management: Concurrency and resources,” in International Conference on Tools with Artificial Intelligence (ICTAI). IEEE, 2022

2022
[10]

Towards blended reactive planning and acting using behavior trees,

M. Colledanchise, D. Almeida, and P. ¨Ogren, “Towards blended reactive planning and acting using behavior trees,” in ICRA, 2019

2019
[11]

Integrated task and motion planning,

C. R. Garrett, R. Chitnis, R. Holladay, et al. , “Integrated task and motion planning,” Annual review of control, robotics, and autonomous systems, vol. 4, no. 1, pp. 265–293, 2021

2021
[12]

Enhancing operational de- liberation in a refinement acting engine with continuous planning,

J. Turi, A. Bit-Monnot, and F. Ingrand, “Enhancing operational de- liberation in a refinement acting engine with continuous planning,” in ICAPS IntEx Workshop, 2023

2023
[13]

Deep object pose estimation for semantic robotic grasping of household objects,

J. Tremblay, T. To, B. Sundaralingam, et al. , “Deep object pose estimation for semantic robotic grasping of household objects,” in CoRL, 2018. [Online]. Available: https://arxiv.org/abs/1809.10790

2018 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Deliberation for autonomous robots: A survey,

F. Ingrand and M. Ghallab, “Deliberation for autonomous robots: A survey,” Artificial Intelligence, vol. 247, pp. 10–44, 2017

work page 2017

[2] [2]

Deliberative acting, planning and learning with hierarchical operational models,

S. Patra, J. Mason, M. Ghallab, D. Nau, and P. Traverso, “Deliberative acting, planning and learning with hierarchical operational models,” Artificial Intelligence, vol. 299, p. 103523, 2021

work page 2021

[3] [3]

Ghallab, D

M. Ghallab, D. Nau, and P. Traverso, Automated planning and acting . Cambridge University Press, 2016

work page 2016

[4] [4]

A physics-based simulated robotics testbed for planning and acting research,

O. Lima, M. G ¨unther, A. Sung, S. Stock, M. Vinci, J. Krause, J. Hertzberg, et al. , “A physics-based simulated robotics testbed for planning and acting research,” in ICAPS PlanRob Workshop, 2023

work page 2023

[5] [5]

ROSPlan: Planning in the robot operating system,

M. Cashmore, M. Fox, D. Long, et al. , “ROSPlan: Planning in the robot operating system,” in ICAPS, vol. 25, 2015, pp. 333–341

work page 2015

[6] [6]

PlanSys2: A Planning System Framework for ROS2,

F. Mart ´ın, J. G. Clavero, et al. , “PlanSys2: A Planning System Framework for ROS2,” in IROS, 2021, pp. 9742–9749

work page 2021

[7] [7]

A closed-loop framework-independent bridge from aiplan4eu’s unified planning platform to embedded sys- tems,

S. H. S. S. Sadanandam, S. Stock, A. Sung, F. Ingrand, O. Lima, M. Vinci, and J. Hertzberg, “A closed-loop framework-independent bridge from aiplan4eu’s unified planning platform to embedded sys- tems,” in ICAPS PlanRob Workshop, 2023

work page 2023

[8] [8]

Unified planning: Modeling, manipulating and solving ai planning problems in python,

A. Micheli, A. Bit-Monnot, G. R ¨oger, et al. , “Unified planning: Modeling, manipulating and solving ai planning problems in python,” SoftwareX, vol. 29, p. 102012, 2025

work page 2025