REVIEW 4 major objections 5 minor 25 references
Automatic Failure Recovery for End-User Programs on Service Mobile Robots
T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Shared runtime repairs hidden robot-task failures automatically.
desk verdict RTPL is a coherent, novel system for automatic failure recovery in end-user robot programs, but the paper's empirical claims outrun the evidence: single-trial runs, no code/data, and expert-set priors that the sensitivity analysis shows to be load-bearing. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is a pair: a belief-update function attached to each action and written by an expert, plus a runtime Bayes net constructed from the execution trace. The belief-update function maps a probabilistic world state to a distribution over next states, parameterized by prior failure probabilities such as $\alpha$. The runtime keeps each world literal as an independent Bernoulli variable, and when an action fails it reconditions the whole trace on the failure evidence using variable elimination. The recovery step then solves a constrained search over a binary perforated trace that says which earlier actions to rerun, minimizing the number of reruns while preserving each rerun action's preconditions; the search is small because every step is either re-execute or skip.
What would settle it
Take a real robot performing the two-package delivery task, set the model's failure prior for pickup low and the prior for loss in transit high, then repeatedly induce a missing-package failure at delivery; if RTPL consistently returns the robot to the mail room to repick instead of reporting that the package was lost, the diagnosis is tracking the prior rather than the evidence. More directly, run the same induced failure many times and count how often the recovered trace re-executes the action that actually failed.
Extended reading notes
Core claim
The central claim is that failure recovery can be generic rather than task-specific. RTPL's runtime incrementally builds a time-indexed Bayes net over a probabilistic world state called Bernoulli-STRIPS, where each action contributes belief-update factors parameterized by expert-set failure probabilities. When a later action's precondition fails, the runtime treats the failure predicate as evidence and performs posterior inference over all past states. It identifies the first time step where the posterior maximum-likelihood state diverges from the forward-predicted state, classifies the cause as a postcondition failure, which is recoverable, or an unintended effect, which is not, and if recoverable, searches for a minimal perforated trace of re-executable actions. The paper reports that four test tasks written without recovery code recovered from diverse errors, and that RTPL was faster than full re-execution on a real robot.
Load-bearing premise
The recovery system trusts that the expert-set probabilities of each failure mode match the real robot and its environment; if those priors are wrong, the most likely cause it computes will not be the actual cause.
Editorial extensions
If this is right
- A single expert-written robot model can give a whole family of end-user programs automatic recovery, so recovery logic does not have to be duplicated per task.
- Failures that become visible only many steps later, such as a human not actually placing a package, can be traced back to the responsible action and repaired.
- The Bayesian diagnosis distinguishes recoverable postcondition failures from unrecoverable unintended effects, so the robot can abort and report a cause rather than loop.
- Because the recovery plan skips actions already known to have succeeded, repairs use less wall-clock time than re-running the whole task from the start.
Reading between the lines
- Beyond the paper: one could test the same two-tier design with a deterministic dependency graph instead of a full probability model; the perforated-trace search would still work, but the choice of which failure to blame would degrade exactly when the expert-set priors are wrong.
- Beyond the paper: the independence assumption between world literals is an approximation; in environments where failures are correlated, such as a crowded room causing both navigation and interaction failures, the inferred root cause could be off, and a richer dependency model is the natural extension.
- Beyond the paper: because the runtime only needs a sequential action trace, the approach could be lifted to visual or demonstration-based programming languages that compile to a linear action sequence, making recovery independent of how the user wrote the program.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes RTPL, a two-tiered programming framework for service mobile robots. An expert roboticist writes a declarative robot model (PDDL-like action specifications with belief-update functions and failure priors), and a non-expert writes an ordinary sequential task program using those actions. At runtime, RTPL maintains a Bernoulli-STRIPS state representation, builds a time-indexed Bayesian network over action outcomes, and, when an error is observed, performs Bayesian inference to identify the most likely root-cause time step and failure predicates. If the failure is judged recoverable, the runtime computes a minimum-length 'perforated trace' that re-executes a subsequence of previously executed actions. The paper presents four example task programs (package delivery, elevator, signature collection, and escort), a parameter-sensitivity study of recovery decisions, and timing measurements on a real robot comparing RTPL with naive re-execution. The authors claim that RTPL allows complex tasks to be written concisely, correctly identifies root causes, and enables recovery without task-specific error-recovery code.
Significance. If the empirical claims are established, RTPL would be a useful contribution to end-user programming for service robots, decoupling task programs from explicit failure-handling logic and providing a principled Bayesian method for diagnosing latent interaction failures. The formal framework in Sections 3-5 is coherent: the BSSR representation, the Bayes-net construction, and the perforated-trace formulation are natural and potentially practical. The paper does not ship code or data, and the evaluation as presented is substantially weaker than the headline claims require. The absence of ground-truth root-cause accuracy, the demonstrated sensitivity to expert-chosen priors, and the unreported trial structure in the timing experiments mean that the central claims are currently plausible but not demonstrated.
major comments (4)
- [§5, §6.2, Eqs. (4)–(5), Figs. 6a/6b] The abstract's claim that RTPL 'correctly identifies the root cause of failure' is not supported by the evaluation. Section 6.2 shows that the same observed evidence—package B missing in 2-PD, visitor not confirming arrival in ES—produces different diagnoses (recoverable re-pickup, unrecoverable lost-in-transit, or predicted unrecoverable) depending on the expert-chosen failure priors α3, α4 and α1, α2 respectively. Since the recovery decision is a threshold on these priors, the central claim requires either a principled calibration procedure for the α values or a sensitivity analysis around the operating point actually used in the Section 6.1 and 6.3 experiments; neither is provided, and no ground-truth labels or repeated-trial accuracy numbers for root-cause identification appear anywhere in the paper.
- [§6.3, Table 2] The timing comparison in Section 6.3 is not statistically usable. The text does not state the number of trials per condition, no variance or confidence intervals are reported, and the experimental protocol (how each failure was induced, what 'naive re-execution' does exactly, and whether all human interactions are repeated) is unspecified. With no error bars, the reported savings (e.g., 1m31s vs 2m28s for EL) cannot be distinguished from noise. Furthermore, the only baseline is full re-execution; there is no comparison with an alternative recovery method such as replanning or hand-written recovery handlers, so the specific benefit of RTPL's Bayesian/perforated-trace mechanism is not established. No code or data are provided to reproduce these measurements.
- [§5, Eqs. (6)–(8)] The perforated-trace repair is presented only as an optimization problem. The paper gives no algorithm for constructing the optimal valid perforated trace τ*, no proof that the search is complete or that the minimum-length trace is always found, and no complexity analysis or discussion of how the runtime solves this optimization online at failure time. Since this search is the mechanism that produces the recovery traces in Table 1, the proposed system is not fully specified as written; this is a load-bearing gap rather than a presentation issue.
- [Abstract, §6.1, Appendix B] The first central claim, that RTPL 'allows complex tasks to be written concisely', is not evaluated. There is no user study, no comparison with other end-user robot programming languages, and no quantitative measure of program complexity or learnability. The appendix code listings show that the example programs are short, but that alone does not establish that non-experts can write, understand, or maintain such programs, and it does not substantiate the 'complex tasks' part of the claim.
minor comments (5)
- [§3.2, Figure 4] The third line of the 2-package delivery program has a typo: `robot.pickup(Package B")` is missing the opening quotation mark; it should be `robot.pickup("Package B")`.
- [§4, Eq. (1)] The product notation in Eq. (1) is difficult to read as printed; please use the standard `\prod_{i=1}^{n}` form.
- [§5, after Eq. (4)] The phrase 're-executing the action atf from that time-step' should use the subscript notation consistently, e.g., 'the action a_{t_f}'.
- [§6.3, Table 2] The naive re-execution baseline should be defined precisely, including whether it restarts from the beginning of the task program and repeats all earlier human interactions; the current description is insufficient for interpreting the reported times.
- [Table 1] The caption of Table 1 should state directly what the red filled nodes, red edges, and dashed/solid borders mean, since these graphical elements are essential for reading the execution traces.
Circularity Check
No circularity: RTPL's inference and recovery are self-contained, with expert-set α priors as explicit inputs and a sensitivity study rather than a fitted post hoc explanation.
full rationale
The paper's derivation chain is not circular. The expert-provided robot model (Section 3.1) supplies action preconditions, postconditions, and belief-update functions whose α parameters are explicitly called prior probabilities (e.g., 'α is the prior probability that the human accidentally fails to give the item to the robot'). The runtime then builds a Bayes net from the executed trace (Section 4), conditions on observed failure evidence via p(W_{0:t} | e_f), and selects a minimum-length perforated trace via Equations (6)-(8). None of these steps defines its own output: the α values are inputs set before the experiments, not parameters fitted to reproduce the shown recoveries. Section 6.2 even varies α1, α2, α3, and α4 over a grid and reports the resulting regions (RV/IF/PF and RP/IF/PF), which is a legitimate parametric sensitivity analysis, not a post hoc re-fit. The two prior self-citations (Holtz, Guha, and Biswas 2018; Rosenthal, Biswas, and Veloso 2010) appear only in related-work context and are not load-bearing inputs to RTPL's inference or recovery derivation. The only substantive concern is that the abstract's claim to 'correctly identifies the root cause' is conditioned on the quality of the expert-set priors; that is a model-accuracy and calibration issue, not a circularity, and the paper openly acknowledges that 'the expert may update the robot model with better priors' (§3.1). The reported recovery traces in Table 1 are observed behaviors under induced failures using a single fixed model, rather than quantities forced by construction. Therefore, no circular step can be exhibited.
Assumptions & free parameters
free parameters (6)
- α_enter_room =
not specified
- α_pickup =
not specified (varied as α3 in Section 6.2)
- α_give =
not specified (varied as α4 in Section 6.2)
- α_askFollow (α1 in ES) =
varied 0.0-0.5 in Section 6.2
- α_escortTo (α2 in ES) =
varied 0.0-0.5 in Section 6.2
- α_other_interactions =
not specified
assumptions (5)
- domain assumption Bernoulli literals in the BSSR world state are independent (Eq. 1).
- domain assumption The expert-written robot model correctly represents all relevant failure modes.
- domain assumption Variable elimination on the linear Bayes net is computationally tractable at runtime.
- ad hoc to paper The perforated-trace search (Eqs. 6-8) finds a minimal valid repair.
- ad hoc to paper Maximum-likelihood predicate evaluation with a 0.5 threshold is a sound decision rule.
invented entities (2)
-
Bernoulli-STRIPS State Representation (BSSR)
-
Perforated trace
Cite this review
Pith. "Pith review of Automatic Failure Recovery for End-User Programs on Service Mobile Robots." pith.science (2026). https://pith.science/paper/62MRQ2WX
@misc{pith2026190902778,
author = {Pith},
title = {Pith review of: Automatic Failure Recovery for End-User Programs on Service Mobile Robots},
year = {2026},
howpublished = {\url{https://pith.science/paper/62MRQ2WX}},
note = {Machine review of arXiv:1909.02778}
}
read the original abstract
For service mobile robots to be most effective, it must be possible for non-experts and even end-users to program them to do new tasks. Regardless of the programming method (e.g., by demonstration or traditional programming), robot task programs are challenging to write, because they rely on multiple actions to succeed, including human-robot interactions. Unfortunately, interactions are prone to fail, because a human may perform the wrong action (e.g., if the robot's request is not clear). Moreover, when the robot cannot directly observe the human action, it may not detect the failure until several steps after it occurs. Therefore, writing fault-tolerant robot tasks is beyond the ability of non-experts. This paper presents a principled approach to detect and recover from a broad class of failures that occur in end-user programs on service mobile robots. We present a two-tiered Robot Task Programming Language (RTPL): 1) an expert roboticist uses a specification language to write a probabilistic model of the robot's actions and interactions, and 2) a non-expert then writes an ordinary sequential program for a particular task. The RTPL runtime system executes the task program sequentially, while using the probabilistic model to build a Bayesian network that tracks possible, unobserved failures. If an error is observed, RTPL uses Bayesian inference to find the likely root cause of the error, and then attempts to re-execute a portion of the program for recovery. Our empirical results show that RTPL 1) allows complex tasks to be written concisely, 2) correctly identifies the root cause of failure, and 3) allows multiple tasks to recover from a variety of errors, without task-specific error-recovery code.
Figures
Reference graph
Works this paper leans on
-
[1]
2014] Alexandrova, S.; Cakmak, M.; Hsiao, K.; and Takayama, L
[Alexandrova et al. 2014] Alexandrova, S.; Cakmak, M.; Hsiao, K.; and Takayama, L
work page 2014
-
[4]
Artificial in- telligence 90(1-2):281–300
Fast planning through planning graph analysis. Artificial in- telligence 90(1-2):281–300. [Brechtel, Gindele, and Dillmann 2011] Brechtel, S.; Gin- dele, T.; and Dillmann, R
work page 2011
-
[9]
In Twenty-Second International Joint Con- ference on Artificial Intelligence
Automatic state abstraction from demonstration. In Twenty-Second International Joint Con- ference on Artificial Intelligence. [Duvallet, Kollar, and Stentz 2013] Duvallet, F.; Kollar, T.; and Stentz, A
work page 2013
-
[11]
In IEEE Workshop on Robot Vision (WORV)
Autonomous nav- igation and sign detector learning. In IEEE Workshop on Robot Vision (WORV). [Featherston et al. 2014] Featherston, E.; Sridharan, M.; Ur- ban, S.; and Urban, J
work page 2014
-
[12]
In AAAI Symposium on Educational Ad- vances in Artificial Intelligence (EAAI)
Dorothy: enhancing bidirec- tional communication between a 3d programming interface and mobile robots. In AAAI Symposium on Educational Ad- vances in Artificial Intelligence (EAAI). [Gonz´alez-Fierro et al. 2013] Gonz ´alez-Fierro, M.; Bal- aguer, C.; Swann, N.; and Nanayakkara, T
work page 2013
-
[13]
In IEEE-RAS International Conference on Humanoid Robots (Humanoids)
A humanoid robot standing up through learning from demonstration using a multimodal reward function. In IEEE-RAS International Conference on Humanoid Robots (Humanoids). [Holtz, Guha, and Biswas 2018] Holtz, J.; Guha, A.; and Biswas, J
work page 2018
-
[14]
Interactive robot transition repair with SMT. In International Joint Conference on Artificial Intel- ligence and the European Conference on Artificial Intelli- gence (IJCAI-ECAI). [Huang, Lau, and Cakmak 2016] Huang, J.; Lau, T.; and Cakmak, M
work page 2016
-
[19]
In Interna- tional Conference on Autonomous Agents and Multi-Agent Systems
An interactive approach for situated task specification through verbal instructions. In Interna- tional Conference on Autonomous Agents and Multi-Agent Systems. International Foundation for Autonomous Agents and Multiagent Systems. [Meric ¸li, Veloso, and Akın 2012] Meric ¸li, C.; Veloso, M.; and Akın, H. L
work page 2012
Show all 25 references
-
[22]
In Annual Meeting on Association for Computational Linguistics
Spoken dialogue management using prob- abilistic reasoning. In Annual Meeting on Association for Computational Linguistics. [Van Der Krogt and De Weerdt 2005] Van Der Krogt, R., and De Weerdt, M
2005
-
[25]
In ACM Con- ference on Human Factors in Computing Systems (CHI)
Evaluating coblox: A comparative study of robotics pro- gramming environments for adult novices. In ACM Con- ference on Human Factors in Computing Systems (CHI). [Wray, Witwicki, and Zilberstein 2017] Wray, K. H.; Witwicki, S. J.; and Zilberstein, S
2017
-
[1542]
[Brenner et al
IEEE. [Brenner et al. 2007] Brenner, M.; Hawes, N.; Kelleher, J. D.; and Wyatt, J. L
2007
-
[1997]
In International Conference on Computer Aided Verification (CAV)
Efficient detection of vacuity in ACTL formulas. In International Conference on Computer Aided Verification (CAV). [Blum and Furst 1997] Blum, A. L., and Furst, M. L
1997
-
[1998]
Technical Report TR-98-003, Yale Center for Computational Vision and Control
PDDL – the Planning Domain Definition Language. Technical Report TR-98-003, Yale Center for Computational Vision and Control. [Meric ¸li et al. 2014] Meric ¸li, C.; Klee, S. D.; Paparian, J.; and Veloso, M
2014
-
[2000]
Journal of Artificial Intelli- gence Research 12:1–34
Planning graph as a (dynamic) csp: Exploiting ebl, ddb and other csp search techniques in graphplan. Journal of Artificial Intelli- gence Research 12:1–34. [Kroemer, Niekum, and Konidaris 2019] Kroemer, O.; Niekum, S.; and Konidaris, G
2019
-
[2005]
In ICAPS
Plan repair as an extension of planning. In ICAPS. [Weintrop et al. 2017] Weintrop, D.; Shepherd, D. C.; Fran- cis, P.; and Franklin, D
2017
-
[2007]
In International Joint Conference on Arti- ficial Intelligence (IJCAI)
Mediating between qualitative and quantitative representations for task-orientated human- robot interaction. In International Joint Conference on Arti- ficial Intelligence (IJCAI). [Chasins, Mueller, and Bodik 2018] Chasins, S. E.; Mueller, M.; and Bodik, R
2018
-
[2010]
In International Conference on Autonomous Agents and Multiagent Systems (AAMAS)
An effective personal mobile robot agent through a symbiotic human-robot interaction. In International Conference on Autonomous Agents and Multiagent Systems (AAMAS). [Roy, Pineau, and Thrun 2000] Roy, N.; Pineau, J.; and Thrun, S
2000
-
[2011]
In 2011 14th International IEEE Confer- ence on Intelligent Transportation Systems (ITSC) , 1537–
Probabilistic mdp-behavior planning for cars. In 2011 14th International IEEE Confer- ence on Intelligent Transportation Systems (ITSC) , 1537–
2011
-
[2012]
Interna- tional Journal of Social Robotics 4(4):423–435
Multi-resolution corrective demon- stration for efficient task execution and refinement. Interna- tional Journal of Social Robotics 4(4):423–435. [Rosenthal, Biswas, and Veloso 2010] Rosenthal, S.; Biswas, J.; and Veloso, M
2010
-
[2013]
In IEEE International Conference on Robotics and Automa- tion (ICRA)
Imitation learning for natural lan- guage direction following through unknown environments. In IEEE International Conference on Robotics and Automa- tion (ICRA). [Ellis et al. 2013] Ellis, L.; Pugeault, N.; ¨Ofj¨all, K.; Hedborg, J.; Bowden, R.; and Felsberg, M
2013
-
[2014]
In Robotics: science and systems
Robot programming by demonstration with interactive action visualizations. In Robotics: science and systems. [Beer et al. 1997] Beer, I.; Ben-David, S.; Eisner, C.; and Rodeh, Y
1997
-
[2016]
In ACM/IEEE Interna- tional Conference on Human-Robot Interaction (HRI)
Design and evaluation of a rapid pro- gramming system for service robots. In ACM/IEEE Interna- tional Conference on Human-Robot Interaction (HRI). [Kambhampati 2000] Kambhampati, S
2000
-
[2017]
In IEEE Blocks and Beyond Workshop (B&B)
Blockly goes to work: Block- based programming for industrial robots. In IEEE Blocks and Beyond Workshop (B&B). [Weintrop et al. 2018] Weintrop, D.; Afzal, A.; Salac, J.; Francis, P.; Li, B.; Shepherd, D. C.; and Franklin, D
2018
-
[2018]
InACM Symposium on User Interface Software and Technology (UIST)
Rousillon: Scraping distributed hierarchical web data. InACM Symposium on User Interface Software and Technology (UIST). [Cobo et al. 2011] Cobo, L. C.; Zang, P.; Isbell Jr, C. L.; and Thomaz, A. L
2011
-
[2019]
arXiv preprint arXiv:1907.03146
A review of robot learning for manipulation: Challenges, representations, and algorithms. arXiv preprint arXiv:1907.03146. [McDermott et al. 1998] McDermott, D.; Ghallab, M.; Howe, A.; Knoblock, C.; Ram, A.; Veloso, M.; Weld, D.; and Wilkins, D
1907 arXiv
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.