Pith. sign in

REVIEW 4 major objections 5 minor 25 references

Automatic Failure Recovery for End-User Programs on Service Mobile Robots

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read Shared runtime repairs hidden robot-task failures automatically.

desk verdict RTPL is a coherent, novel system for automatic failure recovery in end-user robot programs, but the paper's empirical claims outrun the evidence: single-trial runs, no code/data, and expert-set priors that the sensitivity analysis shows to be load-bearing. read the letter →

arxiv 1909.02778 v1 pith:62MRQ2WX submitted 2019-09-06 cs.RO

classification cs.RO
keywords servicemobilerobotsend-userprogrammingfailurerecoveryprobabilisticmodelBayesianinferencerobottaskhuman-robotinteractionperforatedtrace
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a service robot can execute an ordinary sequential task program written by a non-expert and still recover automatically from many failures, including failures the robot cannot see at the moment they happen. The trick is to separate robot-specific knowledge from the task: an expert writes a probabilistic model of each action, and the runtime uses that model to track uncertain state and, on error, to infer the most likely earlier cause. The robot then re-executes only the shortest subsequence of prior actions needed, skipping actions that already succeeded. If this works as reported, end users would not need to write any error-handling logic.

What carries the argument

The machinery is a pair: a belief-update function attached to each action and written by an expert, plus a runtime Bayes net constructed from the execution trace. The belief-update function maps a probabilistic world state to a distribution over next states, parameterized by prior failure probabilities such as $\alpha$. The runtime keeps each world literal as an independent Bernoulli variable, and when an action fails it reconditions the whole trace on the failure evidence using variable elimination. The recovery step then solves a constrained search over a binary perforated trace that says which earlier actions to rerun, minimizing the number of reruns while preserving each rerun action's preconditions; the search is small because every step is either re-execute or skip.

What would settle it

Take a real robot performing the two-package delivery task, set the model's failure prior for pickup low and the prior for loss in transit high, then repeatedly induce a missing-package failure at delivery; if RTPL consistently returns the robot to the mail room to repick instead of reporting that the package was lost, the diagnosis is tracking the prior rather than the evidence. More directly, run the same induced failure many times and count how often the recovered trace re-executes the action that actually failed.

Watch

Extended reading notes

Core claim

The central claim is that failure recovery can be generic rather than task-specific. RTPL's runtime incrementally builds a time-indexed Bayes net over a probabilistic world state called Bernoulli-STRIPS, where each action contributes belief-update factors parameterized by expert-set failure probabilities. When a later action's precondition fails, the runtime treats the failure predicate as evidence and performs posterior inference over all past states. It identifies the first time step where the posterior maximum-likelihood state diverges from the forward-predicted state, classifies the cause as a postcondition failure, which is recoverable, or an unintended effect, which is not, and if recoverable, searches for a minimal perforated trace of re-executable actions. The paper reports that four test tasks written without recovery code recovered from diverse errors, and that RTPL was faster than full re-execution on a real robot.

Load-bearing premise

The recovery system trusts that the expert-set probabilities of each failure mode match the real robot and its environment; if those priors are wrong, the most likely cause it computes will not be the actual cause.

Editorial extensions

If this is right

  • A single expert-written robot model can give a whole family of end-user programs automatic recovery, so recovery logic does not have to be duplicated per task.
  • Failures that become visible only many steps later, such as a human not actually placing a package, can be traced back to the responsible action and repaired.
  • The Bayesian diagnosis distinguishes recoverable postcondition failures from unrecoverable unintended effects, so the robot can abort and report a cause rather than loop.
  • Because the recovery plan skips actions already known to have succeeded, repairs use less wall-clock time than re-running the whole task from the start.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: one could test the same two-tier design with a deterministic dependency graph instead of a full probability model; the perforated-trace search would still work, but the choice of which failure to blame would degrade exactly when the expert-set priors are wrong.
  • Beyond the paper: the independence assumption between world literals is an approximation; in environments where failures are correlated, such as a crowded room causing both navigation and interaction failures, the inferred root cause could be off, and a richer dependency model is the natural extension.
  • Beyond the paper: because the runtime only needs a sequential action trace, the approach could be lifted to visual or demonstration-based programming languages that compile to a linear action sequence, making recovery independent of how the user wrote the program.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes RTPL, a two-tiered programming framework for service mobile robots. An expert roboticist writes a declarative robot model (PDDL-like action specifications with belief-update functions and failure priors), and a non-expert writes an ordinary sequential task program using those actions. At runtime, RTPL maintains a Bernoulli-STRIPS state representation, builds a time-indexed Bayesian network over action outcomes, and, when an error is observed, performs Bayesian inference to identify the most likely root-cause time step and failure predicates. If the failure is judged recoverable, the runtime computes a minimum-length 'perforated trace' that re-executes a subsequence of previously executed actions. The paper presents four example task programs (package delivery, elevator, signature collection, and escort), a parameter-sensitivity study of recovery decisions, and timing measurements on a real robot comparing RTPL with naive re-execution. The authors claim that RTPL allows complex tasks to be written concisely, correctly identifies root causes, and enables recovery without task-specific error-recovery code.

Significance. If the empirical claims are established, RTPL would be a useful contribution to end-user programming for service robots, decoupling task programs from explicit failure-handling logic and providing a principled Bayesian method for diagnosing latent interaction failures. The formal framework in Sections 3-5 is coherent: the BSSR representation, the Bayes-net construction, and the perforated-trace formulation are natural and potentially practical. The paper does not ship code or data, and the evaluation as presented is substantially weaker than the headline claims require. The absence of ground-truth root-cause accuracy, the demonstrated sensitivity to expert-chosen priors, and the unreported trial structure in the timing experiments mean that the central claims are currently plausible but not demonstrated.

major comments (4)
  1. [§5, §6.2, Eqs. (4)–(5), Figs. 6a/6b] The abstract's claim that RTPL 'correctly identifies the root cause of failure' is not supported by the evaluation. Section 6.2 shows that the same observed evidence—package B missing in 2-PD, visitor not confirming arrival in ES—produces different diagnoses (recoverable re-pickup, unrecoverable lost-in-transit, or predicted unrecoverable) depending on the expert-chosen failure priors α3, α4 and α1, α2 respectively. Since the recovery decision is a threshold on these priors, the central claim requires either a principled calibration procedure for the α values or a sensitivity analysis around the operating point actually used in the Section 6.1 and 6.3 experiments; neither is provided, and no ground-truth labels or repeated-trial accuracy numbers for root-cause identification appear anywhere in the paper.
  2. [§6.3, Table 2] The timing comparison in Section 6.3 is not statistically usable. The text does not state the number of trials per condition, no variance or confidence intervals are reported, and the experimental protocol (how each failure was induced, what 'naive re-execution' does exactly, and whether all human interactions are repeated) is unspecified. With no error bars, the reported savings (e.g., 1m31s vs 2m28s for EL) cannot be distinguished from noise. Furthermore, the only baseline is full re-execution; there is no comparison with an alternative recovery method such as replanning or hand-written recovery handlers, so the specific benefit of RTPL's Bayesian/perforated-trace mechanism is not established. No code or data are provided to reproduce these measurements.
  3. [§5, Eqs. (6)–(8)] The perforated-trace repair is presented only as an optimization problem. The paper gives no algorithm for constructing the optimal valid perforated trace τ*, no proof that the search is complete or that the minimum-length trace is always found, and no complexity analysis or discussion of how the runtime solves this optimization online at failure time. Since this search is the mechanism that produces the recovery traces in Table 1, the proposed system is not fully specified as written; this is a load-bearing gap rather than a presentation issue.
  4. [Abstract, §6.1, Appendix B] The first central claim, that RTPL 'allows complex tasks to be written concisely', is not evaluated. There is no user study, no comparison with other end-user robot programming languages, and no quantitative measure of program complexity or learnability. The appendix code listings show that the example programs are short, but that alone does not establish that non-experts can write, understand, or maintain such programs, and it does not substantiate the 'complex tasks' part of the claim.
minor comments (5)
  1. [§3.2, Figure 4] The third line of the 2-package delivery program has a typo: `robot.pickup(Package B")` is missing the opening quotation mark; it should be `robot.pickup("Package B")`.
  2. [§4, Eq. (1)] The product notation in Eq. (1) is difficult to read as printed; please use the standard `\prod_{i=1}^{n}` form.
  3. [§5, after Eq. (4)] The phrase 're-executing the action atf from that time-step' should use the subscript notation consistently, e.g., 'the action a_{t_f}'.
  4. [§6.3, Table 2] The naive re-execution baseline should be defined precisely, including whether it restarts from the beginning of the task program and repeats all earlier human interactions; the current description is insufficient for interpreting the reported times.
  5. [Table 1] The caption of Table 1 should state directly what the red filled nodes, red edges, and dashed/solid borders mean, since these graphical elements are essential for reading the execution traces.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: RTPL's inference and recovery are self-contained, with expert-set α priors as explicit inputs and a sensitivity study rather than a fitted post hoc explanation.

full rationale

The paper's derivation chain is not circular. The expert-provided robot model (Section 3.1) supplies action preconditions, postconditions, and belief-update functions whose α parameters are explicitly called prior probabilities (e.g., 'α is the prior probability that the human accidentally fails to give the item to the robot'). The runtime then builds a Bayes net from the executed trace (Section 4), conditions on observed failure evidence via p(W_{0:t} | e_f), and selects a minimum-length perforated trace via Equations (6)-(8). None of these steps defines its own output: the α values are inputs set before the experiments, not parameters fitted to reproduce the shown recoveries. Section 6.2 even varies α1, α2, α3, and α4 over a grid and reports the resulting regions (RV/IF/PF and RP/IF/PF), which is a legitimate parametric sensitivity analysis, not a post hoc re-fit. The two prior self-citations (Holtz, Guha, and Biswas 2018; Rosenthal, Biswas, and Veloso 2010) appear only in related-work context and are not load-bearing inputs to RTPL's inference or recovery derivation. The only substantive concern is that the abstract's claim to 'correctly identifies the root cause' is conditioned on the quality of the expert-set priors; that is a model-accuracy and calibration issue, not a circularity, and the paper openly acknowledges that 'the expert may update the robot model with better priors' (§3.1). The reported recovery traces in Table 1 are observed behaviors under induced failures using a single fixed model, rather than quantities forced by construction. Therefore, no circular step can be exhibited.

Assumptions & free parameters 6 free parameters · 5 assumptions · 2 invented entities

The central claim rests on the expert's ability to specify accurate failure priors and on two algorithm-design choices (independent Bernoulli literals and the 0.5 ML threshold) that are not empirically validated. These are the main non-standard commitments the paper asks the reader to accept.

free parameters (6)
  • α_enter_room = not specified
    Prior probability that door detection incorrectly reports an open door (Figure 2b).
  • α_pickup = not specified (varied as α3 in Section 6.2)
    Prior probability that a human fails to give the correct item during pickup (Figure 3b).
  • α_give = not specified (varied as α4 in Section 6.2)
    Prior probability that a human takes the wrong package during give (used in 2-PD analysis).
  • α_askFollow (α1 in ES) = varied 0.0-0.5 in Section 6.2
    Prior probability the visitor fails to follow after askFollow.
  • α_escortTo (α2 in ES) = varied 0.0-0.5 in Section 6.2
    Prior probability the visitor loses track of the robot during escortTo.
  • α_other_interactions = not specified
    Each of the remaining interactions (callElevator, selectFloor, confirmFloor, getSignature, confirmArrival) has an analogous independent failure prior, per Appendix A.
assumptions (5)
  • domain assumption Bernoulli literals in the BSSR world state are independent (Eq. 1).
    The product-form p(W) assumes no correlations between predicates such as robot location and held items, which may not hold after sequences of actions.
  • domain assumption The expert-written robot model correctly represents all relevant failure modes.
    RTPL can only infer failures that are encoded in belief-update functions; an unmodeled failure will not be diagnosed.
  • domain assumption Variable elimination on the linear Bayes net is computationally tractable at runtime.
    The paper claims efficiency but provides no complexity analysis or measured inference time.
  • ad hoc to paper The perforated-trace search (Eqs. 6-8) finds a minimal valid repair.
    The graphplan-like backward search is asserted but no algorithm or proof of correctness is given.
  • ad hoc to paper Maximum-likelihood predicate evaluation with a 0.5 threshold is a sound decision rule.
    A near-0.5 probability could flip the inferred world state and trigger spurious failure recovery.
invented entities (2)
  • Bernoulli-STRIPS State Representation (BSSR)
    purpose: Probabilistic world-state representation with independent Bernoulli literals for tracking robot state over time.
    A new formal representation introduced in this paper; it is a modeling construct with no external falsifiable handle.
  • Perforated trace
    purpose: A binary-indicator vector over prior actions that RTPL re-executes to satisfy the failing action's preconditions.
    A new repair concept; it is an algorithmic device, not an empirical entity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Automatic Failure Recovery for End-User Programs on Service Mobile Robots." pith.science (2026). https://pith.science/paper/62MRQ2WX

@misc{pith2026190902778,
  author       = {Pith},
  title        = {Pith review of: Automatic Failure Recovery for End-User Programs on Service Mobile Robots},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/62MRQ2WX}},
  note         = {Machine review of arXiv:1909.02778}
}
read the original abstract

For service mobile robots to be most effective, it must be possible for non-experts and even end-users to program them to do new tasks. Regardless of the programming method (e.g., by demonstration or traditional programming), robot task programs are challenging to write, because they rely on multiple actions to succeed, including human-robot interactions. Unfortunately, interactions are prone to fail, because a human may perform the wrong action (e.g., if the robot's request is not clear). Moreover, when the robot cannot directly observe the human action, it may not detect the failure until several steps after it occurs. Therefore, writing fault-tolerant robot tasks is beyond the ability of non-experts. This paper presents a principled approach to detect and recover from a broad class of failures that occur in end-user programs on service mobile robots. We present a two-tiered Robot Task Programming Language (RTPL): 1) an expert roboticist uses a specification language to write a probabilistic model of the robot's actions and interactions, and 2) a non-expert then writes an ordinary sequential program for a particular task. The RTPL runtime system executes the task program sequentially, while using the probabilistic model to build a Bayesian network that tracks possible, unobserved failures. If an error is observed, RTPL uses Bayesian inference to find the likely root cause of the error, and then attempts to re-execute a portion of the program for recovery. Our empirical results show that RTPL 1) allows complex tasks to be written concisely, 2) correctly identifies the root cause of failure, and 3) allows multiple tasks to recover from a variety of errors, without task-specific error-recovery code.

Figures

Figures reproduced from arXiv: 1909.02778 by the authors.

Figure 1
Figure 1. A canonical robot task program to pickup and de [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. RTPL specification of the pickup action. x with probability 0.9. Any literal that is not defined in the world state is assumed to be identically false. The RTPL run￾time system uses the belief update function to build a prob￾abilistic model of world state. Note that the pre- and post- conditions do not account for real-world execution errors. For example, door detection is imperfect: a temporary obstacle, such as a … view at source ↗
Figure 4
Figure 4. A 2-package delivery program written in RTPL. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: An example Bayes net constructed during normal [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Effect of varying robot model parameters on re [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 24 canonical work pages

  1. [1]

    2014] Alexandrova, S.; Cakmak, M.; Hsiao, K.; and Takayama, L

    [Alexandrova et al. 2014] Alexandrova, S.; Cakmak, M.; Hsiao, K.; and Takayama, L

  2. [4]

    Artificial in- telligence 90(1-2):281–300

    Fast planning through planning graph analysis. Artificial in- telligence 90(1-2):281–300. [Brechtel, Gindele, and Dillmann 2011] Brechtel, S.; Gin- dele, T.; and Dillmann, R

  3. [9]

    In Twenty-Second International Joint Con- ference on Artificial Intelligence

    Automatic state abstraction from demonstration. In Twenty-Second International Joint Con- ference on Artificial Intelligence. [Duvallet, Kollar, and Stentz 2013] Duvallet, F.; Kollar, T.; and Stentz, A

  4. [11]

    In IEEE Workshop on Robot Vision (WORV)

    Autonomous nav- igation and sign detector learning. In IEEE Workshop on Robot Vision (WORV). [Featherston et al. 2014] Featherston, E.; Sridharan, M.; Ur- ban, S.; and Urban, J

  5. [12]

    In AAAI Symposium on Educational Ad- vances in Artificial Intelligence (EAAI)

    Dorothy: enhancing bidirec- tional communication between a 3d programming interface and mobile robots. In AAAI Symposium on Educational Ad- vances in Artificial Intelligence (EAAI). [Gonz´alez-Fierro et al. 2013] Gonz ´alez-Fierro, M.; Bal- aguer, C.; Swann, N.; and Nanayakkara, T

  6. [13]

    In IEEE-RAS International Conference on Humanoid Robots (Humanoids)

    A humanoid robot standing up through learning from demonstration using a multimodal reward function. In IEEE-RAS International Conference on Humanoid Robots (Humanoids). [Holtz, Guha, and Biswas 2018] Holtz, J.; Guha, A.; and Biswas, J

  7. [14]

    In International Joint Conference on Artificial Intel- ligence and the European Conference on Artificial Intelli- gence (IJCAI-ECAI)

    Interactive robot transition repair with SMT. In International Joint Conference on Artificial Intel- ligence and the European Conference on Artificial Intelli- gence (IJCAI-ECAI). [Huang, Lau, and Cakmak 2016] Huang, J.; Lau, T.; and Cakmak, M

  8. [19]

    In Interna- tional Conference on Autonomous Agents and Multi-Agent Systems

    An interactive approach for situated task specification through verbal instructions. In Interna- tional Conference on Autonomous Agents and Multi-Agent Systems. International Foundation for Autonomous Agents and Multiagent Systems. [Meric ¸li, Veloso, and Akın 2012] Meric ¸li, C.; Veloso, M.; and Akın, H. L

Show all 25 references
  1. [22]

    In Annual Meeting on Association for Computational Linguistics

    Spoken dialogue management using prob- abilistic reasoning. In Annual Meeting on Association for Computational Linguistics. [Van Der Krogt and De Weerdt 2005] Van Der Krogt, R., and De Weerdt, M

  2. [25]

    In ACM Con- ference on Human Factors in Computing Systems (CHI)

    Evaluating coblox: A comparative study of robotics pro- gramming environments for adult novices. In ACM Con- ference on Human Factors in Computing Systems (CHI). [Wray, Witwicki, and Zilberstein 2017] Wray, K. H.; Witwicki, S. J.; and Zilberstein, S

  3. [1542]

    [Brenner et al

    IEEE. [Brenner et al. 2007] Brenner, M.; Hawes, N.; Kelleher, J. D.; and Wyatt, J. L

  4. [1997]

    In International Conference on Computer Aided Verification (CAV)

    Efficient detection of vacuity in ACTL formulas. In International Conference on Computer Aided Verification (CAV). [Blum and Furst 1997] Blum, A. L., and Furst, M. L

  5. [1998]

    Technical Report TR-98-003, Yale Center for Computational Vision and Control

    PDDL – the Planning Domain Definition Language. Technical Report TR-98-003, Yale Center for Computational Vision and Control. [Meric ¸li et al. 2014] Meric ¸li, C.; Klee, S. D.; Paparian, J.; and Veloso, M

  6. [2000]

    Journal of Artificial Intelli- gence Research 12:1–34

    Planning graph as a (dynamic) csp: Exploiting ebl, ddb and other csp search techniques in graphplan. Journal of Artificial Intelli- gence Research 12:1–34. [Kroemer, Niekum, and Konidaris 2019] Kroemer, O.; Niekum, S.; and Konidaris, G

  7. [2005]

    In ICAPS

    Plan repair as an extension of planning. In ICAPS. [Weintrop et al. 2017] Weintrop, D.; Shepherd, D. C.; Fran- cis, P.; and Franklin, D

  8. [2007]

    In International Joint Conference on Arti- ficial Intelligence (IJCAI)

    Mediating between qualitative and quantitative representations for task-orientated human- robot interaction. In International Joint Conference on Arti- ficial Intelligence (IJCAI). [Chasins, Mueller, and Bodik 2018] Chasins, S. E.; Mueller, M.; and Bodik, R

  9. [2010]

    In International Conference on Autonomous Agents and Multiagent Systems (AAMAS)

    An effective personal mobile robot agent through a symbiotic human-robot interaction. In International Conference on Autonomous Agents and Multiagent Systems (AAMAS). [Roy, Pineau, and Thrun 2000] Roy, N.; Pineau, J.; and Thrun, S

  10. [2011]

    In 2011 14th International IEEE Confer- ence on Intelligent Transportation Systems (ITSC) , 1537–

    Probabilistic mdp-behavior planning for cars. In 2011 14th International IEEE Confer- ence on Intelligent Transportation Systems (ITSC) , 1537–

  11. [2012]

    Interna- tional Journal of Social Robotics 4(4):423–435

    Multi-resolution corrective demon- stration for efficient task execution and refinement. Interna- tional Journal of Social Robotics 4(4):423–435. [Rosenthal, Biswas, and Veloso 2010] Rosenthal, S.; Biswas, J.; and Veloso, M

  12. [2013]

    In IEEE International Conference on Robotics and Automa- tion (ICRA)

    Imitation learning for natural lan- guage direction following through unknown environments. In IEEE International Conference on Robotics and Automa- tion (ICRA). [Ellis et al. 2013] Ellis, L.; Pugeault, N.; ¨Ofj¨all, K.; Hedborg, J.; Bowden, R.; and Felsberg, M

  13. [2014]

    In Robotics: science and systems

    Robot programming by demonstration with interactive action visualizations. In Robotics: science and systems. [Beer et al. 1997] Beer, I.; Ben-David, S.; Eisner, C.; and Rodeh, Y

  14. [2016]

    In ACM/IEEE Interna- tional Conference on Human-Robot Interaction (HRI)

    Design and evaluation of a rapid pro- gramming system for service robots. In ACM/IEEE Interna- tional Conference on Human-Robot Interaction (HRI). [Kambhampati 2000] Kambhampati, S

  15. [2017]

    In IEEE Blocks and Beyond Workshop (B&B)

    Blockly goes to work: Block- based programming for industrial robots. In IEEE Blocks and Beyond Workshop (B&B). [Weintrop et al. 2018] Weintrop, D.; Afzal, A.; Salac, J.; Francis, P.; Li, B.; Shepherd, D. C.; and Franklin, D

  16. [2018]

    InACM Symposium on User Interface Software and Technology (UIST)

    Rousillon: Scraping distributed hierarchical web data. InACM Symposium on User Interface Software and Technology (UIST). [Cobo et al. 2011] Cobo, L. C.; Zang, P.; Isbell Jr, C. L.; and Thomaz, A. L

  17. [2019]

    arXiv preprint arXiv:1907.03146

    A review of robot learning for manipulation: Challenges, representations, and algorithms. arXiv preprint arXiv:1907.03146. [McDermott et al. 1998] McDermott, D.; Ghallab, M.; Howe, A.; Knoblock, C.; Ram, A.; Veloso, M.; Weld, D.; and Wilkins, D

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.