{"id":"3aaedc27-e75d-45a9-a772-b4579893d38e","arxiv_id":"1908.05059","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Explainable planning can be delivered as a service wrapper around a trusted planner, which answers contrastive questions by compiling them into constrained planning problems.","lead":"This paper proposes wrapping an existing planning system in an explainable planning as a service layer that answers contrastive questions such as \"why did you do A instead of B?\" by re-running the planner under added constraints. It presents a prototype in PDDL2.1 and demonstrates it on a warehouse example.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The service's core compilation can return an HPlan in which the user-suggested action is causally irrelevant; the paper's own Fig. 7 shows this, so the central 'answer contrastive questions' claim is not yet demonstrated.","rationale":"The reader's weakest assumption correctly identifies the causal-essentiality gap, and the paper itself provides direct evidence for it in Fig. 7. My reading of the paper confirms that the central mechanism—compiling a contrastive question into an HModel and replanning—does not guarantee that the forced action appears in a causally meaningful way. The authors explicitly state that the HPlan in Fig. 7 is unsatisfactory because the user action is immediately reversed, and they list as an open question whether it is even possible to compile constraints that make the suggested action essential. This is an internal limitation, not a disagreement with outside consensus, so it is a legitimate load-bearing concern. However, the paper is framed as a prototype and roadmap with open questions, and the service architecture itself is coherent and could be viable if the compilation/refinement problem is solved. The reader's CONDITIONAL verdict is appropriate: the central claim is not falsified, but it is not fully established without a solution to the causal-essentiality problem. I therefore do not change the verdict.","tokens_in":12617,"tokens_out":5734,"duration_ms":57625,"concrete_test":"Implement the §3.2 compilation for the running example and a small set of 'why A rather than B at state S' questions (or use the authors' prototype if released). For each HPlan, run the Chrpa–McCluskey–Osborne redundancy test (cited in §3.2) on the forced action. If the forced action is redundant in a substantial fraction of cases—as it is in Fig. 7—then the service does not yet answer the question it is designed to answer. To go further, test the proposed nogood/refinement loop: impose successive 'don't use undo action' constraints and record whether an essential-B plan is found within a bounded number of iterations; a yes on the paper's example would soften the concern.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is Step 2–3 of the service: compiling a contrastive question into an HModel and replanning so the HPlan answers 'why A rather than B'. For the paper's main question type ('A used in state S rather than B'), the compilation in §3.2 replaces A with B and replans from the resulting state. The paper's own output, Fig. 7, contains the forced action (load_pallet Tom p2 sh6) but immediately undoes it with (unload_pallet Tom p2 sh6). The authors concede this 'would not be a satisfactory explanation' and say it is an open question whether a compilation can guarantee the suggested action is essential to the causal structure (§3.2). This is not a peripheral evaluation gap: it is the central mechanism failing on the running example. A plan whose alternative action is causally redundant does not support the counterfactual the user asked about; it only shows that the action can be inserted at additional cost. Because the service's value proposition is answering contrastive questions with an unmodified planner, the missing guarantee of causal relevance undercuts the central claim as stated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Explainable AI Planning as a Service: a wrapper around an existing PDDL2.1 planner that answers contrastive questions by (1) taking the original planning problem, the current plan, and a user question, (2) compiling the question into a hypothetical model (HModel), (3) invoking the existing planner to produce a hypothetical plan (HPlan), (4) validating the HPlan against the original model, and (5) presenting a comparison between the original plan and the HPlan. The authors categorize several formal question types, describe a modular prototype implementation, and walk through a warehouse running example. They explicitly discuss limitations, including the fact that their main compilation does not guarantee that the user-suggested action is essential to the causal structure of the resulting plan, and they identify several open research directions.","tokens_in":12810,"tokens_out":5008,"duration_ms":47485,"significance":"The paper addresses a timely topic and its core idea—using the user's own trusted planner and model to generate contrastive explanations—is practically appealing, especially for safety-critical domains. The framework's modularity and the use of existing validation tools (VAL) are strengths, as is the candid acknowledgment of the causal-relevance problem. However, the central demonstration fails on the running example: the HPlan in Figure 7 includes the user-suggested action but immediately reverses it, so the explanation does not answer the user's actual counterfactual question. Because the paper is explicitly positioned as early-stage work and identifies this as an open problem, the contribution is best seen as a roadmap and prototype rather than a fully validated solution. The paper does not provide formal correctness guarantees, user studies, or a released implementation, but it clearly names these as future work.","major_comments":[{"comment":"The compilation for the paper's main question type (“Why is action A used in state S, rather than action B?”) produces an HPlan in which the user-suggested action B is immediately undone by (unload_pallet Tom p2 sh6). The paper itself notes that “This would not be a satisfactory explanation to the user” and states that guaranteeing the suggested action is essential to the causal structure is an open question. Since this is the paper's central mechanism and its only fully worked example, the claim that the service answers contrastive questions is not demonstrated. The authors should either provide a compilation that ensures causal relevance for at least this question type, or substantially reframe the contribution as an iterative framework in which the initial compilation is a heuristic and the absence of a causal-relevance guarantee is a core limitation rather than a peripheral one.","section":"§3.2, Figure 7"},{"comment":"The compilation step is described informally and its correctness is not specified. The paper does not define the semantics of “state S” in a temporal plan (e.g., which happenings are included, how simultaneous effects are handled), nor does it prove that the TIL times added in the new initial state are always well-defined and non-conflicting. Moreover, the HModel is introduced as a “revised domain model,” but in the example it is a modified initial state with TILs; the relationship between constraints compiled into the domain versus the problem instance is left unclear. Without a formal statement of what the compilation preserves (e.g., executability with respect to the original model, or a precise notion of “the user's intention”), it is difficult to assess whether the approach generalizes beyond the single example or to verify the validity claims made in Steps 4–5.","section":"§3.2, HModel construction"}],"minor_comments":[{"comment":"In the definition of the timed-initial literals, the parenthetical in “where start (a j) (start (a j)) is the time at which the action a j (B) started execution” appears to be a typo; the second symbol should be start(B).","section":"§3.2, TIL formula"},{"comment":"In the “removed” list, the action “9.001: (goto_waypoint Tom sh1 sh6)” does not appear in the original plan of Figure 4; the original plan at time 9.001 is (goto_waypoint Tom sh1 sh2). This discrepancy should be corrected.","section":"Figure 9"},{"comment":"The actions in Figure 7 are written with spaces (e.g., “load pallet Tom p2 sh6”), while the domain definition uses underscores (load_pallet). The figure should be made consistent with the PDDL syntax.","section":"Figure 7"},{"comment":"The final sentence “we identify open questions for Explainable Planning as a service that identify several possible research directions” is redundant; consider rewording to “we identify open questions and several possible research directions.”","section":"Abstract"},{"comment":"The paper does not state whether the prototype framework or the running-example files will be made available. A brief availability statement or link to a repository would improve reproducibility.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is close to a workshop-style position paper. Its honest treatment of the causal-relevance problem is a strength, but the fact that the primary example fails under the core mechanism is a load-bearing issue that must be addressed before journal publication. A revision that either fixes the compilation for the central question type or explicitly narrows the paper's claims to an architecture-and-roadmap contribution would be appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Let me give you the short version first. This is a sincere, well-written prototype paper on explainable planning as a service, and the service abstraction is a genuinely useful framing. But the paper's own running example fails to demonstrate the central claim: when the planner is forced to include the user's suggested action, the resulting HPlan immediately reverses that action, so it does not answer the contrastive question. The authors know this, say it would not be a satisfactory explanation, and admit that guaranteeing causal essentiality is open. That is not a peripheral weakness; it is the load-bearing step of the approach, and it means the paper is best read as a roadmap with a prototype, not as a working solution.\n\nWhat is new: the wrapper idea—using the user's trusted planner and model to answer contrastive questions by compiling them into hypothetical PDDL2.1 problems—is clearly presented. The five formal question types and the compilation for the 'A in state S rather than B' case, with timed-initial literals to handle concurrent actions, are concrete and go beyond the cited prior work. The authors also correctly insist that any HPlan be validated against the original model, and they sketch an iterative, tree-structured refinement process. All of that is a useful foundation.\n\nWhere it is soft: first, the causal-relevance problem. The paper's Figure 7 shows the forced load_pallet immediately followed by unload_pallet of the same pallet, so the user's alternative is causally irrelevant; the explanation only shows that an action can be inserted at extra cost. The authors gesture at redundant-action detection and nogood constraints, but no algorithm or evaluation is provided. Second, the evaluation is one small warehouse example; there is no code released and no evidence the framework works with other planners or domains. Third, the formal question language is ad hoc—fine for a prototype, but the paper does not claim completeness, so this is minor.\n\nOverall: the paper is honest, non-circular, and gives credit to prior work. The central idea has merit, but the demonstrated failure on the only example means the claims need to be scoped down or the causal-relevance problem needs a real solution. This deserves a serious referee, especially for an XAI-planning venue, because the roadmap and the compilation details will be useful to the community. I would not cite it as a solution, but I might cite it as a statement of open problems.","headline":"A sincere, well-scoped prototype for explainable planning as a service, but the central example fails to demonstrate the main claim and the paper is best read as a roadmap with open problems.","tokens_in":13364,"tokens_out":2560,"would_cite":false,"duration_ms":26119,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Explainable Planning can be a service: a wrapper around an existing planner that compiles 'Why A rather than B?' into a hypothetical model, replans, validates, and returns the cost difference.","keywords":["explainable planning","contrastive explanation","planning as a service","PDDL","hypothetical model","plan validation","human-aware planning","XAI"],"falsifier":"Run the service on the paper's warehouse example with the question 'Why didn't Tom pick up the pallet?' and inspect whether the returned HPlan contains the forced load action in the causal support of a goal. If the compilation alone produces a plan where the forced action is redundant, and the only remedy is hand-adding nogood constraints or refining the question, then the promise of answering contrastive questions as a service is falsified.","tokens_in":12414,"feed_emoji":"🤖","tokens_out":5848,"duration_ms":52455,"temperature":0.7,"pith_summary":"The paper argues that Explainable Planning should be a service rather than a new planning algorithm: a wrapper around the user's existing planner and domain model. When a user asks a contrastive question, typically 'Why did you do A rather than B?', the service compiles that question into a hypothetical planning problem (an HModel), invokes the original planner to solve it, validates the resulting HPlan against the original model, and returns a comparison of the two plans plus the cost difference. The authors implement a prototype for PDDL2.1 domains, demonstrate it on a warehouse robotics example, and show it supports several question types: forbidding an action, forcing an action, forcing a replacement, ordering constraints, and time-window constraints. They are explicit that the hard open problem is guaranteeing the forced action is causally essential to the alternative plan, not merely present in it.","feed_headline":"A wrapper answers 'Why A, not B?' for any planner","feed_subtitle":"It compiles the question into a hypothetical model, replans with your own planner, and shows the cost of your alternative.","key_machinery":"The HModel and its compilation carry the argument. The service translates the formal contrastive question into a set of constraints, compiles them into a modified PDDL2.1 domain, and generates a hypothetical plan using the original planner. For the replacement question, the compilation cuts the original plan at the state where A occurs, replaces A with B, computes the new initial state using the VAL validator's happenings, adds timed-initial literals to simulate actions still executing, and lets the planner solve from that state. The redundancy post-processing, based on Fink and Yang's categories and Chrpa, McCluskey, and Osborne's detection algorithm, is the only mechanism proposed to make forced actions essential, and the paper states that guaranteeing causal essentiality is an open question.","core_discovery":"The central claim is that answering contrastive questions in planning does not require a new explainable planner; it can be layered as a service around any planner that speaks PDDL2.1. The service takes as input the original domain and problem, the plan, and a formalized question; it creates a revised domain model (HModel) that encodes the user's constraint, solves it with the same planner, validates the hypothetical plan against the original model, and forms a contrastive explanation as a tuple of existing, removed, added actions and cost difference. The paper demonstrates this on a warehouse delivery domain, where asking 'why didn't Tom pick up the pallet?' compiles into an HModel that forces the load action, and the planner returns an alternative plan with a small cost penalty. The authors stress that the explanation is only trustworthy because it is produced by the user's own planner and validated against the original model. They also identify a key limitation: forcing an action into a plan is not the same as making it causally necessary for the goals, and the paper shows a generated HPlan where the forced load is immediately undone.","pith_inferences":["The service view could be extended beyond PDDL: any planner that can accept constraint-compiled models could be wrapped, suggesting a standard protocol for 'explanation-by-replanning' across planning systems.","The redundancy problem might be attacked by compiling causal-support constraints directly, such as requiring that B supports a particular goal; the paper notes no current planning language expresses this cleanly.","The cost-difference metric in the explanation assumes plan cost is the right comparison axis; in safety-critical domains, risk or robustness differences might be more meaningful and could be added to the contrast tuple.","The iterative refinement tree suggests an automated explanation search driven by user feedback, turning a one-shot answer into a dialogue where the service proposes and the user critiques."],"forward_implications":["Users get explanations built from their own trusted planner and model, which matters in safety-critical settings such as oil-well drilling.","The service supports several contrastive question types: forbid, force, replace, order, and time-window constraints, plus an iterative tree of follow-up questions.","Because the framework is modular and PDDL2.1-based, it works with any conforming planner and can be integrated into robotics systems like ROSPlan.","The hypothetical plan is validated against the original model, so inconsistencies introduced by model revision are caught before the explanation is shown.","Forcing an action does not guarantee causal essentiality; the paper proposes redundancy detection plus nogood learning, but automating that refinement remains open."],"supporting_citations":[{"why":"Defines the contrastive-question framing ('Why A rather than B?') and the Figure 1 scenario that the service operationalizes.","marker":"Fox, Long, and Magazzeni 2017"},{"why":"Proposes planning as an iterative process, which the service adopts to let users refine questions and HModels.","marker":"Smith 2012"},{"why":"Supplies VAL, used both to compute the state at the replacement point and to validate HPlans against the original model.","marker":"Howey, Long, and Fox 2004"},{"why":"Provides the formal categories of redundant actions used in the post-processing step to check whether a forced action is essential.","marker":"Fink and Yang 1992"},{"why":"Gives the algorithm for detecting redundant actions in sequential plans, the basis for the essentiality check and nogood proposal.","marker":"Chrpa, Mccluskey, and Osborne 2012"},{"why":"The POPF planner used to generate the running-example plans, demonstrating the service with a real PDDL2.1 temporal planner.","marker":"Coles et al. 2010"}],"fun_headline_variants":["Wrap any planner to answer 'Why A, not B?'","Explainable planning as a service, not a new system","Contrastive questions: let your own planner explain","Turn your PDDL planner into an explainer with a wrapper","Ask why not B? Your planner answers via a service"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The user's contrastive question can be faithfully compiled into planning constraints that make the suggested action causally essential to the alternative plan, not merely present in it; the paper's own example (Figure 7) shows a forced action being immediately reversed and states that guaranteeing essentiality is an open question.","fun_headline_variants_meta":{"raw":{"variants":["Wrap any planner to answer 'Why A, not B?'","Explainable planning as a service, not a new system","Contrastive questions: let your own planner explain","Turn your PDDL planner into an explainer with a wrapper","Ask why not B? Your planner answers via a service"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000499,"raw_usage":{"total_tokens":2399,"prompt_tokens":859,"completion_tokens":1540,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":475,"completion_tokens_details":{"reasoning_tokens":1457}},"tokens_in":475,"tokens_out":1540,"duration_ms":14261,"temperature":1.0,"reasoning_tokens":1457,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:24:07.136769+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the service on the paper's warehouse example with the question 'Why didn't Tom pick up the pallet?' and inspect whether the returned HPlan contains the forced load action in the causal support of a goal. If the compilation alone produces a plan where the forced action is redundant, and the only remedy is hand-adding nogood constraints or refining the question, then the promise of answering contrastive questions as a service is falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Proposes planning as an iterative process, which the service adopts to let users refine questions and HModels."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies VAL, used both to compute the state at the replacement point and to validate HPlans against the original model."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the formal categories of redundant actions used in the post-processing step to check whether a forced action is essential."},{"cited_title":"L.; and Osborne , H","cited_arxiv_id":null,"evidence_quote":"Gives the algorithm for detecting redundant actions in sequential plans, the basis for the essentiality check and nogood proposal."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The POPF planner used to generate the running-example plans, demonstrating the service with a real PDDL2.1 temporal planner."}],"review_version":1}