{"id":"44256b9c-5cae-4c3e-989a-4948747eec67","arxiv_id":"2607.15193","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"An expert repairing visible plans rescued 23 of 26 failed GUI automation runs, turning 17 into full and 6 into partial successes.","lead":"Plover is a system that shows GUI automation agents a visible, editable plan while they work, so people can catch and fix mistakes without restarting a task. In tests on previously failed computer tasks, an expert user turned 17 of 26 failures into full successes and 6 into partial ones using an average of two small interventions.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Oracle expert interventions confound the causal claim that plan-centric visibility drives recoverability; without a chat-only control or non-expert users, the 88% recovery may reflect user skill rather than the interface.","rationale":"The reader's weakest assumption is that the oracle-like expert user invalidates the transferability of the recoverability claim. I agree: the first author's knowledge of failure causes and correct targets is a critical confound. My concern sharpens this by emphasizing that the missing control condition prevents even a causal claim about the plan-centric mechanism, not just about typical-user performance. The paper explicitly labels the result an upper bound, so the numerical result itself is not misleading, but the abstract's conclusion that explicit replanning 'helps' overreaches. A controlled user study or an expert ablation with a chat-only condition would resolve whether the interface or the user is responsible. The verdict should remain CONDITIONAL because the upper-bound feasibility result is still valuable; it needs additional evidence to support the stronger design claims. I agree with the reader's assessment and do not propose a different verdict.","tokens_in":29123,"tokens_out":11317,"duration_ms":93508,"concrete_test":"Conduct a between-subjects experiment with non-expert participants (N≈16–20) on a random subset of the 26 failure cases, randomly assigned to either full Plover or a chat-only baseline (same underlying agent but no plan panel, no plan editing, no multimodal annotation; only natural-language corrections). Measure the proportion of tasks reaching success/partial success and the number of interventions required. If Plover's recovery rate significantly exceeds the chat-only baseline, the plan-centric interface is the active ingredient; if recovery is comparable (e.g., within the same 80–100% range), the 88% figure is likely an artifact of oracle-level user knowledge rather than plan visibility.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim (Section 5.1, Table 1) is that mixed-initiative interaction improved 23/26 autonomous non-success cases, and the abstract concludes that 'many GUI-agent failures are structurally repairable when plans remain visible and interventions are localized.' The load-bearing issue is causal attribution: all interventions were supplied by the first author, who knows each failure's cause and the correct target. The paper labels this an upper bound on plan-centric recoverability, which is honest, but the conclusion that explicit replanning and plan visibility 'help' requires a comparison that is absent. There is no condition that strips away the plan-centric features (plan panel, plan editing, annotation) while keeping the same underlying agent and the same expert. Without such an ablation or a non-expert user study, the 88% recovery could be driven entirely by the expert's ability to issue precise natural-language corrections, with the visible plan and localized repair mechanisms adding little. The paper's own limitation admits 'our design assumes users generally know the intended path well enough to correct the agent when needed' (Section 6), but the abstract and conclusion still overstate the causal role of the interface. Thus, the recoverability result is a valid upper bound, but the design conclusions (DG1–DG5, 'explicit replanning helps') are not established by this experiment.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Plover, a plan-centric GUI automation system that externalizes task plans as persistent, editable artifacts and supports mixed-initiative repair through natural-language guidance, multimodal annotation, plan edits, and system-driven replanning. The authors report a formative study with six participants, a benchmark repair study on 38 OSWorld-Verified tasks (26 autonomous non-successes re-run with expert interventions, yielding 23 improved, 17 complete successes, 6 partial successes, 3 failures), and a scenario-based stability analysis with trajectory-derived prompts. The central claim is that many GUI-agent failures are structurally repairable when plans remain visible and interventions are localized, and that explicit replanning improves transparency, controllability, and adaptability.","tokens_in":29473,"tokens_out":4593,"duration_ms":42594,"significance":"If the central claim were fully established, the paper would make a useful contribution to human-agent interaction for GUI automation: it proposes a concrete design space (persistent plans as coordination artifacts, localized repair, visible replanning) and provides a failure taxonomy that could guide future interface design. The paper is honest in labeling the benchmark repair study as an upper bound on plan-centric recoverability, and the appendices contain substantial implementation and evaluation detail, including per-task results and a formative study summary. These are real strengths. However, the main empirical evidence does not currently separate the effect of the plan-centric interface from the effect of an expert oracle user, so the causal design conclusions (DG1–DG5, 'explicit replanning helps') are not yet established by the data.","major_comments":[{"comment":"The load-bearing empirical claim—23/26 non-success cases improved, 88% recovery—is measured with the first author, who knows each failure cause and the correct target, supplying all interventions. There is no control condition that strips away the plan-centric affordances (plan panel, plan editing, annotation, visible replanning) while keeping the same underlying agent and the same expert. As written, the 88% figure is an upper bound on recoverability by an informed oracle, not evidence that plan visibility and localized repair cause the improvement. The paper's own limitation statement in Section 6 ('our design assumes users generally know the intended path well enough to correct the agent') does not carry through to the abstract/conclusion, which assert a causal role for visible plans. Add an ablation or control (e.g., same expert corrections issued as text prompts to the same base age","section":"Section 5.1, Table 1; Abstract"},{"comment":"The scenario stability analysis synthesizes task prompts from sampled interaction trajectories and then compares the replayed plans and final states against the same trajectories. This creates a circularity: the prompt is derived from the reference trajectory, so plan alignment metrics (coverage 0.62, order 0.41, actionability 0.97) partly measure reconstruction from a trajectory-derived instruction rather than general plan quality. The browser-vs-desktop visual fidelity differences are still informative, but the plan alignment results should be presented as a property of this reverse-synthesis setup, not as evidence about Plover's planning under independently authored user instructions. A sanity check with manually authored prompts, or a clear caveat, is needed before these numbers are used to support the 'opportunity for users to inspect and correct' argument.","section":"Section 5.2, 'Trajectory Sampling and Prompt Reconstruction' and Table 2"},{"comment":"The failure-mode analysis is based on the same 26 cases and the same expert interventions. Counts such as 'Execution Drift appeared in 46% (n=12/26)' and 'NL Guidance resolved 11 cases' are reported without any uncertainty or sensitivity analysis. With n=26 and intervention choices made by a single expert who already knows the failure causes, small counts can easily flip; the recovered vs. unrecovered distinction is not robust enough to support the strong claim that 'compound failures' are fundamentally harder. At minimum, report bootstrap or exact binomial confidence intervals and clarify that all recovery counts are conditional on the expert's choice of intervention.","section":"Section 5.3, 'Characterizing Repairable Failures'"}],"minor_comments":[{"comment":"System-Driven IR relies on hardcoded thresholds (REPEAT_SEQ_L3_R3, dHash Hamming distance > 40) with no sensitivity analysis. Since Section 5.3 attributes 9 successful recoveries to System-Driven IR, the threshold choices can materially affect the results; report how varying them changes detection and downstream recovery.","section":"Appendix C, Algorithm 1"},{"comment":"The statement 'no regressions were observed' only covers the 26 autonomous non-success cases; it does not address whether the mixed-initiative interaction could degrade autonomous successes, since those were not re-run in the mixed-initiative condition. Please state this scope explicitly.","section":"Section 5.1, 'no regressions'"},{"comment":"Table 1's 'Improv. Rate' is not formally defined; for Multi-App, 6S+2P out of 10 corresponds to 80%, but the reader must infer the denominator. In Table 2, the 'Overall Average' row for MSE (939.57) is the mean of scenario averages, not the mean over all trials; clarify the aggregation.","section":"Table 1 and Table 2"},{"comment":"The paper uses 'Conference’17' in the ACM reference format and several placeholder-style citations (e.g., the DOI is 'XXXXXXX.XXXXXXX'). Please update the formatting to the final venue style and correct minor typographical issues such as the 'MI (a)' label in Figure 5.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper is likely acceptable after either (a) adding a control condition that separates expert skill from plan-centric affordances, or (b) re-framing the abstract and conclusions to claim only an upper bound on recoverability, not that visible plans and localized interventions are the cause. The scenario analysis should also be reframed to acknowledge the trajectory-derived prompt circularity. The expert-only design is not fatal—the upper-bound framing is honest—but the current abstract and design-goal claims go beyond the evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nRead the Plover paper. Quick take: the system is real, the repair characterization is genuinely new, and the authors are upfront that the expert-repair numbers are an upper bound. But the paper's own abstract draws a causal conclusion the experiment can't support. The headline 88% recovery (23/26) tells you failures are repairable by someone who knows the answer; it does not tell you visible plans are why.\n\nWhat's actually new: integrating plan externalization, multimodal annotation, and visible replanning into one repair loop for vision-based GUI agents, plus the empirical failure taxonomy (execution drift, perception errors, state misinterpretation, compound failures). The compound-failure boundary is a useful finding. The planner-executor separation with the completed-steps invariant is clean, and the design goals are grounded in a real formative study. The scenario-based stability analysis adds a complementary view, showing desktop workflows are far less stable than browser ones.\n\nThe soft spots: Section 5.1 is the load-bearing evidence, and it has no control condition. The interventions come from the first author, who knows the failure and the target, and there's no condition that strips out plan-centric features. So the result is a valid upper bound on recoverability but doesn't establish that explicit replanning \"helps\" — the abstract and conclusion overstate this. The paper admits the oracle assumption in the limitations, which is good, but the framing elsewhere doesn't match that caution. The scenario analysis synthesizes prompts from trajectories, which is fine for generating realistic tasks but can't support claims about agent stability beyond that setup. Minor: no code/data, small N, no confidence intervals.\n\nOverall the central system description and the recovery taxonomy are worth referee time. A serious revision should tone down the causal claims, add a chat-only or plan-invisible control, and ideally run a small non-expert study — even five to eight users would tell you whether ordinary people can detect drift and apply localized corrections. As it stands, I'd read it as a promising feasibility result, not a demonstration that plan visibility is the active ingredient.\n\nI'd send it to review. It deserves a careful referee, mainly to push on the control condition. Good candidate for a reading-group discussion about oracle evaluations in human-AI interaction.","headline":"Plover is a credible systems paper with an honest upper-bound recovery result; just don't let the abstract sell the 88% as proof that plan visibility is what rescues failures.","tokens_in":29942,"tokens_out":1641,"would_cite":true,"duration_ms":15663,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Plover shows that many GUI-agent failures become repairable when the task plan stays visible and corrections stay localized, with 23 of 26 benchmark failures improved by mixed-initiative interaction.","keywords":["GUI agents","mixed-initiative systems","human-AI interaction","plan-centric interaction","intelligent replanning","multimodal annotation","failure recovery","GUI automation"],"falsifier":"Run the same 26 benchmark failures with naive participants who are not told the failure cause, providing only the Plover interface as the intervention channel. If the mixed-initiative recovery rate falls to near the autonomous baseline (no significant improvement over the 0% success on these originally failed tasks), the claim that failures are structurally repairable through visible plans is not supported for realistic users.","tokens_in":29057,"feed_emoji":"🖱️","tokens_out":2526,"duration_ms":24155,"temperature":0.7,"pith_summary":"The paper introduces Plover, a plan-centric GUI automation system that externalizes task plans as persistent, inspectable, and revisable artifacts. Its central claim is that many autonomous GUI-agent failures are not terminal errors but localized breakdowns in grounding, state interpretation, or execution continuity that become structurally recoverable when users can see the plan, intervene surgically, and preserve completed work. In a repair study over 26 autonomous non-success cases from the OSWorld-Verified benchmark, mixed-initiative interaction with an expert user improved 23 cases, converting 17 to complete success and 6 to partial success, with an average of 2.04 interventions per task and no regressions. The paper argues that reliable GUI automation should be treated as an interaction problem as much as a modeling problem: making replanning visible and localized turns silent drift into a collaborative, repairable process.","feed_headline":"88% of GUI-agent failures repaired with visible plans","feed_subtitle":"Plan-centric interaction with editable plans and screenshot annotations turns 17 of 26 autonomous failures into full successes.","key_machinery":"The central mechanism is the persistent plan artifact, a shared state representation with the invariant that executed steps are immutable and only the pending suffix can be revised (C_{t+1}=C_t). On top of this, Plover implements Intelligent Replanning in two modes: User-Driven IR, where natural-language guidance or multimodal annotations (strokes, shapes, text overlays captured as primitives with a bounding box) generate localized plan proposals, and System-Driven IR, where a watchdog detects behavioral loop repetition and visual non-progress (via dHash Hamming distance) and injects a structured failure message that prompts the model to propose a recovery step with rationale. The versioned","core_discovery":"Plover's core claim is that GUI-agent failures are structurally recoverable when plans are externalized and repairs are localized. The system keeps a versioned plan artifact, separates immutable executed steps from an editable pending suffix, and supports user-driven interventions (natural-language guidance, plan edits, and screenshot annotations) plus system-driven replanning triggered by non-progress detection. In the benchmark repair analysis, 26 autonomous failures were re-run in a mixed-initiative setting; 23 improved (17 complete successes, 6 partial), only 3 remained failures, and all 10 autonomous partial successes became complete successes. The paper also characterizes which failure","pith_inferences":["A natural next test, which the paper does not run, is a study with non-expert users who are not told the failure cause: if recovery rates drop to near the autonomous baseline, the 'structurally recoverable' claim would need to be re-scoped from an upper bound to a property that depends on user diagnostic skill.","The plan artifact as a coordination protocol could generalize beyond GUI automation to other long-horizon agent domains (e.g., data-cleaning pipelines or robotics task plans) where partial progress is valuable and corrections must be localized.","The System-Driven IR watchdog (behavioral repetition + perceptual-hash stability) is a concrete, model-agnostic component that could be extracted and benchmarked on its own to measure how many agent stalls it catches before a human would notice.","The paper itself flags that visible plans may inflate user confidence; an empirical study measuring whether users over-accept plan proposals when the system looks confident would be a direct extension."],"forward_implications":["If recoverability holds beyond the expert setting, GUI agents can be deployed in long-horizon, high-friction workflows with a human steering loop instead of requiring near-perfect autonomy.","The plan-invariant design means corrections preserve executed history, so each intervention is cheaper and less disruptive than re-prompting or restarting the whole task.","System-driven non-progress detection (repeated semantic actions plus visual stability) can act as a reusable watchdog that catches drift before it propagates, independent of the specific planner or executor.","The failure taxonomy suggests that perception errors and state misinterpretations are cheaply repairable with language or annotations, while compound failures require catching the initial planning error earlier.","Exposing plans as versioned, diffable artifacts provides a natural audit trail for when and why an agent's behavior changed, which can support verification and post-hoc analysis."],"fun_headline_variants":["Visible plans repair 23 of 26 GUI-agent failures","Editable plans let 17 of 26 failed GUI agents fully succeed","With Plover, 23 of 26 GUI-agent failures improve","Plan-centric interaction rescues GUI agents: 17 full recoveries","Plover's visible plans turn 17 GUI failures into full successes"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The 88% recovery figure depends on an expert user who already knows what went wrong and what the correct target is; if ordinary users cannot detect drift or formulate correct localized corrections, the recoverability claim may not transfer to practice.","fun_headline_variants_meta":{"raw":{"variants":["Visible plans repair 23 of 26 GUI-agent failures","Editable plans let 17 of 26 failed GUI agents fully succeed","With Plover, 23 of 26 GUI-agent failures improve","Plan-centric interaction rescues GUI agents: 17 full recoveries","Plover's visible plans turn 17 GUI failures into full successes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001139,"raw_usage":{"total_tokens":4544,"prompt_tokens":700,"completion_tokens":3844,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":444,"completion_tokens_details":{"reasoning_tokens":3754}},"tokens_in":444,"tokens_out":3844,"duration_ms":21607,"temperature":1.0,"reasoning_tokens":3754,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T23:52:27.268038+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same 26 benchmark failures with naive participants who are not told the failure cause, providing only the Plover interface as the intervention channel. If the mixed-initiative recovery rate falls to near the autonomous baseline (no significant improvement over the 0% success on these originally failed tasks), the claim that failures are structurally repairable through visible plans is not supported for realistic users.","supporting_citations":[],"review_version":1}