{"id":"24239ae3-4082-4abe-afb0-4f96335fbd8d","arxiv_id":"2608.06065","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Gated Hindsight Distillation uses the next screenshot from a successful trajectory as privileged teacher information that corrects a student GUI agent's failed rollouts, improving Android task success over GRPO baselines.","lead":"The paper trains a phone-screen agent to use the next screenshot of a task as a special hint during training, then removes the hint at deployment. This hint lets a teacher model correct the student's wrong next actions, and the method reports higher task success than standard imitation or reinforcement learning on two Android benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 5 gives the teacher the student's sampled prefix y<j; if that prefix already states the action, the gate and distillation pass without the next screenshot. No ablation removes y<j, so the central 'future screenshot' mechanism is not isolated.","rationale":"The reader's weakest assumption correctly identifies the confounding of the teacher's privileged context with the student's own rollout prefix; my read agrees. The paper deserves credit for several strong controls: Table 2 separates gating and dynamic sampling, Table 4 varies the privileged fields while holding the pipeline fixed, and Table 5 shows the core benefit is not just extra rollouts. These controls make the empirical recipe credible. However, none of them strips y<j from the teacher, and Table 2's +Gate result demonstrates that y<j alone carries a measurable distillation signal (+0.71). Because Eq. 5, Eq. 8, and Eq. 9 all pass through the same y<j, the gate can approve a correction that the teacher obtains from the student's own reasoning rather than from ot+1. This does not invalidate the reported gains, but it does mean the paper's central mechanistic attribution—'the next screenshot knows'—is not proven. The requested masking/teacher-decoding ablation would settle it. Since the reader already conditioned acceptance on releasing code and addressing this issue, my concern does not move the verdict; I would keep CONDITIONAL.","tokens_in":12472,"tokens_out":9386,"duration_ms":101768,"concrete_test":"Run the 7B GHD pipeline with Eq. 5's teacher context changed to (xt, ot+1, mask(y<j)), where mask(·) replaces every student-generated reasoning token with a fixed neutral token while keeping the action span intact; keep gating (Eq. 9), dynamic sampling, λ=0.1, K=100, and 200 optimization steps otherwise unchanged. Compare AndroidWorld Pass@1 against the Table 2 '+DS' control (49.56) and full GHD (52.73). If the masked variant does not significantly exceed '+DS', the reported gain flows through y<j rather than the next screenshot; if it matches full GHD, the future-screenshot mechanism is confirmed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"GHD's central claim is that the next screenshot supplies the privileged correction. For that to be established, the teacher's advantage must come from ot+1, not from the student's own rollout. But in Eq. 5 the teacher distribution is π_T = sg[πθ(·|xt, ot+1, y<j)]: it is conditioned on the same sampled prefix y<j as the student. Since y<j includes the student's generated reasoning tokens, those tokens can already state or imply the demonstrated action (e.g., 'to enable USB Debugging, tap System'); the gate in Eq. 9 can then pass and Eq. 7 can distill the student's own correct reasoning even if ot+1 is ignored. Table 2's +Gate control shows this is not hypothetical: an unprivileged teacher with the same y<j adds +0.71 Pass@1 over GRPO. Table 4 varies the privileged fields (a*, r*, ot+1) but never removes y<j, so it cannot isolate the contribution of the future screenshot. The empirical recipe may still work, but the paper's mechanistic headline—that the next screenshot is what knows—is not yet supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Gated Hindsight Distillation (GHD), a training-time method for mobile GUI agents. A student policy generates a reasoning-and-action response from the observable interaction prefix, while a parameter-sharing teacher additionally sees the next screenshot from the successful demonstration trajectory and re-scores the student's sampled tokens. Distillation is gated to rollouts where the prefix-only student fails and the teacher's position-wise top-1 decoding recovers the demonstrated action, and the objective is combined with GRPO. Experiments on AndroidWorld and AndroidLab with Qwen2.5-VL-7B and Qwen3-VL-8B report consistent Pass@1 gains over SFT and GRPO, with ablations for gating, dynamic sampling, and the privileged next-screenshot observation.","tokens_in":12729,"tokens_out":15089,"duration_ms":141484,"significance":"If the reported results hold, GHD is a practical and simple way to exploit future observations that are already present in offline successful trajectories, addressing a genuine supervision gap in GUI-agent training without adding any inference-time component. The paper's strengths are its controlled comparisons with means and standard deviations over three runs, a clean ablation structure that separates gating, dynamic sampling, and the privileged next screenshot, and direct comparisons against alternative uses of future states such as STaR-style target generation and a GUI-Shift-style inverse-dynamics auxiliary task. The method is described in enough detail to be reproducible in principle, and the code/checkpoint availability statement is a positive commitment, though it should be more concrete.","major_comments":[{"comment":"Please clarify the training data for the SFT baseline. The 'GHD Implementation Details' paragraph states that all experiments use the same SFT checkpoint, training split, rollout configuration, and environment for SFT, GRPO, and GHD, but the following sentence says the training data is a hard subset of the OpenMobile trajectories obtained by filtering examples that the corresponding SFT model solves in one attempt. If the SFT row in Table 3 is the checkpoint trained on the full 27,360 examples while GRPO and GHD are trained on the 6,968 (or 5,982) hard examples, then the 'Controlled Comparison' section's claim of 'the same data' is not valid for the SFT row. If the SFT baseline was retrained on the hard subset, state that explicitly. This is needed to interpret the SFT-to-GRPO and SFT-to-GHD comparisons in Table 3.","section":"Experiments / GHD Implementation Details"},{"comment":"The component-wise ablation in Table 2 is reported as single-run numbers. The text claims that the next screenshot provides the largest incremental gain (+3.17 over +DS) and that 'the majority of the improvement comes from transferring future-grounded token-level supervision,' but no variance is reported for these ablation conditions. Given that Table 3 shows run-to-run standard deviations of 0.65-1.51 on the same benchmark, the 0.71, 2.43, and 3.17 point differences could be within noise. Please report means and standard deviations over at least three runs for the ablation conditions, or explicitly temper the mechanistic claim.","section":"Ablation Studies (Table 2)"}],"minor_comments":[{"comment":"Equation (5) conditions the teacher on the student's sampled prefix y<j; the paper should state explicitly that the +DS row in Table 2 is the control for this choice, because it keeps y<j while removing only ot+1. Such a statement would preempt the alternative explanation that the student's own reasoning tokens drive the correction signal.","section":"Method, Eq. (5); Ablation Studies (Table 2)"},{"comment":"In Table 4, the 'Ours' row (only ot+1) outperforms the 'Full' row (a*, r*, ot+1) by 1.80 points; please discuss why adding the ground-truth action and reasoning hurts performance.","section":"Privileged Information and Transfer Mechanism (Table 4)"},{"comment":"In the sentence 'GHD without dynamic sampling still improves over GRPO by by 2.29 points on AndroidWorld,' the word 'by' is duplicated.","section":"Efficiency and Sampling Overhead"},{"comment":"Table 7 reports a single representative run; please indicate whether the per-application numbers are averaged over the three runs and, if not, state that the breakdown is illustrative.","section":"Experiments, Table 7"},{"comment":"The abstract and experiments state that code and checkpoints 'will be made available'; for reproducibility, please provide a link or a concrete release timeline in the submission.","section":"Abstract and Code Availability"},{"comment":"Please describe the hard-subset filtering step in more detail: which model performed the filtering, how 'solves in one attempt' is determined with the step verifier, and whether the SFT baseline in Table 3 was trained before or after filtering.","section":"Experiments / GHD Implementation Details"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical contribution and the headline result (GHD > GRPO) is well supported by three-run comparisons. The main risks are the ambiguity in the SFT baseline's training data and the lack of variance in the key ablation table. The y<j concern raised in the reader's stress-test is largely addressed by the +DS control, which keeps y<j while removing ot+1, so I do not think an additional y<j-removal ablation is strictly necessary for acceptance; an explicit sentence would suffice."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short take: GHD is a real, reproducible-looking gain for offline GUI-agent training, and the experiments are better controlled than most papers in this area. The main caveat is that the teacher conditions on the student's own rollout prefix, so the 'next screenshot knows' framing is stronger than the evidence.\n\nWhat's good: The method combines existing pieces—future-conditioned hindsight (as in GUI-Shift, but used only at training), on-policy self-distillation (SDPO), and a gating rule that only distills when the student fails and the teacher verifiably recovers the demonstrated action. The gating and dynamic sampling are simple and clearly ablated. Tables 2 and 3 are the core: component-wise ablation on AndroidWorld with 7B, and controlled comparison with means/stds over three runs on both benchmarks and two model families. The gains hold up: 5-11 points over GRPO. Table 4 shows that adding the next screenshot gives a larger jump than adding the reference action or reasoning, which supports the idea that the future observation carries useful signal. The comparison with a GUI-Shift-style auxiliary task is also fair and shows GHD's more direct supervision works better.\n\nSoft spots: The main one is the teacher's conditioning on y<j. Equation 5 gives the teacher the student's sampled prefix, including reasoning tokens. If those already state or imply the correct action, the gate can pass and distillation can reinforce the student's own output without the screenshot providing new information. The +Gate control in Table 2 partially addresses this: an unprivileged teacher with the same y<j adds only 0.71 points, so the screenshot adds about 3 points on top. That means the mechanism is real but not fully isolated. An ablation that removes y<j from the teacher's input would be the natural fix, and its absence is a genuine gap. Also, code and checkpoints are promised but not released, and the Table 1 comparison mixes many training setups (they acknowledge this, so it's minor).\n\nWho this is for: anyone working on offline training of GUI agents, especially RL/distillation pipelines. It deserves a serious referee. The right request would be: add the y<j ablation, release the code, and soften the title's mechanistic claim.","headline":"Solid empirical paper with a real, consistent gain for GUI-agent training, but the 'next screenshot knows' mechanism is partially confounded by the teacher's access to the student's own reasoning tokens.","tokens_in":13257,"tokens_out":3876,"would_cite":true,"duration_ms":35855,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A training-time teacher that sees the next screenshot improves mobile GUI agent task success.","keywords":["GUI agents","mobile agents","hindsight distillation","privileged information","reinforcement learning","GRPO","vision-language model","offline training"],"falsifier":"Replace the next screenshot with a randomly chosen screenshot from a different task while keeping the gate and distillation unchanged; if Pass@1 does not fall, the specific content of the next screenshot is not providing the corrective signal, falsifying the claimed hindsight mechanism.","tokens_in":12280,"feed_emoji":"📱","tokens_out":10307,"duration_ms":81793,"temperature":0.7,"pith_summary":"Offline training of GUI agents normally throws away the next screenshot after each demonstrated action, even though that screenshot often contains the only evidence that justifies the action. Gated Hindsight Distillation (GHD) instead treats the next screenshot as privileged training information: a parameter-sharing teacher sees it while re-scoring the student's rollouts, and the corrected token-level distribution is distilled into the student only when the student fails and the teacher's recovery matches the demonstrated action. The paper claims this future-grounded supervision improves task success over SFT and GRPO on AndroidWorld and AndroidLab with both Qwen2.5-VL-7B and Qwen3-VL-8B. If true, GHD offers a training-only way to inject application-specific GUI knowledge into a prefix-only policy, with no extra inference-time module.","feed_headline":"Next-screenshot training lifts mobile GUI agent task success","feed_subtitle":"Distills what the next screen reveals into a prefix-only agent, with zero extra runtime cost.","key_machinery":"The load-bearing mechanism is Gated Hindsight Distillation: a joint objective $L = L_{\\text{GRPO}} + \\lambda L_{\\text{GHD}}$ with $\\lambda = 0.1$, where $L_{\\text{GHD}}$ is a generalized Jensen–Shannon divergence ($\\alpha = 0.5$) between the stop-gradient teacher distribution $\\pi_T(\\cdot \\mid \\tilde{x}_t, y_{<j})$ and the trainable student distribution $\\pi_S(\\cdot \\mid x_t, y_{<j})$, evaluated on the student's top-$K = 100$ tokens plus a residual bucket. The gate $M(y)$ keeps a response only when the step reward is below $\\tau_{\\text{succ}} = 1.45$ (student fails) and the teacher's position-wise top-1 reconstruction matches the demonstrated action within tolerance $\\delta = 20$ for coordinates or exact/edit similarity for text. Dynamic sampling draws up to three rollout-group attempts per prompt to increase the density of gate-passing responses. Because the teacher shares parameters with the student and differs only in receiving the next observation, the entire method adds no inference-time module.","core_discovery":"The paper's central claim is that the next screenshot, observed only during training, converts a hard prediction problem ('what should I do?') into an easier inference problem ('what must I have done?') and that the resulting rationale can be distilled into a prefix-only student. GHD implements this with a parameter-sharing teacher that conditions on the student's own rollout tokens plus the realized next observation, computes teacher-forced token distributions, and applies a gated distillation loss that keeps only the cases where the prefix-only student fails and the teacher's position-wise top-1 action matches the demonstration. The reported results on AndroidWorld and AndroidLab show consistent Pass@1 gains over SFT and GRPO across two vision-language model scales, with the biggest improvements on actions that require application-specific navigation knowledge or implicit prerequisites.","pith_inferences":["A testable extension would isolate the contribution of the future screenshot by removing the student's own reasoning tokens ($y_{<j}$) from the teacher's input; if the gate passes purely on those tokens, the reported gains could be due to self-hints rather than hindsight.","The same gated hindsight scheme could be applied to other sequential decision tasks—web navigation, form filling, robotics—where a verifiable step reward and demonstration trajectories are available and later observations reveal action consequences.","A soft gate that weights distillation by the teacher's confidence or the margin of correction might capture more signal than the hard threshold at $\\tau_{\\text{succ}} = 1.45$, and could reduce the need for the 2.69-attempts-per-batch dynamic sampling overhead.","The per-application breakdown (gains on seven of nine AndroidLab apps, none on Calendar and Zoom) suggests the method's benefit is concentrated in transition types that the gate selects, so a future analysis could predict where hindsight distillation will help from the gate's acceptance statistics."],"forward_implications":["GHD improves task success (Pass@1) over both SFT and GRPO on AndroidWorld and AndroidLab for Qwen2.5-VL-7B and Qwen3-VL-8B, and the gains are largest on steps that require application-specific navigation knowledge or implicit prerequisites.","The step-wise gains come mainly from the future observation itself: ablations show it contributes the largest increment over gating and dynamic sampling, and it outperforms providing the ground-truth action or reasoning alone.","Continuous token-level distribution matching transfers privileged knowledge better than STaR-style off-policy rationalization for every privileged signal tested.","Compared with a GUI-Shift-style inverse-dynamics auxiliary task, GHD's direct future-grounded distillation yields a much larger improvement, indicating that the transfer mechanism matters, not just the future state.","At deployment GHD removes the teacher, the future observation, and the gating procedure, so the prefix-only student runs with no additional inference cost."],"supporting_citations":[{"why":"Defines GRPO, the reinforcement-learning baseline and the RL term in the joint objective.","marker":"Guo et al. 2025"},{"why":"Supplies the AndroidWorld benchmark used for evaluation.","marker":"Rawles et al. 2025"},{"why":"Supplies the AndroidLab benchmark used for evaluation.","marker":"Xu et al. 2025"},{"why":"Provides the SDPO-style generalized Jensen–Shannon divergence used as the distillation objective.","marker":"Hübotter et al. 2026"},{"why":"Provides the OpenMobile training data and SFT recipe from which GHD starts.","marker":"Cheng et al. 2026"},{"why":"Describes the GUI-Shift inverse-dynamics auxiliary task used as a controlled comparison.","marker":"Gao et al. 2026"},{"why":"Describes STaR, the off-policy rationalization transfer mechanism compared in the experiments.","marker":"Zelikman et al. 2022"},{"why":"Provides the Qwen3-VL-8B base model used in the 8B experiments.","marker":"Bai et al. 2025a"},{"why":"Provides the Qwen2.5-VL-7B base model used in the 7B experiments.","marker":"Bai et al. 2025b"}],"fun_headline_variants":[],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's claimed mechanism—that the future screenshot is the source of the correction—is not isolated from the student's own generated reasoning tokens, which the teacher also sees and which could already contain the demonstrated action.","fun_headline_variants_meta":{"error":"Client error '402 Payment Required' for url 'https://api.deepseek.com/chat/completions'\nFor more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/402"},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:36:03.943412+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replace the next screenshot with a randomly chosen screenshot from a different task while keeping the gate and distillation unchanged; if Pass@1 does not fall, the specific content of the next screenshot is not providing the corrective signal, falsifying the claimed hindsight mechanism.","supporting_citations":[],"review_version":1}