{"id":"2611e3b1-e2fe-4fce-81d4-fd64295d6efe","arxiv_id":"2608.11588","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"CoAdapt-GUI jointly adapts a structured workflow context and a LoRA policy adapter from target-app rollouts, reaching 45.0% and 52.9% success on two unseen-app benchmarks.","lead":"This paper introduces CoAdapt-GUI, a method that lets a mobile-app AI agent adapt both its workflow knowledge and its policy from its own attempts in a new app, without needing demonstrations. It reports higher success rates on held-out tasks in unseen apps, which matters for deploying phone assistants to apps they have never seen.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline Setting-1 gain rests on an unreproduced cross-paper Policy-Only TTA baseline (Section 4.2, Appendix B.3); if that baseline shifts under the authors' own harness, the abstract's 45.0-vs-37.5 claim and the joint-adaptation conclusion lose their primary quantitative support.","rationale":"The reader's weakest_assumption focuses on reflector reliability and leakage control. That concern is legitimate, but the paper has internal controls that partially mitigate it: the Category-Novel group in AndroidWorld Plus starts with an empty source context, and target-grounded children must be validated by executable rewards rather than by the reflector's own judgment. If leakage or reflector bias were the dominant failure, one would expect large gains in Category-Novel apps as well, whereas the observed Category-Novel gains are small (31.4% and 34.3% vs. 29.4% base). The more load-bearing issue is that the headline quantitative comparison in Setting 1 is against a baseline from a different paper that the authors did not run, while Setting 2 lacks error bars entirely. This directly affects the abstract's central claim '45.0%... compared with 37.5%' and the conclusion that policy adaptation is complementary to context adaptation. A controlled reproduction and variance reporting could settle the issue, so the appropriate verdict remains CONDITIONAL rather than ACCEPT or REJECT. I agree with the reader's overall CONDITIONAL judgment but identify the baseline-comparability problem rather than the reflector-leakage problem as the single most load-bearing concern; hence 'partial' agreement on the weakest assumption.","tokens_in":21789,"tokens_out":6226,"duration_ms":69121,"concrete_test":"Reproduce the released AndroidWorld-Generalization Policy-Only TTA configuration (Gu et al. 2026) inside the authors' own codebase and evaluation harness, using the same UI-TARS-7B step-500 initialization, the same 40-instance adaptation manifest, and the same frozen 48-instance held-out panel, across at least five seeds; if the reproduced mean is not within about 2 points of 37.5, recompute the claimed 7.5-point joint-adaptation gain and revise the abstract's headline comparison.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that jointly adapting workflow context and a LoRA policy adapter beats policy-only test-time adaptation rests on two pillars, and each has a reproducibility gap. Pillar 1 (AndroidWorld-Generalization, Table 1): CoAdapt-GUI reaches 45.0% +/- 1.86 versus Policy-Only TTA at 37.5%, but Appendix B.3 explicitly states that the 37.5% value was 'not reproduced in our environment' and is taken from Gu et al. 2026. The paper asserts that all adaptive methods use the same 40 adaptation instances and a nominal 50-step schedule, but cross-paper comparability cannot be established for update frequency, LoRA rank, advantage normalization, context rendering, or action decoding; any of these can shift mobile-agent success rates by several points. The internal comparison Context-Only (35.0 +/- 1.74) versus CoAdapt-GUI (45.0 +/- 1.86) is controlled, but the benchmark against the policy-only state is not. Pillar 2 (AndroidWorld Plus, Table 2) uses the authors' own Policy-Only TTA, but reports no variance or repeated-run statistics; the 4.8-point CoAdapt-over-Context-Only gap corresponds to roughly five episodes on a 105-episode panel and could be within run-to-run noise. The concern is not that the authors misreport, but that the central quantitative claim currently depends on a cross-paper baseline in Setting 1 and on an unseeded single-panel result in Setting 2. Even if the reflector and lint checks are fully reliable, these comparisons are not yet sufficient to establish the claimed complementary gain.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CoAdapt-GUI, a test-time adaptation framework for mobile GUI agents deployed to applications absent from source training. The method maintains two adaptive states that are updated from the agent's own target-app rollouts and task rewards: a structured workflow context (a source-derived, transfer-constrained library plus a target-grounded population of context variants) and a LoRA policy adapter on a frozen vision-language model. The context channel uses TrueSkill-rated variants and a frozen reflector that proposes validated workflow revisions, while the policy channel computes task-context-matched group-relative advantages. The authors evaluate on AndroidWorld-Generalization (unseen task instances within unseen apps) and on a newly constructed AndroidWorld Plus benchmark (unseen task templates within unseen apps), reporting 45.0% versus 37.5% for the reported Policy-Only TTA baseline in the first setting and 52.9% versus 38.6% for the Base Policy in the second.","tokens_in":22185,"tokens_out":4569,"duration_ms":48773,"significance":"If the central results hold, the paper makes a meaningful contribution: it explicitly separates transferable workflow knowledge from app-bound source state, coordinates two adaptation channels from shared rollouts, and constructs an evaluation that separates task-instance from task-template generalization. The paper also has notable strengths: Setting 1 reports five-run means and standard deviations, the manifests and leakage controls are described in detail, and the appendix provides concrete prompts, validation contracts, and workflow examples. However, the headline comparison on AndroidWorld-Generalization rests on a Policy-Only TTA baseline that was not reproduced in the authors' own environment, and the AndroidWorld Plus results are presented without variance or repeated-run statistics. These gaps currently prevent the paper from fully supporting its strongest quantitative and conceptual claims.","major_comments":[{"comment":"The central Setting-1 comparison against Policy-Only TTA is not a controlled experiment: the 37.5% value is a reported result from Gu et al. (2026) that, as Appendix B.3 states, was 'not reproduced in our environment.' Because the two methods may differ in update frequency, LoRA rank, advantage normalization, context rendering, and action decoding, the 45.0% versus 37.5% comparison in the abstract and Section 1 cannot be taken as evidence that CoAdapt-GUI outperforms policy-only test-time adaptation in this setting. The controlled internal comparison (Context-Only TTA at 35.0% versus CoAdapt-GUI at 45.0%) supports the context channel's contribution, but the primary benchmark against the policy-only state needs either same-harness reproduction or substantially qualified wording.","section":"§4.2, Table 1, Appendix B.3"},{"comment":"The AndroidWorld Plus results are reported without variance, confidence intervals, or the number of independent runs; the table appears to contain single-panel point estimates. The key gaps are small relative to the 105-episode evaluation panel: CoAdapt-GUI versus Context-Only TTA is 4.8 points (about five episodes), and Policy-Only TTA versus the Base Policy is 1.4 points (about 1.5 episodes). Without repeated seeded runs or an error model, the claims that 'joint policy adaptation further improves held-out performance' and that Policy-Only TTA provides a gain are not statistically supported.","section":"§4.3, Table 2"},{"comment":"The transferability claim depends on the schema/lint checks detecting all source-specific leakage, but no independent audit of the generated workflow entries is provided. The hard linter detects app names, package/resource identifiers, and state identifiers, yet semantic leakage (e.g., a phrase that encodes source-interface geometry or a source-specific interaction pattern without naming the app) could pass. I request a concrete leakage audit—for example, human annotation of a sample of source and target-generated entries for app-bound references, or an ablation that disables the eligibility linter—to confirm that the context channel is genuinely app-agnostic.","section":"§3.2, Appendix F.2"},{"comment":"The paper correctly notes that Context-Only TTA and CoAdapt-GUI run independently and can collect different policy-dependent trajectories, so their difference is not a crossed-state isolation of the LoRA update. However, the abstract and conclusion state that 'joint policy adaptation further improves held-out performance' without this qualification. A controlled estimate of the policy channel, such as applying the LoRA update to the context state produced by Context-Only TTA, or an ablation that isolates the policy update while holding the context trajectory distribution fixed, would support the claim that the policy channel itself, rather than the full joint procedure, is responsible for the additional gain.","section":"§4.1, §4.3, Table 12"}],"minor_comments":[{"comment":"The text uses 'nominal 50-update schedule' in the appendix and 'nominal 50-step schedule' in Section 4.2; please harmonize the terminology. The appendix's caveat that the schedule is not an identical compute budget across methods should also appear next to Table 1, since the column 'Steps/app' invites the opposite reading.","section":"Appendix B.3, Table 1"},{"comment":"Please state explicitly how many independent runs each AndroidWorld Plus number is based on and what seeds were used; without this, the reader cannot assess whether the reported differences are within run-to-run noise.","section":"Table 2"},{"comment":"The frozen synthesizer and reflector are given as 'Claude Opus 4.7,' a proprietary model. Please report the exact model version, any snapshot or API details, and the dates on which artifacts were generated, since results may shift with model updates.","section":"Appendix C.3, C.4"},{"comment":"No quantitative information is given about how many source workflow entries were rejected by the schema/lint checks; reporting rejection rates would strengthen the claim that the eligibility predicate is effective.","section":"Appendix F.2"},{"comment":"Figure 1 is visually dense; adding explicit labels for the two update frequencies (context ratings versus LoRA updates) and for the freeze point before evaluation would make the adaptation loop easier to follow.","section":"Figure 1"}],"recommendation":"major_revision","confidential_remarks":"The paper's primary comparison against Policy-Only TTA (Table 1) comes from a prior paper by the same research group (Gu et al. 2026) and is not reproduced in the submitted environment. I would ask the editor to require either a same-harness reproduction of that baseline or a clear rewording of the abstract and conclusion so that the 45.0% versus 37.5% claim is not presented as a controlled comparison. I see no evidence of misreporting; the issue is that the strength of the comparative claim currently exceeds what the evidence supports."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things up front. The real contribution here is the separation of transferable workflow context from policy, with a linter that strips app-bound state, and the internal ablations are consistent with that design. But the abstract's 45.0-vs-37.5 claim in setting 1 leans on a Policy-Only TTA number taken from the authors' own prior paper and explicitly not reproduced in their environment (Appendix B.3), and setting 2's 52.9% comes with no error bars.\n\nWhat is genuinely new and good: the structured context representation w=⟨c,P,F,V⟩, the eligibility predicate and lint checks that block app-specific strings and coordinates from crossing the transfer boundary, and the task-context-matched group-relative policy objective. Building AndroidWorld Plus with disjoint adaptation and evaluation templates within each target app is a real improvement over the existing AndroidWorld-Generalization split; it tests template transfer rather than only instance transfer. The internal comparisons are controlled: Context-Only TTA and CoAdapt-GUI use the same adaptation tasks and budget, and the gap there is large and stable (35.0±1.74 to 45.0±1.86 over five runs). The appendix is unusually detailed, with complete template manifests, prompt templates, and validation contracts. The limitations section is honest about leakage and sparse-feedback risks.\n\nNow the soft spots, in proportion. The 7.5-point gain over Policy-Only TTA in setting 1 is not established in this paper's own harness. The authors say the schedule and adaptation instances match, but cross-paper comparability fails on things like LoRA rank, update frequency, advantage normalization, and action decoding, any of which can move mobile-agent success by several points. Their internal comparison with Context-Only TTA is the controlled evidence, and it does support the value of joint adaptation, but the headline number against the policy-only state should be treated as a reported value, not a measured one. In setting 2, the 4.8-point gap between CoAdapt-GUI and Context-Only TTA is about five episodes on a 105-episode panel, with no variance reported; that could be run-to-run noise. The Category-Novel gains are smaller and based on roughly 51 episodes. None of this makes the method look fake, but it does mean the complementary-gain claim is weaker than the abstract implies.\n\nThe dependence on Claude Opus 4.7 as the frozen synthesizer/reflector is a reproducibility cost, though the schema and lint checks bound the damage, and the prompts are all printed. Self-citation is not the issue; the unreproduced baseline is. The core evidence is the authors' own controlled configurations, so circularity is low.\n\nWho should read this: anyone working on test-time adaptation for GUI agents, and anyone building cross-app benchmarks. It deserves a serious referee, not a desk reject. I would ask the authors to reproduce Policy-Only TTA in their own harness, report repeated-run statistics for AndroidWorld Plus, and ideally release code and the exact adaptation logs.","headline":"Worth engaging: a clean two-channel TTA design with a genuinely useful new benchmark, but the headline gain rests on an unreproduced cross-paper baseline and a no-variance second panel.","tokens_in":22769,"tokens_out":1724,"would_cite":true,"duration_ms":20404,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CoAdapt-GUI adapts both workflow context and policy from target-app rollouts, raising unseen-app success from 38.6% to 52.9%.","keywords":["mobile GUI agents","test-time adaptation","workflow context adaptation","LoRA","group-relative policy optimization","unseen application generalization","AndroidWorld","vision-language models"],"falsifier":"Replace the reflector's proposed revisions with a no-op (root context) while keeping the LoRA channel and rollout schedule identical; if success on held-out tasks stays near CoAdapt-GUI's reported 52.9% rather than dropping toward the Policy-Only result, the context channel's gain is not caused by the generated revisions. For leakage, run the same pipeline, then delete from the final workflow states every entry whose text contains a string appearing in the target app's UI dumps or task prompts; if held-out success does not fall, the gains are not coming from leaked app-bound details.","tokens_in":21572,"feed_emoji":"📱","tokens_out":13177,"duration_ms":118993,"temperature":0.7,"pith_summary":"CoAdapt-GUI claims that, when a mobile agent meets an app it never saw in training, it can keep improving by adapting two separate things at test time: a structured workflow context (reusable procedures, failure patterns, completion checks) and a lightweight LoRA policy adapter. Both updates are driven only by the agent's own rollout rewards in the target app, with no target demonstrations and no held-out evaluation signal. The paper reports that joint adaptation raises success on AndroidWorld-Generalization to 45.0% versus 37.5% for policy-only adaptation, and on AndroidWorld Plus from 38.6% to 52.9%. A sympathetic reader would take the central claim to be that workflow knowledge and policy parameters are complementary adaptation channels, and that explicitly keeping app-bound source details out of the workflow context is what makes the knowledge transfer.","feed_headline":"Test-time co-adaptation lifts unseen-app GUI success to 52.9%","feed_subtitle":"Context plus policy adapted from the agent's own target-app runs beats policy-only adaptation.","key_machinery":"The load-bearing mechanism is a two-channel, interaction-coupled adaptation loop. A workflow entry is a typed tuple $w=\\langle c,P,F,V\\rangle$ (when the workflow applies, abstract procedure, failure/recovery conditions, verification checks), stored in a transferable library from which app-bound screen-transition states are excluded by an eligibility predicate plus schema and lint checks. The context channel rates candidate workflow variants with TrueSkill and uses a frozen reflector to propose revisions from contrasted successes and failures; each candidate must pass validation before it can be sampled in later rounds. The policy channel applies a group-relative objective: advantage is computed only within groups $G(q,\\kappa)$ sharing the same task and rendered context variant, so context-condition differences cannot masquerade as policy credit, and only the LoRA adapter is updated on a frozen backbone. The two channels are coupled through the shared rollout stream rather than a joint gradient.","core_discovery":"The central claim, stated on the paper's own terms, is that test-time behavior of a GUI agent depends on two complementary adaptive states: an explicit workflow state $M_t(q)$ carrying transferable procedures, failure conditions, and completion checks, and a parametric state $\\theta_t$ controlling visual-action behavior. CoAdapt-GUI updates both from the same target-app rollouts, with two safeguards. The context channel maintains a population of TrueSkill-rated workflow variants; a frozen reflector contrasts successful and failed traces to propose typed revisions $w=\\langle c,P,F,V\\rangle$, which enter the population only after schema, provenance, and lint checks, and the highest-rated validated variant is frozen for evaluation. The policy channel computes group-relative advantages only among rollouts sharing the same task and context condition, updating a LoRA adapter on a frozen vision-language backbone. In the two settings, the jointly adapted agent reaches 45.0% on AndroidWorld-Generalization (37.5% for the reported Policy-Only TTA baseline) and 52.9% on AndroidWorld Plus (38.6% base policy), with the context channel alone contributing most of the AndroidWorld Plus gain (48.1%).","pith_inferences":["A testable extension is to use the TrueSkill rating variance of the workflow-context population as an acquisition signal for choosing which adaptation task to sample next, something the current controller does not exploit.","The AndroidWorld Plus template-disjoint split suggests workflow guidance transfers across task types within an app; a natural stress test is to vary interface similarity across apps in the same category to map where cross-app context transfer breaks down.","Because the reflector's revision quality is load-bearing, an audit that flags workflow entries whose text overlaps the target app's UI dumps would test whether the context channel's gain is genuinely transferable or partly app-bound leakage.","If the gains hold, a practical implication is that the main cost is data collection rather than computation: each LoRA update is cheap, so shortening the per-condition rollout group or reducing context-evaluation rounds would test how much of the benefit depends on the volume of reward-bearing interaction."],"forward_implications":["On AndroidWorld-Generalization, joint adaptation reaches 45.0% versus 37.5% for Policy-Only TTA, a 7.5-point gain from adding the workflow-context channel.","On AndroidWorld Plus, context-only adaptation raises success from 38.6% to 48.1% and joint CoAdapt-GUI to 52.9%, while policy-only adaptation adds only 1.4 points overall, so the context channel carries most of the improvement.","Category-Novel apps, whose source category is absent and therefore receive no source workflow, still improve from 29.4% base to 34.3% with joint adaptation, showing target-grounded context construction works without category-matched source experience.","A recorded adaptation trace shows an evolved context variant exposing a mean task reward of 0.25 in round 12 while the root context stays at zero, illustrating how context evolution can generate the reward signal the policy channel needs.","Appendix E increments show policy adaptation adds 4.8 points overall on top of evolved context but is harmful alone on Category-Novel apps (-3.9 points), supporting the two channels as complementary rather than interchangeable."],"supporting_citations":[{"why":"Supplies the AndroidWorld-Generalization unseen-app split, the Base Policy, and the Policy-Only TTA baseline that CoAdapt-GUI is measured against.","marker":"Gu et al. 2026"},{"why":"Provides the AndroidWorld environment and its programmatic evaluator, which AndroidWorld Plus extends with additional apps and disjoint task templates.","marker":"Rawles et al. 2025"},{"why":"Source of the group-relative policy objective used for the LoRA update, with advantages computed within task-context groups.","marker":"Shao et al. 2024"},{"why":"TrueSkill rating system maintains the population of workflow-context variants and selects the highest-rated validated state for deployment.","marker":"Herbrich, Minka, and Graepel 2006"},{"why":"LoRA is the low-rank adapter mechanism that keeps policy adaptation lightweight on a frozen vision-language backbone.","marker":"Hu et al. 2021"},{"why":"UI-Mem jointly learns memory and policy during source-side training; the paper contrasts its target-side separate adaptation of context and policy.","marker":"Xiao et al. 2026"},{"why":"E-SPL jointly optimizes a global prompt and policy; the paper contrasts its structured validated workflow context with free-text prompt evolution.","marker":"Zhang, Chen, and Stadie 2026"},{"why":"Qwen3-VL-8B-Instruct is the frozen backbone used in the AndroidWorld Plus configuration.","marker":"Bai et al. 2025"},{"why":"UI-TARS-7B step-500 checkpoint is the source-side policy initialization for the AndroidWorld-Generalization setting.","marker":"Qin et al. 2025"}],"fun_headline_variants":["CoAdapt-GUI: joint adaptation lifts unseen-app GUI success to 52.9%","Joint context and policy adaptation beats policy-only on unseen GUI apps","Test-time co-adaptation lifts GUI agents to 52.9% on unseen apps","CoAdapt-GUI: workflow context plus policy adaptation reaches 52.9%","Unseen GUI apps: joint workflow and policy adaptation hits 52.9%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the frozen language model that synthesizes and revises workflow entries reliably produces app-agnostic text and that the schema, provenance, and lint checks catch every app-bound name, label, or identifier left in that text; if either fails, the workflow context leaks source-specific state and the reported improvements would shrink or vanish.","fun_headline_variants_meta":{"raw":{"variants":["CoAdapt-GUI: joint adaptation lifts unseen-app GUI success to 52.9%","Joint context and policy adaptation beats policy-only on unseen GUI apps","Test-time co-adaptation lifts GUI agents to 52.9% on unseen apps","CoAdapt-GUI: workflow context plus policy adaptation reaches 52.9%","Unseen GUI apps: joint workflow and policy adaptation hits 52.9%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000867,"raw_usage":{"total_tokens":3766,"prompt_tokens":966,"completion_tokens":2800,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":582,"completion_tokens_details":{"reasoning_tokens":2694}},"tokens_in":582,"tokens_out":2800,"duration_ms":18758,"temperature":1.0,"reasoning_tokens":2694,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:34:01.074565+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replace the reflector's proposed revisions with a no-op (root context) while keeping the LoRA channel and rollout schedule identical; if success on held-out tasks stays near CoAdapt-GUI's reported 52.9% rather than dropping toward the Policy-Only result, the context channel's gain is not caused by the generated revisions. For leakage, run the same pipeline, then delete from the final workflow states every entry whose text contains a string appearing in the target app's UI dumps or task prompts; if held-out success does not fall, the gains are not coming from leaked app-bound details.","supporting_citations":[],"review_version":1}