{"id":"a4e6f5fd-25cf-4687-9f80-2cdef5b2c403","arxiv_id":"2608.05891","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"AppDeltaWorld predicts mobile GUI transitions as code updates retrieved under action constraints, and its generated trajectories improve an 8B mobile agent on several benchmarks.","lead":"AppDeltaWorld is a new kind of simulated phone environment that predicts the next screen as a small change to existing HTML code instead of generating a whole new image. If the approach holds up, it could let AI agents practice on private or sensitive apps without access to real user data.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Fidelity result may be retrieval memorization: CMGUIBench-500 is sampled from the same CMGUI corpus that supplies 95.47% of the retrieval memory, with no screen-level split described.","rationale":"Reader's weakest assumption is the fidelity of reverse-engineered HTML; I agree that is a serious gap, but I put the overlap issue first because it is directly testable from artifacts the paper already describes (CMGUI source, retrieval memory, eval sample) and because the paper's own retrieval ablation shows how much of the score depends on memory lookup. The two concerns compound: if overlap is present, the fidelity numbers measure consistency with the training reconstruction, not with real apps; if overlap is absent, the reconstruction-fidelity question still remains. I do not think this warrants REJECT: the downstream results on AndroidLens, MobileGym, and MobileWorld are external and give some independent support that AppDeltaWorld rollouts carry learnable signal, and the authors themselves report that ADW-only data degrades performance, which already limits the substitution claim. A CONDITIONAL verdict requiring the split analysis and reconstruction validation is proportionate.","tokens_in":16968,"tokens_out":7416,"duration_ms":70236,"concrete_test":"Build a screen-level disjoint split of CMGUI before constructing the retrieval memory, and re-run CMGUIBench-500 under three conditions: (1) full memory as in the paper; (2) memory with all evaluation screens (and, if available, all screens from the same app session) excluded by DOM hash and retrieval-text similarity; (3) retrieval disabled entirely. If the overall score in condition (2) or (3) falls by more than a few points, or if Sad/Sid approach the image-only baselines, the 73.51 score is attributable to memorized lookup, not to transition-grounded generation. Report the fraction of evaluation screens that have a near-duplicate in the memory.","verdict_should_be":"UNCHANGED","load_bearing_attack":"AppDeltaWorld's headline claim is the 73.51 CMGUIBench-500 fidelity score. The evaluation cannot support this claim until train/eval overlap is ruled out. The world-model memory is built from 95,614 CMGUI transitions (95.47% of training data, Table 1), and the benchmark is '500 random samples from the original CMGUI benchmark' (Settings). The paper specifies no split, no deduplication, and no exclusion of evaluation screens from the memory. Because the method is explicitly a retrieval-and-completion model (Eqs. 4-6), high scores may reflect near-duplicate Level-1 HTML lookup rather than the ability to predict unseen transitions. The w/o RAG ablation supports this reading: removing retrieval drops Sad/Sid from 79.69/77.00 to 65.16/69.40, so the score is largely carried by memory content. Independent of overlap, the reverse-engineered HTML used as training ground truth is never checked against view hierarchies or real screenshots; even a clean split would only certify fidelity to the reconstruction. These two issues together make the world-model fidelity part of the central claim unverified, although they do not invalidate the downstream agent results, which are measured on external benchmarks.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AppDeltaWorld, a world model for mobile GUI agents that predicts the next screen as a transition-grounded delta code update. It retrieves a Level-1 HTML reference under an action-transition constraint, generates Level-2 HTML conditioned on the current screen and predicted next-screen text, inserts synthesized visual assets, and renders the result. The world model is trained on reverse-engineered HTML from multiple datasets (95.47% CMGUI) and evaluated on CMGUIBench-500 under Code2World scoring. The authors report a state-of-the-art overall fidelity score of 73.51, and they use the world model to generate filtered rollouts for SFT and test-time RL, training AppDeltaAgent, which achieves state-of-the-art results on AndroidLens and consistent gains on MobileGym and MobileWorld.","tokens_in":17086,"tokens_out":3749,"duration_ms":34350,"significance":"If the fidelity results survive an evaluation with a proper train/test split, the delta-code retrieval-plus-completion design is a meaningful step toward scalable GUI world models that combine structural stability, modality coverage, and transition consistency. The paper ships public dataset and evaluation code, includes several ablations, and is unusually candid about the limitations of the cluster-based RL reward. However, the headline fidelity claim currently rests on an evaluation setup where the benchmark screens and the retrieval memory come from the same underlying corpus, so the main contribution is not yet verified.","major_comments":[{"comment":"The CMGUIBench-500 evaluation set is described as '500 random samples from the original CMGUI benchmark', while the world-model training data includes 95,614 CMGUI transitions, 95.47% of the training mixture (Table 1). No screen-level or transition-level split is described, and Eq. (6) selects the final Level-1 reference by similarity search over a memory built from that same training data. The high S_ad/S_id scores (79.69/77.00) may therefore reflect retrieval of near-duplicate training screens rather than generalization to unseen transitions. The w/o RAG ablation in Table 4 drops S_ad/S_id to 65.16/69.40, indicating that the functional score is carried largely by the memory. The authors should state explicitly whether any CMGUIBench-500 screen or its associated transitions appear in the world-model training set or retrieval memory, and, if not, describe the deduplication procedure used. They should also report fidelity separately for transitions whose source and target clusters are present in memory versus genuinely novel transitions.","section":"Settings; Eq. (6); Table 1; Table 4"},{"comment":"The world-model training data is 'all reverse engineered using Claude-4.8-Opus and Gemini-3.1-Pro to obtain renderable code', and this reconstructed HTML is treated as ground truth for Level-1/Level-2 labels and as the rendered next state for both training and evaluation. No fidelity check of this reconstruction against real view hierarchies, accessibility trees, or actual device screenshots is reported. Consequently, even a clean evaluation split would certify fidelity only to the reverse-engineered reconstruction, not to real mobile screens. To support the claim that AppDeltaWorld is a faithful world model of real apps, the authors should validate the reconstruction quality on a random sample (e.g., human or model-based comparison of text, coordinates, and element presence against ground-truth view hierarchies) and report agreement metrics.","section":"Training Data Statistics"},{"comment":"The ablation study removes 'RAG' as a single component, but the transition-grounded retrieval described in Eqs. (4)-(6) has two independent parts: the memory-based structural reference and the action-transition constraint. Removing both conflates the contribution of the transition index with that of retrieval. Since transition logic consistency is a stated contribution and a claimed advantage over unconstrained generation, the authors should ablate the transition constraint separately (e.g., unconstrained retrieval over all clusters instead of only C_{t+1}) and, ideally, report invalid-action rejection rates (precision/recall for transitions that should be refused).","section":"World-Model Fidelity Evaluation; Table 4"}],"minor_comments":[{"comment":"The sentence 'It's important to note' contains an apostrophe error; 'Its' would be correct.","section":"Table 2 discussion"},{"comment":"The text states that 'only 1/10 of the data passed the quality verification', but the relationship between the number of seeds, the number of accepted trajectories (33,133 in Table 1), and the 1/10 ratio is not stated; please clarify the denominator.","section":"World-Model-in-the-Loop SFT Data Construction"},{"comment":"The caption mentions a 'Trajectory Audit' stage, but this term is not defined in the main text; the corresponding filtering and QC steps in Algorithm 1 should be referenced explicitly.","section":"Figure 2 caption"},{"comment":"The action-target grid quantization for clicks and long presses into a 6×12 grid is introduced without specifying how the grid is aligned to screen coordinates or how swipe start/end grids are encoded; a sentence with the coordinate convention would improve reproducibility.","section":"Methodology; Eq. (5)"},{"comment":"The related work discusses MobileWorld as a text-level world model, citing Li et al. 2025a, but the experiments section evaluates on 'MobileWorld' as a real-environment benchmark with a different citation (Kong et al. 2026); please disambiguate these two uses to avoid confusion.","section":"Related Work"}],"recommendation":"major_revision","confidential_remarks":"The central blocker is the evaluation split: the CMGUIBench-500 fidelity claim cannot be assessed until overlap with the retrieval memory is ruled out or quantified. If the authors can provide a clean held-out evaluation and the fidelity gap persists, the paper is likely acceptable; if not, the fidelity claim should be substantially softened and the contribution re-centered on the downstream agent training results, which are measured on external benchmarks and appear more robust."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the mechanism is real: constraining Level-1 HTML retrieval by an app-specific action-transition index, then generating Level-2 delta code with image-slot synthesis, is a sensible and novel combination. The ablations show both retrieval and diffusion contribute, and the w/o RAG drop (Sad 79.69 to 65.16) is consistent with the design. Second, the downstream agent results are the strongest part of the paper: AppDeltaAgent improves over its base consistently on AndroidLens, MobileGym, and MobileWorld, all external benchmarks where the world model is not the evaluation surface. Those gains survive even if the world-model fidelity result does not.\n\nThe soft spot is the headline fidelity claim. CMGUIBench-500 is 500 random samples from CMGUI, and CMGUI supplies 95.47% of the world-model training data and presumably most of the retrieval memory. The paper specifies no screen-level split or deduplication. Because Eq. 6 is exactly a retrieval from that memory, high Sad/Sid may be near-duplicate lookup rather than prediction. The paper needs an experiment with evaluation screens excluded from memory, or retrieval disabled on a properly split benchmark. Also, the training HTML was all reverse-engineered by Claude-4.8-Opus and Gemini-3.1-Pro, with no check against view hierarchies; even a clean split would only certify fidelity to the reconstruction.\n\nMinor but worth noting: the paper overclaims that the world model can substitute for real-world environments while its own Figure 7a shows ADW-only data hurts the agent. Soften that. Table 2 has no error bars.\n\nOverall, this is a competent systems paper with a new mechanism and honest ablations, including a clear limitation in the consensus-RL analysis. The fidelity claim needs revision, but the downstream result is valuable. Send to peer review with a request for the split clarification and an overlap-controlled evaluation.","headline":"A genuinely new retrieval-plus-delta-code world model with convincing downstream agent gains, but the headline CMGUIBench-500 fidelity result is at risk from train/eval overlap until the authors show a clean split.","tokens_in":17775,"tokens_out":1865,"would_cite":true,"duration_ms":16771,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AppDeltaWorld claims that predicting a mobile GUI's next screen as a reachable code update—constrained by the action taken—produces more faithful and stable transitions than generating an unconstrained image or fresh HTML, and that the…","keywords":["mobile GUI agents","GUI world models","delta code generation","HTML retrieval","action transition consistency","closed-loop synthetic training","test-time reinforcement learning","screen fidelity"],"falsifier":"Take a fixed set of actions executed on real Android devices where the actual next-screen view hierarchy is recorded, run AppDeltaWorld on the same actions from the same starting screens, and compare the rendered next screen's text and clickable-element coordinates against the recorded hierarchy; if the agreement is no better than image-only baselines, the central claim of transition-grounded fidelity is not supported.","tokens_in":16643,"feed_emoji":"📱","tokens_out":6550,"duration_ms":55146,"temperature":0.7,"pith_summary":"This paper tries to establish that a mobile GUI world model can predict the next screen accurately enough to train agents, without needing real app interactions. The proposal, AppDeltaWorld, treats the next screen as a reachable code update: it retrieves a stored app-specific HTML reference that is reachable under the given action, generates executable HTML for the changed screen, fills image slots with synthetic visuals, and renders the page. On CMGUIBench-500 under the Code2World evaluation, the model reports the highest overall fidelity score, beating both image-generation and code-generation baselines. When rollouts from this world model are filtered and added to public supervision, the resulting agent reaches state-of-the-art action matching on AndroidLens and improves closed-loop task success on MobileGym and MobileWorld. The significance, if true, is that a code-plus-image simulator can supply useful closed-loop training signal where real trajectories are scarce.","feed_headline":"Delta-code world model tops GUI next-screen fidelity","feed_subtitle":"Code-plus-image screen prediction beats image-only models and lifts a mobile agent's success rate without real-app training.","key_machinery":"The carrying mechanism is the two-level delta code formulation with transition-constrained retrieval. Level-1 HTML is a reusable, app-specific page template; Level-2 HTML is the executable next screen written as a change on top of that template. A transition index maps each source cluster, action type, and action-target grid to allowed target clusters, so unsupported actions are rejected rather than hallucinated, and the retrieved Level-1 reference is guaranteed reachable. The final rendered observation is produced by generating Level-2 HTML, synthesizing visual assets for image slots, and rendering in a browser, which is what lets the model combine precise text and layout with photographic visual regions.","core_discovery":"AppDeltaWorld's central claim is that predicting the next GUI as a delta over retrieved, action-reachable code is more stable and more faithful than generating the next screen as an unconstrained image or as fresh HTML from scratch. The world model decomposes each screen into a Level-1 structural reference (layout, widgets, DOM organization) and Level-2 executable HTML (the concrete next screen). At inference, it localizes the current screen to a source cluster, uses a per-app transition index to restrict candidate target clusters to those reachable by the action, retrieves the Level-1 code under that constraint, then generates Level-2 HTML conditioned on the current screen, the action, predicted next-screen text, and the retrieved structure. Visual regions are synthesized by a text-to-image model and inserted into image slots before browser rendering. The paper reports that this pipeline achieves the highest overall fidelity score of 73.51 on CMGUIBench-500 under the Code2World evaluation, with the largest gains in element-level and layout-level visual quality, and that filtered rollouts from the pipeline improve a Qwen3-VL-8B-based agent on AndroidLens, MobileGym, and MobileWorld.","pith_inferences":["Editorial inference: the same retrieval-constrained delta formulation should transfer to web, desktop, or in-car GUI agents whenever a DOM-like structure exists, because the hard part being solved—stable next-state generation under action constraints—is not phone-specific.","Editorial inference: a head-to-head fidelity check against recorded real-device view hierarchies would test whether the reported scores reflect true app consistency or internal consistency with the reverse-engineered HTML; the paper does not report such a check.","Editorial inference: the scaling curve's saturation after roughly 20K added rollouts suggests that simply generating more trajectories will not keep improving the agent; expanding the transition index and Level-1 memory to cover more screens and actions is the likely next lever.","Editorial inference: the consensus-reward RL experiment's own statistics (average winner support 3.295 of 8 rollouts; 23.5% groups with no unique winner) indicate that a learned state-similarity metric, rather than a fixed visual-text threshold, is the key to making self-supervised world-model rewards reliable."],"forward_implications":["Fidelity-wise, the 73.51 overall score on CMGUIBench-500 under Code2World evaluation would make AppDeltaWorld the strongest next-screen predictor among the tested image, code, and hybrid systems, with the largest margins on element and layout scores (56.26/57.43) over image-only models.","Training-wise, 33,133 filtered AppDeltaWorld rollouts mixed with public supervision take AndroidLens Total-LL AMS/ATP from 80.33/34.96 to 90.28/46.63 and Total-HL ATP from 23.30 to 33.05, so closed-loop world-model experience complements static demonstrations.","Deployment-wise, online gains on MobileGym (10.2% to 14.1% SR) and MobileWorld GUI-only (9.4% to 14.9% SR) suggest the benefit carries beyond static action matching into real execution.","Optimization-wise, test-time RL with world-model-rendered successor states improves app-specific AMS by 1.51 to 5.06 points over 48 steps, which implies policies can be adapted without touching real devices."],"supporting_citations":[{"why":"Supplies the Code2World evaluation protocol and renderable-code generation baseline that the fidelity comparison is built on.","marker":"Zheng et al. 2026"},{"why":"Provides CMGUI, the dominant world-model training source and the benchmark from which CMGUIBench-500 is sampled.","marker":"Xie et al. 2026"},{"why":"Supplies AndroidLens, the static action-prediction benchmark on which AppDeltaAgent reports state-of-the-art AMS and ATP.","marker":"Cao et al. 2026a"},{"why":"Supplies MobileGym, the simulated closed-loop environment used for online task-execution evaluation.","marker":"Wu et al. 2026"},{"why":"Supplies MobileWorld, the real-Android environment where the agent's GUI-only success rate is measured.","marker":"Kong et al. 2026"},{"why":"Supplies GUI-Owl action-model supervision and rollout seeds used for world-model-in-the-loop SFT construction.","marker":"Gao et al. 2026"},{"why":"Supplies OpenMobile task and trajectory seeds, another public supervision source and rollout seed set.","marker":"Cheng et al. 2026"},{"why":"Motivates using the world model to preview action outcomes, grounding the test-time RL reward design.","marker":"Xu et al. 2026a"}],"fun_headline_variants":["Delta-code world model tops GUI next-screen fidelity","Code-update prediction beats pixel-only for next GUI","AppDeltaWorld: reachable code deltas for faithful GUI states","World model turns screen transitions into code deltas","Code-delta GUI model boosts agent without real-app data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the reverse-engineered HTML used as training ground truth faithfully represents the real apps; if the AI-generated conversions misstate text, positions, or what an action changes, the simulator teaches agents a distorted version of the apps, and no check against real-device ground truth is reported.","fun_headline_variants_meta":{"raw":{"variants":["Delta-code world model tops GUI next-screen fidelity","Code-update prediction beats pixel-only for next GUI","AppDeltaWorld: reachable code deltas for faithful GUI states","World model turns screen transitions into code deltas","Code-delta GUI model boosts agent without real-app data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000316,"raw_usage":{"total_tokens":1842,"prompt_tokens":1047,"completion_tokens":795,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":663,"completion_tokens_details":{"reasoning_tokens":717}},"tokens_in":663,"tokens_out":795,"duration_ms":8300,"temperature":1.0,"reasoning_tokens":717,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T21:36:15.238226+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a fixed set of actions executed on real Android devices where the actual next-screen view hierarchy is recorded, run AppDeltaWorld on the same actions from the same starting screens, and compare the rendered next screen's text and clickable-element coordinates against the recorded hierarchy; if the agreement is no better than image-only baselines, the central claim of transition-grounded fidelity is not supported.","supporting_citations":[],"review_version":1}