{"id":"ec6f7447-0ca8-4e37-9e1a-c56eaae196c7","arxiv_id":"2507.15846","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Modeling GUI elements as Gaussian distributions instead of binary targets yields 92.0% (ScreenSpot), 93.3% (ScreenSpot-v2), and 47.5% (ScreenSpot-Pro) for a 7B model, outperforming UI-TARS-72B by a relative 24.7% on the professional-software benchmark.","lead":"This paper replaces the all-or-nothing reward used to train GUI-clicking AI models with a smooth Gaussian-shaped reward, and reports large accuracy gains on GUI grounding benchmarks. It is worth reading because it is a practical, well-tested reward-design recipe that visibly improves a 7B model's ability to locate interface elements, despite relying on a hand-tuned scale parameter.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The main evidence for GUI-G2 rests on a reward-shape hyperparameter α=0.5 that is selected by a sweep on ScreenSpot-v2 and then evaluated on that same benchmark; no held-out check shows this choice transfers to ScreenSpot-Pro, where the headline 24.7% improvement over UI-TARS-72B is claimed.","rationale":"The reader's weakest-assumption analysis and mine coincide: the single most load-bearing assumption is that α=0.5, chosen by a ScreenSpot-v2 sweep, transfers to unseen benchmarks. I considered two alternative objections and found them less central. The base-model/training-data confound with UI-TARS-72B is real but does not threaten the same-pipeline ablation in Table 2; it weakens the SOTA claim, not the mechanism. The '24.7%' relative-vs-absolute presentation is a reporting flaw, not a correctness risk. The reward equations (2) and (4) are internally consistent: Eq. 4 correctly evaluates the Bhattacharyya coefficient for two Gaussians, and the adaptive variance rule is well-defined. The Appendix's random-reward controls provide useful independent support that the training signal, not just the RL loop, matters. What remains is that the principal quantitative evidence—Table 2 and the α sweep—is obtained on the same evaluation set, and the headline ScreenSpot-Pro number uses the resulting α without a transfer check. Since the SSPro advantage over the nearest continuous baseline is only 0.2 points, even a small misspecification of α on that distribution would change the conclusion. This is an addressable experimental gap rather than a logical contradiction, so the verdict should remain CONDITIONAL until the transfer check is run.","tokens_in":23431,"tokens_out":8205,"duration_ms":85618,"concrete_test":"Retrain GUI-G2-7B with the same base model, 100K instances, and RL recipe for α ∈ {0.3, 0.5, 0.7}, and evaluate only ScreenSpot and ScreenSpot-Pro (leave ScreenSpot-v2 out of any further tuning). If α=0.5 is not best on ScreenSpot-Pro, or if its margin over α=0.3/0.7 is below the 0.2-point gap to SE-GUI-7B, the headline SSPro advantage is not robust; if α=0.5 beats both by more than 3 points on SSPro, the transfer concern is resolved. Report three seeds to bound variance.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.4 and Figure 5 select α=0.5 by maximizing ScreenSpot-v2 accuracy, and Section 4.1 fixes α=0.5 for all reported models. The same benchmark is then used for the sparse-vs-dense comparison in Table 2 that is the cleanest evidence for the method, making part of the +5.9% gain a selection effect. Because Eq. 5 makes α the only adaptive ingredient in the reward geometry, and the '2σ principle' in Section 4.4 is a post-hoc restatement of α=0.5 rather than a derivation, the internal-validity question is whether an SSv2-optimal reward shape also optimizes the harder, high-resolution ScreenSpot-Pro distribution. This is especially consequential because on ScreenSpot-Pro the reported advantage over the closest continuous baseline, SE-GUI-7B, is only 0.2 points (47.5 vs 47.3), so a modest shift in optimal α on that distribution could erase the headline result. No per-element adaptation beyond the linear rule σ=α·(box size) is provided, so the assumption is precisely that one global scale transfers. The Appendix's spurious-reward experiments strengthen the claim that dense signals help, but they do not test α transfer.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes GUI-G2, a reward-shaping method for RL-based GUI grounding. It replaces binary hit/miss rewards with two dense Gaussian terms: a point reward based on the target-element Gaussian density at the predicted center (Eq. 2) and a coverage reward based on the Bhattacharyya coefficient between predicted and target Gaussians (Eq. 4). The covariance is set adaptively to element size via sigma = alpha times the box dimension (Eq. 5), and the combined reward is used in GRPO training of Qwen2.5-VL-7B. On ScreenSpot, ScreenSpot-v2, and ScreenSpot-Pro, the trained GUI-G2-7B reaches 92.0%, 93.3%, and 47.5%, respectively, with the headline 24.7% relative gain over UI-TARS-72B on ScreenSpot-Pro. Ablations compare sparse point/IoU rewards, an inside-only Gaussian, component weighting, and thinking vs. no-thinking, and an appendix reports random-reward controls.","tokens_in":23604,"tokens_out":8021,"duration_ms":79991,"significance":"Assuming the reported results are reproducible, the paper makes a useful and low-cost contribution: the closed-form Gaussian rewards are simple, easy to implement, and provide dense optimization signals, and the controlled comparison in Table 2 plus the spurious-reward controls in Appendix A.1 are informative evidence that dense, spatially meaningful rewards matter. The announced code release also supports reproducibility. The broader claims, however, are stronger than the evidence: the gain over the closest distance-based continuous baseline on ScreenSpot-Pro is only 0.2 percentage points (Table 3), and the key parameter alpha is tuned on ScreenSpot-v2 and evaluated on the same benchmark, so the quantitative advantage is partly a selection effect. The 'new paradigm' phrasing also overstates what is essentially a reward-shaping design.","major_comments":[{"comment":"The adaptive variance scale alpha is selected by a hyperparameter sweep that maximizes ScreenSpot-v2 accuracy (Figure 5) and is fixed to 0.5 for all reported models (Section 4.1). The cleanest evidence for the central claim, the sparse-vs-dense reward comparison in Table 2, is then reported on the same ScreenSpot-v2 benchmark (93.3% vs. 87.4%). As a result, part of the +5.9 point gap may be a selection effect rather than a property of Gaussian rewards. Please select alpha on a validation split that is not used for the headline evaluation, or add a sweep on ScreenSpot-Pro and another held-out set showing that the relative ranking of reward shapes is stable across alpha. Without such a check, the claim that the optimal reward geometry transfers across interface distributions is not supported.","section":"Section 4.4 / Figure 5 / Section 4.1 / Table 2"},{"comment":"On ScreenSpot-Pro, GUI-G2-7B (47.5%) exceeds the closest distance-based continuous baseline SE-GUI-7B (47.3%) by only 0.2 points, while the headline 24.7% relative gain is against UI-TARS-72B (38.1%), which is neither the strongest RL baseline nor the strongest continuous baseline in the table. The 'substantially outperforms continuous distance-based methods' claim for the high-resolution regime therefore rests on a very thin margin, and SE-GUI is an external model trained with its own data and recipe. Please provide variance or significance information over multiple runs for the ScreenSpot-Pro comparison, or qualify the claim to reflect the margin.","section":"Table 3 / Section 4.2"},{"comment":"The adaptation principle used to justify alpha = 0.5 is internally inconsistent as written. The text defines '2sigma Principle' as sigma_x = width * 2, but then states that alpha = 0.5, which by Eq. (5) gives sigma_x = width / 2, 'aligns with the 2sigma statistical principle'. Moreover, the sigma-ablation results (1sigma, 2sigma, 3sigma, fixed sigma) are said to be 'shown in Table 5', but Table 5 is the nu/gamma weighting ablation, and the sigma results appear only in Figure 5. Please correct the nomenclature and report the complete sigma-ablation table, since this is the central design parameter.","section":"Section 4.4 / Eq. (5) / Table 5"}],"minor_comments":[{"comment":"The improvement over UI-TARS-72B is reported as 24.7% in the abstract and as 9.4% in Section 4.2; please label relative versus absolute gains consistently.","section":"Abstract / Section 4.2"},{"comment":"The token reduction is listed as 87.7% in the table but as 76.9% in the prose; make the two numbers consistent.","section":"Table 6"},{"comment":"The MacKenzie reference entry reads 'I. Scott MacKenzie and.'; the author field is incomplete.","section":"References"},{"comment":"The x-axis is not labeled; please specify the alpha values swept and state clearly that the plotted accuracy is on ScreenSpot-v2.","section":"Figure 5"},{"comment":"The caption contains 'Scrrenspot-v2' and 'bad case分析'; please proofread captions for typos and language consistency.","section":"Figure 10"}],"recommendation":"major_revision","confidential_remarks":"The alpha-selection issue is the main reviewable concern; it is fixable within the manuscript's scope by using a validation split or a second-benchmark sweep. The paper also cites the authors' earlier GUI-grounding work (Tang et al., 2025a,b) appropriately and not excessively. The contribution is incremental but publishable after the internal-validity point is addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read GUI-G2. Core idea is simple: replace binary hit/miss rewards for GUI grounding with a 2D Gaussian reward centered on the target element, variance scaled by element size, plus a Bhattacharyya coverage term. The same-pipeline comparison in Table 2 is the real contribution: Gaussian rewards beat sparse point rewards by 5.9 points on ScreenSpot-v2 (93.3 vs 87.4) and beat an inside-only Gaussian variant by 4.9. That controlled evidence is credible. The reward math is standard and correct, the ablations are clean, and the appendix spurious-reward experiments make a useful point: GUI grounding doesn't benefit from random rewards, so the dense signal matters.\n\nSoft spots. First, the scale parameter alpha is chosen by sweeping on ScreenSpot-v2 (Figure 5), then the same benchmark is used for the headline comparison and the section 4.4 ablations. That is a mild test-set selection effect; the +5.9 figure would be more trustworthy if alpha had been fixed on a held-out split or reported across all benchmarks. Second, the 24.7% improvement over UI-TARS-72B on ScreenSpot-Pro is a relative gain between different base models and training data, not an apples-to-apples comparison. The only same-pipeline continuous baseline on Pro, SE-GUI-7B, is ahead by 0.2 points (47.5 vs 47.3), which is within noise. The abstract's \"substantially outperforming\" overstates this. Third, no error bars or multiple seeds, and the 100K-instance sampling is underspecified. These are fixable.\n\nOverall, the central design choice — dense Gaussian rewards over sparse binary ones — holds up in the controlled setting, and the paper deserves a serious referee. Anyone working on RL for GUI agents should read it. But I would not cite it as evidence that Gaussian rewards beat distance-based continuous rewards; the evidence for that is thin.","headline":"A credible same-pipeline ablation shows Gaussian rewards beat sparse ones for GUI grounding, but the headline numbers are over-sold and the key hyperparameter is tuned on the eval set.","tokens_in":24308,"tokens_out":2635,"would_cite":false,"duration_ms":29718,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GUI-G2 claims that replacing binary hit-or-miss rewards with Gaussian point and coverage rewards lets a 7B model surpass a 72B model on professional-software GUI grounding by a relative 24.7% on ScreenSpot-Pro.","keywords":["GUI grounding","Gaussian rewards","dense continuous optimization","reinforcement learning","GRPO","adaptive variance","ScreenSpot-Pro","vision-language models"],"falsifier":"Train the same 7B model on the same data with identical Gaussian rewards but with $\\alpha = 0.25$ and $\\alpha = 0.75$, and evaluate on ScreenSpot-Pro: if either setting closes the gap over UI-TARS-72B, the claim that $\\alpha = 0.5$ encodes a general '2$\\sigma$' principle is not supported. A complementary check is to collect human click positions on ScreenSpot-Pro elements and verify that the Gaussian reward ranks two near-miss predictions in the same order as human modal click distance; any inversion would falsify the behavioral grounding of the reward.","tokens_in":23107,"feed_emoji":"🎯","tokens_out":7714,"duration_ms":71491,"temperature":0.7,"pith_summary":"The paper argues that GUI grounding underperforms because standard reinforcement-learning rewards are binary: a click inside the target box scores 1, anything outside scores 0, so the model receives no gradient for near-misses. It proposes GUI-G2, which treats each interface element as a two-dimensional Gaussian distribution and rewards predictions with a smooth point term for center alignment plus a coverage term for spatial overlap, with variances scaled to element size. Trained with GRPO on a 7B vision-language model, the system reports 92.0% on ScreenSpot, 93.3% on ScreenSpot-v2, and 47.5% on ScreenSpot-Pro, the last a relative 24.7% improvement over the 72B UI-TARS baseline. The paper's claim is that dense, continuously varying spatial feedback—not model scale—is what drives accurate GUI grounding.","feed_headline":"Gaussian click rewards beat a 72B GUI model at 7B scale","feed_subtitle":"Smooth Gaussian reward signals lift a 7B model past UI-TARS-72B on ScreenSpot-Pro, a 24.7% relative jump.","key_machinery":"The load-bearing object is the Gaussian reward surface defined over the whole interface plane. For a ground-truth box $b = [x_1, y_1, x_2, y_2]$, the paper places a 2D Gaussian with mean $\\mu = ((x_1+x_2)/2, (y_1+y_2)/2)$ and diagonal covariance with $\\sigma_x = \\alpha(x_2-x_1)$, $\\sigma_y = \\alpha(y_2-y_1)$, $\\alpha = 0.5$. The point reward evaluates the predicted center by the Gaussian density at that location, so reward decays smoothly with distance; the coverage reward uses the Bhattacharyya coefficient between predicted and ground-truth Gaussians, with a closed form that penalizes both center offset and shape mismatch. The adaptive variance is what lets one formula serve tiny icons and full-screen panels, and the sum of the two components is normalized into advantages inside GRPO. Together these pieces convert a sparse hit-or-miss signal into dense gradients everywhere, which is the mechanism the paper credits for the reported gains.","core_discovery":"The central discovery, stated on the paper's own terms, is that reward geometry determines GUI grounding quality. Each ground-truth element is converted into a Gaussian $N(x; \\mu, \\Sigma)$ with mean at the element centroid and diagonal covariance $\\Sigma = \\mathrm{diag}(\\sigma_x^2, \\sigma_y^2)$, where $\\sigma_x = \\alpha \\cdot \\text{width}$ and $\\sigma_y = \\alpha \\cdot \\text{height}$ for a fixed $\\alpha = 0.5$. Two rewards are defined: a point reward $R_{\\text{point}} = N(\\mu_p; \\mu_{gt}, \\Sigma_{gt})$ measuring localization, and a coverage reward $R_{\\text{coverage}}$ equal to the Bhattacharyya coefficient between predicted and ground-truth Gaussians, which has a closed form combining center distance and covariance similarity. The total reward $R_{\\text{total}} = \\nu R_{\\text{point}} + \\gamma R_{\\text{coverage}}$ with $\\nu = \\gamma = 1$ feeds GRPO advantage normalization over eight sampled responses. The paper reports that this continuous signal produces monotonic convergence of predictions toward target centers while sparse baselines oscillate, and that cutting the Gaussian feedback off at element boundaries costs 4.9% accuracy, supporting the claim that GUI grounding should be optimized as a dense spatial field rather than a classification task.","pith_inferences":["Beyond the paper, the same point-plus-coverage Gaussian decomposition could serve as a dense auxiliary loss during supervised fine-tuning or as an inference-time reranker, since the reward needs no RL to be computed.","The paper fixes $\\alpha$ by a global sweep; a natural test is to make variance per-element or predicted by the model, which would generalize the mechanism to interfaces where element size does not track click tolerance.","The finding that thinking tokens hurt grounding suggests that for other spatial perceptual tasks—medical image localization, map pointing, robotic reach targets—reward shape may matter more than added reasoning, a claim the paper does not test.","Because the same benchmark gains accompany a 76.9% share of remaining errors from icon semantics, the Gaussian reward may be near a ceiling for localization while semantic understanding stays the bottleneck."],"forward_implications":["A 7B model trained with Gaussian rewards can outperform a 72B model trained with sparse or distance rewards on high-resolution professional interfaces, implying reward design can substitute for model scale in GUI grounding.","Sparse binary and IoU rewards produce erratic training trajectories, while the Gaussian reward converges monotonically (average distance to target center falling from 290px to 150px), implying near-miss feedback is what stabilizes RL for spatial tasks.","Providing Gaussian feedback everywhere, not just inside target boxes, improves accuracy by 4.9%, implying optimization discontinuities at element boundaries are harmful even when the reward is continuous inside.","Removing either the point or the coverage component degrades performance (92.1% and 90.2% vs 93.3%), implying precise localization and regional targeting are complementary signals.","Explicit reasoning tokens hurt grounding accuracy (88.7% vs 93.3% with 87.7% fewer tokens), implying GUI grounding is primarily a perceptual task that does not benefit from chain-of-thought."],"supporting_citations":[{"why":"Supplies the AITW human click statistics (mu=0.111, sigma=0.429) that motivate modeling clicks as Gaussians.","marker":"(Rawles et al., 2023)"},{"why":"Provides the GRPO algorithm into which the Gaussian rewards are plugged for policy optimization.","marker":"(Shao et al., 2024)"},{"why":"Defines the ScreenSpot benchmark and the Widget Captioning training data used for evaluation and training.","marker":"(Cheng et al., 2024)"},{"why":"Contributes ScreenSpot-v2, the benchmark used for ablation and for tuning the alpha scaling factor.","marker":"(Wu et al., 2024)"},{"why":"Defines ScreenSpot-Pro, the high-resolution professional-software benchmark where the largest gains are reported.","marker":"(Li et al., 2025)"},{"why":"Supplies the UI-TARS-72B baseline that GUI-G2-7B claims to surpass by 24.7% relative on ScreenSpot-Pro.","marker":"(Qin et al., 2025)"},{"why":"Provides SE-GUI, the distance-based continuous-reward baseline that GUI-G2 is compared against.","marker":"(Yuan et al., 2025)"},{"why":"Provides the GUI-R1 binary-reward baseline whose sparse-reward dynamics are contrasted with Gaussian rewards.","marker":"(Luo et al., 2025)"},{"why":"Supplies the VLM-R1 framework used for training the 7B vision-language model.","marker":"(Shen et al., 2025)"},{"why":"Describes Qwen2.5-VL-7B-Instruct, the base model the method is built on.","marker":"(Bai et al., 2025)"}],"fun_headline_variants":["Gaussian click rewards lift 7B GUI model past 72B rival","Continuous Gaussian rewards boost GUI grounding by 24.7%","7B model outperforms 72B with Gaussian reward geometry","Gaussian rewards transform GUI grounding: 24.7% gain"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that one global scaling constant, $\\alpha = 0.5$, chosen by sweeping ScreenSpot-v2, transfers to unseen benchmarks and to every element size; if different interfaces or element types need different click tolerances, the Gaussian signal can misdirect training and the reported advantage shrinks or vanishes.","fun_headline_variants_meta":{"raw":{"variants":["Gaussian click rewards lift 7B GUI model past 72B rival","Continuous Gaussian rewards boost GUI grounding by 24.7%","7B model outperforms 72B with Gaussian reward geometry","Gaussian rewards transform GUI grounding: 24.7% gain"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000653,"raw_usage":{"total_tokens":3071,"prompt_tokens":1101,"completion_tokens":1970,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":717,"completion_tokens_details":{"reasoning_tokens":1904}},"tokens_in":717,"tokens_out":1970,"duration_ms":13446,"temperature":1.0,"reasoning_tokens":1904,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T15:23:06.285039+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same 7B model on the same data with identical Gaussian rewards but with $\\alpha = 0.25$ and $\\alpha = 0.75$, and evaluate on ScreenSpot-Pro: if either setting closes the gap over UI-TARS-72B, the claim that $\\alpha = 0.5$ encodes a general '2$\\sigma$' principle is not supported. A complementary check is to collect human click positions on ScreenSpot-Pro elements and verify that the Gaussian reward ranks two near-miss predictions in the same order as human modal click distance; any inversion would falsify the behavioral grounding of the reward.","supporting_citations":[],"review_version":1}