Pith. sign in

REVIEW 5 major objections 4 minor 8 cited by

Uncertainty-Aware GUI Agent: Adaptive Perception through Component Recommendation and Human-in-the-Loop Refinement

T0 review · 5 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read RecAgent improves mobile automation by filtering UI elements to goal-relevant candidates and asking the user when a step is ambiguous.

desk verdict Solid engineering integration, but the interaction module is never benchmarked and the perception filter's recall is load-bearing yet unmeasured. read the letter →

arxiv 2508.04025 v1 pith:HR7TFSVT submitted 2025-08-06 cs.AI

Chao Hao , Shuai Wang , Kaiwen Zhou This is my paper
classification cs.AI
keywords GUIagentmobileautomationcomponentrecommendationperceptualuncertaintydecisionhuman-in-the-loopactiongroundingAndroidWorld
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper seeks to establish that a mobile GUI agent performs better when it does not look at everything on screen. It separates failures into perceptual uncertainty (too much irrelevant UI input) and decision uncertainty (ambiguous user intent). To fight the first, RecAgent treats the current subgoal as a query and retrieves a small top-10 candidate set of UI elements via keyword, semantic, and LLM-based pathways, then uses a retrospection loop to remove failed candidates and retry. To fight the second, an Interaction Agent asks the user for clarification when a step is ambiguous and feeds the answer into the next subgoal. On AndroidWorld the SoM-based agent reaches 47.8% task success versus 40.5% for the full-input M3A baseline; on the new ComplexAction benchmark it reaches 69.3% versus 64.5%, though the paper states the benchmarks do not exercise the Interaction Agent.

What carries the argument

The Component Recommendation Module (CRM) is the load-bearing mechanism: it treats the current subgoal as a query and the UI element list as a candidate pool, runs three parallel recall pathways (keyword/fuzzy matching, BERT-style semantic similarity, and LLM-based functional-role judgment), and passes their union, capped around top-10, to the Decision Agent. The retrospection mechanism is the second half: when the Reflection Agent marks an action unsuccessful, the selected element is removed from the candidate set and the Decision Agent is reinvoked. The Interaction Agent is the third, for decision uncertainty: it predicts when user feedback is needed, generates a natural-language question,

What would settle it

On the 62 ComplexAction scenes, directly check whether the ground-truth target element survives the top-k filter. If the target is often dropped on steps where the unfiltered agent succeeds, filtering cannot be the cause of the gains; if the target is always kept but gains disappear when filtering is disabled, the retrospection mechanism is doing the work instead.

Watch

Extended reading notes

Core claim

The paper's central discovery is that GUI perception can be reframed as a retrieval problem. Instead of feeding the decision model the whole accessibility tree or every SoM-annotated element, RecAgent's Component Recommendation Module treats the current subgoal as a query over the UI element list: keyword matching catches labeled controls, semantic similarity catches paraphrased intents, and an LLM pathway catches functional-role alignment, with the union of the three forming a compact candidate set. The retrospection mechanism then tightens the loop: when reflection says an action had no effect, the chosen element is removed from the candidate set and the Decision Agent is reinvoked, so the

Load-bearing premise

The whole approach stands or falls on whether the recommendation module keeps the correct UI element inside its filtered set; keyword-only or semantic-only filtering can already drop success below the unfiltered baseline, and recall is never measured directly.

Editorial extensions

If this is right

  • GUI agents can operate on a small, goal-relevant subset of the UI tree, lowering token cost and latency without losing accuracy.
  • Grounding errors need not cascade: removing the failed candidate and re-invoking the decision agent recovers from wrong taps without a full replan.
  • Preference-dependent steps such as sweetness, delivery method, or sort order become tractable by querying the user, something purely autonomous agents cannot do.
  • Single-step action accuracy on ComplexAction gives a way to debug perception separately from planning, which could make GUI-agent development more modular.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's benchmarks do not exercise the Interaction Agent, so its benefit is currently supported only by qualitative examples; a preference-heavy task set with user satisfaction ratings would be the direct test.
  • The three-pathway union prevents single-pathway failures but also removes the ability to know when filtering is trustworthy; a confidence-gated variant that falls back to the full UI tree when pathways disagree is a testable extension.
  • All recommendation pathways rely on text or semantic content, so icon-only or image-only buttons are a likely blind spot; a variant of ComplexAction with unlabeled icons would probe this boundary.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes RecAgent, a GPT-4o-based mobile GUI agent that extends the M3A framework with two adaptive mechanisms: a Component Recommendation Module (CRM) that filters UI elements via keyword, semantic, and LLM-based pathways, and an Interaction Agent that solicits user feedback in ambiguous situations. A retrospection mechanism is used to backtrack after failed actions. The authors also introduce ComplexAction, a 62-scene dataset for single-step GUI actions in complex UIs. Experiments on AndroidWorld, MobileMiniWoB++, and ComplexAction report task/action success rates of 47.8%, 69.8%, and 69.3% respectively for the best SoM-based variant, with ablations showing that CRM and retrospection together improve over M3A and that all three CRM pathways are needed on ComplexAction.

Significance. If the results hold, the work makes a useful practical contribution: the multi-channel CRM plus retrospection is a simple, sensible way to reduce input redundancy, and ComplexAction could support more fine-grained evaluation of GUI perception. The paper should be credited for reporting, in Table 5, that keyword-only and semantic-only filtering degrade performance; this is honest negative evidence. However, the current evidence base is not yet commensurate with the abstract's claim that 'extensive experiments validate the effectiveness of our approach.' The central perception component is never directly measured, the Interaction Agent is absent from all quantitative experiments, and no stochasticity or significance information is reported. These gaps are fixable with additional experiments and more carefully scoped claims, so the paper is a credible candidate after major revision.

major comments (5)
  1. [Component Recommendation for Adaptive Perception, Eq. (2); Table 5] The core perceptual claim that CRM improves accuracy is not directly supported because recall of the target element in U'_t is never measured. Since Decision (Eq. 3) selects from U'_t, and Algorithm 1 (lines 13–16) only removes elements on failure, a CRM miss is unrecoverable for click/input actions. Table 5 shows this is a real risk: on ComplexAction, KMP-only and SMP-only drop success from 64.5% to 53.2% and 56.4%, respectively, and the paper attributes this to 'cannot guarantee accurate recall.' The full system's 69.3% depends on the union's recall, but that quantity is not reported. Please report recall@|U'_t| / recall@k against ground-truth elements, an oracle experiment that re-inserts the correct element, and an error analysis separating CRM failures from planning/grounding failures.
  2. [User Feedback via Interaction Agent; Experiments – Datasets] The Interaction Agent is one of the two headline contributions, yet it is excluded from every quantitative experiment. The Datasets subsection explicitly states that evaluation 'does not require the involvement of the Interaction Agent,' and Figure 5 is a single anecdotal visualization. The abstract's claim that 'extensive experiments validate the effectiveness of our approach' is therefore broader than the evidence. Add a task set with ambiguous user preferences, measure success with and without the Interaction Agent, query quality, and user burden; alternatively, revise the contributions and abstract to present the Interaction Agent as a design proposal rather than a quantitatively validated component.
  3. [Tables 1–3] No error bars, standard deviations, or multiple trials are reported for any result. On AndroidWorld, RecAgent (47.8%) is only 0.4 percentage points above GUI-explorer (47.4%), yet the text claims 'best results among all compared methods'; several MobileMiniWoB++ and ComplexAction gaps are also small relative to typical run-to-run variation with GPT-4o. Report means over at least 3–5 repeated runs, or equivalent significance measures, for the main comparisons and ablations, especially for the SOTA claim.
  4. [Component Recommendation for Adaptive Perception, Eq. (2); Introduction] The definition of the final perception input is underspecified. The introduction says only top-ranked elements (e.g., top 10) are passed to the Decision Agent, but Eq. (2) defines U'_t as the union of all pathway outputs with no ranking or threshold. If a top-k truncation is applied after the union, k is a free parameter that should be reported and varied; if no truncation is applied, the claimed input compression is never quantified. This matters because the CRM's benefit is specifically attributed to reducing input redundancy, and Table 5 shows the pathway set choice interacts strongly with success.
  5. [The ComplexAction Dataset] ComplexAction is a contribution, but its description is too high-level for a benchmark: 62 scenes are aggregated across five action types with no per-type counts, no scene/domain distribution, no inter-annotator agreement, and no explicit metric definition (the text says evaluation can be done by target-element match or state transition, but does not state which one produces the reported numbers). The sentence 'More details can be found in the appendix' refers to an appendix that is not present in the submitted manuscript. Please provide the full dataset statistics and evaluation protocol, or include the appendix.
minor comments (4)
  1. [Throughout] Typographical issues: 'Specipically' in the introduction; 'RecAgen' in Tables 1–3 instead of 'RecAgent'; 'Comparision' in the experiments section heading.
  2. [Algorithm 1, line 29] The condition 'if at == [COMPLETE]' is checked after executing action a_t; a COMPLETE action should presumably be detected before execution, or the condition should test the previous action.
  3. [Comparative Baselines] The text mentions MobileAgentV2 as a comparative baseline, but it does not appear in any results table. Either include its numbers or remove the mention.
  4. [Table 5] The row '✗ ✗ ✗' is labeled as 'component recommendation module is not employed,' but it is unclear whether this row corresponds to M3A or to RecAgent with CRM removed. Clarify the configuration so the ablation baseline is unambiguous.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: RecAgent's core claims are evaluated on external benchmarks and its ablations are empirical, not definitional.

full rationale

The paper's central derivation is RecAgent's component recommendation (CRM) filtering U'_t and the decision/reflection loop. No parameter is fitted to the prediction targets; the top-k count and the three recommendation pathways are hand-defined and empirically ablated. The claimed performance gains on AndroidWorld (47.8 vs. 40.5 for M3A), MobileMiniWoB++ (69.8), and ComplexAction (69.3) are measured against external baselines or the authors' own benchmark, but ComplexAction is used as a test set, not to fit CRM. Self-citations (e.g., GUI-explorer, Less is More, Niu et al., Xu et al.) appear only as related work or baseline context and are not load-bearing. The manuscript even discloses a real limitation in the Table 5 ablation: individual KMP/SMP pathways 'cannot guarantee accurate recall of the required components'; this is an empirical weakness, not a circularity. No equation reduces to its input by construction: Eq. 2 (U'_t as union) does not assert success, and Eq. 3 (Decision on U'_t) is an architectural constraint, not a fitted result. The Interaction Agent is explicitly not used in the main benchmark evaluation, so its qualitative demonstration is not a fitted prediction either. Thus no circular step is present.

Assumptions & free parameters 2 free parameters · 2 assumptions · 0 invented entities

This is an empirical systems paper rather than a derivation, so the ledger is small. The main hand-chosen constants are the top-k limit for the component recommendation module and the maximum step count. The work inherits the standard assumptions that the accessibility tree is a faithful state representation and that the chosen LLM's self-reflection and subgoal generation are reliable.

free parameters (2)
  • top-k recommended UI elements = 10
    The paper states 'only the top-ranked elements (e.g., top 10) are passed forward'; no tuning or ablation on k is reported.
  • maximum steps per task Lmax = 30
    The maximum number of steps per task is set to 30; no ablation is provided.
assumptions (2)
  • domain assumption The accessibility tree and screenshot fully capture the GUI state needed for task execution.
    The observation space is limited to parsed UI elements and screenshot; the paper inherits this from M3A/AndroidWorld.
  • domain assumption GPT-4o can reliably reflect on action success and generate correct subgoals.
    The Reflection Agent and Planning Agent rely on LLM judgment; errors in reflection would propagate to rollback decisions and task completion.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Uncertainty-Aware GUI Agent: Adaptive Perception through Component Recommendation and Human-in-the-Loop Refinement." pith.science (2026). https://pith.science/paper/HR7TFSVT

@misc{pith2026250804025,
  author       = {Pith},
  title        = {Pith review of: Uncertainty-Aware GUI Agent: Adaptive Perception through Component Recommendation and Human-in-the-Loop Refinement},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HR7TFSVT}},
  note         = {Machine review of arXiv:2508.04025}
}
read the original abstract

Graphical user interface (GUI) agents have shown promise in automating mobile tasks but still struggle with input redundancy and decision ambiguity. In this paper, we present \textbf{RecAgent}, an uncertainty-aware agent that addresses these issues through adaptive perception. We distinguish two types of uncertainty in GUI navigation: (1) perceptual uncertainty, caused by input redundancy and noise from comprehensive screen information, and (2) decision uncertainty, arising from ambiguous tasks and complex reasoning. To reduce perceptual uncertainty, RecAgent employs a component recommendation mechanism that identifies and focuses on the most relevant UI elements. For decision uncertainty, it uses an interactive module to request user feedback in ambiguous situations, enabling intent-aware decisions. These components are integrated into a unified framework that proactively reduces input complexity and reacts to high-uncertainty cases via human-in-the-loop refinement. Additionally, we propose a dataset called \textbf{ComplexAction} to evaluate the success rate of GUI agents in executing specified single-step actions within complex scenarios. Extensive experiments validate the effectiveness of our approach. The dataset and code will be available at https://github.com/Fanye12/RecAgent.

Figures

Figures reproduced from arXiv: 2508.04025 by the authors.

Figure 1
Figure 1. The two major challenges faced by existing GUI [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Schematic overview of the RecAgent architecture. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Comparison with previous methods. Schematic di [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (3 more)
Figure 3
Figure 3. Figure 3: If Successt is False, indicating that the action failed to achieve the intended effect, the agent reverts to the previ￾ous state st, removes the previously selected action from the filtered candidate set U ′ t and reinvokes the Decision Agent to select an alternative a…
Figure 4
Figure 4. Figure 4: Visualization of SoM outputs before and after us [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Visualization of the Interaction Agent. The left im [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Uncertainty Decomposition for Clarification Seeking in LLM Agents

    cs.AI 2026-06 unverdicted novelty 6.0 of 10

    A prompt-based uncertainty decomposition separates action confidence from request uncertainty to enable clarification seeking in LLM agents, yielding F1 gains of 73% and 36% over baselines on two new underspecified be...

  2. Agent4POI: Agentic Context-Conditioned Affordance Reasoning for Multimodal Point-of-Interest Recommendation

    cs.IR 2026-04 unverdicted novelty 6.0 of 10

    Agent4POI generates context-conditioned multimodal affordance representations via a four-phase LLM agent, achieving 23.2% relative gains over baselines on POI benchmarks with reduced degradation under context shifts.

  3. Dark Patterns Meet GUI Agents: LLM Agent Susceptibility to Manipulative Interfaces and the Role of Human Oversight

    cs.HC 2025-09 conditional novelty 6.0 of 10

    GUI agents frequently fall for deceptive interface designs, often without recognizing them, and human supervision of agents improves avoidance only partially while introducing new attention and workload costs.

  4. Towards Agents That Know When They Don't Know: Uncertainty as a Control Signal for Structured Reasoning

    cs.AI 2025-09 conditional novelty 6.0 of 10

    An LLM agent using retrieval and summary uncertainty as training rewards and inference filters produces more factual, useful multi-omics summaries and better downstream survival predictions.

  5. Dynamic Collaboration of Multi-Language Models based on Minimal Complete Semantic Units

    cs.AI 2025-08 conditional novelty 6.0 of 10

    MCSU-based vocabulary alignment plus distance-based dynamic selection (DDS) lets several LLMs vote token-by-token, beating single models and prior ensemble baselines on multiple reasoning benchmarks without training.

  6. Plover: Steering GUI Agents through Plan-Centric Interaction

    cs.AI 2026-07 conditional novelty 5.0 of 10

    An expert repairing visible plans rescued 23 of 26 failed GUI automation runs, turning 17 into full and 6 into partial successes.

  7. GUI-C$^2$: Coarse-to-Fine GUI Grounding via Difficulty-Aware Reinforcement Learning

    cs.CV 2026-05 unverdicted novelty 5.0 of 10

    GUI-C² pairs a difficulty-scoring data pipeline with an area-gated coarse-to-fine RL mechanism to improve GUI grounding accuracy and training stability.

  8. Distribution-Specific Learning for Joint Salient and Camouflaged Object Detection

    cs.CV 2025-08 conditional novelty 5.0 of 10

    A shared network with about 2,000 decoder-specific parameters and a saliency-filtered, size-balanced training set reaches state-of-the-art accuracy on both salient and camouflaged object detection simultaneously.

Reference graph

Works this paper leans on

42 extracted references · 11 canonical work pages · cited by 8 Pith papers

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al

    Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F. L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  4. [4]

    Bai, J.; Bai, S.; Yang, S.; Wang, S.; Tan, S.; Wang, P.; Lin, J.; Zhou, C.; and Zhou, J. 2023. Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond. arXiv preprint arXiv:2308.12966

    arXiv 2023

  5. [5]

    Bai, S.; Chen, K.; Liu, X.; Wang, J.; Ge, W.; Song, S.; Dang, K.; Wang, P.; Wang, S.; Tang, J.; et al. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923

    arXiv 2025

  6. [6]

    Bobadilla, J.; Ortega, F.; Hernando, A.; and Gutiérrez, A. 2013. Recommender systems survey. Knowledge-Based Systems, 46: 109--132

    work page 2013

  7. [7]

    Chen, G.; Zhou, X.; Shao, R.; Lyu, Y.; Zhou, K.; Wang, S.; Li, W.; Li, Y.; Qi, Z.; and Nie, L. 2025 a . Less is More: Empowering GUI Agent with Context-Aware Simplification. arXiv:2507.03730

    work page Pith review arXiv 2025

  8. [8]

    Chen, J.; Yuen, D.; Xie, B.; Yang, Y.; Chen, G.; Wu, Z.; Yixing, L.; Zhou, X.; Liu, W.; Wang, S.; Zhou, K.; Shao, R.; Nie, L.; Wang, Y.; HAO, J.; Wang, J.; and Shao, K. 2025 b . SPA - BENCH : A COMPREHENSIVE BENCHMARK FOR SMARTPHONE AGENT EVALUATION . In The Thirteenth International Conference on Learning Representations

    work page 2025

Show all 42 references
  1. [9]

    Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv preprint arXiv:1810.04805

    2018 arXiv

  2. [10]

    Gou, B.; Wang, R.; Zheng, B.; Xie, Y.; Chang, C.; Shu, Y.; Sun, H.; and Su, Y. 2025. Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents. In The Thirteenth International Conference on Learning Representations

    2025

  3. [11]

    Hong, W.; Wang, W.; Lv, Q.; Xu, J.; Yu, W.; Ji, J.; Wang, Y.; Wang, Z.; Zhang, Y.; Li, J.; Xu, B.; Dong, Y.; Ding, M.; and Tang, J. 2024. CogAgent: A Visual Language Model for GUI Agents. arXiv:2312.08914

    2024 arXiv

  4. [12]

    Isinkaye, F.; Folajimi, Y.; and Ojokoh, B. 2015. Recommendation systems: Principles, methods and evaluation. Egyptian Informatics Journal, 16(3): 261--273

    2015

  5. [13]

    T.; and Zhang, C

    Jiang, W.; Zhuang, Y.; Song, C.; Yang, X.; Zhou, J. T.; and Zhang, C. 2025. AppAgentX: Evolving GUI Agents as Proficient Smartphone Users. arXiv:2503.02268

    2025 arXiv

  6. [14]

    M.; Zhou, K.; Dong, J.; Guo, K.; Li, X.; Chen, Z.; Wang, J.; and Hao, J

    Li, Y.; Shao, X.; Zhang, J.; Wang, H.; Brunswic, L. M.; Zhou, K.; Dong, J.; Guo, K.; Li, X.; Chen, Z.; Wang, J.; and Hao, J. 2025. Generative Models in Decision Making: A Survey. arXiv:2502.17100

    2025

  7. [15]

    Li, Y.; Zhang, C.; Yang, W.; Fu, B.; Cheng, P.; Chen, X.; Chen, L.; and Wei, Y. 2024. AppAgent v2: Advanced Agent for Flexible Mobile Interactions. arXiv:2408.11824

    2024

  8. [16]

    Z.; Guu, K.; Pasupat, P.; Shi, T.; and Liang, P

    Liu, E. Z.; Guu, K.; Pasupat, P.; Shi, T.; and Liang, P. 2018. Reinforcement Learning on Web Interfaces Using Workflow-Guided Exploration. arXiv:1802.08802

    2018 arXiv

  9. [17]

    Liu, X.; Qin, B.; Liang, D.; Dong, G.; Lai, H.; Zhang, H.; Zhao, H.; Iong, I. L.; Sun, J.; Wang, J.; Gao, J.; Shan, J.; Liu, K.; Zhang, S.; Yao, S.; Cheng, S.; Yao, W.; Zhao, W.; Liu, X.; Liu, X.; Chen, X.; Yang, X.; Yang, Y.; Xu, Y.; Yang, Y.; Wang, Y.; Xu, Y.; Qi, Z.; Dong, ...

    2024 arXiv

  10. [18]

    D.; Xie, Z.; Kim, S.; Zhang, R.; Yu, T.; Tanjim, M.; Ahmed, N

    Nguyen, D.; Chen, J.; Wang, Y.; Wu, G.; Park, N.; Hu, Z.; Lyu, H.; Wu, J.; Aponte, R.; Xia, Y.; Li, X.; Shi, J.; Chen, H.; Lai, V. D.; Xie, Z.; Kim, S.; Zhang, R.; Yu, T.; Tanjim, M.; Ahmed, N. K.; Mathur, P.; Yoon, S.; Yao, L.; Kveton, B.; Nguyen, T. H.; Bui, T.; Zhou, T.; Ro...

    2024

  11. [19]

    Niu, W.; Xie, Z.; Sun, Y.; He, W.; Xu, M.; and Hao, C. 2025. LangTime: A Language-Guided Unified Model for Time Series Forecasting with Proximal Policy Optimization. arXiv:2503.08271

    2025 arXiv

  12. [20]

    Rawles, C.; Clinckemaillie, S.; Chang, Y.; Waltz, J.; Lau, G.; Fair, M.; Li, A.; Bishop, W.; Li, W.; Campbell-Ajala, F.; Toyama, D.; Berry, R.; Tyamagundlu, D.; Lillicrap, T.; and Riva, O. 2024. AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents. arXiv:2405.14573

    2024 arXiv

  13. [21]

    Seed., B. 2025. Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learning. arXiv:2504.13914

    2025

  14. [22]

    Shi, T.; Karpathy, A.; Fan, L.; Hernandez, J.; and Liang, P. 2017. World of Bits: An Open-Domain Platform for Web-Based Agents. In Precup, D.; and Teh, Y. W., eds., Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learni...

    2017

  15. [23]

    I.; and etal

    Team, G.; Georgiev, P.; Lei, V. I.; and etal. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv:2403.05530

    2024 arXiv

  16. [24]

    Wang, J.; Xu, H.; Jia, H.; Zhang, X.; Yan, M.; Shen, W.; Zhang, J.; Huang, F.; and Sang, J. 2024 a . Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaboration. arXiv preprint arXiv:2406.01014

    2024 arXiv

  17. [25]

    Wang, J.; Xu, H.; Ye, J.; Yan, M.; Shen, W.; Zhang, J.; Huang, F.; and Sang, J. 2024 b . Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perception. arXiv preprint arXiv:2401.16158

    2024 arXiv

  18. [26]

    Wang, S.; Liu, W.; Chen, J.; Zhou, Y.; Gan, W.; Zeng, X.; Che, Y.; Yu, S.; Hao, X.; Shao, K.; Wang, B.; Wu, C.; Wang, Y.; Tang, R.; and Hao, J. 2025 a . GUI Agents with Foundation Models: A Comprehensive Survey. arXiv:2411.04890

    2025 arXiv

  19. [27]

    Wang, Y.; Zhang, H.; Tian, J.; and Tang, Y. 2024 c . Ponder & Press: Advancing Visual GUI Agent towards General Computer Control. arXiv:2412.01268

    2024 arXiv

  20. [28]

    Wang, Z.; Xu, H.; Wang, J.; Zhang, X.; Yan, M.; Zhang, J.; Huang, F.; and Ji, H. 2025 b . Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks. arXiv preprint arXiv:2501.11733

    2025 arXiv

  21. [29]

    J.-J.; Jiang, S.; Liu, Y.; Zhang, Y.; and Liu, Y

    Wen, H.; Li, Y.; Liu, G.; Zhao, S.; Yu, T.; Li, T. J.-J.; Jiang, S.; Liu, Y.; Zhang, Y.; and Liu, Y. 2024. AutoDroid: LLM-powered Task Automation in Android. arXiv:2308.15272

    2024 arXiv

  22. [30]

    Wu, D.; Wang, H.; Yu, W.; Zhang, Y.; Chang, K.-W.; and Yu, D. 2025. LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory. In The Thirteenth International Conference on Learning Representations

    2025

  23. [31]

    P.; et al

    Wu, Z.; Wu, Z.; Xu, F.; Wang, Y.; Sun, Q.; Jia, C.; Cheng, K.; Ding, Z.; Chen, L.; Liang, P. P.; et al. 2024. OS-ATLAS: A Foundation Action Model for Generalist GUI Agents. arXiv preprint arXiv:2410.23218

    2024 arXiv

  24. [32]

    Xie, B.; Shao, R.; Chen, G.; Zhou, K.; Li, Y.; Liu, J.; Zhang, M.; and Nie, L. 2025 a . GUI-explorer: Autonomous Exploration and Mining of Transition-aware Knowledge for GUI Agent. In Annual Meeting of the Association for Computational Linguistics (ACL)

    2025

  25. [33]

    Xie, Y.; Li, Z.; Shao, R.; Chen, G.; Zhou, K.; Li, Y.; Jiang, D.; and Nie, L. 2025 b . Mirage-1: Augmenting and Updating GUI Agent with Hierarchical Multimodal Skills. arXiv preprint arXiv:2506.10387

    2025 arXiv

  26. [34]

    Xu, J.; Zhao, Y.; Bao, W.; and Hao, C. 2025 a . Fault diagnosis of motor bearing in complex scenarios based on Mamba and Indicative Contrastive Learning. Engineering Applications of Artificial Intelligence, 146: 110216

    2025

  27. [35]

    Xu, Y.; Wang, Z.; Wang, J.; Lu, D.; Xie, T.; Saha, A.; Sahoo, D.; Yu, T.; and Xiong, C. 2025 b . Aguvis: Unified Pure Vision Agents for Autonomous GUI Interaction. arXiv:2412.04454

    2025 arXiv

  28. [36]

    Yang, J.; Zhang, H.; Li, F.; Zou, X.; Li, C.; and Gao, J. 2023. Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V. arXiv:2310.11441

    2023 arXiv

  29. [37]

    Yang, Y.; Wang, Y.; Li, D.; Luo, Z.; Chen, B.; Huang, C.; and Li, J. 2025. Aria-UI: Visual Grounding for GUI Instructions. arXiv:2412.16256

    2025 arXiv

  30. [38]

    Zhang, C.; Yang, Z.; Liu, J.; Han, Y.; Chen, X.; Huang, Z.; Fu, B.; and Yu, G. 2023. AppAgent: Multimodal Agents as Smartphone Users. arXiv:2312.13771

    2023 arXiv

  31. [39]

    Zhang, Y.; Sun, R.; Chen, Y.; Pfister, T.; Zhang, R.; and Arik, S. 2024. Chain of Agents: Large Language Models Collaborating on Long-Context Tasks. arXiv:2406.02818

    2024 arXiv

  32. [40]

    Zheng, B.; Gou, B.; Kil, J.; Sun, H.; and Su, Y. 2024. GPT-4V(ision) is a Generalist Web Agent, if Grounded. In Forty-first International Conference on Machine Learning

    2024

  33. [41]

    Zhou, Y.; Wang, S.; Dai, S.; Jia, Q.; Du, Z.; Dong, Z.; and Xu, J. 2025. CHOP: Mobile Operating Assistant with Constrained High-frequency Optimized Subtask Planning. arXiv:2503.03743

    2025 arXiv

  34. [42]

    Zhou, Z.; Ning, X.; Hong, K.; Fu, T.; Xu, J.; Li, S.; Lou, Y.; Wang, L.; Yuan, Z.; Li, X.; Yan, S.; Dai, G.; Zhang, X.-P.; Dong, Y.; and Wang, Y. 2024. A Survey on Efficient Inference for Large Language Models. arXiv:2404.14294

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.