Building a Stable Planner: An Extended Finite State Machine Based Planning Module for Mobile GUI Agent

Fanglin Mo; Haoxuan Zhu; Junzhe Chen; Xuming Hu

REVIEW 4 major objections 5 minor 2 cited by

Building a Stable Planner: An Extended Finite State Machine Based Planning Module for Mobile GUI Agent

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SPlanner claims that hand-authored extended finite state machines for mobile apps let a generalist vision-language model plan reliably, lifting AndroidWorld success from 35.0% to 63.8% without any fine-tuning.

desk verdict Plausible and clean engineering idea, but the 28.8-point gain is not yet attributable to EFSM planning rather than human benchmark knowledge — needs artifacts and ablations. read the letter →

arxiv 2505.14141 v1 pith:S2S75YFF submitted 2025-05-20 cs.AI

Fanglin Mo , Junzhe Chen , Haoxuan Zhu , Xuming Hu This is my paper

classification cs.AI

keywords mobileGUIagenttaskplanningextendedfinitestatemachineEFSMvisionlanguagemodelAndroidWorldsymbolicplannerplug-and-play

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that a symbolic planner built from hand-authored app models can solve the planning weakness of mobile GUI agents. It models each app's screens, operations, and core functions as an extended finite state machine, so that a user instruction can be converted into a concrete path through the app and then into a natural-language plan. The authors show that adding this plan to the prompt of a generalist vision-language model raises task success on AndroidWorld from 35.0% to 63.8%, a gain larger than several purpose-built agents achieve. If true, the result matters because it offers a cheap, interpretable, plug-and-play route to more reliable mobile automation without retraining.

What carries the argument

The central object is the extended finite state machine (EFSM): a finite state machine augmented with variables, guard conditions, and update functions, used here so that each app screen is a state, each transition is a GUI event, and each primary function is an output action. The argument is carried by turning a user instruction into a sequence of target primary functions, traversing the EFSM with BFS to get a stable transition path, and then having an LLM polish that path into a natural-language plan that is dropped into the executor's prompt.

What would settle it

Build EFSMs for the 20 AndroidWorld apps using only publicly available app documentation, without seeing the 116 task instructions, then rerun SPlanner with Qwen2.5-VL-72B; if success falls back toward the 35% no-plan baseline, the reported gain depends on task-aware human modeling rather than the EFSM traversal mechanism itself.

Watch

Extended reading notes

Core claim

The central claim, stated on the authors' terms, is that the bottleneck for mobile GUI agents is not visual grounding alone but a lack of knowledge about how to operate apps, and that this knowledge can be encoded as extended finite state machines. Given a user instruction, an LLM identifies the target app and the required primary functions; a BFS-based solver finds a path through the EFSM from launch state to those functions; a second LLM pass rewrites the path into a short natural-language plan. Feeding that plan to Qwen2.5-VL-72B as a prompt produces 63.8% task success on AndroidWorld, compared with 35.0% for the same model without a plan, and beats AgentS2 (54.3%) and V-Droid (59.5%) while nearly matching UI-TARS1.5 (64.2%).

Load-bearing premise

The load-bearing premise is that the manually constructed EFSMs are complete and unbiased, covering every screen and operation the 116 AndroidWorld tasks require, without having been designed with knowledge of those tasks.

Editorial extensions

If this is right

Using SPlanner, a generalist VLM that has never been fine-tuned for Android can outperform several specialized mobile GUI agents on AndroidWorld.
Because the plan is expressed in natural language, the same module should transfer to text-only LLM-based agents, not just VLMs.
The plan reduces the executor's tendency to get lost in recurring screen loops, since each step corresponds to a state transition.
Adding SPlanner requires no architectural changes or task-specific training; only the user instruction and the EFSM knowledge base are needed.
Hand modeling an app takes one to two hours, so expanding to new apps is feasible but remains a manual cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Inference: The size of the gain (28.8 points) likely reflects the injection of human knowledge about app structure, not the EFSM formalism itself; a fair test would compare against another way of encoding the same knowledge.
Inference: If EFSM construction can be automated from app UI dumps or documentation, the approach could scale to arbitrary apps and provide a continuously updatable planning layer that avoids retraining on new app versions.
Inference: The planner's value may be highest for tasks with clear multi-step navigation; for tasks requiring open-ended judgment, the plan reaches its limit and the VLM's own reasoning dominates.
Inference: The EFSM plan could be used as a structured prior to train smaller VLMs, potentially distilling the planning benefit into a model that needs no external planner at inference time.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's my read. The genuinely new thing is packaging app logic as EFSMs and using BFS plus an LLM to turn a parsed instruction into a plan for a VLM. That's a sensible and clean idea, and the formalization in Section 3.1 is precise enough to implement. I also give the authors credit for being candid: Section 5 lists concrete failure reasons and Section 7 admits the manual modeling burden.

The soft spots are the ones you'd guess. The headline result is one bar on one benchmark, with no repeated runs, no confidence intervals, no per-task breakdown, no code, and no EFSM artifacts. Since the EFSMs are hand-authored, the 28.8-point gain could largely be human benchmark knowledge leaking through the state and event definitions. The paper never addresses this. It mentions manual modeling as a cost issue but not as a validity threat.

The missing ablation matters most. If you take the same LLM, skip the EFSM entirely, and have it write a step-by-step plan directly from the instruction, how much of the gain remains? Without that comparison, 'EFSM traversal' and 'generic plan injection' are completely confounded. That single experiment would tell us whether the symbolic machinery is pulling its weight or just providing a nicer prompt.

On the positive side, the method is well specified and the result is plausible. The authors' failure analysis is consistent with what I'd expect from a plan that gets you to the right screen but leaves the last steps to the VLM. And the one-to-two hours per app is a believable cost for a small app suite. I just don't think a single success-rate bar, with no variance and no artifacts, supports the stability claim that's in the title.

The stress-test note is on target: this is a benchmark-specific knowledge problem, not a statistical quibble. It's the kind of issue that can be fixed by releasing the EFSM files, a held-out modeling protocol, and the ablation. If those come, this becomes a useful practical result. For now, treat the magnitude of the gain as unverified.

Recommendation: send it to peer review, but with a clear request for artifacts, error bars, and the LLM-only plan ablation. I'd read it, discuss it in a reading group, and not cite it until the numbers are reproducible.

Referee Report

4 major / 5 minor

Summary. The paper proposes SPlanner, a plug-and-play planning module for mobile GUI agents. SPlanner represents each target mobile application as an extended finite state machine (EFSM) whose states are screens, events are natural-language operation sequences, and output actions are the application's primary functions. At inference time, DeepSeek-V3 parses a user instruction into target primary functions, a BFS-based solver traverses the relevant EFSMs to produce an execution path, and a second DeepSeek-V3 step polishes the path into a natural-language plan. A vision-language model (Qwen2.5-VL-72B) then executes the task by combining the plan with screenshots to emit GUI actions. On AndroidWorld, the authors report a 63.8% task success rate for SPlanner with Qwen2.5-VL-72B, a 28.8 percentage point improvement over the same VLM without planning, outperforming AgentS2 and V-Droid and nearly matching UI-TARS1.5, all without fine-tuning the VLM.

Significance. If the reported result is reproducible, SPlanner would be a notable contribution: it demonstrates that a symbolic, interpretable planning module built on manually authored EFSMs can substantially improve the task-completion rate of a generalist VLM on a dynamic mobile GUI benchmark without any model training. The EFSM formalism is clearly described, the three-stage plan-generation pipeline is simple, and the paper is honest about the manual modeling cost. However, the current evidence does not yet establish the central claim. The single success-rate number lacks statistical grounding, the EFSMs are not released, no ablation separates the EFSM traversal from the injected LLM-written plan, and the benchmark-specificity risk from hand-authored primary functions and events is not addressed. These issues are fixable within the manuscript's scope, but they must be resolved before the headline result can be trusted.

major comments (4)

[Section 4.3] The headline result of 63.8% versus 35.0% is reported as a single number with no confidence intervals, no number of trials, no random seeds, and no per-task breakdown. AndroidWorld is dynamic and the VLM executor is stochastic, so run-to-run variance could be substantial; without repeated runs or a per-task success table, the claimed 28.8 percentage point gain cannot be distinguished from noise. The paper also does not state whether the Qwen2.5-VL-72B baseline of 35.0% was measured by the authors in the same harness or taken from another source, which is essential for a fair comparison.
[Section 3.1 and Section 5] The EFSMs are manually constructed (one to two hours per app, as stated in Section 5) and are not released. The paper provides no state/event/transition counts, no coverage analysis over the 116 AndroidWorld tasks, and no evidence that the modelers were blind to the task list. Because each primary function and event is a natural-language description chosen by hand, the mapping from user instruction to plan could encode benchmark-specific knowledge rather than general application logic. Without releasing the EFSM definitions or evaluating on a held-out set of tasks not seen during modeling, the planned generalization claim is not yet supported.
[Section 3.2 and Section 4.3] No ablation isolates the contribution of the EFSM itself. The full pipeline combines DeepSeek-V3 instruction parsing, BFS traversal over hand-built EFSMs, and DeepSeek-V3 path polishing. A natural control would be to compare against a plan generated directly by an LLM from the user instruction without any EFSM, and against the raw EFSM path without the polishing step. Without such ablations, the observed improvement could be due to the injected natural-language plan alone, rather than to the stable EFSM traversal that the paper emphasizes.
[Section 4.2] Implementation details are insufficient for reproducibility: the paper does not report VLM sampling parameters, the exact action space, the step-limit handling, the prompts used for instruction parsing and path polishing, or the number of evaluation episodes. Since comparisons to AgentS2, V-Droid, and UI-TARS1.5 are taken from their original papers, differences in evaluation protocols may affect comparability; the authors should describe their own evaluation harness precisely and ideally release code and EFSM artifacts.

minor comments (5)

[Abstract and Section 1] The abstract says "control logits" where "control logic" is clearly meant, and the introduction contains the typo "knowlege" and the fragment "guard conditions, and , which enable"; these should be corrected.
[Section 2.1] The sentence "While these methods have yield positive results" should read "have yielded positive results".
[Section 4.1] There is a duplicated period after "planning.." near the end of the benchmark description, and the phrase "on a dynamic and realistic benchmarks" in the contributions list should be singular.
[References] Several reference entries are truncated with phrases such as "and 1 others" (for example, the Qwen2.5-VL, GPT-4o, UI-TARS, and AndroidWorld entries); the author lists should be completed.
[Section 5] The claim that SPlanner "can also be applied to text-only LLM-based GUI agents" is a reasonable conjecture but is not tested anywhere in the paper; it should be explicitly labeled as future work or supported by an experiment.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the plan is a deterministic traversal of manually authored EFSMs, and the reported gain is an empirical benchmark result rather than a restatement of an input or fitted quantity.

full rationale

The derivation chain is compositional and not circular: the LLM parses a user instruction into target primary functions (Eq. 5), a BFS solver traverses hand-authored EFSMs to produce an execution path (Eq. 6), an LLM polishes that path into a natural-language plan, and an external VLM executes the plan. None of these equations define the output in terms of the measured task success rate, and no parameter is fitted to AndroidWorld outcomes. The EFSMs are explicitly manual knowledge inputs (Section 3.1; Section 5: 'The modeling process typically takes one to two hours per application'), and Section 7 acknowledges this as a scalability limitation. No load-bearing self-citation or imported uniqueness theorem appears in the reference list. The absence of released EFSM artifacts and of a held-out modeling protocol raises a genuine external-validity risk—human benchmark-specific knowledge could be embedded in the hand-authored states, events, and primary functions—but that is a potential leakage/correctness concern, not a circular reduction exhibited by the paper's own equations or citations. Since no step reduces by construction to its own inputs, the circularity score is 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 2 invented entities

The system contributes no derived constants; its behavior is determined by hand-written app models and prompt designs, none of which are released. The AndroidWorld evaluation provides external grounding, but the manual modeling step is an unverified, potentially benchmark-informed input.

free parameters (4)

Per-app EFSM definitions (states, events, transitions, primary functions)
Manually authored for each modeled app, 1 to 2 hours per app per Section 5; their content determines every generated plan, and no artifacts are released.
Instruction-parsing prompt for DeepSeek-V3
Hand-designed and unreleased; parsing quality directly controls whether the correct EFSM and primary functions are selected.
Path-polishing prompt
Hand-designed prompt converting raw transitions into natural-language steps; no examples or template provided.
Choice of DeepSeek-V3 and Qwen2.5-VL-72B
Implementation choices that affect the headline number; no comparison with other parser or executor pairs, so the gain is specific to this configuration.

assumptions (5)

domain assumption Mobile app behavior can be fully captured by a finite set of screens (states) and predefined transitions with guard conditions and variable updates.
Adopted in Section 3.1 (Eq. 1) without evidence that all AndroidWorld tasks fit this representation.
ad hoc to paper The manually defined primary function set covers every user intent appearing in the benchmark tasks.
Assumed by the EFSM solving step; no coverage analysis is provided in Sections 3.2 or 5.
domain assumption DeepSeek-V3 reliably maps natural-language instructions to the correct EFSM and target primary functions.
Used in Eq. 5; parsing accuracy is never evaluated.
domain assumption LLM-based path polishing preserves the executable semantics of the raw EFSM path.
Assumed in Section 3.2; no verification that polished plans still correspond to valid transitions.
standard math BFS returns a valid execution path when one exists in the model.
Standard graph search property, uncontroversial.

invented entities (2)

EFSM knowledge base F of modeled apps
purpose: Encodes per-app states, transitions, and primary functions used to generate plans.
No EFSM files, state tables, or transition lists are published, so the artifact's existence and coverage cannot be independently checked.
Primary function vocabulary A
purpose: Interface between parsed user instructions and EFSM transitions.
The vocabulary is author-defined and unreleased; the paper gives only examples such as 'take a photo' and 'record a video'.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Building a Stable Planner: An Extended Finite State Machine Based Planning Module for Mobile GUI Agent." pith.science (2026). https://pith.science/paper/S2S75YFF

@misc{pith2026250514141,
  author       = {Pith},
  title        = {Pith review of: Building a Stable Planner: An Extended Finite State Machine Based Planning Module for Mobile GUI Agent},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/S2S75YFF}},
  note         = {Machine review of arXiv:2505.14141}
}

read the original abstract

Mobile GUI agents execute user commands by directly interacting with the graphical user interface (GUI) of mobile devices, demonstrating significant potential to enhance user convenience. However, these agents face considerable challenges in task planning, as they must continuously analyze the GUI and generate operation instructions step by step. This process often leads to difficulties in making accurate task plans, as GUI agents lack a deep understanding of how to effectively use the target applications, which can cause them to become "lost" during task execution. To address the task planning issue, we propose SPlanner, a plug-and-play planning module to generate execution plans that guide vision language model(VLMs) in executing tasks. The proposed planning module utilizes extended finite state machines (EFSMs) to model the control logits and configurations of mobile applications. It then decomposes a user instruction into a sequence of primary function modeled in EFSMs, and generate the execution path by traversing the EFSMs. We further refine the execution path into a natural language plan using an LLM. The final plan is concise and actionable, and effectively guides VLMs to generate interactive GUI actions to accomplish user tasks. SPlanner demonstrates strong performance on dynamic benchmarks reflecting real-world mobile usage. On the AndroidWorld benchmark, SPlanner achieves a 63.8% task success rate when paired with Qwen2.5-VL-72B as the VLM executor, yielding a 28.8 percentage point improvement compared to using Qwen2.5-VL-72B without planning assistance.

Figures

Figures reproduced from arXiv: 2505.14141 by the authors.

**Figure 2.** Task success rates of SPlanner and baseline methods on AndroidWorld. For clarity of presentation, darker [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

MobiBench: Multi-Branch, Modular Benchmark for Mobile GUI Agents
cs.AI 2025-12 conditional novelty 8.0 of 10

MobiBench reaches near-human offline evaluation fidelity for mobile GUI agents by accepting any valid action at each step, and enables modular attribution of performance to agent components.
SWIRL: A Staged Workflow for Interleaved Reinforcement Learning in Mobile GUI Control
cs.AI 2025-08 conditional novelty 4.0 of 10

A multi-agent RL workflow that interleaves single-agent updates, applied to mobile GUI control, achieves SOTA zero-shot performance and a +14.8 MATH500 gain.

Reference graph

Works this paper leans on

24 extracted references · 1 canonical work pages · cited by 2 Pith papers

[1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Saaket Agashe, Kyle Wong, Vincent Tu, Jiachen Yang, Ang Li, and Xin Eric Wang. 2025. Agent s2: A compositional generalist-specialist framework for computer use agents. arXiv preprint arXiv:2504.00906

arXiv 2025
[4]

Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923

arXiv 2025
[5]

Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, and Zhiyong Wu. 2024. https://arxiv.org/abs/2401.10935 Seeclick: Harnessing gui grounding for advanced visual gui agents . Preprint, arXiv:2401.10935

arXiv 2024
[6]

Gaole Dai, Shiqi Jiang, Ting Cao, Yuanchun Li, Yuqing Yang, Rui Tan, Mo Li, and Lili Qiu. 2025. Advancing mobile gui agents: A verifier-driven approach to practical deployment. arXiv preprint arXiv:2503.15937

arXiv 2025
[7]

Boyu Gou, Ruohan Wang, Boyuan Zheng, Yanan Xie, Cheng Chang, Yiheng Shu, Huan Sun, and Yu Su. 2024. Navigating the digital world as humans do: Universal visual grounding for gui agents. arXiv preprint arXiv:2410.05243

arXiv 2024
[8]

Xu Huang, Weiwen Liu, Xiaolong Chen, Xingmei Wang, Hao Wang, Defu Lian, Yasheng Wang, Ruiming Tang, and Enhong Chen. 2024. https://arxiv.org/abs/2402.02716 Understanding the planning of llm agents: A survey . Preprint, arXiv:2402.02716

arXiv 2024

Show all 24 references

[9]

Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276

2024 arXiv
[10]

Sunjae Lee, Junyoung Choi, Jungjae Lee, Munim Hasan Wasi, Hojun Choi, Steve Ko, Sangeun Oh, and Insik Shin. 2024. Mobilegpt: Augmenting llm with human-like app memory for mobile task automation. In Proceedings of the 30th Annual International Conference on Mobile Computing and...

2024
[11]

Wei Li, William Bishop, Alice Li, Chris Rawles, Folawiyo Campbell-Ajala, Divya Tyamagundlu, and Oriana Riva. 2024. On the effects of data scale on computer control agents. arXiv e-prints, pages arXiv--2406

2024
[12]

Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437

2024 arXiv
[13]

Bo Liu, Yuqian Jiang, Xiaohan Zhang, Qiang Liu, Shiqi Zhang, Joydeep Biswas, and Peter Stone. 2023. Llm+ p: Empowering large language models with optimal planning proficiency. arXiv preprint arXiv:2304.11477

2023 arXiv
[14]

Xinbei Ma, Zhuosheng Zhang, and Hai Zhao. 2024. Coco-agent: A comprehensive cognitive mllm agent for smartphone gui automation. arXiv preprint arXiv:2402.11941

2024 arXiv
[15]

Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, and 1 others. 2025. Ui-tars: Pioneering automated gui interaction with native agents. arXiv preprint arXiv:2501.12326

2025 arXiv
[16]

Christopher Rawles, Sarah Clinckemaillie, Yifan Chang, Jonathan Waltz, Gabrielle Lau, Marybeth Fair, Alice Li, William Bishop, Wei Li, Folawiyo Campbell-Ajala, and 1 others. 2024. Androidworld: A dynamic benchmarking environment for autonomous agents. arXiv preprint arXiv:2405.14573

2024 arXiv
[17]

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

2022
[18]

Hao Wen, Yuanchun Li, Guohong Liu, Shanhui Zhao, Tao Yu, Toby Jia-Jun Li, Shiqi Jiang, Yunhao Liu, Yaqin Zhang, and Yunxin Liu. 2024. https://arxiv.org/abs/2308.15272 Autodroid: Llm-powered task automation in android . Preprint, arXiv:2308.15272

2024 arXiv
[19]

Yifan Xu, Xiao Liu, Xueqiao Sun, Siyi Cheng, Hao Yu, Hanyu Lai, Shudan Zhang, Dan Zhang, Jie Tang, and Yuxiao Dong. 2024. Androidlab: Training and systematic benchmarking of android autonomous agents. arXiv preprint arXiv:2410.24024

2024 arXiv
[20]

Yuhao Yang, Yue Wang, Dongxu Li, Ziyang Luo, Bei Chen, Chao Huang, and Junnan Li. 2024. Aria-ui: Visual grounding for gui instructions. arXiv preprint arXiv:2412.16256

2024 arXiv
[21]

Zhun Yang, Adam Ishay, and Joohyung Lee. 2023. Coupling large language models with logic programming for robust and general reasoning from text. arXiv preprint arXiv:2307.07696

2023 arXiv
[22]

Chaoyun Zhang, Shilin He, Jiaxu Qian, Bowen Li, Liqun Li, Si Qin, Yu Kang, Minghua Ma, Guyue Liu, Qingwei Lin, and 1 others. 2024 a . Large language model-brained gui agents: A survey. arXiv preprint arXiv:2411.18279

2024 arXiv
[23]

Jiwen Zhang, Jihao Wu, Yihua Teng, Minghui Liao, Nuo Xu, Xiao Xiao, Zhongyu Wei, and Duyu Tang. 2024 b . Android in the zoo: Chain-of-action-thought for gui agents. arXiv preprint arXiv:2403.02713

2024 arXiv
[24]

Boyuan Zheng, Boyu Gou, Jihyung Kil, Huan Sun, and Yu Su. 2024. Gpt-4v (ision) is a generalist web agent, if grounded. arXiv preprint arXiv:2401.01614

2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Saaket Agashe, Kyle Wong, Vincent Tu, Jiachen Yang, Ang Li, and Xin Eric Wang. 2025. Agent s2: A compositional generalist-specialist framework for computer use agents. arXiv preprint arXiv:2504.00906

arXiv 2025

[4] [4]

Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923

arXiv 2025

[5] [5]

Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, and Zhiyong Wu. 2024. https://arxiv.org/abs/2401.10935 Seeclick: Harnessing gui grounding for advanced visual gui agents . Preprint, arXiv:2401.10935

arXiv 2024

[6] [6]

Gaole Dai, Shiqi Jiang, Ting Cao, Yuanchun Li, Yuqing Yang, Rui Tan, Mo Li, and Lili Qiu. 2025. Advancing mobile gui agents: A verifier-driven approach to practical deployment. arXiv preprint arXiv:2503.15937

arXiv 2025

[7] [7]

Boyu Gou, Ruohan Wang, Boyuan Zheng, Yanan Xie, Cheng Chang, Yiheng Shu, Huan Sun, and Yu Su. 2024. Navigating the digital world as humans do: Universal visual grounding for gui agents. arXiv preprint arXiv:2410.05243

arXiv 2024

[8] [8]

Xu Huang, Weiwen Liu, Xiaolong Chen, Xingmei Wang, Hao Wang, Defu Lian, Yasheng Wang, Ruiming Tang, and Enhong Chen. 2024. https://arxiv.org/abs/2402.02716 Understanding the planning of llm agents: A survey . Preprint, arXiv:2402.02716

arXiv 2024