Pith. sign in

REVIEW 3 major objections 7 minor 51 references

PG-Agent: An Agent Powered by Page Graph

T0 review · 3 major / 7 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Page graphs lift GUI agent accuracy to 59.5 percent

desk verdict Real but modest systems contribution; the paper shows retrieval helps, but never isolates the page graph structure as the cause. read the letter →

arxiv 2509.03536 v1 pith:QUWCDLQS submitted 2025-08-27 cs.AI cs.HC

Weizhi Chen , Ziwei Wang , Leyang Yang , Sheng Zhou , Xiaoxuan Tang , Jiajun Bu , Yong Li , Wei Jiang This is my paper
classification cs.AIcs.HC
keywords GUIagentpagegraphretrieval-augmentedgenerationmultimodallargelanguagemodeltaskdecompositionnavigationwebmobile
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GUI agents usually learn from linear episode traces, so they miss the fact that screens connect through actions into a graph. This paper argues that reconstructing those episodes into a page graph—where nodes are distinct screens and edges carry the action sequences that move between them—gives an agent a structured memory of what actions lead where. From that graph, retrieval-augmented generation pulls short guidelines (action queues plus the tasks they accomplish) and feeds them into the sub-task planning and decision stages of a multi-agent framework. The paper reports that this raises action accuracy and step success on AITW, Mind2Web, and GUI Odyssey, with gains surviving even when only a small fraction of episodes is used to build the graph. If correct, the page graph is a fine-tuning-free way to transfer navigation knowledge across tasks, websites, and apps.

What carries the argument

The central object is the page graph: a directed graph whose nodes are unique screens, matched by a dual semantic-and-pixel similarity check, and whose edges bundle consecutive in-page actions with the task they serve. Retrieval uses vectorized screen summaries to find similar nodes and breadth-first search to enumerate reachable action queues; the retrieved (action queue, achievable tasks) tuples are the guidelines injected into the sub-task planning and decision agents. The graph is what lets the agent see multiple possible continuations from a screen, not just the next step of one recorded trajectory.

What would settle it

Take a sample of episodes, have human annotators label each action as page-jumping or not and each pair of screenshots as same or different pages, then compare with the model's Eq. (2) and Eq. (6) outputs; if agreement is near chance, or if downstream accuracy does not drop when those judgments are randomized, the page-graph mechanism is not the source of the gains. A cheaper check is to re-run PG-Agent on AITW with the page graph's edges shuffled while keeping the same number of guidelines: if accuracy stays roughly constant, the evidence for graph structure collapses.

Watch

Extended reading notes

Core claim

The paper's central claim is that the page-transition structure of GUIs is reusable prior knowledge, and that converting chain-like episodes into a page graph makes that knowledge accessible to a multimodal agent without retraining. The construction pipeline decides whether an action jumps to a new page, merges visually and semantically similar screens into one node, and stores the intervening in-page actions on the edge. At inference time, the agent summarizes the current screen, retrieves nearby nodes, runs a breadth-first search over outgoing edges, and injects the resulting 'action X can accomplish task Y' guidelines into both the sub-task planner and the decision agent. On all three ben

Load-bearing premise

The entire page graph is built on the multimodal model's unverified yes/no judgments about whether an action changed the page and whether two screenshots are the same page; if those judgments are systematically wrong, every retrieved guideline inherits the error.

Editorial extensions

If this is right

  • GUI navigation knowledge can be transferred across tasks, websites, and domains without fine-tuning, by swapping or augmenting the page graph.
  • Guidelines matter most when the base model is uncertain about non-click actions: SELECT and TYPE operations improve sharply when guidelines are present.
  • A page graph built from a modest random sample of episodes is enough to yield most of the benefit, so collecting large expert traces is not a prerequisite.
  • Injecting guidelines into the decision agent helps more on mobile tasks, while the sub-task planning agent benefits more on web cross-task and cross-domain splits, suggesting placement should be tuned per environment.
  • Page graphs compress episode collections: thousands of screenshots reduce to hundreds of nodes and edges, making the prior knowledge base cheap to store.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural stress test the paper does not run: corrupt the page graph deliberately, merging random nodes or dropping edges, and measure how guideline quality degrades; that would isolate whether the graph structure itself, rather than the number of retrieved guidelines, drives the gains.
  • The graph construction's binary decisions—did the action change the page, and are these two screenshots the same page?—are never scored against human labels; replacing those MLLM judgments with a learned classifier or ground-truth transitions from app activity logs could make the pipeline both cheaper and more reliable.
  • Because the page graph is scenario-specific, the same framework could be pointed at accessibility or enterprise software by building graphs from logs of successful assistive-use sessions, turning the RAG memory into a lightweight personalization layer.
  • The retrieval ranking assumes vectorized screen summaries sort pages by functional similarity; testing retrieval precision directly would tell whether better embeddings or graph-aware retrieval could push the gains further.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes PG-Agent, a GUI navigation agent that converts collected navigation episodes into a page graph, then uses retrieval-augmented generation (RAG) to extract action guidelines from this graph and inject them into a multi-agent planning and decision pipeline. The page graph construction pipeline has three stages: page-jump determination, node-similarity checking, and graph update, all based on MLLM judgments over screenshots and their summaries. The authors evaluate on AITW, Mind2Web, and GUI Odyssey, reporting improvements over API-based baselines and over a no-guidelines ablation. The central claim is that page-graph-based retrieval improves navigation accuracy and step success, including in cross-domain settings, and that the method remains effective with limited training episodes.

Significance. If the claimed results hold, the paper makes a useful practical contribution: it offers a fine-tuning-free way to transfer GUI navigation knowledge across tasks and websites, and it explicitly models page-transition structure rather than treating episodes as independent chains. The cross-domain Mind2Web experiments are a genuine generalization test, and the decision to build graphs from a small sampled subset of episodes is a realistic and commendable setup. The paper also ships pseudocode for the two core modules. However, the central contribution attribution is currently under-supported: the main ablation removes all guidelines, so it does not isolate the value of the graph structure itself, and several headline differences are small enough that statistical noise cannot be ruled out without seed variation or confidence intervals.

major comments (3)
  1. [§4.3, Tables 4 and 6; §3.1, Eqs. (2), (6), (11)] The ablations compare PG-Agent with and without retrieved guidelines, but this does not test whether the page-graph structure is what helps. The guidelines are action queues with task labels, and a flat memory of the same training episodes indexed by the same BGE-M3 screen summaries could produce nearly identical inputs without any graph operations. The paper explicitly distinguishes itself from ExpeL-style trajectory retrieval in §2.1, yet no flat-retrieval baseline is included. Without such a baseline, the observed gains over w/o GL (e.g., AITW overall 59.5 vs 58.3; Mind2Web Cross-Task Step SR 52.9 vs 50.2; Cross-Website 48.7 vs 47.6; Cross-Domain 53.3 vs 50.4) could be attributed to generic retrieval of relevant example actions rather than to node merging, edge construction, or BFS over the graph. This is the central attribution gap for the paper's headline claim.
  2. [§3.1, Eqs. (2) and (6)] The page graph construction depends on two binary MLLM judgments: whether an action triggers a page jump and whether two screenshots show different pages. These decisions gate every node and edge in the graph, but the paper never evaluates their accuracy, e.g., against human annotations. If the model systematically merges distinct pages or fails to detect page jumps, the retrieved guidelines inherit that corruption. Since this is a load-bearing premise for the entire RAG mechanism, the authors should provide a small human-annotated validation set and report precision/recall or agreement rates for these judgments.
  3. [§4.3, Tables 4 and 6] No confidence intervals, multiple seeds, or significance tests are reported, and several key deltas are within 1–3 points. For example, AITW w/o GL is 58.3 vs PG-Agent 59.5, and Mind2Web Cross-Website Step SR is 47.6 vs 48.7. These differences may or may not be stable. Because the core claim rests on these comparisons, the authors should run multiple seeds or at least bootstrap confidence intervals over test episodes, and report whether the observed differences are reliable.
minor comments (7)
  1. [§4.1] The text says 'we evaluate our PG-Agent on two GUI agent datasets' but then lists three datasets (AITW, Mind2Web, GUI Odyssey).
  2. [Table 5 and surrounding text] The metric is inconsistently written as both 'Op.F1' and 'Opt.F1'. Also, the text states that the SELECT-type Opt.F1 decreases, but the table shows PG-Agent's SELECT scores are higher than w/o GL in all three splits; this seems to be a typo and should be corrected.
  3. [§3.2] 'Warper paradigm' appears to be a typo; likely 'wrapper paradigm' is intended.
  4. [§2.1] 'They treat the traffic data as independent trajectory chains' should likely read 'trajectory data'.
  5. [§3.1, Eq. (4) and surrounding notation] The graph is introduced as G = (N, V), but edges are later denoted E; the notation should be made consistent. Algorithm 1 also uses 'G = G ∪ (N_before, E_new, N_new)' without defining the tuple semantics precisely in the main text.
  6. [Figure 1 and Figure 4] Figure 1 contains unexplained text '012690450158' that appears to be an artifact. Figure 4's axis labels and color map are dense; a clearer legend or a textual description of the guideline-count distribution would help.
  7. [§4.4, Table 7] The full-episode page graph does not consistently outperform the random-sampling version; the paper mentions this but does not analyze why. Since this result is somewhat counterintuitive for a 'more data is better' expectation, a brief explanation would be useful.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the page graph is built from training-split episodes and evaluated on held-out benchmark tasks, so the claimed gains are empirical rather than forced by construction.

full rationale

The paper is an empirical systems paper, not a derivation. The page graph is constructed from sampled training-split episodes ("we randomly sample 1/10 episodes from training split to construct concise page graphs") and the agent is evaluated on the standard held-out splits of AITW, Mind2Web, and GUI Odyssey, so the final action predictions are not derived from the test instances by construction. Equations (2) and (6) are model judgments, not parameters fitted to the evaluation metric; the retrieval in Eq. (10) uses a fixed vectorizer and prompts, and the hyperparameters (number of guidelines, BFS layers, retrieval count) are set globally rather than tuned on the test labels. The ablation removes the retrieved guidelines and shows a small accuracy drop, which tests whether injecting retrieved knowledge helps at all; the absence of a flat-trajectory-retrieval baseline means the graph-specific contribution is not fully isolated, but that is an experimental-control limitation, not circularity. No load-bearing self-citation is present: the multi-agent architecture is attributed to external prior work (Mobile-Agent-v2, Mobile-Agent-E), and the page-graph construction is original to this paper. No fitted input is renamed as a prediction, and no known result is merely relabeled. The central claim is therefore self-contained as an empirical comparison, with no step that reduces to its own inputs by definition.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central contribution rests on MLLM judgment quality and retrieval relevance; these are the real costs not paid for upstream. The hyperparameters are hand-tuned per dataset, and the transferability of guidelines is assumed rather than proven.

free parameters (4)
  • retrieved node count n = 4
    Set as the number of retrieved nodes of page similarity search (Section 4.1). No sensitivity analysis is reported.
  • BFS depth l = 3
    Maximum number of layers for BFS during guideline retrieval (Section 4.1). Chosen by hand, no sensitivity analysis.
  • maximum guidelines k = 20 for AITW/GUI Odyssey, 10 for Mind2Web
    Set according to guideline distribution statistics (Figure 4 and Section 4.1), not by validation on held-out data.
  • episode sampling ratio = 1/10 random sample
    Only 1/10 of training episodes are used to construct page graphs (Section 4.1 and Table 8); no random seed is reported.
assumptions (4)
  • domain assumption GUI pages and actions form a graph; each episode is a path in this graph
    Stated in Section 1 and Section 3.1 as the justification for reconstructing episodes into page graphs.
  • domain assumption MLLM determinations of page jump and page similarity are correct
    Equations (2) and (6) gate graph construction; the paper provides no evaluation of the accuracy of these binary judgments.
  • domain assumption Similar screen summaries imply similar pages for retrieval
    Retrieval in Eq. (10) uses vectorized screen summaries; embedding similarity is treated as semantic page similarity without validation.
  • domain assumption Guidelines extracted from training-set episodes transfer to held-out tasks
    The agent relies on retrieved action queues and tasks from the page graph, assuming relevance across tasks and websites even when the target scenario has no page graph (Mind2Web cross-domain).

how reviews work

0 comments
Cite this review

Pith. "Pith review of PG-Agent: An Agent Powered by Page Graph." pith.science (2026). https://pith.science/paper/QUWCDLQS

@misc{pith2026250903536,
  author       = {Pith},
  title        = {Pith review of: PG-Agent: An Agent Powered by Page Graph},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QUWCDLQS}},
  note         = {Machine review of arXiv:2509.03536}
}
read the original abstract

Graphical User Interface (GUI) agents possess significant commercial and social value, and GUI agents powered by advanced multimodal large language models (MLLMs) have demonstrated remarkable potential. Currently, existing GUI agents usually utilize sequential episodes of multi-step operations across pages as the prior GUI knowledge, which fails to capture the complex transition relationship between pages, making it challenging for the agents to deeply perceive the GUI environment and generalize to new scenarios. Therefore, we design an automated pipeline to transform the sequential episodes into page graphs, which explicitly model the graph structure of the pages that are naturally connected by actions. To fully utilize the page graphs, we further introduce Retrieval-Augmented Generation (RAG) technology to effectively retrieve reliable perception guidelines of GUI from them, and a tailored multi-agent framework PG-Agent with task decomposition strategy is proposed to be injected with the guidelines so that it can generalize to unseen scenarios. Extensive experiments on various benchmarks demonstrate the effectiveness of PG-Agent, even with limited episodes for page graph construction.

Figures

Figures reproduced from arXiv: 2509.03536 by the authors.

Figure 1
Figure 1. Illustration of PG-Agent. (i) Convert chain-like episodes into a semantically rich page graph; (ii) With page graph as GUI prior knowledge, RAG technology assists the tailored multi-agent workflow to enhance GUI navigation. from diverse devices, but usually treat them as independent items. For example, the navigation tasks on GUI involving sequences of multi-step operations across different pages, where each step pr… view at source ↗
Figure 2
Figure 2. The overall pipeline of page graph construction. It comprises three stages: page jump determination, node similarity [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The framework of multi-agent workflow. It comprises two parts: RAG pipeline and multi-agent group. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: The data distribution of guidelines in AITW dataset. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Examples of page graph visualizations of scenarios in AITW dataset. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Cases of original images sharing the same node. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: The data distribution of guidelines in Mind2Web [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: The data distribution of guidelines in GUI Odyssey [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Navigation process of PG-Agent. C Case Study In this section, we select a part of cases for detailed analysis. When facing the Click operation, we use red rectangle to mark ground￾truth, and green rectangle to mark the location where PG-Agent clicks. As shown in [PITH…
Figure 10
Figure 10. Figure 10: Cases determined as failure steps in "Single" scenario. [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

51 extracted references · 11 canonical work pages

  1. [1]

    Meta AI. 2024. Llama 3. https://github.com/meta-llama/llama3 Accessed: 2024-11-12

    work page 2024

  2. [2]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. 2025. Qwen2. 5-VL Technical Report. arXiv preprint arXiv:2502.13923 (2025)

    arXiv 2025

  3. [3]

    Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu

  4. [4]

    Minghao Chen, Yihang Li, Yanting Yang, Shiyu Yu, Binbin Lin, and Xiaofei He. 2024. AutoManual: Constructing Instruction Manuals by LLM Agents via Interactive Environmental Learning. arXiv:2405.16247 [cs.AI] https://arxiv.org/ abs/2405.16247

    arXiv 2024

  5. [5]

    Tong Chen, Hongwei Wang, Sihao Chen, Wenhao Yu, Kaixin Ma, Xinran Zhao, and Hongming and Zhang. 2024. Dense X Retrieval: What Retrieval Granularity Should We Use?. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Miami, Florida, USA, 15159–15177. doi:10.18653/v1/2024.emnl...

    doi:10.18653/v1/2024.emnlp-main.845 2024

  6. [6]

    Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. 2024. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. arXiv preprint arXiv:2404.16821 (2024)

    arXiv 2024

  7. [7]

    Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Li YanTao, Jianbing Zhang, and Zhiyong Wu. 2024. SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Bangkok, Thailand, 9313–9332. ht...

    work page 2024

  8. [8]

    Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samual Stevens, Boshi Wang, Huan Sun, and Yu Su. 2023. Mind2Web: Towards a Generalist Agent for the Web. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023

    work page 2023

Show all 51 references
  1. [9]

    Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. 2024. The Faiss library. (2024). arXiv:2401.08281 [cs.LG]

    2024 arXiv

  2. [10]

    Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson. 2025. From Local to Global: A Graph RAG Approach to Query-Focused Summarization. arXiv:2404.16130 [cs.CL] https://arxiv....

    2025 arXiv

  3. [11]

    Yao Fu, Dong-Ki Kim, Jaekyeom Kim, Sungryull Sohn, Lajanugen Logeswaran, Kyunghoon Bae, and Honglak Lee. 2024. AutoGuide: Automated Generation and Selection of Context-Aware Guidelines for Large Language Model Agents. arXiv:2403.08978 [cs.CL] https://arxiv.org/abs/2403.08978

    2024 arXiv

  4. [12]

    Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxiao Dong, Ming Ding, et al. 2024. Cogagent: A visual language model for gui agents. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 14281–14290

    2024

  5. [13]

    Ross, and Alireza Fathi

    Ziniu Hu, Ahmet Iscen, Chen Sun, Zirui Wang, Kai-Wei Chang, Yizhou Sun, Cordelia Schmid, David A. Ross, and Alireza Fathi. 2023. REVEAL: Retrieval- Augmented Visual-Language Pre-Training with Multi-Source Multimodal Knowl- edge Memory. arXiv:2212.05221 [cs.CV] https://arxiv.or...

    2023 arXiv

  6. [14]

    Hanyu Lai, Xiao Liu, Iat Long Iong, Shuntian Yao, Yuxuan Chen, Pengbo Shen, Hao Yu, Hanchen Zhang, Xiaohan Zhang, Yuxiao Dong, and Jie Tang. 2024. AutoWebGLM: A Large Language Model-based Web Navigating Agent. InPro- ceedings of the 30th ACM SIGKDD Conference on Knowledge Disc...

    2024

  7. [15]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2021. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. arXiv:2005.1140...

    2021 arXiv

  8. [16]

    Yanda Li, Chi Zhang, Wanqi Yang, Bin Fu, Pei Cheng, Xin Chen, Ling Chen, and Yunchao Wei. 2024. AppAgent v2: Advanced Agent for Flexible Mobile Interactions. arXiv:2408.11824 [cs.HC] https://arxiv.org/abs/2408.11824

    2024

  9. [17]

    Zijian Li, Qingyan Guo, Jiawei Shao, Lei Song, Jiang Bian, Jun Zhang, and Rui Wang. 2024. Graph Neural Network Enhanced Retrieval for Question Answering of LLMs. arXiv:2406.06572 [cs.CL] https://arxiv.org/abs/2406.06572

    2024 arXiv

  10. [18]

    Kevin Qinghong Lin, Linjie Li, Difei Gao, Zhengyuan Yang, Shiwei Wu, Zechen Bai, Weixian Lei, Lijuan Wang, and Mike Zheng Shou. 2024. Showui: One vision- language-action model for gui visual agent. arXiv preprint arXiv:2411.17465 (2024)

    2024 arXiv

  11. [19]

    Mario Linares-Vásquez, Kevin Moran, and Denys Poshyvanyk. 2017. Continuous, evolutionary and large-scale: A new perspective for automated mobile app testing. In 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 399–410

    2017

  12. [20]

    Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang

    Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2023. Lost in the Middle: How Language Models Use Long Contexts. arXiv:2307.03172 [cs.CL] https://arxiv.org/abs/2307.03172

    2023 arXiv

  13. [21]

    Xinwei Long, Jiali Zeng, Fandong Meng, Zhiyuan Ma, Kaiyan Zhang, Bowen Zhou, and Jie Zhou. 2024. Generative Multi-Modal Knowledge Retrieval with Large Language Models. arXiv:2401.08206 [cs.IR] https://arxiv.org/abs/2401.08206

    2024 arXiv

  14. [22]

    Quanfeng Lu, Wenqi Shao, Zitao Liu, Fanqing Meng, Boxuan Li, Botong Chen, Siyuan Huang, Kaipeng Zhang, Yu Qiao, and Ping Luo. 2024. GUI Odyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devices. arXiv:2406.08451 [cs.CV] https://arxiv.org/abs/2406.08451

    2024 arXiv

  15. [23]

    Yadong Lu, Jianwei Yang, Yelong Shen, and Ahmed Awadallah. 2024. OmniParser for Pure Vision Based GUI Agent. arXiv:2408.00203 [cs.CV] https://arxiv.org/ abs/2408.00203

    2024 arXiv

  16. [24]

    Dehai Min, Nan Hu, Rihui Jin, Nuo Lin, Jiaoyan Chen, Yongrui Chen, Yu Li, Guilin Qi, Yun Li, Nijun Li, and Qianren Wang. 2024. Exploring the Impact of Table- to-Text Methods on Augmenting LLM-based Question Answering with Domain Hybrid Data. In Proceedings of the 2024 Conferen...

    2024

  17. [25]

    Dehai Min, Nan Hu, Rihui Jin, Nuo Lin, Jiaoyan Chen, Yongrui Chen, Yu Li, Guilin Qi, Yun Li, Nijun Li, and Qianren Wang. 2024. Exploring the Impact of Table-to-Text Methods on Augmenting LLM-based Question Answering with Domain Hybrid Data. arXiv:2402.12869 [cs.CL] https://arx...

    2024 arXiv

  18. [26]

    Sai Munikoti, Anurag Acharya, Sridevi Wagle, and Sameera Horawalavithana

  19. [27]

    Ahmed, Puneet Mathur, Seunghyun Yoon, Lina Yao, Branislav Kveton, Thien Huu Nguyen, Trung Bui, Tianyi Zhou, Ryan A

    Dang Nguyen, Jian Chen, Yu Wang, Gang Wu, Namyong Park, Zhengmian Hu, Hanjia Lyu, Junda Wu, Ryan Aponte, Yu Xia, Xintong Li, Jing Shi, Hongjie Chen, Viet Dac Lai, Zhouhang Xie, Sungchul Kim, Ruiyi Zhang, Tong Yu, Mehrab Tanjim, Nesreen K. Ahmed, Puneet Mathur, Seunghyun Yoon, ...

    2024

  20. [28]

    OpenAI. 2023. GPT-4V(ision) System Card. (1 2023). doi:10.26181/25479208.v1

    2023 doi

  21. [29]

    OpenAI. 2024. GPT-4 Technical Report. arXiv:2303.08774 [cs.CL] https://arxiv. org/abs/2303.08774

    2024 arXiv

  22. [30]

    Boci Peng, Yun Zhu, Yongchao Liu, Xiaohe Bo, Haizhou Shi, Chuntao Hong, Yan Zhang, and Siliang Tang. 2024. Graph Retrieval-Augmented Generation: A Survey. arXiv:2408.08921 [cs.AI] https://arxiv.org/abs/2408.08921

    2024 arXiv

  23. [31]

    Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, et al. 2025. UI-TARS: Pioneering Automated GUI Interaction with Native Agents. arXiv preprint arXiv:2501.12326 (2025)

    2025 arXiv

  24. [32]

    Lillicrap

    Christopher Rawles, Alice Li, Daniel Rodriguez, Oriana Riva, and Timothy P. Lillicrap. 2023. Android in the Wild: A Large-Scale Dataset for Android Device Control. CoRR abs/2307.10088 (2023). arXiv:2307.10088

    2023 arXiv

  25. [33]

    Zhengliang Shi, Shuo Zhang, Weiwei Sun, Shen Gao, Pengjie Ren, Zhumin Chen, and Zhaochun Ren. 2024. Generate-then-Ground in Retrieval-Augmented Gen- eration for Multi-hop Question Answering. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguis...

    2024 doi

  26. [34]

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems 36 (2023), 8634–8652

    2023

  27. [35]

    Junyang Wang, Haiyang Xu, Haitao Jia, Xi Zhang, Ming Yan, Weizhou Shen, Ji Zhang, Fei Huang, and Jitao Sang. 2025. Mobile-agent-v2: Mobile device operation assistant with effective navigation via multi-agent collaboration. Advances in Neural Information Processing Systems 37 (...

    2025

  28. [36]

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Jirong Wen. 2024. A survey on large language model based autonomous agents. Frontiers of Computer Science 18, 6 (March 2024). d...

    2024 doi

  29. [37]

    Rossi, Alexa Siu, Ruiyi Zhang, and Tyler Derr

    Yu Wang, Nedim Lipka, Ryan A. Rossi, Alexa Siu, Ruiyi Zhang, and Tyler Derr

  30. [38]

    Zhenhailong Wang, Haiyang Xu, Junyang Wang, Xi Zhang, Ming Yan, Ji Zhang, Fei Huang, and Heng Ji. 2025. Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks. arXiv:2501.11733 [cs.CL] https://arxiv.org/abs/2501.11733

    2025 arXiv

  31. [39]

    Zhiyong Wu, Chengcheng Han, Zichen Ding, Zhenmin Weng, Zhoumianze Liu, Shunyu Yao, Tao Yu, and Lingpeng Kong. 2024. Os-copilot: Towards generalist computer agents with self-improvement. arXiv preprint arXiv:2402.07456 (2024)

    2024 arXiv

  32. [40]

    arXiv:2308.11730 [cs.CL] https://arxiv.org/abs/2308.11730 MM ’25, October 27–31, 2025, Dublin, Ireland Weizhi Chen et al

    Knowledge Graph Prompting for Multi-Document Question Answering. arXiv:2308.11730 [cs.CL] https://arxiv.org/abs/2308.11730 MM ’25, October 27–31, 2025, Dublin, Ireland Weizhi Chen et al

    2025 arXiv

  33. [41]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR)

    2023

  34. [42]

    Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. 2024. MiniCPM-V: A GPT-4V Level MLLM on Your Phone. arXiv preprint arXiv:2408.01800 (2024)

    2024 arXiv

  35. [43]

    Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao

  36. [44]

    arXiv:2310.11441 [cs.CV] https://arxiv.org/abs/2310.11441

    Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V. arXiv:2310.11441 [cs.CV] https://arxiv.org/abs/2310.11441

    arXiv

  37. [45]

    Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. 2024. Expel: Llm agents are experiential learners. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 38. 19632–19642

    2024

  38. [46]

    # Episodes

    Siyun Zhao, Yuqing Yang, Zilong Wang, Zhiyuan He, Luna K. Qiu, and Lili Qiu. 2024. Retrieval Augmented Generation (RAG) and Beyond: A Compre- hensive Survey on How to Make your LLMs use External Data More Wisely. arXiv:2409.14924 [cs.CL] https://arxiv.org/abs/2409.14924 A Stru...

    2024 arXiv

  39. [47]

    Antonio Jimeno Yepes, Yao You, Jan Milczek, Sebastian Laverde, and Renyu Li

  40. [48]

    arXiv:2402.05131 [cs.CL] https://arxiv.org/abs/2402.05131

    Financial Report Chunking for Effective Retrieval Augmented Generation. arXiv:2402.05131 [cs.CL] https://arxiv.org/abs/2402.05131

    arXiv

  41. [49]

    Chi Zhang, Zhao Yang, Jiaxuan Liu, Yucheng Han, Xin Chen, Zebiao Huang, Bin Fu, and Gang Yu. 2023. Appagent: Multimodal agents as smartphone users. arXiv preprint arXiv:2312.13771 (2023)

    2023 arXiv

  42. [2023]

    arXiv:2311.12289 [cs.CL] https://arxiv.org/abs/2311

    ATLANTIC: Structure-Aware Retrieval-Augmented Language Model for Interdisciplinary Science. arXiv:2311.12289 [cs.CL] https://arxiv.org/abs/2311. 12289

    arXiv

  43. [2024]

    arXiv:2402.03216 [cs.CL]

    BGE M3-Embedding: Multi-Lingual, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation. arXiv:2402.03216 [cs.CL]

    arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.