Pith. sign in

REVIEW 4 major objections 6 minor 24 references

For local computer-use agents, extra inference compute mostly reshuffles failures instead of raising success.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 03:22 UTC pith:TR7MXGHE

load-bearing objection Solid empirical map of local CUA inference-time scaling: extra compute mostly redistributes failures (loops/stalls → premature false success) rather than raising the success ceiling, with clear H/S/structure/P tradeoffs on OSWorld. the 4 major comments →

arxiv 2607.28573 v1 pith:TR7MXGHE submitted 2026-07-30 cs.AI

Rethinking Inference-Time Scaling in Local Computer-Use Agents: Failure Modes and Compute Tradeoffs

Woongkyu Lee , Jungwook Choi This is my paper
classification cs.AI
keywords computer-use agentsinference-time scalinglocal multimodal modelsOSWorldfailure modescontext lengthtwo-stage planningcompute tradeoffs
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Local computer-use agents matter for privacy and cost, but it is unclear whether the inference-time scaling tricks that help big proprietary agents also help small open models under tight hardware limits. This paper runs a systematic study on several 7B–30B multimodal agents on OSWorld, varying history length, max steps, single- versus two-stage structure, and number of parallel plans. It finds that a little history stabilizes trajectories and lifts accuracy, yet longer context and longer horizons quickly saturate: loops and step-limit stalls fall, while premature false successes rise, and token cost climbs. Two-stage planning plus grounding can add planning and format-parsing failures that parallel plans only partly buy back at high cost. The practical claim is that efficient local agents need selective compute, failure-aware control, and designs matched to what local models can actually do—not uniform scaling.

Core claim

Additional inference-time computation in local computer-use agents often yields diminishing returns while changing failure modes rather than raising the success ceiling. Contextual scaling (history) and temporal scaling (max steps) improve trajectory stability by cutting repetitive loops and max-step stalls, but gains saturate as failures shift toward premature false successes; structural two-stage decomposition can introduce planning and formatting overhead that parallel plan scaling only partially offsets at substantial token cost.

What carries the argument

Four inference-time scaling dimensions—contextual (history image length H), temporal (max decoding steps S), structural (single agent vs planning–grounding), and parallel (number of candidate plans P)—paired with step and prompt-token cost metrics and a failure-mode cohort analysis that tracks how errors redistribute as compute grows.

Load-bearing premise

The benchmark’s screenshot-only success labels and the paper’s failure categories, especially “premature false success,” cleanly measure the models’ task-completion judgment rather than partly reflecting how the evaluator or termination rules are defined.

What would settle it

On the same OSWorld tasks and models, if increasing H beyond 4 or S beyond 50 produced large, sustained success-rate gains without a matching rise in premature false successes—or if a carefully prompted local two-stage agent beat its single-agent baseline at similar token cost—the central diminishing-returns and failure-shift story would be undermined.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Prefer moderate history (around one to four screenshots) over unbounded context expansion for local GUI agents.
  • Treat max-step budgets as bounded by local reasoning capacity; longer horizons alone will not fix wrong trajectories.
  • Default to structurally simple single-pass agents unless parallel plan diversity is worth the token cost.
  • Invest in loop detection, trajectory revision, and completion verification rather than uniform extra decode compute.
  • Design multi-stage local frameworks around format reliability and planning skill of the actual small models used.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Termination and “done” judgment may be a higher-leverage bottleneck for local CUAs than raw action selection, so training or verifying completion signals could beat more history tokens.
  • Selective memory (keep task-relevant screens, drop redundant ones) is a natural next experiment given saturation past H=4.
  • The same failure-shift pattern may appear in other long-horizon tool agents when small models get more steps without better self-correction.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper empirically studies inference-time scaling for resource-constrained local computer-use agents (CUAs) on OSWorld (361 Ubuntu tasks, screenshot-only). It factors scaling into contextual history length H∈{0,1,4,8}, temporal max steps S∈{15,50,100}, structural form (single-agent vs two-stage planning–grounding with fixed GTA-1-7B grounder), and parallel plan count P, evaluating Qwen3-VL-8B/30B-A3B, UI-TARS-1.5-7B, and OpenCUA-7B. The central claim is that extra local compute often yields diminishing returns while redistributing failures: modest history stabilizes trajectories and lifts success, but gains saturate near H=4 as failures shift from loops/max-step stalls toward premature false successes; larger S similarly reduces stalls without raising the success ceiling; two-stage decomposition can add planning/format overhead that parallel plans only partly offset at high token cost. The authors conclude that efficient local CUAs need selective compute allocation, failure-aware control, and frameworks matched to local-model limits.

Significance. Local CUAs are practically important for privacy, cost, and deployability, and most prior inference-time scaling evidence comes from stronger proprietary agents. A multi-model, multi-axis ablation on a standard benchmark with explicit accuracy–cost and failure-cohort analysis is a useful contribution if the measurement chain holds. Strengths include a clear experimental taxonomy, consistent patterns across several open models, qualitative trajectory/format-error examples, and concrete deployment guidelines (moderate H, bounded S, structural simplicity, selective P). The distinctive scientific claim is not ordinary saturation alone, but systematic failure-mode redistribution under local compute; that claim, if cleanly supported, would usefully reorient local-agent design toward context quality and termination/progress control rather than uniform test-time scaling.

major comments (4)
  1. [§3.3–3.4; §4.1 Findings 2–3; Fig. 2(b), 4(d)] §3.3–3.4 and Findings 2–3 (Fig. 2(b), Fig. 4(d)): the load-bearing failure-mode story depends on “premature false success” cleanly measuring weak task-completion judgment. The manuscript does not fully specify the labeling rule—e.g., whether this is (i) the agent emitting an explicit DONE/terminate action on an incomplete state, (ii) OSWorld verifier rejection of a near-complete state after agent stop, or (iii) interaction with prompt/history conventions that encourage early stop. Please define the cohort taxonomy operationally, report how often false successes are agent-initiated terminations versus verifier/termination artifacts, and, if possible, add a small manual audit or action-level breakdown. Without this separation, the claim that contextual/temporal scaling mainly redistributes failures (rather than changing when stalls stop being counted) is under-supported, and §5’s failure-a
  2. [§3.1; §4.2 Finding 4; Fig. 5–6] §3.1, §4.2 Finding 4 and Fig. 5–6: the conclusion that structural decomposition “introduces overhead” in local agents is only partly identified. The two-stage setup fixes GTA-1-7B as grounder and uses Qwen3 planners that may not be instruction-aligned to the required plan schema; Fig. 6 shows format/parsing failures as a material failure mass. Please disentangle (a) genuine planning/reasoning limits, (b) format-following/schema compliance, and (c) planner–grounder interface mismatch—e.g., by reporting format-error rates conditional on valid parses, a same-model single-agent vs planner-only comparison, or a lightweight format-repair/reprompt baseline. Otherwise the anti-decomposition recommendation may mainly reflect brittle scaffolding rather than an inherent local two-stage limit.
  3. [§3.4; §4; Fig. 2, 4, 5] §3.4 / §4 results overall: success rates and cohort shifts are reported as point estimates on 361 tasks with no uncertainty (bootstrap CIs, seed variation, or task-level variance). Several key comparisons are small in absolute terms (e.g., H=4 vs H=8; S=15 vs 50/100; two-stage recovery with P). For a systems empirical paper whose central claim is saturation plus failure redistribution, please add uncertainty estimates for main bars/cohort fractions and state whether runs are single-shot. This is needed to judge whether saturation and mode shifts are stable effects or within noise.
  4. [§3.2.3; §4.2 Finding 5; Fig. 5(c)] §3.2.3 and §4.2 Finding 5: parallel scaling is studied only as multiple planner candidates inside the two-stage stack, while related work (Agent S3 trajectory rollouts, etc.) also uses trajectory-level selection for single agents. Given that single-agent baselines already dominate on the accuracy–cost frontier (Fig. 5(c)), the paper should either (i) include a modest single-agent parallel/trajectory-selection control at comparable token budget, or (ii) explicitly narrow the claim to “parallel plan selection inside local two-stage agents,” and avoid general wording that parallel scaling is the recovery mechanism for local CUAs as a class.
minor comments (6)
  1. [Fig. 4; §4.1] Fig. 4 caption/body inconsistency: the prose refers to panels for success, steps, tokens, and failure cohorts, but the in-text labels (a–d) and the caption ordering are easy to misread; align panel letters with the narrative in §4.1 Finding 3.
  2. [Fig. 2; Fig. 4; §3.3] Normalize or absolute-report token/step metrics more transparently. “Normalized” average steps and prompt tokens (Fig. 2(c,d), Fig. 4) need an explicit baseline (e.g., H=0 or S=15 = 1.0) in every caption.
  3. [§3.4] State decoding hyperparameters (temperature, top-p, whether planner samples are i.i.d.), vLLM settings, and whether history includes actions only, screenshots only, or both interleaved; this affects reproducibility of H and P ablations.
  4. [Abstract; §4.1; Fig. 2(b), 4(d)] The abstract and introduction say failures shift toward “premature false successes” as a general local-CUA pattern, but detailed cohort plots are shown primarily for Qwen3-VL-30B-A3B. Briefly note whether UI-TARS and OpenCUA show the same cohort shift or only the success/cost pattern.
  5. [§3.2.1; References] Minor prose/typo cleanup: spacing in “T emporal Scaling,” inconsistent capitalization of scaling dimension names, and a few long reference-list author blocks that appear duplicated or minimally truncated.
  6. [§2] Related Work could more sharply contrast OSWorld-Human efficiency gaps and GUI context-compression work (GUI-KV, PAL-UI) with what is new in the failure-cohort analysis, to avoid reading as a pure systems bake-off.

Circularity Check

0 steps flagged

No significant circularity: empirical OSWorld measurements, not definitional or fitted predictions.

full rationale

This paper is a systems-style empirical study of inference-time scaling factors (history length H, max steps S, two-stage vs single-agent structure, parallel plan count P) on local CUAs evaluated on the external OSWorld benchmark. Task success, steps-per-task, prompt tokens, and failure-mode cohorts are measured outcomes under controlled experimental factors; they are not quantities defined to equal the claimed conclusions, nor parameters fitted on a subset and then relabeled as predictions. Prior frameworks (e.g., GTA-1-style planning/grounding and parallel plans) are reused as methodological baselines and experimental conditions, which is ordinary citation practice and does not force the reported accuracy–cost curves or failure-mode shifts by construction. There are no self-definitional identities, uniqueness theorems imported from the authors, or ansatzes smuggled in that make the central claims true by definition. Measurement concerns about how OSWorld labels premature false successes affect correctness/validity risk, not circularity of a derivation chain. Score 0 with empty steps is the appropriate honest finding.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 2 invented entities

Load-bearing content is experimental design and measurement, not a formal derivation. The claim rests on standard GUI-agent evaluation assumptions, a chosen factorization of ‘inference-time scaling,’ and fixed model/tooling choices. There are no fitted physical constants; experimental grid values are design choices that define the measured regime.

free parameters (3)
  • History lengths H ∈ {0,1,4,8} = grid {0,1,4,8}
    Discrete contextual budgets chosen by the authors; the recommended ‘H=4 best tradeoff’ is relative to this hand-picked grid, not a derived optimum.
  • Max steps S ∈ {15,50,100} = grid {15,50,100}
    Temporal budgets taken from OSWorld-style standards and varied by hand; conclusions about temporal scaling inefficiency are conditioned on these caps.
  • Parallel plan count P (varied, e.g. 1→4) = varied; highlighted 1 to 4
    Number of candidate plans is an author-chosen test-time parameter following GTA-1-style setups; recovery claims are tied to the sampled P values.
axioms (4)
  • domain assumption OSWorld screenshot-only task success is a valid primary measure of local CUA capability and completion judgment.
    §3.4: 361 Ubuntu tasks, no a11y trees or SoM; all accuracy and false-success claims depend on this evaluator.
  • ad hoc to paper Inference-time compute for CUAs can be adequately factored into contextual (H), temporal (S), structural (single vs two-stage), and parallel (P) dimensions.
    §3.1–3.2 introduce this taxonomy as the experimental frame; other axes (temperature sampling, self-consistency voting, memory modules, coding tools) are set aside.
  • domain assumption Prompt-token count plus step count are sufficient operational cost proxies for local deployment tradeoffs.
    §3.3 defines cost via steps and cumulative prompt tokens; wall-clock, energy, and peak VRAM are not primary metrics.
  • domain assumption A two-stage local agent with Qwen3 planners and fixed GTA-1-7B grounding is representative of structural decomposition for local CUAs.
    §3.4 and §4.2 compare single-agent baselines to this specific pairing; Finding 4–5 generalize from that setup.
invented entities (2)
  • Four-dimensional inference-time scaling taxonomy for local CUAs (contextual/temporal/structural/parallel) no independent evidence
    purpose: Organize ablations and unify discussion of compute allocation and failure-mode shifts.
    Not a physical entity; an author-defined analytical scaffold built from existing agent practices. Independent evidence is only whether others find the same axes predictive.
  • Failure-mode cohorts (repetitive loops, max-step stalls, premature false successes, format/parsing errors) no independent evidence
    purpose: Explain why success saturates while cost rises under scaling.
    Operational labels on trajectories; useful if coding is stable and externally checkable, but defined inside this evaluation pipeline.

pith-pipeline@v1.2.0-daily-grok45 · 17112 in / 3645 out tokens · 71533 ms · 2026-07-31T03:22:27.005020+00:00 · methodology

0 comments
read the original abstract

Deploying autonomous computer-use agents (CUAs) locally is increasingly important for privacy, cost efficiency, and practical usability, yet improving their performance under strict hardware constraints remains challenging. While recent studies show that inference-time scaling can improve frontier computer-use agents through additional computation during execution, its effectiveness for resource-constrained local models remains poorly understood. We present a systematic empirical study of inference-time scaling in local CUAs across contextual, temporal, structural, and parallel dimensions. We evaluate Qwen3-VL-8B/30B-A3B, UI-TARS-1.5-7B, and OpenCUA-7B on the OSWorld benchmark. Our results show that additional computation often yields diminishing returns while changing failure modes. Contextual scaling provides historical grounding that improves trajectory stability and task accuracy, but its gains saturate as token cost increases and failures shift from repetitive or stalled trajectories toward premature false successes. Temporal scaling similarly reduces max-step stalls, yet does not substantially improve task success, indicating that longer horizons often extend erroneous trajectories rather than correct them. We further find that structural decomposition can introduce planning and formatting overhead in local two-stage agents, while parallel scaling partially mitigates these failures at a substantial computational cost. Overall, our findings suggest that efficient local CUAs require selective compute allocation, failure-aware control mechanisms, and agentic frameworks designed around the capabilities and limitations of local models.

Figures

Figures reproduced from arXiv: 2607.28573 by Jungwook Choi, Woongkyu Lee.

Figure 1
Figure 1. Figure 1: Illustration of CUA frameworks and inference-time scaling dimensions. (a) The [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Impact of history length on single-agent performance, efficiency, and failure modes. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Comparison of agent trajectories by contextual scaling. Without contextual [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Impact of temporal scaling on single-agent. (a) Task success rates across different [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Impact of parallel scaling on two-stage agent. (a,b) Task success rates of Qwen3-8B [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Example of a planner format error. Despite explicit formatting instructions, the [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 18 linked inside Pith

  1. [3]

    Anthropic

    URL https://arxiv.org/abs/2504.00906. Anthropic. Introducing claude sonnet 4.5. https://www.anthropic.com/news/ claude-sonnet-4-5,

    Pith/arXiv arXiv

  2. [4]

    Rogerio Bonatti, Dan Zhao, Francesco Bonacci, Dillon Dupont, Sara Abdali, Yinheng Li, Yadong Lu, Justin Wagle, Kazuhito Koishida, Arthur Bucker, Lawrence Jang, and Zack Hui

    URL https://arxiv.org/abs/2511.21631. Rogerio Bonatti, Dan Zhao, Francesco Bonacci, Dillon Dupont, Sara Abdali, Yinheng Li, Yadong Lu, Justin Wagle, Kazuhito Koishida, Arthur Bucker, Lawrence Jang, and Zack Hui. Windows agent arena: Evaluating multi-modal os agents at scale,

    Pith/arXiv arXiv

  3. [5]

    Gongwei Chen, Xurui Zhou, Rui Shao, Yibo Lyu, Kaiwen Zhou, Shuai Wang, Wentao Li, Yinchuan Li, Zhongang Qi, and Liqiang Nie

    URL https://arxiv.org/abs/2409.08264. Gongwei Chen, Xurui Zhou, Rui Shao, Yibo Lyu, Kaiwen Zhou, Shuai Wang, Wentao Li, Yinchuan Li, Zhongang Qi, and Liqiang Nie. Less is more: Empowering gui agent with context-aware simplification,

    Pith/arXiv arXiv

  4. [6]

    Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su

    URLhttps://arxiv.org/abs/2507.03730. Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2web: Towards a generalist agent for the web,

    Pith/arXiv arXiv

  5. [9]

    Kevin Qinghong Lin, Linjie Li, Difei Gao, Zhengyuan Yang, Shiwei Wu, Zechen Bai, Weixian Lei, Lijuan Wang, and Mike Zheng Shou

    URL https: //arxiv.org/abs/2510.00536. Kevin Qinghong Lin, Linjie Li, Difei Gao, Zhengyuan Yang, Shiwei Wu, Zechen Bai, Weixian Lei, Lijuan Wang, and Mike Zheng Shou. Showui: One vision-language-action model for gui visual agent,

    arXiv

  6. [10]

    Zikang Liu, Junyi Li, Wayne Xin Zhao, Dawei Gao, Yaliang Li, and Ji rong Wen

    URLhttps://arxiv.org/abs/2411.17465. Zikang Liu, Junyi Li, Wayne Xin Zhao, Dawei Gao, Yaliang Li, and Ji rong Wen. Pal-ui: Planning with active look-back for vision-based gui agents,

    Pith/arXiv arXiv

  7. [11]

    org/abs/2510.00413

    URL https://arxiv. org/abs/2510.00413. Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, Wanjun Zhong, Kuanye Li, Jiale Yang, Yu Miao, Woyu Lin, Longxiang Liu, Xu Jiang, Qianli Ma, Jingyu Li, Xiaojun Xiao, Kai Cai, Chuang Li, Yaowei Zheng, Chaolin Jin, Chen Li, Xiao Zhou, Minchao W...

    arXiv

  8. [12]

    10 Preprint

    URL https://arxiv.org/ abs/2501.12326. 10 Preprint. Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, Akshay Nathan, Alan Luo, Alec Helyar, Aleksander Madry, Aleksandr Efremov, Aleksandra Spyra, Alex Baker- Whitcomb, Alex Beutel, Alex Karpenko, Alex Makelov, Alex Neitz...

    Pith/arXiv arXiv

  9. [13]

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar

    URLhttps://arxiv.org/abs/2601.03267. Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters,

    Pith/arXiv arXiv

  10. [14]

    Linxin Song, Yutong Dai, Viraj Prabhu, Jieyu Zhang, Taiwei Shi, Li Li, Junnan Li, Silvio Savarese, Zeyuan Chen, Jieyu Zhao, Ran Xu, and Caiming Xiong

    URL https:// arxiv.org/abs/2408.03314. Linxin Song, Yutong Dai, Viraj Prabhu, Jieyu Zhang, Taiwei Shi, Li Li, Junnan Li, Silvio Savarese, Zeyuan Chen, Jieyu Zhao, Ran Xu, and Caiming Xiong. Coact-1: Computer- using multi-agent system with coding actions,

    Pith/arXiv arXiv

  11. [15]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou

    URLhttps://arxiv.org/abs/2508.09123. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models,

    arXiv

  12. [16]

    Jinbiao Wei, Kangqi Ni, Yilun Zhao, Guo Gan, and Arman Cohan

    URLhttps://arxiv.org/abs/2203.11171. Jinbiao Wei, Kangqi Ni, Yilun Zhao, Guo Gan, and Arman Cohan. Step-level optimization for efficient computer-use agents,

    Pith/arXiv arXiv

  13. [17]

    URLhttps://arxiv.org/abs/2604.27151. Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, and Tao Yu. Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments,

    Pith/arXiv arXiv

  14. [18]

    URL https://arxiv.org/abs/2404.07972. Tianbao Xie, Jiaqi Deng, Xiaochuan Li, Junlin Yang, Haoyuan Wu, Jixuan Chen, Wenjing Hu, Xinyuan Wang, Yuhui Xu, Zekun Wang, Yiheng Xu, Junli Wang, Doyen Sahoo, Tao Yu, and Caiming Xiong. Scaling computer-use grounding via user interface decomposition and synthesis,

    Pith/arXiv arXiv

  15. [19]

    Taofeng Xue, Chong Peng, Mianqiu Huang, Linsen Guo, Tiancheng Han, Haozhe Wang, Jianing Wang, Xiaocheng Zhang, Xin Yang, Dengchang Zhao, Jinrui Ding, Xiandi Ma, 12 Preprint

    URLhttps://arxiv.org/abs/2505.13227. Taofeng Xue, Chong Peng, Mianqiu Huang, Linsen Guo, Tiancheng Han, Haozhe Wang, Jianing Wang, Xiaocheng Zhang, Xin Yang, Dengchang Zhao, Jinrui Ding, Xiandi Ma, 12 Preprint. Yuchen Xie, Peng Pei, Xunliang Cai, and Xipeng Qiu. Evocua: Evolving computer use agents via learning from scalable synthetic experience,

    arXiv

  16. [20]

    Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao

    URL https://arxiv.org/ abs/2601.15876. Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. Set- of-mark prompting unleashes extraordinary visual grounding in gpt-4v,

    arXiv

  17. [21]

    URL https://arxiv.org/abs/2310.11441. Yan Yang, Dongxu Li, Yutong Dai, Yuhao Yang, Ziyang Luo, Zirui Zhao, Zhiyuan Hu, Junzhe Huang, Amrita Saha, Zeyuan Chen, Ran Xu, Liyuan Pan, Silvio Savarese, Caiming Xiong, and Junnan Li. Gta1: Gui test-time scaling agent,

    Pith/arXiv arXiv

  18. [22]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L

    URL https://arxiv.org/ abs/2507.05791. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models,

    Pith/arXiv arXiv

  19. [23]

    Shuyan Zhou, Frank F

    URLhttps://arxiv.org/abs/2305.10601. Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. Webarena: A realistic web environment for building autonomous agents,

    Pith/arXiv arXiv

  20. [24]

    org/abs/2307.13854

    URL https://arxiv. org/abs/2307.13854. 13

    Pith/arXiv arXiv

  21. [2023]

    Gonzalo Gonzalez-Pumariega, Vincent Tu, Chih-Lun Lee, Jiachen Yang, Ang Li, and Xin Eric Wang

    URL https: //arxiv.org/abs/2306.06070. Gonzalo Gonzalez-Pumariega, Vincent Tu, Chih-Lun Lee, Jiachen Yang, Ang Li, and Xin Eric Wang. Scaling agents for computer use,

    Pith/arXiv arXiv

  22. [2024]

    Saaket Agashe, Kyle Wong, Vincent Tu, Jiachen Yang, Ang Li, and Xin Eric Wang

    URL https: //arxiv.org/abs/2410.08164. Saaket Agashe, Kyle Wong, Vincent Tu, Jiachen Yang, Ang Li, and Xin Eric Wang. Agent s2: A compositional generalist-specialist framework for computer use agents,

    Pith/arXiv arXiv

  23. [2025]

    Saaket Agashe, Jiuzhou Han, Shuyu Gan, Jiachen Yang, Ang Li, and Xin Eric Wang

    URLhttps://arxiv.org/abs/2506.16042. Saaket Agashe, Jiuzhou Han, Shuyu Gan, Jiachen Yang, Ang Li, and Xin Eric Wang. Agent s: An open agentic framework that uses computers like a human,

    Pith/arXiv arXiv

  24. [2026]

    Kung-Hsiang Huang, Haoyi Qiu, Yutong Dai, Caiming Xiong, and Chien-Sheng Wu

    URLhttps://arxiv.org/abs/2510.02250. Kung-Hsiang Huang, Haoyi Qiu, Yutong Dai, Caiming Xiong, and Chien-Sheng Wu. Gui- kv: Efficient gui agents via kv cache with spatio-temporal awareness,

    arXiv