Pith. sign in

REVIEW 2 major objections 5 minor 81 references

LLM agents can finish the visible task after the authority that licensed it has already expired, so endpoint success is not a security property.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 11:20 UTC pith:QXQ74HRW

load-bearing objection Clear definitional + empirical systems paper: endpoint success and authorized commit really do diverge under controlled invalidation; rates are stress-test, not prevalence, and the harness coupling is a real but bounded caveat. the 2 major comments →

arxiv 2607.10487 v1 pith:QXQ74HRW submitted 2026-07-11 cs.CR cs.AI

Temporary Authority, Permanent Effects: Commit-Time Authorization for LLM Agents

Igor Santos-Grueiro This is my paper
classification cs.CR cs.AI
keywords commit-time authorizationLLM agentsauthorized commitTOCTOUboundary monitortemporary authorityagent securityendpoint success
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper studies the moment when temporary authority carried by an LLM agent—page snapshots, approval tokens, version witnesses, branch markers, or worker results—stops entitling a later durable effect. It defines commit-time authorization: a durable effect is authorized only if the witness that licensed the derived state is still fresh, causally prior, bound to the same effect, and eligible at commit. In a controlled-invalidation suite of 54 browser, tool/API, and multi-agent tasks that keep the user goal fixed while breaking the authority relation before durability, 262 of 270 runs still reach the visible result, but only 55 are authorized completions; among 216 invalidating rows, 207 commit after the licensing path has failed. Clean controls stay authorized, and authority-preserving timing changes produce no unauthorized commits. Prompt caution and single-condition checks are not enough because different hazards break different boundary conditions; defenses work when they refresh, rebind, replan, or refuse at the durability boundary. CommitGuard, a fail-closed boundary monitor, blocks stale durable-effect attempts when runtimes emit the needed witness, dependency, binding, and eligibility signals. The reporting lesson is that endpoint success measures utility; authorized commit is the security property.

Core claim

Under controlled authority invalidation that preserves the user goal and payload, LLM agents routinely complete the visible endpoint after the path that licensed the durable effect has failed. In the primary 54-task matrix, 262/270 runs reach the endpoint while only 55/270 are authorized; 207 of 216 invalidating rows commit unauthorized. A durable effect is authorized only when the witness that licensed its derived state remains fresh, causally prior, bound to the same effect, and eligible at commit time.

What carries the argument

Commit-time authorization: four boundary checks—freshness, causal priority, effect binding, and commit eligibility—that must all hold when derived state becomes a durable effect. CommitGuard is the fail-closed monitor that enforces those checks on protected commit surfaces when the runtime emits witness, dependency, binding, and eligibility signals and couples the final check to the write.

Load-bearing premise

The claim depends on the runtime honestly emitting witness, dependency, binding, and eligibility signals and coupling the final check to the durable write; without that, unauthorized commits cannot be blocked or even observed.

What would settle it

If the same controlled-invalidation suite—same goals and payloads, authority broken only before durability—shows unguarded agents almost never committing after the licensing path fails, or if a boundary monitor without complete emitted signals still reliably prevents unauthorized durable effects, the claimed separation of endpoint success from authorized commit would fail.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Agent evaluations should report authorized completion, unauthorized commit, safe non-completion, and infrastructure failure separately from endpoint success.
  • Prompt-only caution and single-condition rechecks cannot cover all hazard classes because different failures break different boundary conditions.
  • Runtimes need an enforcement point between derived state and durable effect that can refresh, rebind, replan, or refuse at commit.
  • When required boundary signals are missing, systems should fail closed or report non-observability rather than treat endpoint success as safety.
  • With emitted signals and an atomic check-and-commit primitive, a fail-closed monitor can convert stale durable-effect attempts into guarded aborts.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Agent platforms that score only task completion may systematically overstate safety wherever UI, approvals, versions, or workers can change between observation and commit.
  • The same witness-to-derived-state-to-commit gap likely appears in non-LLM automation that caches handles, leases, or approvals across asynchronous steps.
  • Production agent stacks may need standard emission of epochs, ETags, branch eligibility, and conditional commits before commit-time authorization can be claimed at scale.
  • Benchmarks that only run clean, authority-preserving traces will hide the authorization gap this suite is designed to measure.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper defines commit-time authorization for LLM agents: a durable effect is authorized only if the witness that licensed its derived state remains fresh, causally prior, bound to the same effect, and eligible at the durability boundary. Using a 54-task controlled-invalidation suite (browser, tool/API, multi-agent; 270 unguarded runs), it reports that endpoint success stays high (262/270) while authorized completion is low (55/270), with 207/216 invalidating rows committing after the authorizing path fails. Clean controls (54/54) and authority-preserving negatives (0 unauthorized) support the evaluator. Mitigation probes show prompt caution and single-condition checks are incomplete; CommitGuard, a fail-closed boundary monitor, blocks stale commits when runtimes emit the required signals. The reporting lesson is that endpoint success is a utility metric and authorized commit is a security property.

Significance. If the result holds, the paper supplies a useful security object and measurement discipline for agent systems in mutable environments: separate endpoint success from authorized completion, and gate durability on four boundary conditions rather than final visible state. Strengths include a fixed primary matrix with clean and authority-preserving controls, fixed paired comparisons that hold endpoint success fixed while authorization flips, a clear threat model and conditional CommitGuard guarantee, an open artifact with manifests and a standalone monitor prototype, and an honest non-prevalence framing. Relative to classical TOCTOU and concurrent agent-TOCTOU work, the contribution is the witness o derived-state o commit framing and the empirical demonstration that agents often still externalize effects after controlled authority loss. That is a defensible systems/security contribution for agent runtime design and evaluation reporting.

major comments (2)
  1. [Sections 4–5, Tables 7–8, Appendix B] Sections 4–5 and Appendix B: the primary unauthorized-commit rate (207/216 invalidating rows) is measured in a harness that both injects the invalidator and emits the witness/dependency/binding/eligibility fields the evaluator later reads. Clean controls and authority-preserving negatives show the evaluator is not randomly rejecting long runs, but they do not fully establish independence between invalidation injection and the authorization label. The load-bearing behavioral claim is that agents still reach the durable endpoint after authority loss; please state explicitly, with a short independence argument or instrumentation diagram, that endpoint scoring uses only the final visible effect and never the invalidator flag, and discuss residual definitional coupling so readers cannot read 207/216 as restating successful injection alone.
  2. [Section 6, Table 2, Figure 3] Section 6 (CommitGuard conditional guarantee) and Table 2: the security claim holds only for protected surfaces that emit trustworthy boundary evidence and couple the final check to durability. The paper states this, but the main empirical defense results are largely shadow/replay and matched enforcement on instrumented surfaces. For the runtime-design lesson to carry, add a concrete discussion of which real commit surfaces (browser handles, ETags, approval epochs, branch capabilities) already emit the four signals, what fails closed when they do not, and how often non-observability would replace blocking in less instrumented stacks. Without that, the mitigation result risks overstating deployability relative to the measurement apparatus.
minor comments (5)
  1. [Abstract; Table 8] Table 8 vs. abstract: the abstract’s 55/270 authorized and 207 unauthorized among invalidating rows match the matrix arithmetic (55 authorized + 207 unauthorized + 8 safe non-completions = 270); keep that identity explicit in the main text near Table 8 so readers do not re-derive it.
  2. [Figure 2] Figure 2 caption and body: clarify that black ticks mark endpoint success and that aggregate rate claims come only from the full 270-row matrix, not the paired inspection, to avoid double-counting impressions.
  3. [Section 5] Section 5 route/model diagnostics: primary rates are on LangGraph+Qwen2.5-7B; the paper already treats other routes as diagnostics, but a one-sentence reminder next to the headline rates would reduce over-generalization of the 262/270 and 55/270 figures.
  4. [Section 8; Table 1] Related Work: position more sharply against concurrent agent-TOCTOU papers (e.g., Mind the Gap; Atomicity for Agents) in one paragraph that states what is measured differently (witness→derived state→commit entitlement vs. stale action after check).
  5. [Throughout] Typographical consistency: COMMITGUARD / CommitGuard capitalization varies between abstract, body, and figure captions; pick one house style.

Circularity Check

1 steps flagged

No load-bearing circular derivation: the main result is an empirical stress-test of agent behavior under injected invalidation, not a prediction forced by fitted inputs or self-citation.

specific steps
  1. other [§6 Diagnostic repair-map comparison; Appendix C mitigation selector]
    "The repair-map comparison is intentionally diagnostic. It starts from a selected unguarded case in the 36-task mitigation panel, maps that case’s adjudicated initiating boundary violation to one pre-specified repair family, and inspects the corresponding defense run... Controls remain clean on all 36/36 tasks, selected unguarded cases are unauthorized on all 36/36 tasks, and mapped defenses move all 36/36 cases into safe non-completion."

    This secondary comparison is label-aware by construction: the initiating violation is known, mapped to the defense family designed for that violation class, and then reported as fully repaired (0/36 unauthorized). That outcome is largely forced by the selector/mapping rather than an independent out-of-sample prediction. The paper itself marks it as diagnostic and not CommitGuard’s decision rule, so it does not make the primary matrix or the main separability claim circular.

full rationale

This is a systems/security measurement paper, not a first-principles derivation. Commit-time authorization is defined by four explicit boundary checks; unauthorized commit is then scored from emitted traces when a durable effect still lands after those checks fail. The primary 262/270 vs 55/270 split is therefore an observation about agent control paths under controlled invalidation (agents keep committing), not a quantity fitted from the same data and re-presented as a prediction. Clean controls (54/54 authorized) and authority-preserving negatives (0/54 unauthorized) further separate the evaluator from mere schedule noise. There is no self-citation chain, uniqueness theorem imported from the same author, ansatz smuggled via prior work, or renaming of a known closed-form result. The only mild circularity-adjacent element is the intentionally label-aware repair-map diagnostic (Appendix C / §6), which maps a known initiating violation to a pre-specified defense family and then reports that family repairing it—useful as a mechanism check, but not the load-bearing claim. CommitGuard’s enforce-mode aborts when its own predicates fail is implementation verification of a fail-closed monitor, not a circular scientific prediction. Overall circularity is negligible.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 3 invented entities

This is an empirical systems/security paper. The load-bearing content is a definitional security property, a constructed stress suite, and a monitor contract that assumes runtime observability and atomic commit coupling. There are essentially no fitted scientific constants; the free choices are design parameters of the suite and monitor. Invented entities are conceptual/runtime constructs, not physical postulates.

free parameters (2)
  • 54-task suite composition (3 families × 6 buckets × 3 tasks; 4 invalidators each)
    Task selection and invalidator library are author-constructed to cover witness/invalidator/effect cells; rates depend on this design rather than a random sample of production workflows.
  • Reference stack (LangGraph + Qwen2.5-7B-Instruct) and unguarded baseline prompt
    Primary rates are measured on one instrumented stack; other models/routes are diagnostics gated by clean-control/infrastructure checks.
axioms (4)
  • ad hoc to paper A durable effect is authorized only if freshness, causal priority, effect binding, and commit eligibility all hold at the durability boundary.
    Core definition of commit-time authorization introduced in Abstract/§3; the measurement and CommitGuard decision rule are built around these four checks.
  • domain assumption Runtime, trace emission, and boundary monitor are trusted; adversary/drift can change environment state but cannot forge monitor evidence or compromise the host.
    Threat model Table 2 and §2; without this, unauthorized commits and monitor guarantees are not well-defined.
  • domain assumption Controlled invalidation that preserves user goal and payload shape isolates authorization failure from task-difficulty confounds.
    Experimental design premise in §4; paired and negative-control checks rest on this matching constraint.
  • domain assumption Classical TOCTOU and transactional agent control are insufficient alone because authority propagates through derived state (plans, cached targets, staged writes, branch outputs, shared memory).
    §2 Table 1 and Related Work; motivates measuring witness→derived state→commit rather than only check/use of one resource.
invented entities (3)
  • Commit-time authorization (four boundary conditions) independent evidence
    purpose: Define when temporary-authority-derived state retains entitlement to make a durable effect.
    Central conceptual contribution; operationalized via evaluator and CommitGuard rule.
  • CommitGuard fail-closed boundary monitor no independent evidence
    purpose: Block or repair stale durable-effect attempts on protected commit surfaces when signals are emitted.
    Prototype monitor with shadow/enforce/repair modes; evidence is replay and enforcement runs in the artifact, not external deployment data.
  • Controlled-invalidation suite (54 tasks / 270 rows) no independent evidence
    purpose: Measure divergence between endpoint success and authorized completion under matched goals.
    Constructed benchmark spanning browser, tool/state, multi-agent families; not an external standard benchmark.

pith-pipeline@v1.1.0-grok45 · 29080 in / 3300 out tokens · 39779 ms · 2026-07-14T11:20:00.595335+00:00 · methodology

0 comments
read the original abstract

LLM agents can commit durable effects from authority evidence that was valid earlier in execution: a DOM snapshot, approval epoch, version witness, branch token, or worker result. We study the commit boundary at which earlier authority evidence no longer authorizes a durable effect. We call this property commit-time authorization: a durable effect is authorized only if the witness that licensed its derived state remains fresh, causally prior, bound to the same effect, and eligible at commit time. We build a controlled-invalidation suite spanning browser, tool/API, and multi-agent workflows. The suite preserves the user goal and payload shape while invalidating the authority relation before durability. In the primary 54-task matrix, endpoint success remains high: 262/270 runs reach the visible result. Only 55/270 are authorized completions; among the 216 invalidating rows, 207 commit after the authorizing path has failed. All 54 clean controls remain authorized, and a separate 54-run authority-preserving check produces no unauthorized commits. We then evaluate mitigation families. Prompt caution and single-condition checks are insufficient because different hazards break different boundary conditions. Defenses work when they refresh, rebind, replan, or refuse at the durability boundary. CommitGuard, a fail-closed boundary monitor, blocks stale durable-effect attempts on protected commit surfaces when runtimes emit witness, dependency, binding, and eligibility signals. The result is a reporting and runtime-design lesson: endpoint success is a utility metric; authorized commit is a security property.

Figures

Figures reproduced from arXiv: 2607.10487 by Igor Santos-Grueiro.

Figure 1
Figure 1. Figure 1: The problem in one picture. Two runs can reach a [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Primary controlled-invalidation evidence on the [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: COMMITGUARD pseudocode. The final line must be implemented by a runtime primitive that couples the refreshed evidence to the durable effect, such as a conditional write, lease, ETag, element handle, branch capability, or tool-side transaction. The guarantee does not cover hidden dependencies, forged evidence, compromised hosts, side effects that bypass the protected boundary, or runtimes without an atomic … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

81 extracted references · 4 canonical work pages

  1. [1]

    Agent S: An open agen- tic framework that uses computers like a human

    Saaket Agashe, Jiuzhou Han, Shuyu Gan, Jiachen Yang, Ang Li, and Xin Eric Wang. Agent S: An open agen- tic framework that uses computers like a human. In The Thirteenth International Conference on Learning Representations, 2025

    2025

  2. [2]

    ScreenAI: A vision-language model for UI and infographics un- derstanding

    Gilles Baechler, Srinivas Sunkara, Maria Wang, Fedir Zubach, Hassan Mansoor, Vincent Etter, et al. ScreenAI: A vision-language model for UI and infographics un- derstanding. InProceedings of the Thirty-Third In- ternational Joint Conference on Artificial Intelligence, pages 3058–3068, 2024. DOI: https://doi.org/10. 24963/ijcai.2024/339

    2024

  3. [3]

    Why and where: A characterization of data provenance

    Peter Buneman, Sanjeev Khanna, and Wang-Chiew Tan. Why and where: A characterization of data provenance. InDatabase Theory – ICDT 2001, volume 1973 ofLec- ture Notes in Computer Science, pages 316–330, 2001. DOI: https://doi.org/10.1007/3-540-44503-X_ 20

    work page doi:10.1007/3-540-44503-x_ 2001

  4. [4]

    Agentverse: Facili- tating multi-agent collaboration and exploring emergent behaviors

    Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chi-Min Chan, et al. Agentverse: Facili- tating multi-agent collaboration and exploring emergent behaviors. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  5. [5]

    Who tests the testers? systematic enumeration and coverage audit of LLM agent tool call safety, 2026

    Xuan Chen, Lu Yan, Ruqi Zhang, and Xiangyu Zhang. Who tests the testers? systematic enumeration and coverage audit of LLM agent tool call safety, 2026. arXiv:2603.18245

    arXiv 2026

  6. [6]

    SeeClick: Harnessing GUI grounding for advanced visual GUI agents

    Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, and Zhiyong Wu. SeeClick: Harnessing GUI grounding for advanced visual GUI agents. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9313–9332, 2024. DOI: https: //doi.org/10.18653/v1/2024.acl-long.505

    doi:10.18653/v1/2024.acl-long.505 2024

  7. [7]

    AI runtime infrastructure, 2026

    Christopher Cruz. AI runtime infrastructure, 2026. arXiv:2603.00495

    Pith/arXiv arXiv 2026

  8. [8]

    AgentDojo: A dynamic environment to evaluate prompt injection attacks and defenses for LLM agents

    Edoardo Debenedetti, Jie Zhang, Mislav Balunovi ´c, Luca Beurer-Kellner, Marc Fischer, and Florian Tramèr. AgentDojo: A dynamic environment to evaluate prompt injection attacks and defenses for LLM agents. In NeurIPS 2024 Track Datasets and Benchmarks, 2024. DOI:https://doi.org/10.52202/079017-2636. 14

    doi:10.52202/079017-2636 2024

  9. [9]

    Mind2Web: Towards a generalist agent for the web

    Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2Web: Towards a generalist agent for the web. InAdvances in Neural Information Processing Sys- tems 36 (Datasets and Benchmarks Track), 2023. DOI: https://doi.org/10.52202/075280-1220

    doi:10.52202/075280-1220 2023

  10. [10]

    AI agents under threat: A survey of key security challenges and future pathways.ACM Computing Surveys, 57(7),

    Zehang Deng, Yongjian Guo, Changzhou Han, Wanlun Ma, Junwu Xiong, Sheng Wen, and Yang Xiang. AI agents under threat: A survey of key security challenges and future pathways.ACM Computing Surveys, 57(7),

  11. [11]

    DOI:https://doi.org/10.1145/3716628

    doi:10.1145/3716628

  12. [12]

    Laradji, Manuel Del Verme, Tom Marty, et al

    Alexandre Drouin, Maxime Gasse, Massimo Caccia, Is- sam H. Laradji, Manuel Del Verme, Tom Marty, et al. WorkArena: How capable are web agents at solving common knowledge work tasks? InProceedings of the 41st International Conference on Machine Learn- ing, volume 235 ofProceedings of Machine Learning Research, pages 11642–11662. PMLR, 2024

    2024

  13. [13]

    AgentProcessBench: Di- agnosing step-level process quality in tool-using agents,

    Shengda Fan, Xuyan Ye, Yupeng Huo, Zhi-Yuan Chen, Yiju Guo, Shenzhi Yang, et al. AgentProcessBench: Di- agnosing step-level process quality in tool-using agents,

  14. [14]

    Tooltalk: Eval- uating tool-usage in a conversational setting, 2023

    Nicholas Farn and Richard Shin. Tooltalk: Eval- uating tool-usage in a conversational setting, 2023. arXiv:2311.10775

    Pith/arXiv arXiv 2023

  15. [15]

    Multimodal web navigation with instruction-finetuned foundation models

    Hiroki Furuta, Kuang-Huei Lee, Ofir Nachum, Yu- taka Matsuo, Aleksandra Faust, Shixiang Shane Gu, and Izzeddin Gur. Multimodal web navigation with instruction-finetuned foundation models. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  16. [16]

    ASSISTGUI: Task-oriented PC graphical user interface automation

    Difei Gao, Lei Ji, Zechen Bai, Mingyu Ouyang, Peiran Li, Dongxing Mao, et al. ASSISTGUI: Task-oriented PC graphical user interface automation. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13289–13298, 2024. DOI: https://doi.org/10.1109/CVPR52733.2024. 01262

    doi:10.1109/cvpr52733.2024 2024

  17. [17]

    Gray and David R

    Cary G. Gray and David R. Cheriton. Leases: An efficient fault-tolerant mechanism for distributed file cache consistency.ACM SIGOPS Operating Systems Re- view, 23(5):202–210, 1989. DOI: https://doi.org/ 10.1145/74851.74870

    doi:10.1145/74851.74870 1989

  18. [18]

    A real-world WebAgent with planning, long context understanding, and program synthesis

    Izzeddin Gur, Hiroki Furuta, Austin Huang, Mustafa Safdari, Yutaka Matsuo, Douglas Eck, and Aleksandra Faust. A real-world WebAgent with planning, long context understanding, and program synthesis. InThe Twelfth International Conference on Learning Represen- tations, 2024

    2024

  19. [20]

    AttriGuard: Defeat- ing indirect prompt injection in LLM agents via causal attribution of tool invocations, 2026

    Yu He, Haozhe Zhu, Yiming Li, Shuo Shao, Hongwei Yao, Zhihao Liu, and Zhan Qin. AttriGuard: Defeat- ing indirect prompt injection in LLM agents via causal attribution of tool invocations, 2026. arXiv:2603.10749

    Pith/arXiv arXiv 2026

  20. [21]

    Metagpt: Meta programming for a multi-agent collaborative framework

    Sirui Hong, Mingchen Zhuge, Jiaqi Chen, Xiawu Zheng, Yuheng Cheng, Ceyao Zhang, et al. Metagpt: Meta programming for a multi-agent collaborative framework. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  21. [22]

    CogAgent: A visual language model for GUI agents

    Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, et al. CogAgent: A visual language model for GUI agents. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14281–14290, 2024. DOI: https: //doi.org/10.1109/CVPR52733.2024.01354

    doi:10.1109/cvpr52733.2024.01354 2024

  22. [23]

    Atomicity for agents: Exposing, exploiting, and miti- gating TOCTOU vulnerabilities in browser-use agents,

    Linxi Jiang, Zhijie Liu, Haotian Luo, and Zhiqiang Lin. Atomicity for agents: Exposing, exploiting, and miti- gating TOCTOU vulnerabilities in browser-use agents,

  23. [24]

    You told me to do it: Measuring instructional text- induced private data leakage in LLM agents, 2026

    Ching-Yu Kao, Xinfeng Li, Shenyu Dai, Tianze Qiu, Pengcheng Zhou, Eric Hanchen Jiang, and Philip Sperl. You told me to do it: Measuring instructional text- induced private data leakage in LLM agents, 2026. arXiv:2603.11862

    arXiv 2026

  24. [25]

    OmniACT: A dataset and bench- mark for enabling multimodal generalist autonomous agents for desktop and web

    Raghav Kapoor, Yash Parag Butala, Melisa Russak, Jing Yu Koh, Kiran Kamble, Waseem AlShikh, and Rus- lan Salakhutdinov. OmniACT: A dataset and bench- mark for enabling multimodal generalist autonomous agents for desktop and web. InComputer Vision – ECCV 2024, volume 15126 ofLecture Notes in Computer Sci- ence, pages 161–178. Springer, 2024. DOI: https: //...

    doi:10.1007/978-3-031-73113-6_10 2024

  25. [26]

    Vi- sualWebArena: Evaluating multimodal agents on real- istic visual web tasks

    Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Chong Lim, Po-Yu Huang, et al. Vi- sualWebArena: Evaluating multimodal agents on real- istic visual web tasks. InProceedings of the 62nd An- nual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), 2024. DOI: https: //doi.org/10.18653/v1/2024.acl-long.50

    doi:10.18653/v1/2024.acl-long.50 2024

  26. [27]

    H. T. Kung and John T. Robinson. On optimistic meth- ods for concurrency control.ACM Transactions on 15 Database Systems, 6(2):213–226, 1981. DOI: https: //doi.org/10.1145/319566.319567

    doi:10.1145/319566.319567 1981

  27. [28]

    AutoWebGLM: A large language model-based web navigating agent

    Hanyu Lai, Xiao Liu, Iat Long Iong, Shuntian Yao, Yuxuan Chen, Pengbo Shen, et al. AutoWebGLM: A large language model-based web navigating agent. InProceedings of the 30th ACM SIGKDD Confer- ence on Knowledge Discovery and Data Mining, pages 5295–5306, 2024. DOI: https://doi.org/10.1145/ 3637528.3671620

    arXiv 2024

  28. [29]

    Benchmarking mobile de- vice control agents across diverse configurations, 2024

    Juyong Lee, Taywon Min, Minyong An, Changyeon Kim, and Kimin Lee. Benchmarking mobile de- vice control agents across diverse configurations, 2024. arXiv:2404.16660

    Pith/arXiv arXiv 2024

  29. [30]

    Camel: Com- municative agents for “mind” exploration of large lan- guage model society

    Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Com- municative agents for “mind” exploration of large lan- guage model society. InAdvances in Neural Information Processing Systems 36, pages 51991–52008, 2023. DOI: https://doi.org/10.52202/075280-2264

    doi:10.52202/075280-2264 2023

  30. [31]

    Api-bank: A compre- hensive benchmark for tool-augmented LLMs

    Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, et al. Api-bank: A compre- hensive benchmark for tool-augmented LLMs. InPro- ceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023. DOI: https: //doi.org/10.18653/v1/2023.emnlp-main.187

    doi:10.18653/v1/2023.emnlp-main.187 2023

  31. [32]

    Security considerations for artificial intelligence agents,

    Ninghui Li, Kaiyuan Zhang, Kyle Polley, and Jerry Ma. Security considerations for artificial intelligence agents,

  32. [33]

    A zero-shot language agent for computer control with structured reflection

    Tao Li, Gang Li, Zhiwei Deng, Bryan Wang, and Yang Li. A zero-shot language agent for computer control with structured reflection. InFindings of the Asso- ciation for Computational Linguistics: EMNLP 2023, pages 11261–11274, 2023. DOI: https://doi.org/ 10.18653/v1/2023.findings-emnlp.753

    doi:10.18653/v1/2023.findings-emnlp.753 2023

  33. [34]

    Mind the gap: Time-of-check to time-of-use vulnerabilities in LLM- enabled agents, 2025

    Derek Lilienthal and Sanghyun Hong. Mind the gap: Time-of-check to time-of-use vulnerabilities in LLM- enabled agents, 2025. arXiv:2508.17155

    Pith/arXiv arXiv 2025

  34. [35]

    Agentbench: Evaluating LLMs as agents

    Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, et al. Agentbench: Evaluating LLMs as agents. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  35. [36]

    Visual confused deputy: Ex- ploiting and defending perception failures in computer- using agents, 2026

    Xunzhuo Liu, Bowei He, Xue Liu, Andy Luo, Haichen Zhang, and Huamin Chen. Visual confused deputy: Ex- ploiting and defending perception failures in computer- using agents, 2026. arXiv:2603.14707

    arXiv 2026

  36. [37]

    A dynamic LLM-powered agent network for task- oriented agent collaboration

    Zijun Liu, Yanzhe Zhang, Peng Li, Yang Liu, and Diyi Yang. A dynamic LLM-powered agent network for task- oriented agent collaboration. InThe First Conference on Language Modeling, 2024

    2024

  37. [38]

    We- bLINX: Real-world website navigation with multi-turn dialogue

    Xing Han Lu, Zden ˇek Kasner, and Siva Reddy. We- bLINX: Real-world website navigation with multi-turn dialogue. InProceedings of the 41st International Conference on Machine Learning, volume 235 ofPro- ceedings of Machine Learning Research, pages 33007– 33056. PMLR, 2024

    2024

  38. [39]

    GAIA: A benchmark for general AI assistants

    Grégoire Mialon, Clémentine Fourrier, Craig Swift, Thomas Wolf, Yann LeCun, and Thomas Scialom. GAIA: A benchmark for general AI assistants. InThe Twelfth International Conference on Learning Represen- tations, 2024

    2024

  39. [40]

    Atomix: Timely, transactional tool use for reliable agentic work- flows, 2026

    Bardia Mohammadi, Nearchos Potamitis, Lars Klein, Akhil Arora, and Laurent Bindschaedler. Atomix: Timely, transactional tool use for reliable agentic work- flows, 2026. arXiv:2602.14849

    Pith/arXiv arXiv 2026

  40. [41]

    A trace-based assurance framework for agentic AI orchestration: Contracts, testing, and governance

    Ciprian Paduraru, Petru-Liviu Bouruc, and Alin Ste- fanescu. A trace-based assurance framework for agentic AI orchestration: Contracts, testing, and governance. In Proceedings of the 21st International Conference on Evaluation of Novel Approaches to Software Engineer- ing, pages 420–427, 2026. DOI: https://doi.org/ 10.5220/0014840300004015

    work page doi:10.5220/0014840300004015 2026

  41. [42]

    Patil, Tianjun Zhang, Xin Wang, and Joseph E

    Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. Gorilla: Large language model connected with massive apis. InAdvances in Neural Information Processing Systems 37, 2024. DOI:https://doi.org/ 10.52202/079017-4020

    doi:10.52202/079017-4020 2024

  42. [43]

    TinyClick: Single-turn agent for empowering GUI automation

    Pawel Pawlowski, Krystian Zawistowski, Wojciech La- pacz, Adam Wiacek, Marcin Skorupa, Sebastien Postan- sque, and Jakub Hoscilowicz. TinyClick: Single-turn agent for empowering GUI automation. InInterspeech 2025, pages 3035–3039, 2025. DOI: https://doi. org/10.21437/Interspeech.2025-176

    work page doi:10.21437/interspeech.2025-176 2025

  43. [44]

    Nguyen, and Nghi D

    Huy Nhat Phan, Phong X. Nguyen, and Nghi D. Q. Bui. HyperAgent: Generalist software engineering agents to solve coding tasks at scale, 2024. arXiv:2409.16299

    Pith/arXiv arXiv 2024

  44. [46]

    Toolllm: Facilitating large language models to master 16000+ real-world apis

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, et al. Toolllm: Facilitating large language models to master 16000+ real-world apis. InThe Twelfth International Conference on Learning Representations, 2024. 16

    2024

  45. [47]

    An- droidWorld: A dynamic benchmarking environment for autonomous agents

    Christopher Rawles, Sarah Clinckemaillie, Yifan Chang, Jonathan Waltz, Gabrielle Lau, Marybeth Fair, et al. An- droidWorld: A dynamic benchmarking environment for autonomous agents. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  46. [48]

    Saltzer and Michael D

    Jerome H. Saltzer and Michael D. Schroeder. The protection of information in computer systems.Pro- ceedings of the IEEE, 63(9):1278–1308, 1975. DOI: https://doi.org/10.1109/PROC.1975.9939

    doi:10.1109/proc.1975.9939 1975

  47. [49]

    Toolformer: Language models can teach themselves to use tools

    Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Can- cedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. InAd- vances in Neural Information Processing Systems 36, pages 68539–68551, 2023. DOI: https://doi.org/ 10.52202/075280-2997

    doi:10.52202/075280-2997 2023

  48. [50]

    A survey on agen- tic security: Applications, threats and defenses, 2025

    Asif Shahriar, Md Nafiu Rahman, Sadif Ahmed, Farig Sadeque, and Md Rizwan Parvez. A survey on agen- tic security: Applications, threats and defenses, 2025. arXiv:2510.06445

    arXiv 2025

  49. [52]

    A survey on autonomy- induced security risks in large model-based agents

    Hang Su, Jun Luo, Chang Liu, Xiao Yang, Yichi Zhang, Yinpeng Dong, and Jun Zhu. A survey on autonomy- induced security risks in large model-based agents. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1–20, 2026. DOI: https://doi. org/10.1109/TPAMI.2026.3688650

    work page doi:10.1109/tpami.2026.3688650 2026

  50. [53]

    MMInA: Benchmarking multihop multimodal in- ternet agents

    Shulin Tian, Ziniu Zhang, Liangyu Chen, and Ziwei Liu. MMInA: Benchmarking multihop multimodal in- ternet agents. InFindings of the Association for Com- putational Linguistics: ACL 2025, pages 13682–13697,

    2025

  51. [54]

    findings-acl.703

    DOI: https://doi.org/10.18653/v1/2025. findings-acl.703

    doi:10.18653/v1/2025 2025

  52. [55]

    AppWorld: A controllable world of apps and people for benchmarking interactive coding agents

    Harsh Trivedi, Tushar Khot, Mareike Hartmann, Ruskin Manku, Vinty Dong, Edward Li, et al. AppWorld: A controllable world of apps and people for benchmarking interactive coding agents. InProceedings of the 62nd An- nual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), pages 16022–16076,

  53. [57]

    Wagner, and Dilma Da Silva

    Dan Tsafrir, Tomer Hertz, David A. Wagner, and Dilma Da Silva. Portably solving file races with hard- ness amplification.ACM Transactions on Storage, 4(3):1–30, 2008. DOI: https://doi.org/10.1145/ 1416944.1416948

    arXiv 2008

  54. [58]

    OpenAgentSafety: A comprehensive framework for evaluating real-world AI agent safety,

    Sanidhya Vijayvargiya, Aditya Bharat Soni, Xuhui Zhou, Zora Zhiruo Wang, Nouha Dziri, Graham Neubig, and Maarten Sap. OpenAgentSafety: A comprehensive framework for evaluating real-world AI agent safety,

  55. [59]

    A comprehensive survey in LLM(-agent) full stack safety: Data, training and deployment, 2025

    Kun Wang, Guibin Zhang, Zhenhong Zhou, Jiahao Wu, Miao Yu, Shiqian Zhao, et al. A comprehensive survey in LLM(-agent) full stack safety: Data, training and deployment, 2025. arXiv:2504.15585

    Pith/arXiv arXiv 2025

  56. [60]

    A survey on large lan- guage model based autonomous agents.Frontiers of Computer Science, 2024

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, et al. A survey on large lan- guage model based autonomous agents.Frontiers of Computer Science, 2024. DOI: https://doi.org/10. 1007/s11704-024-40231-1

    2024

  57. [61]

    Xu, Xiangru Tang, Mingchen Zhuge, et al

    Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge, et al. OpenHands: An open platform for AI software developers as generalist agents. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  58. [62]

    The OpenHands software agent SDK: A composable and extensible foundation for production agents, 2025

    Xingyao Wang, Simon Rosenberg, Juan Michelini, Calvin Smith, Hoang Tran, Engel Nyst, et al. The OpenHands software agent SDK: A composable and extensible foundation for production agents, 2025. arXiv:2511.03690

    Pith/arXiv arXiv 2025

  59. [63]

    AgentTrace: Causal graph tracing for root cause analysis in deployed multi-agent systems, 2026

    Zhaohui Geoffrey Wang. AgentTrace: Causal graph tracing for root cause analysis in deployed multi-agent systems, 2026. arXiv:2603.14688

    arXiv 2026

  60. [64]

    WebWalker: Benchmark- ing LLMs in web traversal

    Jialong Wu, Wenbiao Yin, Yong Jiang, Zhenglin Wang, Zekun Xi, Runnan Fang, et al. WebWalker: Benchmark- ing LLMs in web traversal. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10290– 10305, 2025. DOI: https://doi.org/10.18653/v1/ 2025.acl-long.508

    doi:10.18653/v1/ 2025

  61. [65]

    ChainFuzzer: Greybox fuzzing for workflow- level multi-tool vulnerabilities in LLM agents, 2026

    Jiangrong Wu, Zitong Yao, Yuhong Nan, and Zibin Zheng. ChainFuzzer: Greybox fuzzing for workflow- level multi-tool vulnerabilities in LLM agents, 2026. arXiv:2603.12614

    arXiv 2026

  62. [66]

    Autogen: Enabling next- gen LLM applications via multi-agent conversation

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, et al. Autogen: Enabling next- gen LLM applications via multi-agent conversation. In The First Conference on Language Modeling, 2024. 17

    2024

  63. [67]

    The rise and potential of large language model based agents: A survey.Science China Information Sciences, 68(2):121101, 2025

    Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, et al. The rise and potential of large language model based agents: A survey.Science China Information Sciences, 68(2):121101, 2025. DOI: https://doi.org/10.1007/s11432-024-4222-0

    doi:10.1007/s11432-024-4222-0 2025

  64. [68]

    OSWorld: Bench- marking multimodal agents for open-ended tasks in real computer environments

    Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, et al. OSWorld: Bench- marking multimodal agents for open-ended tasks in real computer environments. InNeurIPS 2024 Track Datasets and Benchmarks, 2024. DOI: https://doi. org/10.52202/079017-1650

    doi:10.52202/079017-1650 2024

  65. [69]

    From storage to steering: Memory control flow attacks on LLM agents, 2026

    Zhenlin Xu, Xiaogang Zhu, Yu Yao, Minhui Xue, and Yiliao Song. From storage to steering: Memory control flow attacks on LLM agents, 2026. arXiv:2603.15125

    Pith/arXiv arXiv 2026

  66. [70]

    τ-bench: A benchmark for tool-agent-user interaction in real-world domains

    Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. τ-bench: A benchmark for tool-agent-user interaction in real-world domains. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  67. [71]

    Narasimhan, and Yuan Cao

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. InThe Eleventh International Conference on Learning Representations, 2023

    2023

  68. [72]

    CCTU: A bench- mark for tool use under complex constraints, 2026

    Junjie Ye, Guoqiang Zhang, Wenjie Fu, Tao Gui, Qi Zhang, and Xuanjing Huang. CCTU: A bench- mark for tool use under complex constraints, 2026. arXiv:2603.15309

    arXiv 2026

  69. [73]

    Survey on evaluation of LLM-based agents, 2025

    Asaf Yehudai, Lilach Eden, Alan Li, Guy Uziel, Yilun Zhao, Roy Bar-Haim, Arman Cohan, and Michal Shmueli-Scheuer. Survey on evaluation of LLM-based agents, 2025. arXiv:2503.16416

    Pith/arXiv arXiv 2025

  70. [74]

    AssistantBench: Can web agents solve realistic and time-consuming tasks? InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024

    Ori Yoran, Samuel Joseph Amouyal, Chaitanya Malaviya, Ben Bogin, Ofir Press, and Jonathan Berant. AssistantBench: Can web agents solve realistic and time-consuming tasks? InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024. DOI: https: //doi.org/10.18653/v1/2024.emnlp-main.505

    doi:10.18653/v1/2024.emnlp-main.505 2024

  71. [75]

    A survey on trustworthy LLM agents: Threats and countermeasures

    Miao Yu, Fanci Meng, Xinyun Zhou, Shilong Wang, Jun- yuan Mao, Linsey Pan, et al. A survey on trustworthy LLM agents: Threats and countermeasures. InProceed- ings of the 31st ACM SIGKDD Conference on Knowl- edge Discovery and Data Mining, pages 6216–6226,

  72. [76]

    DOI: https://doi.org/10.1145/3711896. 3736561

    doi:10.1145/3711896

  73. [77]

    UFO: A UI-focused agent for windows OS interaction

    Chaoyun Zhang, Liqun Li, Shilin He, Xu Zhang, Bo Qiao, Si Qin, et al. UFO: A UI-focused agent for windows OS interaction. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 597–622, 2025. DOI: https://doi.org/10.18653/ v1/2...

    2025

  74. [78]

    AppAgent: Multimodal agents as smartphone users

    Chi Zhang, Zhao Yang, Jiaxuan Liu, Yanda Li, Yucheng Han, Xin Chen, et al. AppAgent: Multimodal agents as smartphone users. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pages 70:1–70:20, 2025. DOI: https://doi.org/10. 1145/3706598.3713600

    arXiv 2025

  75. [79]

    Android in the zoo: Chain-of-action-thought for GUI agents

    Jiwen Zhang, Jihao Wu, Teng Yihua, Minghui Liao, Nuo Xu, Xiao Xiao, Zhongyu Wei, and Duyu Tang. Android in the zoo: Chain-of-action-thought for GUI agents. InFindings of the Association for Computa- tional Linguistics: EMNLP 2024, pages 12016–12031,

    2024

  76. [80]

    findings-emnlp.702

    DOI: https://doi.org/10.18653/v1/2024. findings-emnlp.702

    doi:10.18653/v1/2024 2024

  77. [81]

    VeriGrey: Greybox agent validation, 2026

    Yuntong Zhang, Sungmin Kang, Ruijie Meng, Marcel Böhme, and Abhik Roychoudhury. VeriGrey: Greybox agent validation, 2026. arXiv:2603.17639

    arXiv 2026

  78. [82]

    Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, et al

    Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, et al. WebArena: A real- istic web environment for building autonomous agents. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  79. [83]

    Teams of LLM agents can exploit zero-day vulnerabilities

    Yuxuan Zhu, Antony Kellermann, Akul Gupta, Philip Li, Richard Fang, Rohan Bindu, and Daniel Kang. Teams of LLM agents can exploit zero-day vulnerabilities. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguis- tics (Volume 1: Long Papers), pages 23–35, Rabat, Mo- rocco, 2026. Association for Computatio...

    doi:10.18653/v1/2026 2026

  80. [84]

    Toolqa: A dataset for LLM question answering with external tools

    Yuchen Zhuang, Yue Yu, Kuan Wang, Haotian Sun, and Chao Zhang. Toolqa: A dataset for LLM question answering with external tools. InAdvances in Neu- ral Information Processing Systems 36, pages 22119– 22164, 2023. DOI: https://doi.org/10.52202/ 075280-2180

    2023

Showing first 80 references.