Pith. sign in

REVIEW 2 major objections 5 minor 6 cited by

Levels of Autonomy for AI Agents

T0 review · 2 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Autonomy for AI agents can be a deliberate design choice, separate from capability.

desk verdict A clear, genuinely useful five-level framework for agent autonomy that deserves serious engagement, but its own evaluation proposal conflates capability with design choice. read the letter →

arxiv 2506.12469 v2 pith:WCSZUJZB submitted 2025-06-14 cs.HC cs.AI

K. J. Kevin Feng , David W. McDonald , Amy X. Zhang This is my paper
classification cs.HCcs.AI
keywords AIagentsautonomylevelsuserroleshuman-AIinteractioncertificatesassistedevaluationagentgovernancemulti-agentsystems
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Autonomy is usually treated as a byproduct of an AI agent's capability: the smarter the model, the more autonomously it can act. This paper argues the opposite: for a fixed set of capabilities and a fixed operating environment, an agent's level of autonomy is a deliberate design decision developers can make and should be held accountable for. It proposes five levels, distinguished by the role a user plays—operator, collaborator, consultant, approver, or observer—and describes the control mechanisms and design questions that come with each role. The payoff is practical: autonomy levels could be certified by a third party, letting developers, regulators, and other agents reason about risk and coordination without first reverse-engineering a model's raw ability. The paper also sketches an 'assisted evaluation' meant to measure an agent's autonomy separately from its task performance.

What carries the argument

The load-bearing object is the five-level autonomy framework centered on user roles. An agent's autonomy is defined as the extent to which it is designed to operate without user involvement; the 'user' may be a human or another agent. Each of the five levels—operator, collaborator, consultant, approver, observer—carries a distinct set of must-have controls (e.g., user-managed planning and invocation-or-approval at L1; approval elicitation for consequential actions at L4; an emergency off-switch at L5). This role-centered ladder does the work of making autonomy a design dimension rather than a capability outcome, and it grounds the two further mechanisms: autonomy certificates (a way to publish the chosen level) and assisted evaluations (a way to measure it).

What would settle it

Run an assisted evaluation on an agent whose design is L5 (no user-involvement mechanisms beyond monitoring) but whose raw performance on the benchmark sits below the threshold T. The procedure will add user involvement until T is met and will certify the agent below L5, even though its design autonomy was L5—contradicting the claim that the measured level tracks design rather than capability.

Watch

Extended reading notes

Core claim

The central claim is that the degree to which an AI agent works without user involvement can be separated from how capable the agent is, and that this degree should be set explicitly rather than inferred from benchmarks. Concretely, the paper defines five escalating levels of autonomy L1–L5, each named for the user's role: operator (user directs and decides, agent acts on demand), collaborator (user and agent plan, delegate, and execute together), consultant (agent leads but consults the user for expertise and preferences), approver (agent acts on its own except for pre-specified or risky decisions), and observer (agent acts fully autonomously, with monitoring and an emergency off-switch but no other user input). The authors further propose 'autonomy certificates'—documents issued by a third-party body stating the maximum level at which an agent may operate given its specifications and environment—and 'assisted evaluations,' in which a standby user helps the agent only as much as needed to pass a task threshold, with the recorded nature of that help determining the level. If the framing holds, autonomy becomes a governable property in its own right, one that can be audited and coordinated across single- and multi-agent systems independently of model quality.

Load-bearing premise

The assisted-evaluation method assumes that the user help an agent needs to pass a task threshold reveals its designed autonomy level, rather than merely its lack of capability.

Editorial extensions

If this is right

  • Developers can build two agents with identical models, tools, and environments that behave at different autonomy levels, and those differences are legitimate design choices, not failures of capability.
  • Regulators and auditors can target evaluations at level-specific risks, e.g., scrutinizing long-horizon planning for L4/L5 agents and feedback-elicitation design for L2/L3 agents.
  • Autonomy certificates give multi-agent systems a machine-readable signal for predicting which agents can work together, e.g., a system of all-L1 agents would stall waiting for operators, while all-L5 agents would be hard to steer or audit.
  • Assisted evaluations could certify an agent's autonomy level without requiring the evaluator to inspect its internal code, using the nature and amount of user help needed to pass a benchmark threshold.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not develop: an autonomy certificate could be combined with a safety case so that behavioral guarantees (e.g., 'this agent will consult before taking consequential actions') are checked together with capability claims, potentially strengthening both.
  • The framework suggests a testable design space: for a fixed model and task, systematically varying only the control constraints (e.g., mandatory approval for certain action classes) should move an agent across levels; if it does not, the 'design' variable has no purchase.
  • The assisted-evaluation procedure could, in principle, be reversed: instead of starting from no help and adding help until success, an evaluator could start from full help and reduce it, which might better expose agents whose design relies on subtle human initiative.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. This essay argues that an AI agent's level of autonomy should be treated as a deliberate design decision, separable from its capability and operational environment, rather than as an inevitable byproduct of increasing capability. The authors propose a five-level taxonomy centered on the user's role—operator, collaborator, consultant, approver, observer—and illustrate each level with a running example and open design questions. They then introduce autonomy certificates as a governance mechanism, in which a third-party body issues certificates based on developer-submitted autonomy cases and private evaluations, and conclude with a proposed 'assisted evaluation' procedure that escalates user involvement until the agent exceeds a task threshold T, assigning an autonomy level from the nature of that involvement. The paper is explicitly exploratory: it offers definitions, examples, and early ideas rather than empirical validation or a fully specified certification protocol.

Significance. If the framework is adopted, it gives developers, auditors, and regulators a common vocabulary for discussing agent autonomy and a way to separate questions about capability from questions about user interaction policy. The taxonomy is clearly presented, the agency/autonomy distinction is useful, and the running example successfully demonstrates that a fixed model, toolset, and environment can support different interaction designs. The autonomy-certificate proposal is timely and connects well to existing safety-case literature. Notable strengths are the explicit definitions, the summary table, and the candid acknowledgment of open questions at every level. The main weakness is the assisted-evaluation proposal in Section 5, which as stated conflates capability with designed autonomy; this is the load-bearing link between the conceptual framework and the claim that autonomy can be measured independently of capability, and it needs substantial reworking.

major comments (2)
  1. [§5 (assisted evaluation, Steps 1–3)] The proposed evaluation measures the minimum amount of user assistance required for the agent to exceed threshold T, which conflates designed autonomy with capability. A low-capability agent whose interaction policy is L5 will fail the initial no-involvement round and will be reclassified at a lower level once the standby user supplies help; conversely, a high-capability agent designed for L1 will fail the initial round because its policy requires approval before action, and the procedure's later rounds, which inject L4-style approvals, can then classify it at L4. The outcome is therefore co-determined by model capability, benchmark difficulty, threshold T, and the assistive policy of the standby user, not by the designer's chosen user role. This directly undermines the stated goal in §5 ('measure the extent of user involvement requested by an agent to successfully complete a task') and the conclusion that autonomy can be evaluated separately from capability. Because certificate issuance in §4.2 relies on such evaluations, the certificate's behavioral guarantee is not established. The paper should either reframe the assisted evaluation as a measure of assistance-required, add capability-control conditions and design-probe tasks that isolate the interaction policy, or explicitly acknowledge this confound as an open problem.
  2. [§4.2 and §5] Autonomy certificates are defined as prescriptions of the maximum level at which an agent 'can operate' (§4), and issuance requires an autonomy case 'with proof of the agent's interactive behaviors' (§4.2). The paper does not specify what counts as proof or how the governing body's private evaluations establish the behavioral guarantee; Section 5 itself begins by calling evaluation 'an important, yet still missing, piece of the puzzle.' This is acceptable for an exploratory essay, but the manuscript should state more explicitly that the certificate mechanism is not yet operationalizable, and that the assisted evaluation does not independently validate the taxonomy because it classifies agents using the framework's own 'Characteristics' column rather than an external criterion.
minor comments (5)
  1. [§3.5, Table 1] The L5 description says the agent comes 'with no means for user involvement' while also allowing activity-log monitoring and an emergency off-switch; since §2.2 defines involvement as including 'light supervision,' the relationship between monitoring and involvement should be clarified to avoid an apparent inconsistency.
  2. [§5, Step 2] The procedure assumes that 'approvals and rejections' uniquely indicate L4 interactions, yet L1 also includes 'Invocation or approval before actions' in Table 1; the mapping from recorded interaction types to autonomy levels needs an explicit disambiguation rule.
  3. [§5, threshold T] The choice of threshold T is left unspecified, and classification can change with T—for example, a looser threshold might let an agent pass the initial no-involvement round and be labeled L5, while a stricter threshold forces escalation. The authors should state how T is set and reported, or at least note that the resulting autonomy level is relative to T.
  4. [Table 1] The example systems are classified 'based on their public releases and demos as of June 2025'; given rapid product changes, these classifications may become stale quickly, so a versioning or 'as of' note would help readers interpret the table.
  5. [§4.2 (autonomy cases)] The analogy to safety cases is useful, but the paper does not discuss an important difference: safety cases argue that risks are acceptable under specified conditions, whereas autonomy cases argue that an agent's behavior stays below a behavioral bound; a sentence clarifying how arguments and evidence differ between the two would strengthen the proposal.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the framework is stipulated rather than derived, and no fitted input or load-bearing self-citation is present.

full rationale

This paper is a position/essay that introduces a five-level autonomy taxonomy and an early evaluation sketch. It contains no derivation chain, no fitted parameters, and no equation-level reduction. The central claim that autonomy can be a deliberate design decision is an argumentative position, not a result derived from prior premises. The assisted evaluation in Section 5 operationalizes the paper's own definition of autonomy as the extent to which an agent is designed to operate without user involvement; using the framework's 'Characteristics' column to classify agents is a proposed measurement procedure, not a derivation of the taxonomy from itself. The authors' prior work (e.g., Cocoa, IdeaSynth, Co-Gym) appears only as illustrative example systems in Table 1 and is not load-bearing for any central argument. The possible conflation of 'required assistance' with 'designed autonomy' is a validity or correctness concern about the evaluation proposal, not a circularity in the paper's reasoning. Therefore, no significant circularity is found.

Assumptions & free parameters 1 free parameters · 4 assumptions · 2 invented entities

The paper is a conceptual essay, so it introduces no fitted constants. The central load-bearing assumptions are the separability of autonomy from capability, the exhaustiveness of the five roles, the measurability of involvement, and the feasibility of third-party certification. The certificate and autonomy case are newly named governance artifacts without independent evidence.

free parameters (1)
  • Success threshold T
    In the assisted evaluation (Section 5), the task accuracy or completion threshold T determines which autonomy level an agent receives. Its value is left unspecified and is not derived from any principle or prior data.
assumptions (4)
  • domain assumption Autonomy is a design property separable from capability and operational environment.
    Stated as the central argument in Sections 1 and 3, and supported only by hypothetical examples rather than controlled evidence or a formal derivation.
  • domain assumption The five user roles form an exhaustive and ordinal spectrum of user involvement.
    Section 3 defines five levels without showing that all possible interaction patterns fall cleanly into one of these roles or that the levels are strictly ordered in the way the framework requires.
  • domain assumption User involvement can be captured in logs or diaries and mapped to the framework's characteristics.
    Section 5 assumes that post-hoc analysis of involvement logs or diaries can reliably classify an agent into an autonomy level, but no inter-rater reliability or validation of such classification is provided.
  • domain assumption A third-party governing body can run evaluations sufficient to certify an agent's maximum autonomy.
    Section 4 assumes the certificate process is feasible and that autonomy cases can provide convincing demonstrations. The analogy to safety cases is argued, but no implementation or pilot exists.
invented entities (2)
  • Autonomy certificate
    purpose: A digital document that prescribes the maximum autonomy level at which an agent may operate, issued by a third-party governing body and stored with agent metadata.
    Proposed in Section 4 with no implementation, pilot, or falsifiable test. It is a governance artifact whose value depends on the framework being operationalized.
  • Autonomy case
    purpose: An evidence-based argument, analogous to a safety case, that an agent behaves at or below a stated autonomy level.
    Introduced in Section 4.2 and not deployed. It relies on the same framework it is meant to certify, so it cannot serve as an independent check on the framework's validity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Levels of Autonomy for AI Agents." pith.science (2026). https://pith.science/paper/WCSZUJZB

@misc{pith2026250612469,
  author       = {Pith},
  title        = {Pith review of: Levels of Autonomy for AI Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WCSZUJZB}},
  note         = {Machine review of arXiv:2506.12469}
}
read the original abstract

Autonomy is a double-edged sword for AI agents, simultaneously unlocking transformative possibilities and serious risks. How can agent developers calibrate the appropriate levels of autonomy at which their agents should operate? We argue that an agent's level of autonomy can be treated as a deliberate design decision, separate from its capability and operational environment. In this work, we define five levels of escalating agent autonomy, characterized by the roles a user can take when interacting with an agent: operator, collaborator, consultant, approver, and observer. Within each level, we describe the ways by which a user can exert control over the agent and open questions for how to design the nature of user-agent interaction. We then highlight a potential application of our framework towards AI autonomy certificates to govern agent behavior in single- and multi-agent systems. We conclude by proposing early ideas for evaluating agents' autonomy. Our work aims to contribute meaningful, practical steps towards responsibly deployed and useful AI agents in the real world.

Figures

Figures reproduced from arXiv: 2506.12469 by the authors.

Figure 1
Figure 1. Our five levels of autonomy for AI agents, centered around the roles a user (human or AI) may take [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Our proposed procedure for the issuance of autonomy certificates. The agent developer would [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Dark Patterns Meet GUI Agents: LLM Agent Susceptibility to Manipulative Interfaces and the Role of Human Oversight

    cs.HC 2025-09 conditional novelty 6.0 of 10

    GUI agents frequently fall for deceptive interface designs, often without recognizing them, and human supervision of agents improves avoidance only partially while introducing new attention and workload costs.

  2. Agentic AI Autonomy Assessment: A Decision-Support Framework Towards Governed Supply Chain Systems

    cs.HC 2026-07 conditional novelty 5.0 of 10

    A task-level Autonomy Score built from initiative and consultation rates is proposed; a beer-game simulation suggests upstream supply chain tiers benefit from high autonomy while downstream tiers do not.

  3. Separating Capability from Permission: A Governance Framework for Agentic AI Autonomy Levels

    cs.AI 2026-07 conditional novelty 5.0 of 10

    Agentic AI governance should assign Allowed Autonomy Levels separately from Autonomous Capability Levels so high capability need not imply high permission.

  4. Interaction as Intelligence: Deep Research With Human-AI Partnership

    cs.CL 2025-07 reject novelty 5.0 of 10

    A human-in-the-loop deep research system with transparent, interruptible interaction is claimed to outperform commercial baselines, but the evidence is weakened by small samples and biased instructions.

  5. The Path to Self-Evolving Clinical Systems: Scaling Medical Agents from Assistance to Autonomy

    cs.AI 2026-07 conditional novelty 4.5 of 10

    Medical agents should be scaled mainly by richer clinical environments and self-evolution loops, not parameter growth alone, under a three-level autonomy taxonomy.

  6. Do we have the knowledge we need? Rethinking human-AI decision-making in corporations

    cs.AI 2026-04 conditional novelty 4.5 of 10

    Agency between humans and AI should be allocated dynamically from knowledge grounding (descriptive vs. prescriptive) and problem attributes (regularity, consequences, goals, solution space) via a TDE-loop.

Reference graph

Works this paper leans on

44 extracted references · 23 canonical work pages · cited by 6 Pith papers

  1. [1]

    Developing a computer use model

    Anthropic. Developing a computer use model. https://www.anthropic.com/news/ developing-computer-use, 2024

    work page 2024

  2. [2]

    Responsible Scaling Policy

    Anthropic. Responsible Scaling Policy. https://assets.anthropic.com/m/ 24a47b00f10301cd/original/Anthropic-Responsible-Scaling-Policy-2024-10-15. pdf, 2024

    work page 2024

  3. [3]

    G. A. Bekey. Autonomous robots: from biological inspiration to implementation and control. MIT press, 2005

    work page 2005

  4. [4]

    Bennett, O

    D. Bennett, O. Metatla, A. Roudaut, and E. D. Mekler. How does hci understand human agency and autonomy? In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–18, 2023

    work page 2023

  5. [5]

    M. D. Buhl, G. Sett, L. Koessler, J. Schuett, and M. Anderljung. Safety cases for frontier ai. arXiv preprint arXiv:2410.21572, 2024

    arXiv 2024

  6. [6]

    A. Chan, R. Salganik, A. Markelius, C. Pang, N. Rajkumar, D. Krasheninnikov, L. Langosco, Z. He, Y . Duan, M. Carroll, et al. Harms from increasingly agentic algorithmic systems. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pages 651–666, 2023

    work page 2023

  7. [7]

    Cihon, M

    P. Cihon, M. Stein, G. Bansal, S. Manning, and K. Xu. Measuring ai agent autonomy: Towards a scalable approach with code inspection. arXiv preprint arXiv:2502.15212, 2025

    arXiv 2025

  8. [8]

    K. M. Collins, I. Sucholutsky, U. Bhatt, K. Chandra, L. Wong, M. Lee, C. E. Zhang, T. Zhi-Xuan, M. Ho, V . Mansinghka, et al. Building machines that learn and think with people. arXiv preprint arXiv:2408.03943, 2024

    arXiv 2024

Show all 44 references
  1. [9]

    R. Crisp. Aristotle: nicomachean ethics. Cambridge University Press, 2014

    2014

  2. [10]

    Dafoe, A

    A. Dafoe, A. Dragan, F. Flynn, H. King, T. Lue, L. Ho, and R. Shah. Updating the frontier safety framework, Feb. 2025

    2025

  3. [11]

    Try Deep Research and our new experimental model in Gemini, your AI assistant

    Dave Citron. Try Deep Research and our new experimental model in Gemini, your AI assistant. https://blog.google/products/gemini/google-gemini-deep-research/ , 2024

    2024

  4. [12]

    R. Fang, D. Bowman, and D. Kang. V oice-enabled ai agents can perform common scams. arXiv preprint arXiv:2410.15650, 2024

    2024 arXiv

  5. [13]

    K. Feng, K. Pu, M. Latzke, T. August, P. Siangliulue, J. Bragg, D. S. Weld, A. X. Zhang, and J. C. Chang. Cocoa: Co-planning and co-execution with ai agents. arXiv preprint arXiv:2412.10999, 2024

    2024

  6. [14]

    What are AI agents?, 2025

    GitHub. What are AI agents?, 2025

    2025

  7. [15]

    Goemans, M

    A. Goemans, M. D. Buhl, J. Schuett, T. Korbak, J. Wang, B. Hilton, and G. Irving. Safety case template for frontier ai: A cyber inability argument. arXiv preprint arXiv:2411.08088, 2024

    2024 arXiv

  8. [16]

    C. Huyen. Agents. huyenchip.com, January 2025

    2025

  9. [17]

    Jansen, M.-A

    P. Jansen, M.-A. Côté, T. Khot, E. Bransom, B. D. Mishra, B. P. Majumder, O. Tafjord, and P. Clark. Discoveryworld: A virtual environment for developing and evaluating automated scientific discovery agents. arXiv preprint arXiv:2406.06769, 2024

    2024 arXiv

  10. [18]

    C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. Narasimhan. Swe-bench: Can language models resolve real-world github issues? arXiv preprint arXiv:2310.06770, 2023. 13 Levels of Autonomy for AI Agents

    2023 arXiv

  11. [19]

    Kapoor, B

    S. Kapoor, B. Stroebl, Z. S. Siegel, N. Nadgir, and A. Narayanan. Ai agents that matter. arXiv preprint arXiv:2407.01502, 2024

    2024 arXiv

  12. [20]

    S. Kim, J. R. Anthis, and S. Sebo. A taxonomy of robot autonomy for human-robot interaction. In Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction, HRI ’24, page 381–393, New York, NY , USA, 2024. Association for Computing Machinery

    2024

  13. [21]

    Korbak, J

    T. Korbak, J. Clymer, B. Hilton, B. Shlegeris, and G. Irving. A sketch of an ai control safety case. arXiv preprint arXiv:2501.17315, 2025

    2025 arXiv

  14. [22]

    Kulveit, R

    J. Kulveit, R. Douglas, N. Ammann, D. Turan, D. Krueger, and D. Duvenaud. Gradual disempowerment: Systemic existential risks from incremental ai development. arXiv preprint arXiv:2501.16946, 2025

    2025 arXiv

  15. [23]

    A. Li, Y . Zhou, V . C. Raghuram, T. Goldstein, and M. Goldblum. Commercial llm agents are already vulnerable to simple yet dangerous attacks. arXiv preprint arXiv:2502.08586, 2025

    2025 arXiv

  16. [24]

    Lieberman

    H. Lieberman. Autonomous interface agents. Proceedings of the ACM SIGCHI Conference on Human factors in computing systems, 1997

    1997

  17. [25]

    C. Lu, C. Lu, R. T. Lange, J. Foerster, J. Clune, and D. Ha. The ai scientist: Towards fully automated open-ended scientific discovery. arXiv preprint arXiv:2408.06292, 2024

    2024 arXiv

  18. [26]

    P. Maes. Agents that reduce work and information overload. Commun. ACM, 37:30–40, 1994

    1994

  19. [27]

    Manus AI - The AI Assistant That Turns Thoughts Into Actions.https://www.manusai.io/, 2025

    Manus. Manus AI - The AI Assistant That Turns Thoughts Into Actions.https://www.manusai.io/, 2025

    2025

  20. [28]

    Mitchell, A

    M. Mitchell, A. Ghosh, A. S. Luccioni, and G. Pistilli. Fully autonomous ai agents should not be developed. arXiv preprint arXiv:2502.02649, 2025

    2025

  21. [29]

    M. R. Morris, J. Sohl-Dickstein, N. Fiedel, T. Warkentin, A. Dafoe, A. Faust, C. Farabet, and S. Legg. Position: Levels of agi for operationalizing progress on the path to agi. In Forty-first International Conference on Machine Learning, 2024

    2024

  22. [30]

    Computer-Using Agent, 2025

    OpenAI. Computer-Using Agent, 2025

    2025

  23. [31]

    Introducing Operator

    OpenAI. Introducing Operator. https://openai.com/index/introducing-operator/, 2025

    2025

  24. [32]

    K. Pu, K. Feng, T. Grossman, T. Hope, B. D. Mishra, M. Latzke, J. Bragg, J. C. Chang, and P. Siangliu- lue. Ideasynth: Iterative research idea development through evolving and composing idea facets with literature-grounded feedback. arXiv preprint arXiv:2410.04025, 2024

    arXiv

  25. [33]

    Turn your ideas into apps with ai, 2025

    Replit. Turn your ideas into apps with ai, 2025

    2025

  26. [34]

    Russell and P

    S. Russell and P. Norvig. Artificial Intelligence: A Modern Approach. Prentice Hall, Englewood Cliffs, NJ, 1995

    1995

  27. [35]

    Schick, J

    T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Cancedda, and T. Scialom. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36, 2024

    2024

  28. [36]

    Y . Shao, V . Samuel, Y . Jiang, J. Yang, and D. Yang. Collaborative gym: A framework for enabling and evaluating human-agent collaboration. arXiv preprint arXiv:2412.15701, 2024

    2024

  29. [37]

    Stanford Encyclopedia of Philosophy. Agency. https://plato.stanford.edu/entries/ agency/, 2015

    2015

  30. [38]

    G. Wang, Y . Xie, Y . Jiang, A. Mandlekar, C. Xiao, Y . Zhu, L. J. Fan, and A. Anandkumar. V oyager: An open-ended embodied agent with large language models. ArXiv, abs/2305.16291, 2023

    2023 arXiv

  31. [39]

    S. Wu. Introducing Devin, the first AI software engineer. https://www.cognition.ai/blog/ introducing-devin, 2024

    2024

  32. [40]

    T. Xie, D. Zhang, J. Chen, X. Li, S. Zhao, R. Cao, J. H. Toh, Z. Cheng, D. Shin, F. Lei, et al. Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments. Advances in Neural Information Processing Systems, 37:52040–52094, 2024. 14 Levels of Aut...

    2024

  33. [41]

    F. F. Xu, Y . Song, B. Li, Y . Tang, K. Jain, M. Bao, Z. Z. Wang, X. Zhou, Z. Guo, M. Cao, et al. Theagentcompany: benchmarking llm agents on consequential real world tasks. arXiv preprint arXiv:2412.14161, 2024

    2024 arXiv

  34. [42]

    J. Yang, C. E. Jimenez, A. Wettig, K. Lieret, S. Yao, K. Narasimhan, and O. Press. Swe-agent: Agent-computer interfaces enable automated software engineering. arXiv preprint arXiv:2405.15793, 2024

    2024 arXiv

  35. [43]

    S. Yao, H. Chen, J. Yang, and K. Narasimhan. Webshop: Towards scalable real-world web interaction with grounded language agents. Advances in Neural Information Processing Systems, 35:20744–20757, 2022

    2022

  36. [44]

    A. Ye, J. Moore, R. Novick, and A. X. Zhang. Language models as critical thinking tools: A case study of philosophers. arXiv preprint arXiv:2404.04516, 2024. 15 Levels of Autonomy for AI Agents Level (User Role) Must-Have Controls Example Characteristics Example Systems Open Q...

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.