Pith.

Machine Learning · arXiv:2610.35845 v1 · submitted 26 Oct 2026 · 295 references

Work Agents: Systems, Capabilities, and Optimization

Zishan Xu, Enjun Du, Mingqi Gao, Cheng Yang, Junhao Shen, Zirong Chen, Zhiyuan Yao, Yifu Guo

show 8 more authors

Yuquan Lu, Jinyang Huang, Zhengxi Lu, Siyi Liu, Ziyu Zheng, Yongqi Zhang, Difan Zou, Weiwen Liu

The paper says

“As foundation models evolve from information generation to digital work execution, agents are increasingly capable of completing complex tasks across tools, applications, and workflows. However, existing research lacks a unified perspective on how agents reliably accomplish delegated work.”

more of the abstract

To address this gap, we present a work-centric, system-level perspective that views work agents as systems combining foundation models and runtime harnesses to execute tasks in stateful workspaces and deliver verifiable outcomes. We introduce two complementary dimensions, work persistence and work openness, to characterize working conditions, and classify digital work into five forms according to task scope and organization. Building on this framework, we identify five core capability families required for reliable work execution and examine approaches to model optimization, harness optimization, and their joint optimization. We further analyze existing evaluation methods and metrics from the perspective of work outcomes. Finally, we discuss future research directions, including sustained work delegation, adaptation to open environments, and the interaction between work execution and capability improvement. This paper provides a unified framework for understanding, optimizing, and evaluating work agents, supporting the development of systems capable of delivering reliable, verifiable, and practically valuable work outcomes.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 27 Oct 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

The passes disagreed. Reader: holds up. Skeptic: holds up only with conditions. Referee: major revision.

A careful, genuinely useful survey-with-framework whose persistence/openness axes mostly hold up; the weakest spot is the unstated boundary criterion behind the τ-bench quadrant assignment. This line is wrong

2 major objections · 6 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 295 references say.

The paper says, at §3.5, with §2.2.2 and §2.2.3

The review did not quote this passage.

Pith says: Objection 1 of 2 · major

The four-quadrant claim in Section 2.2.3 ('all four combinations appear in existing research') is load-bearing for the paper's assertion that work persistence and work openness are 'conceptually distinct', and that claim currently rests on exactly one high-persistence/low-openness example, τ-bench, with no other setting assigned to that quadrant in the Figure 2 discussion.

more of objection 1

The classification of τ-bench is not supported by the paper's own openness definition as written. Section 2.2.2 defines openness over 'the objects, interfaces, rules, and actors relevant to the task' and says the agent 'may also need to find out where the relevant state is held, how other actors have changed it, and which actions it is authorized to take'; the τ-bench user's confirmation gates the database update, so on a literal reading the agent must discover actor-supplied information during execution, and the same phenomenon is used in the same subsection to justify τ²-bench's high openness. The text never states the criterion that would exclude interaction contents within a known protocol (confirmations, messages) from the openness axis while still counting the τ²-bench user's state changes as open; the Figure 2 caption ('return paths through other participants') even suggests that participant actions drive persistence, which is the same phenomenon. A charitable reading is available — openness concerns the structure of the task world, not event contents exchanged within a specified protocol — but it must be stated, with a test that lets a reader classify any interactive benchmark. I recommend adding this criterion to Section 2.2.2 and then re-checking the quadrant assignments, since a reader who adopts the literal reading would see the two dimensions collapse in exactly the case that is supposed to separate them.

The paper says, at §3.5 (with §2.2.2 and §2.2.3)

The review did not quote this passage.

Pith says: Objection 2 of 2 · major

The paper's distinction between work forms and working conditions is not maintained at the Workspace Level. Section 2.2.3 states that the persistence/openness combinations 'describe the conditions of work rather than its form', while Section 3.5 defines the Workspace Level by the fact that 'the request no longer specifies every object involved' and the agent 'must establish which files, records, and …

more of objection 2

… applications bear on the work and how they are related' — phrasing nearly identical to the Section 2.2.2 definition of openness (the agent must establish during execution which objects are relevant and how they are represented and related). Section 3.5's closing sentence then maps the level's two concerns directly onto the openness and persistence axes. If the Workspace Level is defined by discovery, one of the five forms is defined by exactly the property assigned to the conditions axis, making the two parts of the framework partially redundant. Please sharpen Section 3.5's opening so that the level is defined by the scope and organization of the delegated object graph (multi-object consistency, version reconciliation, dependency tracing as the substance of the work) rather than by whether the request specified the objects; a brief worked example of a Workspace-Level task with a fully predefined object graph would settle the boundary.

6 minor objections

The paper says, at Throughout

Pith says: minor objection 1 of 6

The manuscript text contains numerous doubled-letter and spacing artifacts (for example, the running header rendered as 'EExxppaannddiinngg wwoorrkk ssccooppee aanndd ddeeppeennddeenncciieess' and a garbled repository/homepage line above the abstract); these should be cleaned in the production version.

The paper says, at Figure 4

Pith says: minor objection 2 of 6

The Figure 4 caption doubles as a classification appendix, listing per-entry rationale for dozens of works; moving that rationale to a supplementary table would make the caption readable and the level assignments easier to audit.

show 4 more

The paper says, at Table 10

Pith says: minor objection 3 of 6

Table 10 spans a very large portion of Section 5; moving it to an appendix and keeping in the main text the three-level summary (targets in 5.2.1, methods in 5.2.2, metrics in 5.3) would improve readability without losing content.

The paper says, at §5.3

Pith says: minor objection 4 of 6

The distinction between pass@k and passk is introduced in a single clause; a one-sentence worked example (for example, k=2 with one successful and one failed attempt) would make the all-attempts-must-pass reading of passk unambiguous.

The paper says, at §2.1, Eqs. (1)–(2)

Pith says: minor objection 5 of 6

Equations (1) and (2) use informal tuple-like notation for what is really a schematic account; a sentence noting that the notation is illustrative rather than a formal operational semantics would prevent over-reading.

The paper says, at §4.4.2

Pith says: minor objection 6 of 6

The proposed (M0, H0), (M1, H0), (M0, H1), (M1, H1) comparison protocol is a useful and honest contribution; consider making it a highlighted 'proposed protocol' call-out so that readers can cleanly separate the survey's synthesis from its forward-looking proposals.

What would settle it

Prong 1: apply the §2.2.2 openness definition to τ-bench by enumerating every task-relevant piece of information the agent must obtain from the simulated user during execution (requests, confirmations, corrections), and compare this with the paper's treatment of the τ²-bench user's actions as openness. If confirmations count as discovered information, Figure 2's top-left quadrant needs a new canonical instance or a stated exclusion criterion; if they do not count, state that criterion explicitly and re-check the other quadrant examples (SQuAD, DocVQA, GAIA, Workspace-Bench, OSWorld 2.0, VibeLifeBench) against it.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 27 Oct 2026

  1. Reader holds up confidence high
  2. Skeptic partly agrees: says it holds up only with conditions 3 min 19 s · 45,420 tokens read, 19,419 written Under §2.2.2's own definition, τ-bench, the canonical high-persistence/low-openness instance (§2.2.3), is partially open: user confirmations that keep responsibility open are actor information discovered during execution.
  3. Editor wrote the verdict above 1 min 14 s · 46,453 tokens read, 7,659 written
  4. Referee major revision: 2 major objections, 6 minor 3 min 20 s · 45,817 tokens read, 21,444 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

The takeaway: this is worth reading, and the part worth reading is Section 2. The paper defines a "work agent" as six elements (task, model, harness, workspace, outcome, verification) and introduces two working-condition axes — persistence (when responsibility closes) and openness (how much of the task world must be discovered during execution) — plus a five-form taxonomy of work scope. The rest is a broad survey of training, harness optimization, and evaluation, with roughly ninety recent works mapped onto the framework. It makes no empirical claims of its own; all numbers are attributed to cited sources, and the authors are explicitly honest about comparability limits (dataset sizes, trajectory content, cross-benchmark scores). The citation pattern looks careful, and the paper marks its own proposals as proposals, which is more than most surveys manage.

What it does well: the persistence/openness distinction genuinely cuts across long-horizon, computer-use, and office-agent research, and the five work forms organize the benchmark landscape usefully. The placements mostly felt right.

Soft spots, in proportion. The main one is the orthogonality of the two axes, which is the load-bearing novelty. Section 2.2.3 claims all four quadrant combinations appear in existing research, with τ-bench as the high-persistence/low-openness instance. The stress-test is half right: under the paper's own openness definition, τ-bench's user confirmations are actor information discovered during execution, so the cleanest low-openness/high-persistence example sits on an unstated boundary criterion. A charitable reading is available — openness concerns the structure of the task world the agent must establish, not contingent outcomes within a known protocol — and the τ-bench/τ²-bench contrast at the end of §2.2.3 gestures exactly that way. But the criterion is never stated. Fixable with a paragraph, not fatal; the axes remain analytically distinct. The part of the stress-test about closed-world monitoring sliding into openness I would reject: discovering which entries an incident affects is openness, the obligation to respond is persistence, and the paper's decomposition there is coherent.

The taxonomy is subjective by nature — no proof of exhaustiveness; the five forms and five capability families are chosen by judgment. That is characteristic of conceptual frameworks, not a flaw. The paper is long and somewhat repetitive.

Who it is for: anyone building or evaluating agents in software, office, or computer-use settings, and survey writers who need a vocabulary. It will not change what agents can do, but it gives the field shared coordinates. I would send it to peer review, with the referee asked to push on the openness boundary and the "all four combinations" claim. My verdict: accept with revision, not a reject.

The strongest objection

The central claim is the unification of agent research through two 'conceptually distinct yet complementary' dimensions, work persistence and work openness, with all four combinations instantiated in existing research (§2.2.3, Figure 2). The four-quadrant argument is load-bearing: if the axes are not cleanly separable, the framework's main novelty reduces to a single dimension with a confound. The paper's own definitions do not separate them. §2.2.2 defines openness over 'the objects, interfaces, rules, and actors relevant to the task' and counts learning 'how other actors have changed' the shared state as openness, as in τ²-bench. §2.2.1 defines persistence to include responsibility kept open because it 'depends on other participants,' citing τ-bench's withheld database updates 'until the user confirms them.' The user's confirmation is task-relevant information supplied by an actor that the agent must discover during execution; on a literal reading of the openness definition, τ-bench is at least partially open, and the paper never states the criterion that would exclude interaction contents (confirmations, messages) from the openness axis while still counting the τ²-bench user's actions as open. A charitable reading is available — openness concerns the structure of the task world, not contents exchanged within a known protocol — but the text does not state it, and the Figure 2 caption ('return paths through other participants') suggests participants' actions drive persistence, not openness. The closed-world monitoring obligation in §2.2.1 also slides toward openness once incidents arrive, since tracing affected components is itself classified as openness. So the high-persistence/low-openness quadrant may lack a clean empirical instance in the surveyed corpus.

I agree with the reader that the paper is a comprehensive, honest position piece whose central framework is definitional and largely coherent. The reader's flagged weakest assumption (sufficiency of the two axes) is real; my concern is a more specific, internally testable defect within the same chapter: the boundary between 'predefined task world' and 'discovered during execution' is under-specified, and the paper's single canonical instance of the high-persistence/low-openness quadrant, τ-bench, becomes partially open under the paper's own openness definition. If this lands, the four-quadrant claim (Figure 2) does not separate cleanly, and the claimed unification rests on a one-dimensional axis plus an unacknowledged interaction with actor-supplied information. This matters because the two dimensions are one of the two named conceptual contributions in the abstract, not a peripheral example. I do not consider this rejection-level: equations (1)–(2), the five work forms, the extensive benchmark mapping, and the authors' candid limitation statements — dataset-size non-comparability, verifier unreliability, single-run VERSE evidence, context-window limits — all support acceptance after a targeted revision. Hence CONDITIONAL: accept if the authors state the missing criterion distinguishing interaction contents from task-world discovery and verify that each quadrant retains a clean instance in the surveyed literature; if the criterion cannot be stated without conceding that persistence and openness co-vary, the contribution should be reframed accordingly. Agreement with the reader is partial: we both locate the risk in the two-axis chapter, but the reader emphasizes missing coordinates (exhaustiveness) while I emphasize a definitional ambiguity that threatens the axes' independence as stated.

The referee's report

If its definitions hold, this framework gives the field a useful common vocabulary: the six-element system view corrects the habit of scoring models in isolation, the five capability families are well illustrated by the running business-report example in Section 2.3, Section 3.4's criterion for distinguishing workflow stages from system-internal execution stages is careful, and the Section 5 trichotomy of evaluation targets, dimensions, and methods is practically usable. The reporting discipline is a genuine strength: numbers are attributed to cited sources (for example, the WorkForge ALE gains and the Nemotron SWE-bench gains in Section 4), and the authors state the limits of their own suggestions, including single-run recursive-self-improvement results in Section 4.4.3 and dataset-size comparability in Section 4.1.1. The intellectual risk is definitional: the persistence/openness axes and the boundary between work forms and working conditions are the paper's novel analytical apparatus, and their current wording invites misclassification of the paper's own canonical examples. These are fixable clarification issues, not flaws in the survey's empirical content.

Pith's reading of the paper (a machine's summary, not the authors')

This paper tries to establish a single definition of what an AI agent is when it does real work: a foundation model that decides, plus a runtime harness that observes, acts, and remembers, operating in a stateful workspace and judged by verifiable outcomes. It argues that this work-centric view unifies research that currently splits into long-horizon, computer-use, and office agents, because those lines each track one facet of the same delegated task. To make the unification usable, the paper proposes two coordinates for working conditions — persistence (does responsibility close at delivery or stay open across events?) and openness (is the task world given in advance or discovered during execution?) — and five forms of digital work, from single answers to whole-workspace operations. A sympathetic reader would care because the framework promises a common language for comparing benchmarks, tracing failures to one of five capability families, and deciding whether an improvement belongs in the model or in the harness.

What carries the argument. The load-bearing object is the six-element system definition with its execution–verification loop. At an assessment point the loop splits into two roles: execution writes $(T, E_0) \xrightarrow{M,H} (\tau_{0:t}, E_t, O_t)$, meaning the model–harness pair acts in the initial workspace and produces an action record, a new workspace state, and an outcome, while verification evaluates $S_t = V(T, E_0, \tau_{0:t}, E_t, O_t)$, meaning the evidence is checked against the task requirements by programs, humans, or model-based judges. Two dimensions order the working conditions, five work forms order the scope of the delegated work, and the argument is carried by showing that existing systems occupy distinct positions in this space: responsibility closes at delivery in question answering, remains open awaiting approval in service workflows, requires discovery in information seeking, and combines open discovery with open responsibility in ongoing work under changing conditions.

Core claim. On the paper's own terms, the central claim is that a work agent is best understood not as a model that generates outputs but as a system of six elements — task, model, harness, workspace, work outcome, and verification — bound together in a closed loop. The harness assembles context from the task and the workspace for the model; the model proposes actions; the harness executes them and returns observations; and verification assesses the resulting evidence against the task requirements. The same loop runs in every work-agent setting, and what varies are the conditions, which the paper captures with two dimensions: work persistence, from responsibility that closes at delivery to responsibility that stays open across stages, feedback cycles, and later events, and work openness, from a predefined task world to one the agent must discover while acting. The paper classifies digital work into five forms by scope — conversation, atomic action, multi-step, workflow, and workspace — and derives five capability families from the work process: perception and understanding, reasoning and decision making, planning and coordination, execution and state management, and verification, recovery, and replanning. It then reads the field's optimization and evaluation literature through this frame, arguing that capabilities can be improved by training the model, by upgrading the harness, or by adapting the two jointly.

The premise it rests on. The load-bearing premise is that two axes — how long responsibility stays open and how much of the task world the agent must discover — are the right and sufficient coordinates for working conditions, and that all digital knowledge work fits the five proposed forms; if a third factor such as collaboration structure or risk is equally primary, the promised unification is weaker than the paper claims.

If the paper is right (Pith's inferences, not the authors')

Question answering, single-file edits, multi-step projects, staged workflows, and whole-workspace operations become one spectrum of work forms, so methods and benchmarks designed at one level can be positioned, compared, and reused at another.

Failures can be routed to a specific capability — missing evidence to perception, a questionable decision to reasoning, a changed prerequisite to planning, an execution error to state management — which directs engineers to train the model or change the harness.

Evaluation should inspect outcomes through evidence — artifacts, workspace states, workflow records, and maintained conditions — rather than the final answer alone, because acceptance depends on the object actually delivered.

Optimization decomposes into model optimization, harness optimization, and joint adaptation, and the paper proposes comparing $(M_0,H_0)$, $(M_1,H_0)$, $(M_0,H_1)$, and $(M_1,H_1)$ under consistent task conditions to attribute improvements to the right component.

Because persistence and openness raise the demands on all five capability families, benchmarks that are high on both axes become the testbed for sustained delegation, which the paper identifies as the field's frontier.

If the two dimensions were adopted as a standard annotation, benchmark families could be plotted on a single map, making progress across software, office, and enterprise domains comparable without a shared test set.

A prediction the paper leaves implicit: agents trained in high-persistence, high-openness environments should transfer to ordinary office work better than agents trained on delivery-close, predefined tasks, because the former must practice verification, dependency tracking, and state management.

The five work forms suggest a curriculum ordering — workspace-level competence presupposes the mechanisms of the lower forms — so staged training that introduces openness and persistence gradually may be more sample-efficient than uniform task mixing.

Because verification is defined as part of the system rather than an external judge, the framework implies that agents which close the verification loop themselves are closer to deployment-ready; this could be tested by ablating an agent's internal verification loop and comparing against an external evaluator.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

As foundation models evolve from information generation to digital work execution, agents are increasingly capable of completing complex tasks across tools, applications, and workflows. However, existing research lacks a unified perspective on how agents reliably accomplish delegated work. To address this gap, we present a work-centric, system-level perspective that views work agents as systems combining foundation models and runtime harnesses to execute tasks in stateful workspaces and deliver verifiable outcomes. We introduce two complementary dimensions, work persistence and work openness, to characterize working conditions, and classify digital work into five forms according to task scope and organization. Building on this framework, we identify five core capability families required for reliable work execution and examine approaches to model optimization, harness optimization, and their joint optimization. We further analyze existing evaluation methods and metrics from the perspective of work outcomes. Finally, we discuss future research directions, including sustained work delegation, adaptation to open environments, and the interaction between work execution and capability improvement. This paper provides a unified framework for understanding, optimizing, and evaluating work agents, supporting the development of systems capable of delivering reliable, verifiable, and practically valuable work outcomes.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-27

The authors' main claim, as the classifier picked it out 1.00

“This paper provides a unified framework for understanding, optimizing, and evaluating work agents, supporting the development of systems capable of delivering reliable, verifiable, and practically valuable work outcomes.”

How the main claim is stated: asserted 0.99

What the abstract itself states, as probabilities: claims first 0.13 · states limitation 0.10 · human participants 0.05 · controlled comparison 0.02 · randomized assignment 0.01 · states quantitative result 0.37 · compares with prior results 0.13 · mentions code or data availability 0.02

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
As foundation models evolve from information generation to digital work execution, agents are increasingly capable of completing complex tasks across tools, applications, and workflows. background 1.00 0.06 0.03 asserted 0.94
However, existing research lacks a unified perspective on how agents reliably accomplish delegated work. gap 1.00 0.08 0.02 asserted 1.00
To address this gap, we present a work-centric, system-level perspective that views work agents as systems combining foundation models and runtime harnesses to execute tasks in stateful workspaces and deliver verifiable outcomes. contribution 1.00 0.88 0.15 asserted 0.99
We introduce two complementary dimensions, work persistence and work openness, to characterize working conditions, and classify digital work into five forms according to task scope and organization. contribution 0.96 0.94 0.04 asserted 0.99
Building on this framework, we identify five core capability families required for reliable work execution and examine approaches to model optimization, harness optimization, and their joint optimization. contribution 0.99 0.87 0.06 asserted 1.00
We further analyze existing evaluation methods and metrics from the perspective of work outcomes. contribution 0.95 0.60 0.03 asserted 0.92
Finally, we discuss future research directions, including sustained work delegation, adaptation to open environments, and the interaction between work execution and capability improvement. future work 1.00 0.40 0.05 asserted 0.53
This paper provides a unified framework for understanding, optimizing, and evaluating work agents, supporting the development of systems capable of delivering reliable, verifiable, and practically valuable work outcomes. contribution 0.90 0.81 0.92 asserted 0.99

What kind of paper the classifier reports this is: position or commentary 0.52 · evidence from argument 0.68

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-27: 1 request, 6,206 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (11)
Figure 1 of the paper
Figure 1. A system view of work agents. The model and harness form the work agent within the dashed boundary. The task specifies the objective, constraints, resources, permissions, and completion conditions. The harness supplies context to the model, carries out proposed actions in the stateful workspace, and receives observations. Outcomes include answers, artifacts, state changes, and process outcomes. For ongoing responsibilities, they also include maintained conditions and event responses over an agreed interval. Verification assesses evidence from these outcomes, workspace states, and…
Figure 2 of the paper
Figure 2. Work persistence and work openness as conceptually distinct dimensions of working conditions. Persistence ranges from responsibility that closes at delivery to responsibility that remains open across stages and events; openness ranges from a predefined task world to one discovered during execution. The four scenes show question answering (bottom left), information seeking (bottom right), service workflows (top left), and ongoing work under changing conditions (top right). Supplied packets distinguish the predefined task worlds; scattered material and magnifiers indicate that…
Figure 3 of the paper
Figure 3. Five core capabilities supporting an illustrative business-report update. New data in the source workbook motivate revisions of the report, charts, and slides. The numbered panels identify the five functions, and solid arrows trace evidence, approaches, plans, actions, observations, and results. Shared work context retains task requirements, evidence and object relationships, plans and pending dependencies, and observed state with check status. Verification supports delivery when current requirements are met. Orange dashed paths indicate possible recovery, replanning, or continued…
Figure 4 of the paper
Figure 4. Evolution of agent work through 90 representative works in 2023–2026. Colors follow the five work forms in Section 3 and classify selected task settings, not entire methods or frameworks. Method settings include Generative Agents interviews, Self-Refine FED dialogue, MemGPT MSC dialogue, numerical answers in Toolformer, DSPy, AFlow and Agent Lightning, CRITIC web-evidence QA, AutoGen and Reflexion ALFWorld tasks, Magentic-One GAIA tasks, a specified UI-TARS slide edit, ReasoningBank full purchase-history lookup, and an EnvCraft released literature-synthesis task. Benchmark settings…
Figure 5 of the paper
Figure 5. Optimization targets in a work-agent system. Training environments, corpora, trajectories, and outcome feedback inform candidate changes to the model, harness, or their coordination. Model optimization changes learned knowledge and behavior; harness optimization changes interfaces, workflows, memory, and skills. Joint adaptation coordinates model and harness changes. Defects in the training environment may require environment repair. Comparisons on subsequent work under consistent task conditions and compatible interfaces test whether a candidate update helps. The optimizer proposes…
Figure 6 of the paper
Figure 6. Training emphases that develop the model. Illustrative samples distinguish visible context from prediction targets and feedback. Pre-training builds semantic, computational, structural, and visual knowledge. Mid-training enriches domain knowledge, introduces interaction patterns, and develops long-context use in preparation for subsequent optimization. Post-training uses demonstrations, execution rewards, teacher guidance, and accumulated feedback to supervise decisions, trajectories, artifacts, or earlier attempts. Links to the model denote learning signals, not live execution.…
Figure 7 of the paper
Figure 7. Model–harness coupling and improvement across training executions. The current harness shapes experience used to train the model; model updates can motivate harness adaptation. Evidence informs candidate changes to executor weights θ, harness h, or training environment E. Optimizer G proposes and tests changes, and accepted versions supply subsequent rollouts. The paths illustrate possible component updates, without prescribing their order or requiring every target to change. A separate relation denotes recursive self-improvement of G’s search and diagnostic/testing procedures.…
Figure 8 of the paper
Figure 8. Work domains, settings, and representative knowledge requirements across 79 benchmarks. Studies are grouped by their primary domain or work setting, following the groups of
Figure 9 of the paper
Figure 9. Research directions for next-generation work agents. Continuing service around a user’s goals, representation of a particular person’s judgment, and coordination across organizational roles motivate advances in environment, persistence, and verification. Interacting execution and improvement loops support these responsibilities, while accepted work, value in use, and total effort guide the assessment of beneficial delegation. 6.1 Future Applications: From Assistance to Delegated Work The three applications concern different parts of the same reporting process: the manager’s…
Figure 10 of the paper
Figure 10. Work execution and capability improvement during continuing service. A persistent work record retains agreed requirements, source versions, unfinished work, and completed actions. Execution history and feedback from downstream use inform targeted updates. Independent tests assess capability gains, while checks on current work identify affected results and commitments. This evidence guides whether to apply an update, migrate affected work, or retain the current procedure. The benefit of an update depends on its gains relative to the resulting rework and coordination costs. 6.3.1 The…
Figure 11 of the paper
Figure 11. Assessing beneficial delegation across work conditions. The four scenarios illustrate variation in work openness and persistence: single reports from given sources or in an unfamiliar workspace, and recurring reports within a fixed workflow or a changing workspace. Across all four settings, accepted work, value in use, and total human and computational effort provide complementary evidence for assessing delegation. The division of work between people and agents is part of this comparison. In a product-innovation experiment, individuals with AI assistance matched human teams without…

Figures are the authors', reproduced from the paper.

References (295)
  1. [1] Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the Opportunities and Risks of Foundation Models.arXiv preprint arXiv:2108.07258, 2021. URLhttps://arxiv.o... on PithPith's reviewarXiv
  2. [2] Tong Zheng, Xidong Wu, Zheng Zhang, Zhankui He, Chaoyi Zhang, Benjamin Coleman, Ruoqiao Wei, Di Bai, Haolin Liu, Rui Liu, et al. Dream-rsi: Recursive self-improvement through evolving worlds.arXiv preprint arXiv:2609.14858, 2026 on PithPith's reviewarXiv
  3. [3] John Yang, Carlos Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering. In Advances in Neural Information Processing Systems, volume 37, pages 50528–50652. Curran As... on Pithdoi:10.52202/079017-1601
  4. [4] Hongxin Li, Jingran Su, Yuntao Chen, Qing Li, and ZHAO-XIANG ZHANG. SheetCopilot: Bringing Software Productivity to the Next Level through Large Language Models. InAdvances in Neural Information Processing Systems, volume 36, pages 4952–4984. Curran Associates, Inc., 2023. doi... on Pithdoi:10.52202/075280-0220
  5. [5] Hao Zheng, Xinyan Guan, Hao Kong, Wenkai Zhang, Jia Zheng, Weixiang Zhou, Hongyu Lin, Yaojie Lu, Xianpei Han, and Le Sun. PPTAgent: Generating and Evaluating Presentations Beyond Text-to-Slides. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Pro... on Pithdoi:10.18653/v1/2025.emnlp-main.728
  6. [6] Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. WebArena: A Realistic Web Environment for Building Autonomous Agents. InInternational Conference on Learning Representat... on Pith
  7. [7] Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, and Tao Yu. OSWorld: Benchmarking Multimodal Agents for Op... on Pithdoi:10.52202/079017-1650
  8. [8] Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxuan Zhang, Juanzi Li, et al. CogAgent: A Visual Language Model for GUI Agents. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. URLhttps... on PithPith's reviewarXiv
  9. [9] Alexandre Drouin, Maxime Gasse, Massimo Caccia, Issam H. Laradji, Manuel Del Verme, Tom Marty, David Vazquez, Nicolas Chapados, and Alexandre Lacoste. WorkArena: How Capable are Web Agents at Solving Common Knowledge Work Tasks? InProceedings of the 41st International Conferen... on Pith
  10. [10] Ziru Chen, Shijie Chen, Yuting Ning, Qianheng Zhang, Boshi Wang, Botao Yu, Yifei Li, Zeyi Liao, Chen Wei, Zitong Lu, Vishal Dey, Mingyi Xue, Frazier N. Baker, Benjamin Burns, Daniel Adu-Ampratwum, Xuhui Huang, Xia Ning, Song Gao, Yu Su, and Huan Sun. ScienceAgentBench: Toward ... on Pith
  11. [11] OpenAI. Codex Manual, n.d. URL https://developers.openai.com/codex/codex-manual.md . Accessed October 22, 2026 on Pith
  12. [12] Anthropic. Claude Code Overview, n.d. URLhttps://code.claude.com/docs/en/overview. Accessed October 22, 2026. 63 on Pith
  13. [13] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing Reasoning and Acting in Language Models. InInternationalConference on Learning Representations, on Pith
  14. [14] Theodore R. Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas L. Griffiths. Cognitive Architectures for Language Agents. Transactions on Machine Learning Research, 2024. URL https://arxiv.org/abs/2309.024 27 on Pith
  15. [15] Mingchen Zhuge, Changsheng Zhao, Dylan R. Ashley, Wenyi Wang, Dmitrii Khizbullin, Yunyang Xiong, Zechun Liu, Ernie Chang, Raghuraman Krishnamoorthi, Yuandong Tian, Yangyang Shi, Vikas Chandra, and Jürgen Schmidhuber. Agent-as-a-Judge: Evaluate Agents with Agents. In Proceeding... on Pith
  16. [16] Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. SWE-bench: Can Language Models Resolve Real-world Github Issues? InInternational Conference on Learning Representations, pages 54107–54157, 2024. URLhttps://proceedings.ic... on Pith
  17. [17] Xin-Sheng Chen, Jiayu Zhu, Pei-lin Li, et al. PresentBench: A Fine-Grained Rubric-Based Benchmark for Slide Generation. arXiv preprint, 2026. URLhttps://arxiv.org/abs/2603.07244 on PitharXiv
  18. [18] Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. SQuAD: 100,000+ Questions for Machine Comprehension of Text. InProceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392. Association for Computational Linguistics,... on Pithdoi:10.18653/v1/d16-1264
  19. [19] Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. InProceedings of the 2018 Conference on Empirical Methods in Natural Language Proces... on Pithdoi:10.18653/v1/d18-1259
  20. [20] Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, and Tat- Seng Chua. TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance. In Proceedings of the 59th Annual Meeting of the Association for Computa... on Pithdoi:10.18653/v1/2021.acl-long.254
show 80 more
  1. [21] Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, and William Yang Wang. FinQA: A Dataset of Numerical Reasoning over Financial Data. In Proceedings of the 2021 Conference on Empirical M... on Pithdoi:10.18653/v1/2021
  2. [22] Minesh Mathew, Dimosthenis Karatzas, and C. V. Jawahar. DocVQA: A Dataset for VQA on Document Images. In 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 2199–2208. IEEE, 2021. doi: 10.1109/wacv48630.2021.00225. URLhttps://doi.org/10.1109/wacv48630.... on Pithdoi:10.1109/wacv48630.2021.00225
  3. [23] Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning. InFindings of the Association for Computational Linguistics: ACL 2022, pages 2263–2279. Association for Computati... on Pithdoi:10.18653/v1/2022.findings-acl.177
  4. [24] Timo Schick, Jane Dwivedi-Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language Models Can Teach Themselves to Use Tools. In Advances in Neural Information Processing Systems, volume 36, page... on Pithdoi:10.52202/07528
  5. [25] Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. API-Bank: AComprehensiveBenchmarkforTool-AugmentedLLMs. In Proceedings ofthe2023Conferenceon Empirical Methods in Natural Language Processing, pages 3102–3116. Associ... on Pith
  6. [26] Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. Gorilla: Large Language Model Connected with Massive APIs. InAdvances in Neural Information Processing Systems, volume 37, pages 126544–126565, on Pith
  7. [27] Chris Rawles, Sarah Clinckemaillie, Yifan Chang, Jonathan Waltz, Gabrielle Lau, Marybeth Fair, Alice Li, William Bishop, Wei Li, Folawiyo Campbell-Ajala, Daniel Toyama, Robert Berry, Divya Tyamagundlu, Timothy Lillicrap, and Oriana Riva. AndroidWorld: A Dynamic Benchmarking En... on Pith
  8. [28] doi: 10.18653/v1/2023.emnlp-main.187. URLhttps://aclanthology.org/2023.emnlp-main.187/. 64 on Pithdoi:10.18653/v1/2023.emnlp-main.187
  9. [29] Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan.τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. InInternational Conference on Learning Representations, pages 9965–10017, on Pith
  10. [30] doi: 10.52202/079017-4020. URL https://proceedings.neurips.cc/paper_files/paper/2024/hash/e 4c61f578ff07830f5c37378dd3ecb0d-Abstract-Conference.html on Pithdoi:10.52202/079017-4020
  11. [31] Frank (Fangzheng) Xu, Yufan Song, Boxuan Li, Yuxuan Tang, Kritanjali Jain, Mengxue Bao, Zora Wang, Xuhui Zhou, Zhitong Guo, Murong Cao, Mingyang Yang, Hao Yang Lu, Amaad Martin, Zhe Su, Leander Maben, Raj Mehta, Wayne Chi, Lawrence Jang, Yiqing Xie, Shuyan Zhou, and Graham Neu... on Pithdoi:10.52202/085713-0315
  12. [32] Xingyao Wang, Zihan Wang, Jiateng Liu, Yangyi Chen, Lifan Yuan, Hao Peng, and Heng Ji. MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback. InInternational Conference on Learning Representations, pages 32593–32627, 2024. URLhttps://proceedings.iclr... on Pith
  13. [33] Wei Fan, Xinjie Shen, Xudong Guo, et al. E-Commerce Bench: Evaluating LLM Agents on Long-Horizon Autonomous Business Operation.arXiv preprint arXiv:2608.30730, 2026. URL https://arxiv.org/abs/2608 .30730 on PithPith's reviewarXiv
  14. [34] Xiaohongshu Dots Studio and Evolvent AI. VibeLifeBench: Can your life agent be proactive and persistent in a living world? arXiv preprint arXiv:2608.10875, 2026. URLhttps://arxiv.org/abs/2608.10875 on PithPith's reviewarXiv
  15. [35] Victor Barres, Honghua Dong, Soham Ray, Xujie Si, and Karthik R Narasimhan. τ 2-Bench: Evaluating Conversational Agents in a Dual-Control Environment. InProceedings of the 43rd International Conference on Machine Learning, volume 306 ofProceedings of Machine Learning Research,... on Pith
  16. [36] Zirui Tang, Xuanhe Zhou, Yumou Liu, Linchun Li, Weizheng Wang, Hongzhang Huang, Jun Zhou, Jiachen Song, Shaoli Yu, Jinqi Wang, Zihang Zhou, Hongyi Zhou, Yuting Lv, Jinyang Li, Jiashuo Liu, Ruoyu Chen, Chunwei Liu, GuoLiang Li, Jihua Kang, and Fan Wu. Workspace-Bench 1.0: Bench... on Pith
  17. [37] Mengqi Yuan, Zilong Zhou, Xinzhuang Xiong, et al. OSWorld 2.0: Benchmarking Computer Use Agents on Long- Horizon Real-World Tasks.arXiv preprint arXiv:2606.29537, 2026. URL https://arxiv.org/abs/2606.29537 on PithPith's reviewarXiv
  18. [38] Tao Yu, Hao Wang, Changyu Li, Shenghua Chai, Minghui Zhang, Zhongtian Luo, Yuxuan Zhou, Haopeng Jin, Zhaolu Kang, Jiabing Yang, YiFan Zhang, Xinming Wang, Hongzhu Yi, Zheqi He, Jing-Shu Zheng, Xi Yang, Yan Huang, and Liang Wang. Beyond the All-in-One Agent: Benchmarking Role-S... on PithPith's reviewarXiv
  19. [39] Quan Shi, Alexandra Zytek, Pedram Razavi, Karthik Narasimhan, and Victor Barres.τ-Knowledge: Evaluating Conversational Agents over Unstructured Knowledge.arXiv preprint arXiv:2603.04370, 2026. doi: 10.48550/a rXiv.2603.04370. URLhttps://arxiv.org/abs/2603.04370 on Pithdoi:10.48550/arxiv.2603.04370
  20. [40] Shishir G. Patil, Huanzhi Mao, Charlie Cheng-Jie Ji, Fanjia Yan, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. The berkeley function calling leaderboard (bfcl): From tool use to agentic evaluation of large language models. InForty-secondInternational Conference on Machine... on Pith
  21. [41] Houxing Ren, Mingjie Zhan, Zimu Lu, Ke Wang, Yunqiao Yang, Haotian Hou, and Hongsheng Li. Towards Robust Real-World Spreadsheet Understanding with Multi-Agent Multi-Format Reasoning. InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Vol... on Pithdoi:10.18653/v1/2026.acl-long.86
  22. [42] Harsh Trivedi, Tushar Khot, Mareike Hartmann, Ruskin Manku, Vinty Dong, Edward Li, Shashank Gupta, Ashish Sabharwal, and Niranjan Balasubramanian. AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents. InProceedings of the 62nd Annual Mee... on Pithdoi:10.18653/v1/2024.acl-long.850
  23. [43] Yuntong Zhang, Haifeng Ruan, Zhiyu Fan, and Abhik Roychoudhury. AutoCodeRover: Autonomous Program Improvement. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, pages 1592–1604. ACM, 2024. doi: 10.1145/3650212.3680384. URLhttps://... on Pithdoi:10.1145/3650212.3680384
  24. [44] Tianyang Liu, Canwen Xu, and Julian McAuley. RepoBench: Benchmarking Repository-Level Code Auto- Completion Systems. InInternational Conference on Learning Representations, pages 47832–47850, 2024. URL https://proceedings.iclr.cc/paper_files/paper/2024/file/d191ba4c8923ed8fd89... on Pith
  25. [45] Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. GAIA: a benchmark for General AI Assistants. InInternational Conference on Learning Representations, pages 9025–9049, 2024. URL https://proceedings.iclr.cc/paper_files/paper/2024/file/25ae35b5b1... on Pith
  26. [46] Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. Executable Code Actions Elicit Better LLM Agents. In Proceedings of the 41st International Conference on Machine Learning, volume 235 ofProceedings of Machine Learning Research, pages 50208–5... on Pith
  27. [47] Haoyu Dong, Jianbo Zhao, Yuzhang Tian, Junyu Xiong, Mengyu Zhou, Yun Lin, José Cambronero, Yeye He, Shi Han, and Dongmei Zhang. Encoding Spreadsheets for Large Language Models. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 2072... on Pithdoi:10.18653/v1/2024.emnlp-main.1154
  28. [48] Guancheng Zeng, Xueyi Chen, Jiawang Hu, Shaohua Qi, Yaxuan Mao, Zhantao Wang, Yifan Nie, Shuang Li, Qiuyang Feng, Pengxu Qiu, Yujia Wang, Wenqiang Han, Linyan Huang, Gang Li, Jingjing Mo, and Haowen Hu. Routine: A structural planning framework for llm agent system in enterpris... on PithPith's reviewarXiv
  29. [49] Hanchao Liu, Rongjun Li, Weimin Xiong, Ziyu Zhou, and Wei Peng. WorkTeam: Constructing Workflows from Natural Language with Multi-Agents. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Languag... on Pithdoi:10.18653/v1/2025.naacl-industry.3
  30. [50] Yibin Chen, Yifu Yuan, Zeyu Zhang, YAN ZHENG, Jinyi Liu, Fei Ni, Jianye HAO, Hangyu Mao, and Fuzheng Zhang. SheetAgent: Towards a Generalist Agent for Spreadsheet Reasoning and Manipulation via Large Language Models. In THE WEB CONFERENCE 2025, 2025. URLhttps://openreview.net/... on Pith
  31. [51] Sirui Hong, Yizhang Lin, Bang Liu, Bangbang Liu, Binhao Wu, Ceyao Zhang, Danyang Li, Jiaqi Chen, Jiayi Zhang, Jinlin Wang, Li Zhang, Lingyao Zhang, Min Yang, Mingchen Zhuge, Taicheng Guo, Tuo Zhou, Wei Tao, Robert Tang, Xiangtao Lu, Xiawu Zheng, Xinbing Liang, Yaying Fei, Yuhe... on Pith
  32. [52] Bo Liu, Yuqian Jiang, Xiaohan Zhang, Qiang Liu, Shiqi Zhang, Joydeep Biswas, and Peter Stone. Llm+p: Empowering large language models with optimal planning proficiency.arXiv preprint arXiv:2304.11477, 2023. URLhttps://arxiv.org/abs/2304.11477 on PithPith's reviewarXiv
  33. [53] Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, Ahmed Hassan Awadallah, Ryen W White, Doug Burger, and Chi Wang. AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversations. InFirst Confer... on Pith
  34. [54] Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, zili wang, Steven Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber. MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework. ... on Pith
  35. [55] Thomas W. Malone and Kevin Crowston. The interdisciplinary study of coordination.ACM Computing Surveys, 26(1):87–119, 1994. doi: 10.1145/174666.174668 on Pithdoi:10.1145/174666.174668
  36. [56] Zeyao Ma, Bohan Zhang, Jing Zhang, Jifan Yu, Xiaokang Zhang, Xiaohan Zhang, Sijia Luo, Xi Wang, and Jie Tang. SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulation. InAdvances in Neural Information Processing Systems, volume 37, pages 94871–94908, 2024. doi... on Pithdoi:10.52202/079017-3007
  37. [57] Saaket Agashe, Kyle Wong, Vincent Tu, Jiachen Yang, Ang Li, and Xin Eric Wang. Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents. InSecond Conference on Language Modeling, 2025. URLhttps://openreview.net/forum?id=zg5is4GJ3R on Pith
  38. [58] Hao Zheng, Guozhao Mo, Xinru Yan, Qianhao Yuan, Wenkai Zhang, Xuanang Chen, Yaojie Lu, Hongyu Lin, Xianpei Han, and Le Sun. DeepPresenter: Environment-Grounded Reflection for Agentic Presentation Generation. In Findings of the Association for Computational Linguistics: ACL 202... on Pithdoi:10.18653/v1/2026.findings-acl.1578
  39. [59] AmanMadaan, NiketTandon, PrakharGupta, SkylerHallinan, LuyuGao, SarahWiegreffe, UriAlon, NouhaDziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-Refine: Iterative Refine... on Pithdoi:10.52202/075280-2019
  40. [60] Chenglin Yu, Yuchen Wang, Songmiao Wang, Hongxia Yang, and Li Ming. InfiAgent: An Infinite-Horizon Frame- work for General-Purpose Autonomous Agents. InFindings of the Association for Computational Linguistics: ACL 2026, pages 35884–35894. Association for Computational Linguis... on Pithdoi:10.18653/v1/2026.finding
  41. [61] Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman, Haohan Wang, and Yu-Xiong Wang. Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models. InProceedings of the 41st International Conference on Machine Learning, volume 235 ofProceedings of Machine Le... on Pith
  42. [62] Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. CRITIC: Large language models can self-correct with tool-interactive critiquing. InInternational Conference on Learning Representations, 2024. URLhttps://arxiv.org/abs/2305.11738 on PithPith's reviewarXiv
  43. [63] Thomas Kwa, Ben West, Joel Becker, Amy Deng, Katharyn Garcia, Max Hasin, Sami Jawhar, Megan Kinniment, Nate Rush, Sydney Von Arx, Ryan Bloom, Thomas Broadley, Haoxing Du, Brian Goodrich, Nikola Jurkovic, Luke Miles, Seraphina Nix, Tao Lin, Neev Parikh, David Rein, Lucas Jun Ko... on Pith
  44. [64] Hjalmar Wijk, Tao Roa Lin, Joel Becker, Sami Jawhar, Neev Parikh, Thomas Broadley, Lawrence Chan, Michael Chen, Joshua M Clymer, Jai Dhyani, Elena Ericheva, Katharyn Garcia, Brian Goodrich, Nikola Jurkovic, Megan Kinniment, Aron Lajko, Seraphina Nix, Lucas Jun Koba Sato, Willi... on Pith
  45. [65] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: language agents with verbal reinforcement learning. InAdvances in Neural Information Processing Systems, volume 36, pages 8634–8652, 2023. doi: 10.52202/075280-0377. URLhttps://procee... on Pithdoi:10.52202/075280-0377
  46. [66] Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. Evaluating Very Long-Term Conversational Memory of LLM Agents. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p... on Pithdoi:10.18653/v1/2024.acl-long.747
  47. [67] Guanting Dong, Xiaoshuai Song, Yuyang Hu, et al. Towards Long-Horizon Agents: A Survey — Foundation, Evolution, Harness, Optimization, Application, and Frontier, 2026. URLhttps://github.com/RUC-NLPIR/Awe some-Long-Horizon-Agents on Pith
  48. [68] Qianhui Wu, Kanzhi Cheng, Rui Yang, Chaoyun Zhang, Jianwei Yang, Huiqiang Jiang, Jian Mu, Baolin Peng, Bo Qiao, Reuben Tan, Si Qin, Lars Liden, Qingwei Lin, Huan Zhang, Tong Zhang, Jianbing Zhang, Dongmei Zhang, and Jianfeng Gao. GUI-Actor: Coordinate-Free Visual Grounding for... on Pithdoi:10.52202/085713-0508
  49. [69] Yadong Lu, Jianwei Yang, Yelong Shen, and Ahmed Awadallah. OmniParser for Pure Vision Based GUI Agent. arXiv preprint arXiv:2408.00203, 2024. URLhttps://arxiv.org/abs/2408.00203 on PithPith's reviewarXiv
  50. [70] Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai-Wei Chang, and Dong Yu. LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory. InInternational Conference on Learning Representations, pages 86809–86836, 2025. URLhttps://proceedings.iclr.cc/paper_files/paper... on Pith
  51. [71] Zhengxi Lu, Jiabo Ye, Fei Tang, Yongliang Shen, Haiyang Xu, Ziwei Zheng, Weiming Lu, Ming Yan, Fei Huang, Jun Xiao, et al. Ui-s1: Advancing gui automation via semi-online reinforcement learning.arXiv preprint arXiv:2509.11543, 2025 on PitharXiv
  52. [72] Boyu Gou, Ruohan Wang, Boyuan Zheng, Yanan Xie, Cheng Chang, Yiheng Shu, Huan Sun, and Yu Su. Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents. InInternational Conference on Learning Representations, 2025. URLhttps://arxiv.org/abs/2410.05243 on PithPith's reviewarXiv
  53. [73] Hongliang He, Wenlin Yao, Kaixin Ma, Wenhao Yu, Yong Dai, Hongming Zhang, Zhenzhong Lan, and Dong Yu. WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1:... on Pithdoi:10.18653/v1/2024.acl-long.371
  54. [74] Zhiyong Wu, Chengcheng Han, Zichen Ding, Zhenmin Weng, Zhoumianze Liu, Shunyu Yao, Tao Yu, and Lingpeng Kong. OS-Copilot: Towards Generalist Computer Agents with Self-Improvement. InICLR 2024 Workshop on Large Language Model Agents, 2024. URLhttps://openreview.net/forum?id=3WWFrg8UjJ on Pith
  55. [75] Zhengxi Lu, Yuxiang Chai, Yaxuan Guo, Xi Yin, Liang Liu, Hao Wang, Han Xiao, Shuai Ren, Pengxiang Zhao, Guangyi Liu, et al. Ui-r1: Enhancing efficient action prediction of gui agents by reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, v... on Pith
  56. [76] Zilong Wang, Yuedong Cui, Li Zhong, et al. OfficeBench: Benchmarking Language Agents across Multiple Applications for Office Automation.arXiv preprint, 2024. URLhttps://arxiv.org/abs/2407.19056 on PithPith's reviewarXiv
  57. [77] Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Li YanTao, Jianbing Zhang, and Zhiyong Wu. SeeClick: Harnessing GUI grounding for advanced visual GUI agents. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association... on Pithdoi:10.18653/v1/2024.acl-long.505
  58. [78] Renze Lou, Hanzi Xu, Sijia Wang, Jiangshu Du, Ryo Kamoi, Xiaoxin Lu, Jian Xie, Yuxuan Sun, Yusen Zhang, Jihyun Janice Ahn, Hongchao Fang, Zhuoyang Zou, Wenchao Ma, Xi Li, Kai Zhang, Congying Xia, Lifu Huang, and Wenpeng Yin. AAAR-1.0: Assessing AI’s Potential to Assist Researc... on Pith
  59. [79] Yitao Long, Tiansheng Hu, Yilun Zhao, Arman Cohan, and Chen Zhao. FinLFQA: Evaluating Attributed Text Generation of LLMs in Financial Long-Form Question Answering. InFindings of EMNLP 2025, 2025. URL https://aclanthology.org/2025.findings-emnlp.908/ on Pith
  60. [80] Huawei Cloud. Huawei Cloud OfficeAce Office Agent Opens for Invitation Testing, 2026. URL https: //www.huaweicloud.com/news/2026/20260416163440660.html. Official product announcement, April 16, 2026. Accessed October 26, 2026. 68 on Pith
  61. [81] Zhaowei Wang, Wenhao Yu, Xiyu Ren, Jipeng Zhang, Yu Zhao, Rohit Saxena, Liang Cheng, Ginny Wong, Simon See, Pasquale Minervini, Yangqiu Song, and Mark Steedman. MMLongBench: Benchmarking Long-Context Vision-Language Models Effectively and Thoroughly. InNeurIPS 2025 Datasets an... on Pith
  62. [82] Junjie Xing, Yeye He, Mengyu Zhou, Haoyu Dong, Shi Han, Lingjiao Chen, Dongmei Zhang, Surajit Chaudhuri, and H. V. Jagadish. MMTU: A Massive Multi-Task Table Understanding and Reasoning Benchmark. InNeurIPS 2025 Datasets and Benchmarks Track, 2025. URL https://papers.neurips.c... on Pith
  63. [83] Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, and Yu Su. TravelPlanner: A Benchmark for Real-World Planning with Language Agents. InProceedings of the 41st International Conference on Machine Learning, volume 235 ofProceedings of Mach... on Pith
  64. [84] Shuofei Qiao, Runnan Fang, Zhisong Qiu, Xiaobin Wang, Ningyu Zhang, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. Benchmarking Agentic Workflow Generation. InICLR 2025, 2025. URL https://proceedings. iclr.cc/paper_files/paper/2025/hash/adbe936993aa7cf41e45054d8b72f183-A... on Pith
  65. [85] Afra Feyza Akyürek, Advait Gosai, Chen Bo Calvin Zhang, Vipul Gupta, Jaehwan Jeong, Anisha Gunjal, Tahseen Rabbani, Maria Mazzone, David Randolph IV, Mohammad Mahmoudi Meymand, Gurshaan Chattha, Paula Rodriguez, Diego A. Mares Buendia, Pavit Singh, Michael Liu, Subodh Chawla, ... on Pith
  66. [86] Anna C. Doris, Jacob Thomas Sony, Ghadi Nehme, Era Syla, Amin Heyrani Nobari, and Faez Ahmed. CADBench: A Multimodal Benchmark for AI-Assisted CAD Program Generation.arXiv preprint arXiv:2605.10873, 2026. URLhttps://arxiv.org/abs/2605.10873 on PithPith's reviewarXiv
  67. [87] Jin Huang, Silviu Cucerzan, Sujay Kumar Jauhar, and Ryen W. White. Idea2Plan: Exploring AI-Powered Research Planning. arXiv preprint arXiv:2510.24891, 2025. URLhttps://arxiv.org/abs/2510.24891 on PithPith's reviewarXiv
  68. [88] Tianshu Zhang, Xiang Yue, Yifei Li, and Huan Sun. TableLlama: Towards Open Large Generalist Mod- els for Tables. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Pa... on Pithdoi:10.18653/v1/2024.naacl-long.335
  69. [89] Bin Wang, Chao Xu, Xiaomeng Zhao, Linke Ouyang, Fan Wu, Zhiyuan Zhao, Rui Xu, Kaiwen Liu, Yuan Qu, Fukai Shang, et al. MinerU: An Open-Source Solution for Precise Document Content Extraction.arXiv preprint arXiv:2409.18839, 2024. URLhttps://arxiv.org/abs/2409.18839 on PithPith's reviewarXiv
  70. [90] Yuhao Du, Shunian Chen, Wenbo Zan, Peizhao Li, Mingxuan Wang, Dingjie Song, Bo Li, Yan Hu, and Benyou Wang. BlenderLLM: Training Large Language Models for Computer-Aided Design with Self-improvement.arXiv preprint arXiv:2412.14203, 2024. URLhttps://arxiv.org/abs/2412.14203 on PithPith's reviewarXiv
  71. [91] Suhaas Garre, Emily Ritchie, Sushant Mehta, and Edwin Chen. GDP.pdf: Benchmarking Grounded Multimodal Reasoning over Professional PDF Documents.arXiv preprint arXiv:2607.11192, 2026. URL https://arxiv.or g/abs/2607.11192 on PithPith's reviewarXiv
  72. [92] Jonathan Herzig, Pawel Krzysztof Nowak, Thomas Müller, Francesco Piccinno, and Julian Eisenschlos. TaPas: Weakly Supervised Table Parsing via Pre-training. InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4320–4333. Association f... on Pith
  73. [93] Pranab Islam, Anand Kannappan, Douwe Kiela, Rebecca Qian, Nino Scherrer, and Bertie Vidgen. FinanceBench: A New Benchmark for Financial Question Answering.arXiv preprint, 2023. URL https://arxiv.org/abs/23 11.11944 on Pith
  74. [94] Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding. InProceedings of the 62nd Annual Meeting ... on Pithdoi:10.18653/v1/2024.acl-long.172
  75. [95] Ahmed Masry, Mohammed Saidul Islam, Mahir Ahmed, Aayush Bajaj, Firoz Kabir, Aaryaman Kartha, Md Tahmid Rahman Laskar, Mizanur Rahman, Shadikur Rahman, Mehrad Shahmohammadi, Megh Thakkar, Md Rizwan Parvez, Enamul Hoque, and Shafiq Joty. ChartQAPro: A More Diverse and Challengin... on Pith
  76. [96] Rubèn Tito, Dimosthenis Karatzas, and Ernest Valveny. Hierarchical multimodal transformers for Multipage DocVQA. Pattern Recognition, 144:109834, 2023. doi: 10.1016/j.patcog.2023.109834. URLhttps://doi.org/ 10.1016/j.patcog.2023.109834 on Pithdoi:10.1016/j.patcog.2023.109834
  77. [97] Yushi Bai, Shangqing Tu, Jiajie Zhang, Hao Peng, Xiaozhi Wang, Xin Lv, Shulin Cao, Jiazheng Xu, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks. In Proceedings of the 63rd Annual Meeti... on Pithdoi:10.18653/v1/2025.acl-long.183
  78. [98] Zhiyu Chen, Shiyang Li, Charese Smiley, Zhiqiang Ma, Sameena Shah, and William Yang Wang. ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processin... on Pithdoi:10.18653/v1/2022.emnlp-main.421
  79. [99] Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. PAL: Program-aided Language Models. InInternational Conference on Machine Learning, pages 10764–10799, on Pith
  80. [100] Hongchen Wei, Yuanzhe Wang, Bei Liu, Yifan Yang, Qi Dai, Ruichun Ma, Kai Qiu, Yunsheng Li, Dongdong Chen, Chong Luo, Zhenzhong Chen, and Baining Guo. XL-DocBench: Benchmarking Evidence-Grounded Extra-Long Document Understanding. In Advances in Neural Information Processing Sys... on Pith

The first 100 of 295 references are listed here. All of them are in this paper's citation record.

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment