Machine Learning · arXiv:2610.35845 v1 · submitted 26 Oct 2026 · 295 references
Work Agents: Systems, Capabilities, and Optimization
The paper says
“As foundation models evolve from information generation to digital work execution, agents are increasingly capable of completing complex tasks across tools, applications, and workflows. However, existing research lacks a unified perspective on how agents reliably accomplish delegated work.”
more of the abstract
To address this gap, we present a work-centric, system-level perspective that views work agents as systems combining foundation models and runtime harnesses to execute tasks in stateful workspaces and deliver verifiable outcomes. We introduce two complementary dimensions, work persistence and work openness, to characterize working conditions, and classify digital work into five forms according to task scope and organization. Building on this framework, we identify five core capability families required for reliable work execution and examine approaches to model optimization, harness optimization, and their joint optimization. We further analyze existing evaluation methods and metrics from the perspective of work outcomes. Finally, we discuss future research directions, including sustained work delegation, adaptation to open environments, and the interaction between work execution and capability improvement. This paper provides a unified framework for understanding, optimizing, and evaluating work agents, supporting the development of systems capable of delivering reliable, verifiable, and practically valuable work outcomes.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 27 Oct 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
The passes disagreed. Reader: holds up. Skeptic: holds up only with conditions. Referee: major revision.
A careful, genuinely useful survey-with-framework whose persistence/openness axes mostly hold up; the weakest spot is the unstated boundary criterion behind the τ-bench quadrant assignment. This line is wrong
2 major objections · 6 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 295 references say.
The paper says, at §3.5, with §2.2.2 and §2.2.3
The review did not quote this passage.
Pith says: Objection 1 of 2 · major
The four-quadrant claim in Section 2.2.3 ('all four combinations appear in existing research') is load-bearing for the paper's assertion that work persistence and work openness are 'conceptually distinct', and that claim currently rests on exactly one high-persistence/low-openness example, τ-bench, with no other setting assigned to that quadrant in the Figure 2 discussion.
more of objection 1
The classification of τ-bench is not supported by the paper's own openness definition as written. Section 2.2.2 defines openness over 'the objects, interfaces, rules, and actors relevant to the task' and says the agent 'may also need to find out where the relevant state is held, how other actors have changed it, and which actions it is authorized to take'; the τ-bench user's confirmation gates the database update, so on a literal reading the agent must discover actor-supplied information during execution, and the same phenomenon is used in the same subsection to justify τ²-bench's high openness. The text never states the criterion that would exclude interaction contents within a known protocol (confirmations, messages) from the openness axis while still counting the τ²-bench user's state changes as open; the Figure 2 caption ('return paths through other participants') even suggests that participant actions drive persistence, which is the same phenomenon. A charitable reading is available — openness concerns the structure of the task world, not event contents exchanged within a specified protocol — but it must be stated, with a test that lets a reader classify any interactive benchmark. I recommend adding this criterion to Section 2.2.2 and then re-checking the quadrant assignments, since a reader who adopts the literal reading would see the two dimensions collapse in exactly the case that is supposed to separate them.
The paper says, at §3.5 (with §2.2.2 and §2.2.3)
The review did not quote this passage.
Pith says: Objection 2 of 2 · major
The paper's distinction between work forms and working conditions is not maintained at the Workspace Level. Section 2.2.3 states that the persistence/openness combinations 'describe the conditions of work rather than its form', while Section 3.5 defines the Workspace Level by the fact that 'the request no longer specifies every object involved' and the agent 'must establish which files, records, and …
more of objection 2
… applications bear on the work and how they are related' — phrasing nearly identical to the Section 2.2.2 definition of openness (the agent must establish during execution which objects are relevant and how they are represented and related). Section 3.5's closing sentence then maps the level's two concerns directly onto the openness and persistence axes. If the Workspace Level is defined by discovery, one of the five forms is defined by exactly the property assigned to the conditions axis, making the two parts of the framework partially redundant. Please sharpen Section 3.5's opening so that the level is defined by the scope and organization of the delegated object graph (multi-object consistency, version reconciliation, dependency tracing as the substance of the work) rather than by whether the request specified the objects; a brief worked example of a Workspace-Level task with a fully predefined object graph would settle the boundary.
6 minor objections
The paper says, at Throughout
Pith says: minor objection 1 of 6
The manuscript text contains numerous doubled-letter and spacing artifacts (for example, the running header rendered as 'EExxppaannddiinngg wwoorrkk ssccooppee aanndd ddeeppeennddeenncciieess' and a garbled repository/homepage line above the abstract); these should be cleaned in the production version.
The paper says, at Figure 4
Pith says: minor objection 2 of 6
The Figure 4 caption doubles as a classification appendix, listing per-entry rationale for dozens of works; moving that rationale to a supplementary table would make the caption readable and the level assignments easier to audit.
show 4 more
The paper says, at Table 10
Pith says: minor objection 3 of 6
Table 10 spans a very large portion of Section 5; moving it to an appendix and keeping in the main text the three-level summary (targets in 5.2.1, methods in 5.2.2, metrics in 5.3) would improve readability without losing content.
The paper says, at §5.3
Pith says: minor objection 4 of 6
The distinction between pass@k and passk is introduced in a single clause; a one-sentence worked example (for example, k=2 with one successful and one failed attempt) would make the all-attempts-must-pass reading of passk unambiguous.
The paper says, at §2.1, Eqs. (1)–(2)
Pith says: minor objection 5 of 6
Equations (1) and (2) use informal tuple-like notation for what is really a schematic account; a sentence noting that the notation is illustrative rather than a formal operational semantics would prevent over-reading.
The paper says, at §4.4.2
Pith says: minor objection 6 of 6
The proposed (M0, H0), (M1, H0), (M0, H1), (M1, H1) comparison protocol is a useful and honest contribution; consider making it a highlighted 'proposed protocol' call-out so that readers can cleanly separate the survey's synthesis from its forward-looking proposals.
What would settle it
Prong 1: apply the §2.2.2 openness definition to τ-bench by enumerating every task-relevant piece of information the agent must obtain from the simulated user during execution (requests, confirmations, corrections), and compare this with the paper's treatment of the τ²-bench user's actions as openness. If confirmations count as discovered information, Figure 2's top-left quadrant needs a new canonical instance or a stated exclusion criterion; if they do not count, state that criterion explicitly and re-check the other quadrant examples (SQuAD, DocVQA, GAIA, Workspace-Bench, OSWorld 2.0, VibeLifeBench) against it.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 27 Oct 2026
- Reader holds up
- Skeptic partly agrees: says it holds up only with conditions Under §2.2.2's own definition, τ-bench, the canonical high-persistence/low-openness instance (§2.2.3), is partially open: user confirmations that keep responsibility open are actor information discovered during execution.
- Editor wrote the verdict above
- Referee major revision: 2 major objections, 6 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
The takeaway: this is worth reading, and the part worth reading is Section 2. The paper defines a "work agent" as six elements (task, model, harness, workspace, outcome, verification) and introduces two working-condition axes — persistence (when responsibility closes) and openness (how much of the task world must be discovered during execution) — plus a five-form taxonomy of work scope. The rest is a broad survey of training, harness optimization, and evaluation, with roughly ninety recent works mapped onto the framework. It makes no empirical claims of its own; all numbers are attributed to cited sources, and the authors are explicitly honest about comparability limits (dataset sizes, trajectory content, cross-benchmark scores). The citation pattern looks careful, and the paper marks its own proposals as proposals, which is more than most surveys manage.
What it does well: the persistence/openness distinction genuinely cuts across long-horizon, computer-use, and office-agent research, and the five work forms organize the benchmark landscape usefully. The placements mostly felt right.
Soft spots, in proportion. The main one is the orthogonality of the two axes, which is the load-bearing novelty. Section 2.2.3 claims all four quadrant combinations appear in existing research, with τ-bench as the high-persistence/low-openness instance. The stress-test is half right: under the paper's own openness definition, τ-bench's user confirmations are actor information discovered during execution, so the cleanest low-openness/high-persistence example sits on an unstated boundary criterion. A charitable reading is available — openness concerns the structure of the task world the agent must establish, not contingent outcomes within a known protocol — and the τ-bench/τ²-bench contrast at the end of §2.2.3 gestures exactly that way. But the criterion is never stated. Fixable with a paragraph, not fatal; the axes remain analytically distinct. The part of the stress-test about closed-world monitoring sliding into openness I would reject: discovering which entries an incident affects is openness, the obligation to respond is persistence, and the paper's decomposition there is coherent.
The taxonomy is subjective by nature — no proof of exhaustiveness; the five forms and five capability families are chosen by judgment. That is characteristic of conceptual frameworks, not a flaw. The paper is long and somewhat repetitive.
Who it is for: anyone building or evaluating agents in software, office, or computer-use settings, and survey writers who need a vocabulary. It will not change what agents can do, but it gives the field shared coordinates. I would send it to peer review, with the referee asked to push on the openness boundary and the "all four combinations" claim. My verdict: accept with revision, not a reject.
The strongest objection
The central claim is the unification of agent research through two 'conceptually distinct yet complementary' dimensions, work persistence and work openness, with all four combinations instantiated in existing research (§2.2.3, Figure 2). The four-quadrant argument is load-bearing: if the axes are not cleanly separable, the framework's main novelty reduces to a single dimension with a confound. The paper's own definitions do not separate them. §2.2.2 defines openness over 'the objects, interfaces, rules, and actors relevant to the task' and counts learning 'how other actors have changed' the shared state as openness, as in τ²-bench. §2.2.1 defines persistence to include responsibility kept open because it 'depends on other participants,' citing τ-bench's withheld database updates 'until the user confirms them.' The user's confirmation is task-relevant information supplied by an actor that the agent must discover during execution; on a literal reading of the openness definition, τ-bench is at least partially open, and the paper never states the criterion that would exclude interaction contents (confirmations, messages) from the openness axis while still counting the τ²-bench user's actions as open. A charitable reading is available — openness concerns the structure of the task world, not contents exchanged within a known protocol — but the text does not state it, and the Figure 2 caption ('return paths through other participants') suggests participants' actions drive persistence, not openness. The closed-world monitoring obligation in §2.2.1 also slides toward openness once incidents arrive, since tracing affected components is itself classified as openness. So the high-persistence/low-openness quadrant may lack a clean empirical instance in the surveyed corpus.
I agree with the reader that the paper is a comprehensive, honest position piece whose central framework is definitional and largely coherent. The reader's flagged weakest assumption (sufficiency of the two axes) is real; my concern is a more specific, internally testable defect within the same chapter: the boundary between 'predefined task world' and 'discovered during execution' is under-specified, and the paper's single canonical instance of the high-persistence/low-openness quadrant, τ-bench, becomes partially open under the paper's own openness definition. If this lands, the four-quadrant claim (Figure 2) does not separate cleanly, and the claimed unification rests on a one-dimensional axis plus an unacknowledged interaction with actor-supplied information. This matters because the two dimensions are one of the two named conceptual contributions in the abstract, not a peripheral example. I do not consider this rejection-level: equations (1)–(2), the five work forms, the extensive benchmark mapping, and the authors' candid limitation statements — dataset-size non-comparability, verifier unreliability, single-run VERSE evidence, context-window limits — all support acceptance after a targeted revision. Hence CONDITIONAL: accept if the authors state the missing criterion distinguishing interaction contents from task-world discovery and verify that each quadrant retains a clean instance in the surveyed literature; if the criterion cannot be stated without conceding that persistence and openness co-vary, the contribution should be reframed accordingly. Agreement with the reader is partial: we both locate the risk in the two-axis chapter, but the reader emphasizes missing coordinates (exhaustiveness) while I emphasize a definitional ambiguity that threatens the axes' independence as stated.
The referee's report
If its definitions hold, this framework gives the field a useful common vocabulary: the six-element system view corrects the habit of scoring models in isolation, the five capability families are well illustrated by the running business-report example in Section 2.3, Section 3.4's criterion for distinguishing workflow stages from system-internal execution stages is careful, and the Section 5 trichotomy of evaluation targets, dimensions, and methods is practically usable. The reporting discipline is a genuine strength: numbers are attributed to cited sources (for example, the WorkForge ALE gains and the Nemotron SWE-bench gains in Section 4), and the authors state the limits of their own suggestions, including single-run recursive-self-improvement results in Section 4.4.3 and dataset-size comparability in Section 4.1.1. The intellectual risk is definitional: the persistence/openness axes and the boundary between work forms and working conditions are the paper's novel analytical apparatus, and their current wording invites misclassification of the paper's own canonical examples. These are fixable clarification issues, not flaws in the survey's empirical content.
Pith's reading of the paper (a machine's summary, not the authors')
This paper tries to establish a single definition of what an AI agent is when it does real work: a foundation model that decides, plus a runtime harness that observes, acts, and remembers, operating in a stateful workspace and judged by verifiable outcomes. It argues that this work-centric view unifies research that currently splits into long-horizon, computer-use, and office agents, because those lines each track one facet of the same delegated task. To make the unification usable, the paper proposes two coordinates for working conditions — persistence (does responsibility close at delivery or stay open across events?) and openness (is the task world given in advance or discovered during execution?) — and five forms of digital work, from single answers to whole-workspace operations. A sympathetic reader would care because the framework promises a common language for comparing benchmarks, tracing failures to one of five capability families, and deciding whether an improvement belongs in the model or in the harness.
What carries the argument. The load-bearing object is the six-element system definition with its execution–verification loop. At an assessment point the loop splits into two roles: execution writes $(T, E_0) \xrightarrow{M,H} (\tau_{0:t}, E_t, O_t)$, meaning the model–harness pair acts in the initial workspace and produces an action record, a new workspace state, and an outcome, while verification evaluates $S_t = V(T, E_0, \tau_{0:t}, E_t, O_t)$, meaning the evidence is checked against the task requirements by programs, humans, or model-based judges. Two dimensions order the working conditions, five work forms order the scope of the delegated work, and the argument is carried by showing that existing systems occupy distinct positions in this space: responsibility closes at delivery in question answering, remains open awaiting approval in service workflows, requires discovery in information seeking, and combines open discovery with open responsibility in ongoing work under changing conditions.
Core claim. On the paper's own terms, the central claim is that a work agent is best understood not as a model that generates outputs but as a system of six elements — task, model, harness, workspace, work outcome, and verification — bound together in a closed loop. The harness assembles context from the task and the workspace for the model; the model proposes actions; the harness executes them and returns observations; and verification assesses the resulting evidence against the task requirements. The same loop runs in every work-agent setting, and what varies are the conditions, which the paper captures with two dimensions: work persistence, from responsibility that closes at delivery to responsibility that stays open across stages, feedback cycles, and later events, and work openness, from a predefined task world to one the agent must discover while acting. The paper classifies digital work into five forms by scope — conversation, atomic action, multi-step, workflow, and workspace — and derives five capability families from the work process: perception and understanding, reasoning and decision making, planning and coordination, execution and state management, and verification, recovery, and replanning. It then reads the field's optimization and evaluation literature through this frame, arguing that capabilities can be improved by training the model, by upgrading the harness, or by adapting the two jointly.
The premise it rests on. The load-bearing premise is that two axes — how long responsibility stays open and how much of the task world the agent must discover — are the right and sufficient coordinates for working conditions, and that all digital knowledge work fits the five proposed forms; if a third factor such as collaboration structure or risk is equally primary, the promised unification is weaker than the paper claims.
If the paper is right (Pith's inferences, not the authors')
Question answering, single-file edits, multi-step projects, staged workflows, and whole-workspace operations become one spectrum of work forms, so methods and benchmarks designed at one level can be positioned, compared, and reused at another.
Failures can be routed to a specific capability — missing evidence to perception, a questionable decision to reasoning, a changed prerequisite to planning, an execution error to state management — which directs engineers to train the model or change the harness.
Evaluation should inspect outcomes through evidence — artifacts, workspace states, workflow records, and maintained conditions — rather than the final answer alone, because acceptance depends on the object actually delivered.
Optimization decomposes into model optimization, harness optimization, and joint adaptation, and the paper proposes comparing $(M_0,H_0)$, $(M_1,H_0)$, $(M_0,H_1)$, and $(M_1,H_1)$ under consistent task conditions to attribute improvements to the right component.
Because persistence and openness raise the demands on all five capability families, benchmarks that are high on both axes become the testbed for sustained delegation, which the paper identifies as the field's frontier.
If the two dimensions were adopted as a standard annotation, benchmark families could be plotted on a single map, making progress across software, office, and enterprise domains comparable without a shared test set.
A prediction the paper leaves implicit: agents trained in high-persistence, high-openness environments should transfer to ordinary office work better than agents trained on delivery-close, predefined tasks, because the former must practice verification, dependency tracking, and state management.
The five work forms suggest a curriculum ordering — workspace-level competence presupposes the mechanisms of the lower forms — so staged training that introduces openness and persistence gradually may be more sample-efficient than uniform task mixing.
Because verification is defined as part of the system rather than an external judge, the framework implies that agents which close the verification loop themselves are closer to deployment-ready; this could be tested by ablating an agent's internal verification loop and comparing against an external evaluator.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
As foundation models evolve from information generation to digital work execution, agents are increasingly capable of completing complex tasks across tools, applications, and workflows. However, existing research lacks a unified perspective on how agents reliably accomplish delegated work. To address this gap, we present a work-centric, system-level perspective that views work agents as systems combining foundation models and runtime harnesses to execute tasks in stateful workspaces and deliver verifiable outcomes. We introduce two complementary dimensions, work persistence and work openness, to characterize working conditions, and classify digital work into five forms according to task scope and organization. Building on this framework, we identify five core capability families required for reliable work execution and examine approaches to model optimization, harness optimization, and their joint optimization. We further analyze existing evaluation methods and metrics from the perspective of work outcomes. Finally, we discuss future research directions, including sustained work delegation, adaptation to open environments, and the interaction between work execution and capability improvement. This paper provides a unified framework for understanding, optimizing, and evaluating work agents, supporting the development of systems capable of delivering reliable, verifiable, and practically valuable work outcomes.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-27
The authors' main claim, as the classifier picked it out 1.00
“This paper provides a unified framework for understanding, optimizing, and evaluating work agents, supporting the development of systems capable of delivering reliable, verifiable, and practically valuable work outcomes.”
How the main claim is stated: asserted 0.99
What the abstract itself states, as probabilities: claims first 0.13 · states limitation 0.10 · human participants 0.05 · controlled comparison 0.02 · randomized assignment 0.01 · states quantitative result 0.37 · compares with prior results 0.13 · mentions code or data availability 0.02
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| As foundation models evolve from information generation to digital work execution, agents are increasingly capable of completing complex tasks across tools, applications, and workflows. | background 1.00 | 0.06 | 0.03 | asserted 0.94 |
| However, existing research lacks a unified perspective on how agents reliably accomplish delegated work. | gap 1.00 | 0.08 | 0.02 | asserted 1.00 |
| To address this gap, we present a work-centric, system-level perspective that views work agents as systems combining foundation models and runtime harnesses to execute tasks in stateful workspaces and deliver verifiable outcomes. | contribution 1.00 | 0.88 | 0.15 | asserted 0.99 |
| We introduce two complementary dimensions, work persistence and work openness, to characterize working conditions, and classify digital work into five forms according to task scope and organization. | contribution 0.96 | 0.94 | 0.04 | asserted 0.99 |
| Building on this framework, we identify five core capability families required for reliable work execution and examine approaches to model optimization, harness optimization, and their joint optimization. | contribution 0.99 | 0.87 | 0.06 | asserted 1.00 |
| We further analyze existing evaluation methods and metrics from the perspective of work outcomes. | contribution 0.95 | 0.60 | 0.03 | asserted 0.92 |
| Finally, we discuss future research directions, including sustained work delegation, adaptation to open environments, and the interaction between work execution and capability improvement. | future work 1.00 | 0.40 | 0.05 | asserted 0.53 |
| This paper provides a unified framework for understanding, optimizing, and evaluating work agents, supporting the development of systems capable of delivering reliable, verifiable, and practically valuable work outcomes. | contribution 0.90 | 0.81 | 0.92 | asserted 0.99 |
What kind of paper the classifier reports this is: position or commentary 0.52 · evidence from argument 0.68
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-27: 1 request, 6,206 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (11)
Figures are the authors', reproduced from the paper.
References (295)
- [1] Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the Opportunities and Risks of Foundation Models.arXiv preprint arXiv:2108.07258, 2021. URLhttps://arxiv.o... on PithPith's reviewarXiv
- [2] Tong Zheng, Xidong Wu, Zheng Zhang, Zhankui He, Chaoyi Zhang, Benjamin Coleman, Ruoqiao Wei, Di Bai, Haolin Liu, Rui Liu, et al. Dream-rsi: Recursive self-improvement through evolving worlds.arXiv preprint arXiv:2609.14858, 2026 on PithPith's reviewarXiv
- [3] John Yang, Carlos Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering. In Advances in Neural Information Processing Systems, volume 37, pages 50528–50652. Curran As... on Pithdoi:10.52202/079017-1601
- [4] Hongxin Li, Jingran Su, Yuntao Chen, Qing Li, and ZHAO-XIANG ZHANG. SheetCopilot: Bringing Software Productivity to the Next Level through Large Language Models. InAdvances in Neural Information Processing Systems, volume 36, pages 4952–4984. Curran Associates, Inc., 2023. doi... on Pithdoi:10.52202/075280-0220
- [5] Hao Zheng, Xinyan Guan, Hao Kong, Wenkai Zhang, Jia Zheng, Weixiang Zhou, Hongyu Lin, Yaojie Lu, Xianpei Han, and Le Sun. PPTAgent: Generating and Evaluating Presentations Beyond Text-to-Slides. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Pro... on Pithdoi:10.18653/v1/2025.emnlp-main.728
- [6] Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. WebArena: A Realistic Web Environment for Building Autonomous Agents. InInternational Conference on Learning Representat... on Pith
- [7] Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, and Tao Yu. OSWorld: Benchmarking Multimodal Agents for Op... on Pithdoi:10.52202/079017-1650
- [8] Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxuan Zhang, Juanzi Li, et al. CogAgent: A Visual Language Model for GUI Agents. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. URLhttps... on PithPith's reviewarXiv
- [9] Alexandre Drouin, Maxime Gasse, Massimo Caccia, Issam H. Laradji, Manuel Del Verme, Tom Marty, David Vazquez, Nicolas Chapados, and Alexandre Lacoste. WorkArena: How Capable are Web Agents at Solving Common Knowledge Work Tasks? InProceedings of the 41st International Conferen... on Pith
- [10] Ziru Chen, Shijie Chen, Yuting Ning, Qianheng Zhang, Boshi Wang, Botao Yu, Yifei Li, Zeyi Liao, Chen Wei, Zitong Lu, Vishal Dey, Mingyi Xue, Frazier N. Baker, Benjamin Burns, Daniel Adu-Ampratwum, Xuhui Huang, Xia Ning, Song Gao, Yu Su, and Huan Sun. ScienceAgentBench: Toward ... on Pith
- [11] OpenAI. Codex Manual, n.d. URL https://developers.openai.com/codex/codex-manual.md . Accessed October 22, 2026 on Pith
- [12] Anthropic. Claude Code Overview, n.d. URLhttps://code.claude.com/docs/en/overview. Accessed October 22, 2026. 63 on Pith
- [13] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing Reasoning and Acting in Language Models. InInternationalConference on Learning Representations, on Pith
- [14] Theodore R. Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas L. Griffiths. Cognitive Architectures for Language Agents. Transactions on Machine Learning Research, 2024. URL https://arxiv.org/abs/2309.024 27 on Pith
- [15] Mingchen Zhuge, Changsheng Zhao, Dylan R. Ashley, Wenyi Wang, Dmitrii Khizbullin, Yunyang Xiong, Zechun Liu, Ernie Chang, Raghuraman Krishnamoorthi, Yuandong Tian, Yangyang Shi, Vikas Chandra, and Jürgen Schmidhuber. Agent-as-a-Judge: Evaluate Agents with Agents. In Proceeding... on Pith
- [16] Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. SWE-bench: Can Language Models Resolve Real-world Github Issues? InInternational Conference on Learning Representations, pages 54107–54157, 2024. URLhttps://proceedings.ic... on Pith
- [17] Xin-Sheng Chen, Jiayu Zhu, Pei-lin Li, et al. PresentBench: A Fine-Grained Rubric-Based Benchmark for Slide Generation. arXiv preprint, 2026. URLhttps://arxiv.org/abs/2603.07244 on PitharXiv
- [18] Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. SQuAD: 100,000+ Questions for Machine Comprehension of Text. InProceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392. Association for Computational Linguistics,... on Pithdoi:10.18653/v1/d16-1264
- [19] Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. InProceedings of the 2018 Conference on Empirical Methods in Natural Language Proces... on Pithdoi:10.18653/v1/d18-1259
- [20] Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, and Tat- Seng Chua. TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance. In Proceedings of the 59th Annual Meeting of the Association for Computa... on Pithdoi:10.18653/v1/2021.acl-long.254
show 80 more
- [21] Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, and William Yang Wang. FinQA: A Dataset of Numerical Reasoning over Financial Data. In Proceedings of the 2021 Conference on Empirical M... on Pithdoi:10.18653/v1/2021
- [22] Minesh Mathew, Dimosthenis Karatzas, and C. V. Jawahar. DocVQA: A Dataset for VQA on Document Images. In 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 2199–2208. IEEE, 2021. doi: 10.1109/wacv48630.2021.00225. URLhttps://doi.org/10.1109/wacv48630.... on Pithdoi:10.1109/wacv48630.2021.00225
- [23] Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning. InFindings of the Association for Computational Linguistics: ACL 2022, pages 2263–2279. Association for Computati... on Pithdoi:10.18653/v1/2022.findings-acl.177
- [24] Timo Schick, Jane Dwivedi-Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language Models Can Teach Themselves to Use Tools. In Advances in Neural Information Processing Systems, volume 36, page... on Pithdoi:10.52202/07528
- [25] Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. API-Bank: AComprehensiveBenchmarkforTool-AugmentedLLMs. In Proceedings ofthe2023Conferenceon Empirical Methods in Natural Language Processing, pages 3102–3116. Associ... on Pith
- [26] Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. Gorilla: Large Language Model Connected with Massive APIs. InAdvances in Neural Information Processing Systems, volume 37, pages 126544–126565, on Pith
- [27] Chris Rawles, Sarah Clinckemaillie, Yifan Chang, Jonathan Waltz, Gabrielle Lau, Marybeth Fair, Alice Li, William Bishop, Wei Li, Folawiyo Campbell-Ajala, Daniel Toyama, Robert Berry, Divya Tyamagundlu, Timothy Lillicrap, and Oriana Riva. AndroidWorld: A Dynamic Benchmarking En... on Pith
- [28] doi: 10.18653/v1/2023.emnlp-main.187. URLhttps://aclanthology.org/2023.emnlp-main.187/. 64 on Pithdoi:10.18653/v1/2023.emnlp-main.187
- [29] Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan.τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. InInternational Conference on Learning Representations, pages 9965–10017, on Pith
- [30] doi: 10.52202/079017-4020. URL https://proceedings.neurips.cc/paper_files/paper/2024/hash/e 4c61f578ff07830f5c37378dd3ecb0d-Abstract-Conference.html on Pithdoi:10.52202/079017-4020
- [31] Frank (Fangzheng) Xu, Yufan Song, Boxuan Li, Yuxuan Tang, Kritanjali Jain, Mengxue Bao, Zora Wang, Xuhui Zhou, Zhitong Guo, Murong Cao, Mingyang Yang, Hao Yang Lu, Amaad Martin, Zhe Su, Leander Maben, Raj Mehta, Wayne Chi, Lawrence Jang, Yiqing Xie, Shuyan Zhou, and Graham Neu... on Pithdoi:10.52202/085713-0315
- [32] Xingyao Wang, Zihan Wang, Jiateng Liu, Yangyi Chen, Lifan Yuan, Hao Peng, and Heng Ji. MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback. InInternational Conference on Learning Representations, pages 32593–32627, 2024. URLhttps://proceedings.iclr... on Pith
- [33] Wei Fan, Xinjie Shen, Xudong Guo, et al. E-Commerce Bench: Evaluating LLM Agents on Long-Horizon Autonomous Business Operation.arXiv preprint arXiv:2608.30730, 2026. URL https://arxiv.org/abs/2608 .30730 on PithPith's reviewarXiv
- [34] Xiaohongshu Dots Studio and Evolvent AI. VibeLifeBench: Can your life agent be proactive and persistent in a living world? arXiv preprint arXiv:2608.10875, 2026. URLhttps://arxiv.org/abs/2608.10875 on PithPith's reviewarXiv
- [35] Victor Barres, Honghua Dong, Soham Ray, Xujie Si, and Karthik R Narasimhan. τ 2-Bench: Evaluating Conversational Agents in a Dual-Control Environment. InProceedings of the 43rd International Conference on Machine Learning, volume 306 ofProceedings of Machine Learning Research,... on Pith
- [36] Zirui Tang, Xuanhe Zhou, Yumou Liu, Linchun Li, Weizheng Wang, Hongzhang Huang, Jun Zhou, Jiachen Song, Shaoli Yu, Jinqi Wang, Zihang Zhou, Hongyi Zhou, Yuting Lv, Jinyang Li, Jiashuo Liu, Ruoyu Chen, Chunwei Liu, GuoLiang Li, Jihua Kang, and Fan Wu. Workspace-Bench 1.0: Bench... on Pith
- [37] Mengqi Yuan, Zilong Zhou, Xinzhuang Xiong, et al. OSWorld 2.0: Benchmarking Computer Use Agents on Long- Horizon Real-World Tasks.arXiv preprint arXiv:2606.29537, 2026. URL https://arxiv.org/abs/2606.29537 on PithPith's reviewarXiv
- [38] Tao Yu, Hao Wang, Changyu Li, Shenghua Chai, Minghui Zhang, Zhongtian Luo, Yuxuan Zhou, Haopeng Jin, Zhaolu Kang, Jiabing Yang, YiFan Zhang, Xinming Wang, Hongzhu Yi, Zheqi He, Jing-Shu Zheng, Xi Yang, Yan Huang, and Liang Wang. Beyond the All-in-One Agent: Benchmarking Role-S... on PithPith's reviewarXiv
- [39] Quan Shi, Alexandra Zytek, Pedram Razavi, Karthik Narasimhan, and Victor Barres.τ-Knowledge: Evaluating Conversational Agents over Unstructured Knowledge.arXiv preprint arXiv:2603.04370, 2026. doi: 10.48550/a rXiv.2603.04370. URLhttps://arxiv.org/abs/2603.04370 on Pithdoi:10.48550/arxiv.2603.04370
- [40] Shishir G. Patil, Huanzhi Mao, Charlie Cheng-Jie Ji, Fanjia Yan, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. The berkeley function calling leaderboard (bfcl): From tool use to agentic evaluation of large language models. InForty-secondInternational Conference on Machine... on Pith
- [41] Houxing Ren, Mingjie Zhan, Zimu Lu, Ke Wang, Yunqiao Yang, Haotian Hou, and Hongsheng Li. Towards Robust Real-World Spreadsheet Understanding with Multi-Agent Multi-Format Reasoning. InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Vol... on Pithdoi:10.18653/v1/2026.acl-long.86
- [42] Harsh Trivedi, Tushar Khot, Mareike Hartmann, Ruskin Manku, Vinty Dong, Edward Li, Shashank Gupta, Ashish Sabharwal, and Niranjan Balasubramanian. AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents. InProceedings of the 62nd Annual Mee... on Pithdoi:10.18653/v1/2024.acl-long.850
- [43] Yuntong Zhang, Haifeng Ruan, Zhiyu Fan, and Abhik Roychoudhury. AutoCodeRover: Autonomous Program Improvement. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, pages 1592–1604. ACM, 2024. doi: 10.1145/3650212.3680384. URLhttps://... on Pithdoi:10.1145/3650212.3680384
- [44] Tianyang Liu, Canwen Xu, and Julian McAuley. RepoBench: Benchmarking Repository-Level Code Auto- Completion Systems. InInternational Conference on Learning Representations, pages 47832–47850, 2024. URL https://proceedings.iclr.cc/paper_files/paper/2024/file/d191ba4c8923ed8fd89... on Pith
- [45] Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. GAIA: a benchmark for General AI Assistants. InInternational Conference on Learning Representations, pages 9025–9049, 2024. URL https://proceedings.iclr.cc/paper_files/paper/2024/file/25ae35b5b1... on Pith
- [46] Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. Executable Code Actions Elicit Better LLM Agents. In Proceedings of the 41st International Conference on Machine Learning, volume 235 ofProceedings of Machine Learning Research, pages 50208–5... on Pith
- [47] Haoyu Dong, Jianbo Zhao, Yuzhang Tian, Junyu Xiong, Mengyu Zhou, Yun Lin, José Cambronero, Yeye He, Shi Han, and Dongmei Zhang. Encoding Spreadsheets for Large Language Models. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 2072... on Pithdoi:10.18653/v1/2024.emnlp-main.1154
- [48] Guancheng Zeng, Xueyi Chen, Jiawang Hu, Shaohua Qi, Yaxuan Mao, Zhantao Wang, Yifan Nie, Shuang Li, Qiuyang Feng, Pengxu Qiu, Yujia Wang, Wenqiang Han, Linyan Huang, Gang Li, Jingjing Mo, and Haowen Hu. Routine: A structural planning framework for llm agent system in enterpris... on PithPith's reviewarXiv
- [49] Hanchao Liu, Rongjun Li, Weimin Xiong, Ziyu Zhou, and Wei Peng. WorkTeam: Constructing Workflows from Natural Language with Multi-Agents. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Languag... on Pithdoi:10.18653/v1/2025.naacl-industry.3
- [50] Yibin Chen, Yifu Yuan, Zeyu Zhang, YAN ZHENG, Jinyi Liu, Fei Ni, Jianye HAO, Hangyu Mao, and Fuzheng Zhang. SheetAgent: Towards a Generalist Agent for Spreadsheet Reasoning and Manipulation via Large Language Models. In THE WEB CONFERENCE 2025, 2025. URLhttps://openreview.net/... on Pith
- [51] Sirui Hong, Yizhang Lin, Bang Liu, Bangbang Liu, Binhao Wu, Ceyao Zhang, Danyang Li, Jiaqi Chen, Jiayi Zhang, Jinlin Wang, Li Zhang, Lingyao Zhang, Min Yang, Mingchen Zhuge, Taicheng Guo, Tuo Zhou, Wei Tao, Robert Tang, Xiangtao Lu, Xiawu Zheng, Xinbing Liang, Yaying Fei, Yuhe... on Pith
- [52] Bo Liu, Yuqian Jiang, Xiaohan Zhang, Qiang Liu, Shiqi Zhang, Joydeep Biswas, and Peter Stone. Llm+p: Empowering large language models with optimal planning proficiency.arXiv preprint arXiv:2304.11477, 2023. URLhttps://arxiv.org/abs/2304.11477 on PithPith's reviewarXiv
- [53] Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, Ahmed Hassan Awadallah, Ryen W White, Doug Burger, and Chi Wang. AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversations. InFirst Confer... on Pith
- [54] Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, zili wang, Steven Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber. MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework. ... on Pith
- [55] Thomas W. Malone and Kevin Crowston. The interdisciplinary study of coordination.ACM Computing Surveys, 26(1):87–119, 1994. doi: 10.1145/174666.174668 on Pithdoi:10.1145/174666.174668
- [56] Zeyao Ma, Bohan Zhang, Jing Zhang, Jifan Yu, Xiaokang Zhang, Xiaohan Zhang, Sijia Luo, Xi Wang, and Jie Tang. SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulation. InAdvances in Neural Information Processing Systems, volume 37, pages 94871–94908, 2024. doi... on Pithdoi:10.52202/079017-3007
- [57] Saaket Agashe, Kyle Wong, Vincent Tu, Jiachen Yang, Ang Li, and Xin Eric Wang. Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents. InSecond Conference on Language Modeling, 2025. URLhttps://openreview.net/forum?id=zg5is4GJ3R on Pith
- [58] Hao Zheng, Guozhao Mo, Xinru Yan, Qianhao Yuan, Wenkai Zhang, Xuanang Chen, Yaojie Lu, Hongyu Lin, Xianpei Han, and Le Sun. DeepPresenter: Environment-Grounded Reflection for Agentic Presentation Generation. In Findings of the Association for Computational Linguistics: ACL 202... on Pithdoi:10.18653/v1/2026.findings-acl.1578
- [59] AmanMadaan, NiketTandon, PrakharGupta, SkylerHallinan, LuyuGao, SarahWiegreffe, UriAlon, NouhaDziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-Refine: Iterative Refine... on Pithdoi:10.52202/075280-2019
- [60] Chenglin Yu, Yuchen Wang, Songmiao Wang, Hongxia Yang, and Li Ming. InfiAgent: An Infinite-Horizon Frame- work for General-Purpose Autonomous Agents. InFindings of the Association for Computational Linguistics: ACL 2026, pages 35884–35894. Association for Computational Linguis... on Pithdoi:10.18653/v1/2026.finding
- [61] Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman, Haohan Wang, and Yu-Xiong Wang. Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models. InProceedings of the 41st International Conference on Machine Learning, volume 235 ofProceedings of Machine Le... on Pith
- [62] Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. CRITIC: Large language models can self-correct with tool-interactive critiquing. InInternational Conference on Learning Representations, 2024. URLhttps://arxiv.org/abs/2305.11738 on PithPith's reviewarXiv
- [63] Thomas Kwa, Ben West, Joel Becker, Amy Deng, Katharyn Garcia, Max Hasin, Sami Jawhar, Megan Kinniment, Nate Rush, Sydney Von Arx, Ryan Bloom, Thomas Broadley, Haoxing Du, Brian Goodrich, Nikola Jurkovic, Luke Miles, Seraphina Nix, Tao Lin, Neev Parikh, David Rein, Lucas Jun Ko... on Pith
- [64] Hjalmar Wijk, Tao Roa Lin, Joel Becker, Sami Jawhar, Neev Parikh, Thomas Broadley, Lawrence Chan, Michael Chen, Joshua M Clymer, Jai Dhyani, Elena Ericheva, Katharyn Garcia, Brian Goodrich, Nikola Jurkovic, Megan Kinniment, Aron Lajko, Seraphina Nix, Lucas Jun Koba Sato, Willi... on Pith
- [65] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: language agents with verbal reinforcement learning. InAdvances in Neural Information Processing Systems, volume 36, pages 8634–8652, 2023. doi: 10.52202/075280-0377. URLhttps://procee... on Pithdoi:10.52202/075280-0377
- [66] Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. Evaluating Very Long-Term Conversational Memory of LLM Agents. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p... on Pithdoi:10.18653/v1/2024.acl-long.747
- [67] Guanting Dong, Xiaoshuai Song, Yuyang Hu, et al. Towards Long-Horizon Agents: A Survey — Foundation, Evolution, Harness, Optimization, Application, and Frontier, 2026. URLhttps://github.com/RUC-NLPIR/Awe some-Long-Horizon-Agents on Pith
- [68] Qianhui Wu, Kanzhi Cheng, Rui Yang, Chaoyun Zhang, Jianwei Yang, Huiqiang Jiang, Jian Mu, Baolin Peng, Bo Qiao, Reuben Tan, Si Qin, Lars Liden, Qingwei Lin, Huan Zhang, Tong Zhang, Jianbing Zhang, Dongmei Zhang, and Jianfeng Gao. GUI-Actor: Coordinate-Free Visual Grounding for... on Pithdoi:10.52202/085713-0508
- [69] Yadong Lu, Jianwei Yang, Yelong Shen, and Ahmed Awadallah. OmniParser for Pure Vision Based GUI Agent. arXiv preprint arXiv:2408.00203, 2024. URLhttps://arxiv.org/abs/2408.00203 on PithPith's reviewarXiv
- [70] Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai-Wei Chang, and Dong Yu. LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory. InInternational Conference on Learning Representations, pages 86809–86836, 2025. URLhttps://proceedings.iclr.cc/paper_files/paper... on Pith
- [71] Zhengxi Lu, Jiabo Ye, Fei Tang, Yongliang Shen, Haiyang Xu, Ziwei Zheng, Weiming Lu, Ming Yan, Fei Huang, Jun Xiao, et al. Ui-s1: Advancing gui automation via semi-online reinforcement learning.arXiv preprint arXiv:2509.11543, 2025 on PitharXiv
- [72] Boyu Gou, Ruohan Wang, Boyuan Zheng, Yanan Xie, Cheng Chang, Yiheng Shu, Huan Sun, and Yu Su. Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents. InInternational Conference on Learning Representations, 2025. URLhttps://arxiv.org/abs/2410.05243 on PithPith's reviewarXiv
- [73] Hongliang He, Wenlin Yao, Kaixin Ma, Wenhao Yu, Yong Dai, Hongming Zhang, Zhenzhong Lan, and Dong Yu. WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1:... on Pithdoi:10.18653/v1/2024.acl-long.371
- [74] Zhiyong Wu, Chengcheng Han, Zichen Ding, Zhenmin Weng, Zhoumianze Liu, Shunyu Yao, Tao Yu, and Lingpeng Kong. OS-Copilot: Towards Generalist Computer Agents with Self-Improvement. InICLR 2024 Workshop on Large Language Model Agents, 2024. URLhttps://openreview.net/forum?id=3WWFrg8UjJ on Pith
- [75] Zhengxi Lu, Yuxiang Chai, Yaxuan Guo, Xi Yin, Liang Liu, Hao Wang, Han Xiao, Shuai Ren, Pengxiang Zhao, Guangyi Liu, et al. Ui-r1: Enhancing efficient action prediction of gui agents by reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, v... on Pith
- [76] Zilong Wang, Yuedong Cui, Li Zhong, et al. OfficeBench: Benchmarking Language Agents across Multiple Applications for Office Automation.arXiv preprint, 2024. URLhttps://arxiv.org/abs/2407.19056 on PithPith's reviewarXiv
- [77] Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Li YanTao, Jianbing Zhang, and Zhiyong Wu. SeeClick: Harnessing GUI grounding for advanced visual GUI agents. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association... on Pithdoi:10.18653/v1/2024.acl-long.505
- [78] Renze Lou, Hanzi Xu, Sijia Wang, Jiangshu Du, Ryo Kamoi, Xiaoxin Lu, Jian Xie, Yuxuan Sun, Yusen Zhang, Jihyun Janice Ahn, Hongchao Fang, Zhuoyang Zou, Wenchao Ma, Xi Li, Kai Zhang, Congying Xia, Lifu Huang, and Wenpeng Yin. AAAR-1.0: Assessing AI’s Potential to Assist Researc... on Pith
- [79] Yitao Long, Tiansheng Hu, Yilun Zhao, Arman Cohan, and Chen Zhao. FinLFQA: Evaluating Attributed Text Generation of LLMs in Financial Long-Form Question Answering. InFindings of EMNLP 2025, 2025. URL https://aclanthology.org/2025.findings-emnlp.908/ on Pith
- [80] Huawei Cloud. Huawei Cloud OfficeAce Office Agent Opens for Invitation Testing, 2026. URL https: //www.huaweicloud.com/news/2026/20260416163440660.html. Official product announcement, April 16, 2026. Accessed October 26, 2026. 68 on Pith
- [81] Zhaowei Wang, Wenhao Yu, Xiyu Ren, Jipeng Zhang, Yu Zhao, Rohit Saxena, Liang Cheng, Ginny Wong, Simon See, Pasquale Minervini, Yangqiu Song, and Mark Steedman. MMLongBench: Benchmarking Long-Context Vision-Language Models Effectively and Thoroughly. InNeurIPS 2025 Datasets an... on Pith
- [82] Junjie Xing, Yeye He, Mengyu Zhou, Haoyu Dong, Shi Han, Lingjiao Chen, Dongmei Zhang, Surajit Chaudhuri, and H. V. Jagadish. MMTU: A Massive Multi-Task Table Understanding and Reasoning Benchmark. InNeurIPS 2025 Datasets and Benchmarks Track, 2025. URL https://papers.neurips.c... on Pith
- [83] Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, and Yu Su. TravelPlanner: A Benchmark for Real-World Planning with Language Agents. InProceedings of the 41st International Conference on Machine Learning, volume 235 ofProceedings of Mach... on Pith
- [84] Shuofei Qiao, Runnan Fang, Zhisong Qiu, Xiaobin Wang, Ningyu Zhang, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. Benchmarking Agentic Workflow Generation. InICLR 2025, 2025. URL https://proceedings. iclr.cc/paper_files/paper/2025/hash/adbe936993aa7cf41e45054d8b72f183-A... on Pith
- [85] Afra Feyza Akyürek, Advait Gosai, Chen Bo Calvin Zhang, Vipul Gupta, Jaehwan Jeong, Anisha Gunjal, Tahseen Rabbani, Maria Mazzone, David Randolph IV, Mohammad Mahmoudi Meymand, Gurshaan Chattha, Paula Rodriguez, Diego A. Mares Buendia, Pavit Singh, Michael Liu, Subodh Chawla, ... on Pith
- [86] Anna C. Doris, Jacob Thomas Sony, Ghadi Nehme, Era Syla, Amin Heyrani Nobari, and Faez Ahmed. CADBench: A Multimodal Benchmark for AI-Assisted CAD Program Generation.arXiv preprint arXiv:2605.10873, 2026. URLhttps://arxiv.org/abs/2605.10873 on PithPith's reviewarXiv
- [87] Jin Huang, Silviu Cucerzan, Sujay Kumar Jauhar, and Ryen W. White. Idea2Plan: Exploring AI-Powered Research Planning. arXiv preprint arXiv:2510.24891, 2025. URLhttps://arxiv.org/abs/2510.24891 on PithPith's reviewarXiv
- [88] Tianshu Zhang, Xiang Yue, Yifei Li, and Huan Sun. TableLlama: Towards Open Large Generalist Mod- els for Tables. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Pa... on Pithdoi:10.18653/v1/2024.naacl-long.335
- [89] Bin Wang, Chao Xu, Xiaomeng Zhao, Linke Ouyang, Fan Wu, Zhiyuan Zhao, Rui Xu, Kaiwen Liu, Yuan Qu, Fukai Shang, et al. MinerU: An Open-Source Solution for Precise Document Content Extraction.arXiv preprint arXiv:2409.18839, 2024. URLhttps://arxiv.org/abs/2409.18839 on PithPith's reviewarXiv
- [90] Yuhao Du, Shunian Chen, Wenbo Zan, Peizhao Li, Mingxuan Wang, Dingjie Song, Bo Li, Yan Hu, and Benyou Wang. BlenderLLM: Training Large Language Models for Computer-Aided Design with Self-improvement.arXiv preprint arXiv:2412.14203, 2024. URLhttps://arxiv.org/abs/2412.14203 on PithPith's reviewarXiv
- [91] Suhaas Garre, Emily Ritchie, Sushant Mehta, and Edwin Chen. GDP.pdf: Benchmarking Grounded Multimodal Reasoning over Professional PDF Documents.arXiv preprint arXiv:2607.11192, 2026. URL https://arxiv.or g/abs/2607.11192 on PithPith's reviewarXiv
- [92] Jonathan Herzig, Pawel Krzysztof Nowak, Thomas Müller, Francesco Piccinno, and Julian Eisenschlos. TaPas: Weakly Supervised Table Parsing via Pre-training. InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4320–4333. Association f... on Pith
- [93] Pranab Islam, Anand Kannappan, Douwe Kiela, Rebecca Qian, Nino Scherrer, and Bertie Vidgen. FinanceBench: A New Benchmark for Financial Question Answering.arXiv preprint, 2023. URL https://arxiv.org/abs/23 11.11944 on Pith
- [94] Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding. InProceedings of the 62nd Annual Meeting ... on Pithdoi:10.18653/v1/2024.acl-long.172
- [95] Ahmed Masry, Mohammed Saidul Islam, Mahir Ahmed, Aayush Bajaj, Firoz Kabir, Aaryaman Kartha, Md Tahmid Rahman Laskar, Mizanur Rahman, Shadikur Rahman, Mehrad Shahmohammadi, Megh Thakkar, Md Rizwan Parvez, Enamul Hoque, and Shafiq Joty. ChartQAPro: A More Diverse and Challengin... on Pith
- [96] Rubèn Tito, Dimosthenis Karatzas, and Ernest Valveny. Hierarchical multimodal transformers for Multipage DocVQA. Pattern Recognition, 144:109834, 2023. doi: 10.1016/j.patcog.2023.109834. URLhttps://doi.org/ 10.1016/j.patcog.2023.109834 on Pithdoi:10.1016/j.patcog.2023.109834
- [97] Yushi Bai, Shangqing Tu, Jiajie Zhang, Hao Peng, Xiaozhi Wang, Xin Lv, Shulin Cao, Jiazheng Xu, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks. In Proceedings of the 63rd Annual Meeti... on Pithdoi:10.18653/v1/2025.acl-long.183
- [98] Zhiyu Chen, Shiyang Li, Charese Smiley, Zhiqiang Ma, Sameena Shah, and William Yang Wang. ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processin... on Pithdoi:10.18653/v1/2022.emnlp-main.421
- [99] Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. PAL: Program-aided Language Models. InInternational Conference on Machine Learning, pages 10764–10799, on Pith
- [100] Hongchen Wei, Yuanzhe Wang, Bei Liu, Yifan Yang, Qi Dai, Ruichun Ma, Kai Qiu, Yunsheng Li, Dongdong Chen, Chong Luo, Zhenzhong Chen, and Baining Guo. XL-DocBench: Benchmarking Evidence-Grounded Extra-Long Document Understanding. In Advances in Neural Information Processing Sys... on Pith
The first 100 of 295 references are listed here. All of them are in this paper's citation record.
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment