{"id":"e9317048-d7a8-4bf6-93f6-29035efbc141","arxiv_id":"2610.35845","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"A survey-framework paper that defines work agents, organizes digital work into five levels, and reviews optimization and evaluation through a unified lens.","lead":"This paper gives a unified description of AI agents that do digital work, defining them as a model plus a runtime system that acts in a stateful workspace and is checked against the task. It sorts digital work into five levels and introduces two axes, persistence and openness, as a way to compare working conditions across the field.","discovery_kind":"unification","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"Under §2.2.2's own definition, τ-bench, the canonical high-persistence/low-openness instance (§2.2.3), is partially open: user confirmations that keep responsibility open are actor information discovered during execution.","rationale":"I agree with the reader that the paper is a comprehensive, honest position piece whose central framework is definitional and largely coherent. The reader's flagged weakest assumption (sufficiency of the two axes) is real; my concern is a more specific, internally testable defect within the same chapter: the boundary between 'predefined task world' and 'discovered during execution' is under-specified, and the paper's single canonical instance of the high-persistence/low-openness quadrant, τ-bench, becomes partially open under the paper's own openness definition. If this lands, the four-quadrant claim (Figure 2) does not separate cleanly, and the claimed unification rests on a one-dimensional axis plus an unacknowledged interaction with actor-supplied information. This matters because the two dimensions are one of the two named conceptual contributions in the abstract, not a peripheral example. I do not consider this rejection-level: equations (1)–(2), the five work forms, the extensive benchmark mapping, and the authors' candid limitation statements — dataset-size non-comparability, verifier unreliability, single-run VERSE evidence, context-window limits — all support acceptance after a targeted revision. Hence CONDITIONAL: accept if the authors state the missing criterion distinguishing interaction contents from task-world discovery and verify that each quadrant retains a clean instance in the surveyed literature; if the criterion cannot be stated without conceding that persistence and openness co-vary, the contribution should be reframed accordingly. Agreement with the reader is partial: we both locate the risk in the two-axis chapter, but the reader emphasizes missing coordinates (exhaustiveness) while I emphasize a definitional ambiguity that threatens the axes' independence as stated.","tokens_in":45420,"tokens_out":19419,"duration_ms":199183,"concrete_test":"Prong 1: apply the §2.2.2 openness definition to τ-bench by enumerating every task-relevant piece of information the agent must obtain from the simulated user during execution (requests, confirmations, corrections), and compare this with the paper's treatment of the τ²-bench user's actions as openness. If confirmations count as discovered information, Figure 2's top-left quadrant needs a new canonical instance or a stated exclusion criterion; if they do not count, state that criterion explicitly and re-check the other quadrant examples (SQuAD, DocVQA, GAIA, Workspace-Bench, OSWorld 2.0, VibeLifeBench) against it.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is the unification of agent research through two 'conceptually distinct yet complementary' dimensions, work persistence and work openness, with all four combinations instantiated in existing research (§2.2.3, Figure 2). The four-quadrant argument is load-bearing: if the axes are not cleanly separable, the framework's main novelty reduces to a single dimension with a confound. The paper's own definitions do not separate them. §2.2.2 defines openness over 'the objects, interfaces, rules, and actors relevant to the task' and counts learning 'how other actors have changed' the shared state as openness, as in τ²-bench. §2.2.1 defines persistence to include responsibility kept open because it 'depends on other participants,' citing τ-bench's withheld database updates 'until the user confirms them.' The user's confirmation is task-relevant information supplied by an actor that the agent must discover during execution; on a literal reading of the openness definition, τ-bench is at least partially open, and the paper never states the criterion that would exclude interaction contents (confirmations, messages) from the openness axis while still counting the τ²-bench user's actions as open. A charitable reading is available — openness concerns the structure of the task world, not contents exchanged within a known protocol — but the text does not state it, and the Figure 2 caption ('return paths through other participants') suggests participants' actions drive persistence, not openness. The closed-world monitoring obligation in §2.2.1 also slides toward openness once incidents arrive, since tracing affected components is itself classified as openness. So the high-persistence/low-openness quadrant may lack a clean empirical instance in the surveyed corpus.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"This is a survey-and-framework paper about foundation-model agents that perform delegated digital work. Section 2 defines a work agent as a system composed of a foundation-model decision core and a runtime harness operating in a stateful workspace, embedded in six elements: task, model, harness, workspace, work outcome, and verification (Figure 1). The paper's main conceptual proposals are two working-condition dimensions — work persistence (when responsibility may close) and work openness (how much of the task world must be discovered during execution) — whose four combinations are claimed to be instantiated in existing research (Section 2.2, Figure 2), and a five-level taxonomy of work forms from conversation-level to workspace-level (Section 3). It then surveys five capability families (Section 2.3), model, harness, and joint optimization with training-environment construction and feedback design (Section 4), and an evaluation analysis covering benchmark construction, judging methods, evaluation targets, and metric classes (Section 5), closing with sustained-delegation and recursive-self-improvement directions (Section 6). The paper reports numerical results only from cited works and labels its own evaluation-protocol proposals as proposals.","tokens_in":45817,"tokens_out":21444,"duration_ms":200145,"significance":"If its definitions hold, this framework gives the field a useful common vocabulary: the six-element system view corrects the habit of scoring models in isolation, the five capability families are well illustrated by the running business-report example in Section 2.3, Section 3.4's criterion for distinguishing workflow stages from system-internal execution stages is careful, and the Section 5 trichotomy of evaluation targets, dimensions, and methods is practically usable. The reporting discipline is a genuine strength: numbers are attributed to cited sources (for example, the WorkForge ALE gains and the Nemotron SWE-bench gains in Section 4), and the authors state the limits of their own suggestions, including single-run recursive-self-improvement results in Section 4.4.3 and dataset-size comparability in Section 4.1.1. The intellectual risk is definitional: the persistence/openness axes and the boundary between work forms and working conditions are the paper's novel analytical apparatus, and their current wording invites misclassification of the paper's own canonical examples. These are fixable clarification issues, not flaws in the survey's empirical content.","major_comments":[{"comment":"The four-quadrant claim in Section 2.2.3 ('all four combinations appear in existing research') is load-bearing for the paper's assertion that work persistence and work openness are 'conceptually distinct', and that claim currently rests on exactly one high-persistence/low-openness example, τ-bench, with no other setting assigned to that quadrant in the Figure 2 discussion. The classification of τ-bench is not supported by the paper's own openness definition as written. Section 2.2.2 defines openness over 'the objects, interfaces, rules, and actors relevant to the task' and says the agent 'may also need to find out where the relevant state is held, how other actors have changed it, and which actions it is authorized to take'; the τ-bench user's confirmation gates the database update, so on a literal reading the agent must discover actor-supplied information during execution, and the same phenomenon is used in the same subsection to justify τ²-bench's high openness. The text never states the criterion that would exclude interaction contents within a known protocol (confirmations, messages) from the openness axis while still counting the τ²-bench user's state changes as open; the Figure 2 caption ('return paths through other participants') even suggests that participant actions drive persistence, which is the same phenomenon. A charitable reading is available — openness concerns the structure of the task world, not event contents exchanged within a specified protocol — but it must be stated, with a test that lets a reader classify any interactive benchmark. I recommend adding this criterion to Section 2.2.2 and then re-checking the quadrant assignments, since a reader who adopts the literal reading would see the two dimensions collapse in exactly the case that is supposed to separate them.","section":"§3.5, with §2.2.2 and §2.2.3","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The paper's distinction between work forms and working conditions is not maintained at the Workspace Level. Section 2.2.3 states that the persistence/openness combinations 'describe the conditions of work rather than its form', while Section 3.5 defines the Workspace Level by the fact that 'the request no longer specifies every object involved' and the agent 'must establish which files, records, and applications bear on the work and how they are related' — phrasing nearly identical to the Section 2.2.2 definition of openness (the agent must establish during execution which objects are relevant and how they are represented and related). Section 3.5's closing sentence then maps the level's two concerns directly onto the openness and persistence axes. If the Workspace Level is defined by discovery, one of the five forms is defined by exactly the property assigned to the conditions axis, making the two parts of the framework partially redundant. Please sharpen Section 3.5's opening so that the level is defined by the scope and organization of the delegated object graph (multi-object consistency, version reconciliation, dependency tracing as the substance of the work) rather than by whether the request specified the objects; a brief worked example of a Workspace-Level task with a fully predefined object graph would settle the boundary.","section":"§3.5 (with §2.2.2 and §2.2.3)","authors_acknowledge":null,"authors_acknowledge_p":null}],"minor_comments":[{"comment":"The manuscript text contains numerous doubled-letter and spacing artifacts (for example, the running header rendered as 'EExxppaannddiinngg wwoorrkk ssccooppee aanndd ddeeppeennddeenncciieess' and a garbled repository/homepage line above the abstract); these should be cleaned in the production version.","section":"Throughout","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The Figure 4 caption doubles as a classification appendix, listing per-entry rationale for dozens of works; moving that rationale to a supplementary table would make the caption readable and the level assignments easier to audit.","section":"Figure 4","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Table 10 spans a very large portion of Section 5; moving it to an appendix and keeping in the main text the three-level summary (targets in 5.2.1, methods in 5.2.2, metrics in 5.3) would improve readability without losing content.","section":"Table 10","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The distinction between pass@k and passk is introduced in a single clause; a one-sentence worked example (for example, k=2 with one successful and one failed attempt) would make the all-attempts-must-pass reading of passk unambiguous.","section":"§5.3","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Equations (1) and (2) use informal tuple-like notation for what is really a schematic account; a sentence noting that the notation is illustrative rather than a formal operational semantics would prevent over-reading.","section":"§2.1, Eqs. (1)–(2)","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The proposed (M0, H0), (M1, H0), (M0, H1), (M1, H1) comparison protocol is a useful and honest contribution; consider making it a highlighted 'proposed protocol' call-out so that readers can cleanly separate the survey's synthesis from its forward-looking proposals.","section":"§4.4.2","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"major_revision"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"The takeaway: this is worth reading, and the part worth reading is Section 2. The paper defines a \"work agent\" as six elements (task, model, harness, workspace, outcome, verification) and introduces two working-condition axes — persistence (when responsibility closes) and openness (how much of the task world must be discovered during execution) — plus a five-form taxonomy of work scope. The rest is a broad survey of training, harness optimization, and evaluation, with roughly ninety recent works mapped onto the framework. It makes no empirical claims of its own; all numbers are attributed to cited sources, and the authors are explicitly honest about comparability limits (dataset sizes, trajectory content, cross-benchmark scores). The citation pattern looks careful, and the paper marks its own proposals as proposals, which is more than most surveys manage.\n\nWhat it does well: the persistence/openness distinction genuinely cuts across long-horizon, computer-use, and office-agent research, and the five work forms organize the benchmark landscape usefully. The placements mostly felt right.\n\nSoft spots, in proportion. The main one is the orthogonality of the two axes, which is the load-bearing novelty. Section 2.2.3 claims all four quadrant combinations appear in existing research, with τ-bench as the high-persistence/low-openness instance. The stress-test is half right: under the paper's own openness definition, τ-bench's user confirmations are actor information discovered during execution, so the cleanest low-openness/high-persistence example sits on an unstated boundary criterion. A charitable reading is available — openness concerns the structure of the task world the agent must establish, not contingent outcomes within a known protocol — and the τ-bench/τ²-bench contrast at the end of §2.2.3 gestures exactly that way. But the criterion is never stated. Fixable with a paragraph, not fatal; the axes remain analytically distinct. The part of the stress-test about closed-world monitoring sliding into openness I would reject: discovering which entries an incident affects is openness, the obligation to respond is persistence, and the paper's decomposition there is coherent.\n\nThe taxonomy is subjective by nature — no proof of exhaustiveness; the five forms and five capability families are chosen by judgment. That is characteristic of conceptual frameworks, not a flaw. The paper is long and somewhat repetitive.\n\nWho it is for: anyone building or evaluating agents in software, office, or computer-use settings, and survey writers who need a vocabulary. It will not change what agents can do, but it gives the field shared coordinates. I would send it to peer review, with the referee asked to push on the openness boundary and the \"all four combinations\" claim. My verdict: accept with revision, not a reject.","headline":"A careful, genuinely useful survey-with-framework whose persistence/openness axes mostly hold up; the weakest spot is the unstated boundary criterion behind the τ-bench quadrant assignment.","tokens_in":46453,"tokens_out":7659,"duration_ms":74367},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"This paper defines a work agent as a foundation model paired with a runtime harness acting in a stateful workspace, and argues that this work-centric frame unifies long-horizon, computer-use, and office agent research.","keywords":["work agents","foundation models","runtime harness","work persistence","work openness","agent capabilities","agent optimization","agent evaluation"],"falsifier":"Annotate a sample of existing agent tasks by their position on the persistence and openness axes. The claim would be undercut if a substantial share of tasks resists placement in the four persistence–openness combinations, or if tasks at the same coordinates demand systematically different capabilities — for example, tasks that differ only in collaboration structure or regulatory risk but require mechanisms the two axes do not predict.","tokens_in":46898,"feed_emoji":"🤖","tokens_out":10801,"duration_ms":94961,"temperature":0.7,"pith_summary":"This paper tries to establish a single definition of what an AI agent is when it does real work: a foundation model that decides, plus a runtime harness that observes, acts, and remembers, operating in a stateful workspace and judged by verifiable outcomes. It argues that this work-centric view unifies research that currently splits into long-horizon, computer-use, and office agents, because those lines each track one facet of the same delegated task. To make the unification usable, the paper proposes two coordinates for working conditions — persistence (does responsibility close at delivery or stay open across events?) and openness (is the task world given in advance or discovered during execution?) — and five forms of digital work, from single answers to whole-workspace operations. A sympathetic reader would care because the framework promises a common language for comparing benchmarks, tracing failures to one of five capability families, and deciding whether an improvement belongs in the model or in the harness.","feed_headline":"One definition for all agents: model plus harness, judged by outcomes","feed_subtitle":"Persistence and openness map every agent task onto two axes, from simple Q&A to open-ended office work.","key_machinery":"The load-bearing object is the six-element system definition with its execution–verification loop. At an assessment point the loop splits into two roles: execution writes $(T, E_0) \\xrightarrow{M,H} (\\tau_{0:t}, E_t, O_t)$, meaning the model–harness pair acts in the initial workspace and produces an action record, a new workspace state, and an outcome, while verification evaluates $S_t = V(T, E_0, \\tau_{0:t}, E_t, O_t)$, meaning the evidence is checked against the task requirements by programs, humans, or model-based judges. Two dimensions order the working conditions, five work forms order the scope of the delegated work, and the argument is carried by showing that existing systems occupy distinct positions in this space: responsibility closes at delivery in question answering, remains open awaiting approval in service workflows, requires discovery in information seeking, and combines open discovery with open responsibility in ongoing work under changing conditions.","core_discovery":"On the paper's own terms, the central claim is that a work agent is best understood not as a model that generates outputs but as a system of six elements — task, model, harness, workspace, work outcome, and verification — bound together in a closed loop. The harness assembles context from the task and the workspace for the model; the model proposes actions; the harness executes them and returns observations; and verification assesses the resulting evidence against the task requirements. The same loop runs in every work-agent setting, and what varies are the conditions, which the paper captures with two dimensions: work persistence, from responsibility that closes at delivery to responsibility that stays open across stages, feedback cycles, and later events, and work openness, from a predefined task world to one the agent must discover while acting. The paper classifies digital work into five forms by scope — conversation, atomic action, multi-step, workflow, and workspace — and derives five capability families from the work process: perception and understanding, reasoning and decision making, planning and coordination, execution and state management, and verification, recovery, and replanning. It then reads the field's optimization and evaluation literature through this frame, arguing that capabilities can be improved by training the model, by upgrading the harness, or by adapting the two jointly.","pith_inferences":["If the two dimensions were adopted as a standard annotation, benchmark families could be plotted on a single map, making progress across software, office, and enterprise domains comparable without a shared test set.","A prediction the paper leaves implicit: agents trained in high-persistence, high-openness environments should transfer to ordinary office work better than agents trained on delivery-close, predefined tasks, because the former must practice verification, dependency tracking, and state management.","The five work forms suggest a curriculum ordering — workspace-level competence presupposes the mechanisms of the lower forms — so staged training that introduces openness and persistence gradually may be more sample-efficient than uniform task mixing.","Because verification is defined as part of the system rather than an external judge, the framework implies that agents which close the verification loop themselves are closer to deployment-ready; this could be tested by ablating an agent's internal verification loop and comparing against an external evaluator."],"forward_implications":["Question answering, single-file edits, multi-step projects, staged workflows, and whole-workspace operations become one spectrum of work forms, so methods and benchmarks designed at one level can be positioned, compared, and reused at another.","Failures can be routed to a specific capability — missing evidence to perception, a questionable decision to reasoning, a changed prerequisite to planning, an execution error to state management — which directs engineers to train the model or change the harness.","Evaluation should inspect outcomes through evidence — artifacts, workspace states, workflow records, and maintained conditions — rather than the final answer alone, because acceptance depends on the object actually delivered.","Optimization decomposes into model optimization, harness optimization, and joint adaptation, and the paper proposes comparing $(M_0,H_0)$, $(M_1,H_0)$, $(M_0,H_1)$, and $(M_1,H_1)$ under consistent task conditions to attribute improvements to the right component.","Because persistence and openness raise the demands on all five capability families, benchmarks that are high on both axes become the testbed for sustained delegation, which the paper identifies as the field's frontier."],"supporting_citations":[{"why":"Supplies the multi-step software task in which a code patch is accepted only if repository tests pass, used to exemplify responsibility closing at delivery and artifact-level verification.","marker":"[16]"},{"why":"Gives the service-workflow setting where updates are withheld until a user confirms, anchoring the high-persistence, low-openness quadrant.","marker":"[29]"},{"why":"Provides the setting where a simulated user changes shared state the agent cannot operate, supporting the openness and workflow-coordination discussion.","marker":"[30]"},{"why":"The web environment in which task-relevant information must be located by navigation, anchoring the openness dimension and the computer-use line of work.","marker":"[6]"},{"why":"Extends openness to real desktop applications and supplies the state-based evaluation model for whole sequences of interface operations.","marker":"[7]"},{"why":"The case where revised budget and vendor constraints arrive mid-task and force reworking earlier entries, motivating persistence across later events.","marker":"[32]"},{"why":"The information-seeking task where responsibility closes at delivery but answers require discovering sources, occupying the low-persistence, high-openness quadrant.","marker":"[40]"},{"why":"The workspace-level benchmark where locating relevant files and tracing their dependencies is the core of the task, defining the top work form and its agent-judge evaluation.","marker":"[36]"},{"why":"The harness that reorganizes repository operations and output filtering for the model, the lead example of harness optimization.","marker":"[3]"},{"why":"The benchmark of proactive assistance over simulated weeks, used to define obligations over an agreed interval with high persistence.","marker":"[34]"}],"fun_headline_variants":["Agents are model-plus-harness loops judged by verifiable outcomes","Two axes, persistence and openness, sort every agent task","Six elements close the loop from task to verified work","Work agents: model, harness, workspace, and verification in one loop","From simple Q&A to open-ended work, one loop with five capability families"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The load-bearing premise is that two axes — how long responsibility stays open and how much of the task world the agent must discover — are the right and sufficient coordinates for working conditions, and that all digital knowledge work fits the five proposed forms; if a third factor such as collaboration structure or risk is equally primary, the promised unification is weaker than the paper claims.","fun_headline_variants_meta":{"raw":{"variants":["Agents are model-plus-harness loops judged by verifiable outcomes","Two axes, persistence and openness, sort every agent task","Six elements close the loop from task to verified work","Work agents: model, harness, workspace, and verification in one loop","From simple Q&A to open-ended work, one loop with five capability families"]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":7.4e-05,"raw_usage":{"cost":7.3528e-05,"is_byok":false,"cost_details":{"upstream_inference_cost":7.3528e-05,"upstream_inference_prompt_cost":3.0576e-05,"upstream_inference_completions_cost":4.2952e-05},"total_tokens":1312,"prompt_tokens":1076,"completion_tokens":236,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":925,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":159}},"tokens_in":1076,"tokens_out":236,"cost_source":"openrouter_usage","duration_ms":4044,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":159,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-10-27T05:19:16.474666+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":"Annotate a sample of existing agent tasks by their position on the persistence and openness axes. The claim would be undercut if a substantial share of tasks resists placement in the four persistence–openness combinations, or if tasks at the same coordinates demand systematically different capabilities — for example, tasks that differ only in collaboration structure or regulatory risk but require mechanisms the two axes do not predict.","supporting_citations":[{"cited_title":"SWE-bench: Can Language Models Resolve Real-world Github Issues? InInternational Conference on Learning Representations, pages 54107–54157, 2024","cited_arxiv_id":null,"evidence_quote":"Supplies the multi-step software task in which a code patch is accepted only if repository tests pass, used to exemplify responsibility closing at delivery and artifact-level verification."},{"cited_title":"InInternational Conference on Learning Representations, pages 9965–10017,","cited_arxiv_id":null,"evidence_quote":"Gives the service-workflow setting where updates are withheld until a user confirms, anchoring the high-persistence, low-openness quadrant."},{"cited_title":"and Zhang, Tianjun and Wang, Xin and Gonzalez, Joseph E","cited_arxiv_id":null,"evidence_quote":"Provides the setting where a simulated user changes shared state the agent cannot operate, supporting the openness and workflow-coordination discussion."},{"cited_title":"WebArena: A Realistic Web Environment for Building Autonomous Agents","cited_arxiv_id":null,"evidence_quote":"The web environment in which task-relevant information must be located by navigation, anchoring the openness dimension and the computer-use line of work."},{"cited_title":"OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments , url =","cited_arxiv_id":null,"evidence_quote":"Extends openness to real desktop applications and supplies the state-based evaluation model for whole sequences of interface operations."},{"cited_title":"MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback","cited_arxiv_id":null,"evidence_quote":"The case where revised budget and vendor constraints arrive mid-task and force reworking earlier entries, motivating persistence across later events."},{"cited_title":"Patil, Huanzhi Mao, Charlie Cheng-Jie Ji, Fanjia Yan, Vishnu Suresh, Ion Stoica, and Joseph E","cited_arxiv_id":null,"evidence_quote":"The information-seeking task where responsibility closes at delivery but answers require discovering sources, occupying the low-persistence, high-openness quadrant."},{"cited_title":"Workspace-Bench 1.0: Benchmarking AI Agents on Workspace Tasks with Large-Scale File Dependencies","cited_arxiv_id":null,"evidence_quote":"The workspace-level benchmark where locating relevant files and tracing their dependencies is the core of the task, defining the top work form and its agent-judge evaluation."},{"cited_title":"Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press","cited_arxiv_id":null,"evidence_quote":"The harness that reorganizes repository operations and output filtering for the model, the lead example of harness optimization."},{"cited_title":"VibeLifeBench: Can Your Life Agent Be Proactive and Persistent in a Living World?","cited_arxiv_id":"2608.10875","evidence_quote":"The benchmark of proactive assistance over simulated weeks, used to define obligations over an agreed interval with high persistence."}],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}