Pith.

Artificial Intelligence · arXiv:2608.15089 v1 · submitted 15 Aug 2026 · 14 references

StateM: Reaching 95.3% Raw Accuracy, or a $15 Frontier Run, on Terminal-Bench 2.1 via Harness Scaling

Ziheng Qin, Yaxin Lu, Zhangyang Atlas Wang, Kai Wang

The paper says

“Long-horizon agents can fail even when their underlying models can solve the constituent steps. They may lose track of mutable state, fail to reactivate lessons from earlier executions, skip known procedures, or stop prematurely.”

more of the abstract

We bet on harness scaling to improve the execution system around an agent without changing its model weights. We introduce StateM, an agent-native runtime that organizes execution around durable states, phase-local context, checked transitions, recoverable runbooks, and versioned procedural practices that agents and users can inspect together. On Terminal-Bench 2.1, StateM raises GPT-5.5 xhigh to 92.1%, versus 83.1% reference and GPT-5.6 Sol Ultra at 91.9%. The runbook transfers unchanged to GPT-5.6. With GPT-5.6 Sol xhigh, StateM reaches 95.3% raw accuracy across 445 trials and succeeds on all 89 tasks at least once. The frozen profile raises GPT-5.6 Luna from 76.7 to 85.4%, above the 84.9% Sol xhigh reference. Using the same runtime, runbook structure, and golden rules, less than $38 of adaptation raises DeepSeek-V4 Flash from 82.7 to 88.1% under standard timeouts and to 89.1% on an 88-task common core. Extending only the remaining latency-sensitive task matches the reported 88.8% GPT-5.6 Sol max result. Final-score API usage is about $15 versus $574.68 for the GPT reference; total DeepSeek expenditure is $52.22. On BusinessBench, family-specific runbooks built on development sets yield held-out gains of 0.55 macro and 1.34 micro points; two mechanism-matched families improve by 10.04 points. Concrete rules generalize when tasks share execution structure, while the control methodology applies broadly. StateM turns selected postmortem findings into persistent, executable preconditions and practices, making learned controls explicit and enforceable through stateful controls. Code at github.com/henryqin1997/statem.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 27 Aug 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. A serious, unusually transparent harness-scaling paper whose headline gain rests on an unmatched baseline; Appendix B may undercut the central +9-point claim. This line is wrong

4 major objections · 6 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 14 references say.

The paper says, at 4.2 and Appendix B

The review did not quote this passage.

Pith says: Objection 1 of 4 · major

The paper's central +9.0-point fixed-model claim is computed against the published 83.1% Codex reference (footnote 3), not a matched rerun, and Appendix B states that 'On a personal Macbook Pro 2025, GPT-5.5 xhigh can reaches around 91% pass rate on Terminal Bench 2.1.'

more of objection 1

If that sentence refers to the bare GPT-5.5 xhigh Codex agent without StateM, the StateM-specific gain is about one point, not nine; if it refers to a StateM run, the text needs to say so explicitly and to give the same-environment baseline. The manuscript must clarify this sentence and add a matched A/B comparison (same agent version, same hardware, same timeouts) before the headline harness-scaling claim can stand.

The paper says, at 4.2 (GPT-5.6 rows)

The review did not quote this passage.

Pith says: Objection 2 of 4 · major

The +10.4-point GPT-5.6 Sol xhigh gain (84.9% to 95.28%) is likewise computed against a public reference row, with no evidence that the reference was obtained under the same agent version, environment, or timeout conditions. Since the paper's own Appendix B notes that hardware and sandbox differences (AWS vs. MacBook, Daytona issues) materially affect Terminal-Bench runs, a same-environment bare-GPT-5.6 baseline is required to attribute the difference to StateM rather than to environment or agent-version effects.

The paper says, at 4.1 and 4.7

The review did not quote this passage.

Pith says: Objection 3 of 4 · major

The development boundary in Section 4.1 does not exclude benchmark adaptation. Section 4.7 explicitly states that video-task frame-precision defaults were introduced 'after observing benchmark behavior' and that the DNA insertion conventions were learned from repeated evaluator feedback, and Section 3.1 says the Terminal-Bench results should not be interpreted as isolating the state-machine abstraction.

more of objection 3

Because the profile was optimized on the same 89 tasks that are then scored, the 95.28% and 92.1% figures may partly measure calibration to Terminal-Bench's verifier conventions. The frozen GPT-5.6 transfer shows the profile is not model-specific, but it remains benchmark-specific; a concrete test, such as applying the frozen profile to another terminal benchmark or to truly held-out tasks, or removing the two admitted evaluator-derived practices and reporting the change, is needed.

The paper says, at 4.4

The review did not quote this passage.

Pith says: Objection 4 of 4 · major

The cost-frontier comparison is not matched in operating conditions. The '88.8%' match is a descriptive aggregate that replaces the five gpt2-codegolf trials with an extended-timeout evaluation, while the $574.68 figure is the submission-reported model cost of a different public evaluation (footnote 5).

more of objection 4

The claim that StateM delivers a frontier result at roughly 1/40 the cost therefore mixes two different evaluations and one selectively time-extended run. The paper should present a fully matched cost experiment, or explicitly restrict the claim to the standard-timeout 88.09% result and the disclosed $15.20 evaluation cost.

6 minor objections

The paper says, at Appendix B

Pith says: minor objection 1 of 6

Appendix B contains typos ('resouce', 'can reaches') and, more importantly, should specify whether the 91% figure is for the bare Codex agent or the StateM system; this ambiguity is directly relevant to the main claim.

The paper says, at Abstract and footnote 1

Pith says: minor objection 2 of 6

The abstract and Section 1 present 95.3% as 'raw accuracy'; the manuscript already discloses in footnote 1 that scoring all flagged reward-hacking trajectories as zero gives 93.26%. Consider moving the adjudication-adjusted range into the main text so the reader sees it before the headline.

show 4 more

The paper says, at Table 1

Pith says: minor objection 3 of 6

The two-family slice is labeled 'Exploratory mechanism-matched'; the text should state explicitly that this slice was selected after inspecting results, so it is not confirmatory evidence.

The paper says, at Figure 5

Pith says: minor objection 4 of 6

Figure 5 plots cost and accuracy for points that come from different operating conditions (one extended-timeout task); the caption should repeat the disclosure that 88.76% is a descriptive aggregate.

The paper says, at 4.7

Pith says: minor objection 5 of 6

Section 4.7 lists the video-task and DNA conventions as examples of post hoc decisions; a table enumerating all such decisions and their effect on the reported scores would strengthen the transparency claim.

The paper says, at Appendix A

Pith says: minor objection 6 of 6

Appendix A's runbook example is helpful; consider adding a note that the example is illustrative and not the full Terminal-Bench profile.

What would settle it

Run a matched A/B evaluation on Terminal-Bench 2.1: same Codex agent version, same AWS m7i.4xlarge environment, same timeouts, same 445-trial protocol, with and without StateM, for GPT-5.5 xhigh and GPT-5.6 Sol xhigh. Compare the bare-model scores to the 83.1% and 84.9% references and to StateM's 92.1% and 95.28%. If bare GPT-5.5 xhigh scores near 91% as Appendix B suggests, the harness-specific gain is about a percentage point and the 'model-generation-sized' central claim fails; if it scores near 83%, the gain is confirmed.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 27 Aug 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic partly agrees 48 s · 20,865 tokens read, 5,314 written
  3. Editor wrote the verdict above 22 s · 21,716 tokens read, 2,315 written
  4. Referee major revision: 4 major objections, 6 minor 1 min 15 s · 21,139 tokens read, 8,284 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Short version: this is a serious paper worth reading, but the marquee result—the "model-generation-sized" harness gain—is not backed by a matched control, and one informal line in Appendix B suggests it could be mostly an artifact.

What's genuinely new is the system: an agent-native, YAML state-machine runbook that the agent and user can both edit, with checked transitions, persistent state, and recoverable runbooks. That combination is new in the cited prior work, and the paper is careful to separate the generic runtime from the benchmark-adapted control profile. I also give real credit for the frozen cross-model transfer to GPT-5.6 with zero runbook changes—that is a genuinely independent piece of evidence. The cost frontier with DeepSeek (~$15 for the final-score evidence) is striking and plausibly real, though the baseline comparisons there are also unmatched. The paper is unusually candid: it discloses raw vs adjudicated scores, the open PR, the benchmark-adaptation boundary, and the post hoc defaults in Section 4.7. That transparency earns trust.

The soft spots are proportionate but significant. The 92.1% vs 83.1% comparison uses a published reference with an older Codex agent version, not a matched A/B rerun—footnote 3 admits this. Appendix B then says a bare GPT-5.5 xhigh on a MacBook reaches ~91% on Terminal-Bench 2.1. If that is without StateM, the harness-specific gain is about one point, not nine, and the "model-generation-sized" narrative collapses. The 95.28% GPT-5.6 score is pre-adjudication and they say so, but it is still the headline. The benchmark-adaptation concern is real but disclosed; the frozen transfer mitigates it, yet the profile was developed on the same 89 tasks, so the GPT-5.6 number is still partly fitted to that benchmark. The DeepSeek cost accounting looks careful, but the 88.8% match to a "Sol max" reference is descriptive, not a controlled comparison.

Overall: the contribution is the system and the methodology, not the specific accuracy numbers. Those should be treated as conditional until matched baselines are run. A serious referee should engage with this paper, but the authors should be pushed to run proper A/B controls on the same agent version, environment, and hardware before the headline claims are taken at face value.

The strongest objection

The central quantitative evidence for harness scaling is the +9.0-point GPT-5.5 lift (83.1% to 92.1%) and the +10.4-point GPT-5.6 lift (84.9% to 95.28%). Both are computed against published reference rows, not matched A/B controls: footnote 3 admits that the 83.1% reference uses Codex agent v0.125.0 and is not an agent-version-matched rerun. Appendix B then states that 'On a personal Macbook Pro 2025, GPT-5.5 xhigh can reaches around 91% pass rate on Terminal Bench 2.1.' If that 91% refers to the bare Codex agent without StateM, the StateM-specific gain is roughly one point, not nine, and the paper's model-generation-sized claim collapses. The GPT-5.6 Sol xhigh 84.9% baseline and the 91.9% Sol ultra reference are likewise unrerun public numbers; environment, agent version, timeout, and hardware differences (AWS vs. MacBook, Daytona sandbox issues in Appendix B) can plausibly move Terminal-Bench scores by several points. The benchmark-adaptation disclosures in Section 4.7 compound this concern: because the runbook was developed against the same benchmark, part of the residual gain could be adaptation to evaluator conventions rather than generic harness value. The load-bearing assumption is that the public references are comparable under current conditions, and nothing in the paper verifies that.

The reader's stated weakest assumption was the benchmark-adaptation boundary, which is a real concern given the Section 4.7 disclosures. I agree that adaptation to evaluator conventions could inflate the headline numbers. However, the more load-bearing and more immediately checkable problem is the baseline mismatch: the paper computes its flagship harness gains against public references that are not matched in agent version, environment, or evaluation conditions, and Appendix B contains an informal bare GPT-5.5 xhigh pass rate near 91% that is within one point of the 92.1% StateM result. If accurate, that single sentence undercuts the central empirical claim far more than the adaptation issues do. The paper is unusually transparent: it discloses raw versus adjudicated scores, unmatched baselines, post hoc defaults, and its own near-violations, and the frozen GPT-5.6 transfer and open code provide independent support. But transparency does not substitute for a matched control. The recommendation remains conditional because the decisive A/B has not been run: if the matched baseline lands near 83%, the harness-scaling claim is strongly supported; if it lands near 91%, the conclusion must be substantially revised. The concrete test above would settle which world we are in.

The referee's report

If the quantitative claims survive matched controls, the paper makes a useful contribution: it formulates harness scaling as a distinct capability axis, implements an inspectable state-machine artifact, and provides a transfer hierarchy plus unusually transparent cost and adjudication disclosure. The frozen GPT-5.5-to-GPT-5.6 transfer, the public submission PR, and the open code base are concrete assets for verification. However, the headline numbers currently rest on public-reference baselines and on a benchmark-adapted profile, and the internal inconsistency with Appendix B means the central gain magnitude is not established. The significance therefore depends on the matched A/B evidence that the paper does not yet provide.

Pith's reading of the paper (a machine's summary, not the authors')

This paper argues that much of what looks like agent failure is actually harness failure. Holding model weights completely fixed, it wraps a command-line agent in StateM, a runtime that organizes long tasks into durable states with checked transitions, phase-local instructions, and versioned procedural lessons. On the 89-task Terminal-Bench 2.1, the same model jumps from 83.1% to 92.1%, roughly the size of a model-generation upgrade, and a profile frozen across the upgrade carries GPT-5.6 Sol to a raw 95.28% (424 of 445 trials) versus an 84.9% reference. On the cost frontier, the same runbook structure re-adapted for under $38 lifts DeepSeek-V4 Flash from 82.7% to 88.1%, with final-score evidence costing about $15 in API charges against the $574.68 reported for the GPT reference run. If the paper is right, harness scaling, not model scaling, is the main bottleneck holding back long-horizon agents, and procedural knowledge can be accumulated outside model weights.

What carries the argument. The load-bearing object is the runbook: a human-readable YAML state machine the agent operates through ordinary CLI commands and a human supervisor can edit, version, and audit in the same workspace. Each state is a context-and-contract boundary: an in_hook loads phase-local instructions and durable progress on entry, an out_hook persists receipts, and a before_transfer block evaluates exit conditions (host-run command or predicate checks, manual decisions, structured self-attestations, or separate llm_review judgments) before a goto commits a transition. The runtime keeps the mutable execution record, including current state, transition history, hook outcomes, and evidence files, outside the model context, so a restarted or context-compacted agent resumes from an explicit phase rather than reconstructing it from an append-only transcript. Around this core sit versioned practices (prompts, checks, and activation conditions that encode lessons from earlier failures) and a stop hook that blocks premature termination unless the run is terminal or externally blocked. This representation makes the harness itself the search space: after a failed run, a hyper-agent proposes changes to states, checks, and recovery rules, and the model weights never change.

Core claim. StateM externalizes the procedural state of a long-running agent into a versioned YAML runbook: a directed graph of coarse phase-states, each a 'context-and-contract boundary' whose entry refreshes the phase-local instructions and durable progress and whose exit is gated by explicit checks the host runtime can evaluate. Transitions use a checked goto protocol that verifies the edge, evaluates before_transfer checks, runs persistence hooks, and only then commits the new state, logging failures so the run stays repairable. Versioned 'practices' store selected postmortem lessons as executable prompts, checks, and activation rules, closing the procedural-memory gap in which a lesson is learned but never reactivated. The paper's evidence is a transfer hierarchy: exact runbooks carry frozen from GPT-5.5 to GPT-5.6 (+9.0 and +10.4 points over references), development principles carry across providers (under $38 of adaptation lifts DeepSeek-V4 Flash from 82.7% to 88.1%), and the control-boundary method carries across task families on BusinessBench, where mechanism-matched families gain about 10 points and mismatched families lose a few. The headline 95.28% is a raw public submission pending adjudication; the paper itself notes that zeroing four flagged trajectories gives 94.38% and zeroing all nine gives 93.26%, and that the results measure the combined runtime plus a benchmark-adapted profile rather than the state-machine abstraction alone.

The premise it rests on. The whole result rests on one boundary: the people who wrote the runbooks never looked at the hidden tests, the answer-verifier code, the task hashes, or the evaluation outcomes of the measured models to decide what the runbook should do, and the paper itself records two near-violations (image frame-precision defaults and DNA insertion conventions) adopted only after watching benchmark behavior.

If the paper is right (Pith's inferences, not the authors')

If the claim holds, agent benchmark scores are system scores, not model scores: the same model can be made to look a generation stronger or weaker purely by changing the runtime around it.

Procedural knowledge can live outside model weights: a frozen runbook carried unchanged from GPT-5.5 to GPT-5.6, so execution lessons can survive model upgrades without retraining.

The cost-accuracy frontier moves by an order of magnitude: a cheap model with an adapted harness matched a frontier reference score at roughly one-38th of the reported API cost, changing which deployment choices are economical.

Reliability lessons become enforceable: a postmortem encoded as a checked transition or activation rule cannot be silently skipped, which is what converts recurring failures into completed jobs.

The transfer hierarchy sets expectations: exact runbooks transfer across nearby models, principles transfer across providers, and control-boundary abstractions transfer across tasks only when the execution structure matches, while mismatches produce small negative transfer.

My read: the paper's own adjudication arithmetic brackets the true GPT-5.6 Sol effect between 93.26% and 95.28% raw, still roughly 8 to 10 points over the 84.9% reference, so the defensible claim is the size of the harness gain, not its exact decimal.

A testable extension the paper does not run: apply the frozen GPT-5.5 profile to a later GPT model generation and measure whether the residual gain shrinks as baseline harnesses improve; the shrinking residual would quantify how much of current frontier headroom is actually harness headroom.

The selective-memory principle generalizes beyond runbooks: it predicts that agent memory systems retain more usable lessons when failures are filtered through a control-boundary attribution step, which could be tested by comparing retention policies on the same long-horizon suite.

The cost result invites a cheap extrapolation: the same 'adapt a profile, not a model' loop should reproduce an order-of-magnitude cost ratio on other long-horizon task suites whenever the cheaper model sits in the same price tier as DeepSeek-V4 Flash.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Long-horizon agents can fail even when their underlying models can solve the constituent steps. They may lose track of mutable state, fail to reactivate lessons from earlier executions, skip known procedures, or stop prematurely. We bet on harness scaling to improve the execution system around an agent without changing its model weights. We introduce StateM, an agent-native runtime that organizes execution around durable states, phase-local context, checked transitions, recoverable runbooks, and versioned procedural practices that agents and users can inspect together. On Terminal-Bench 2.1, StateM raises GPT-5.5 xhigh to 92.1%, versus 83.1% reference and GPT-5.6 Sol Ultra at 91.9%. The runbook transfers unchanged to GPT-5.6. With GPT-5.6 Sol xhigh, StateM reaches 95.3% raw accuracy across 445 trials and succeeds on all 89 tasks at least once. The frozen profile raises GPT-5.6 Luna from 76.7 to 85.4%, above the 84.9% Sol xhigh reference. Using the same runtime, runbook structure, and golden rules, less than $38 of adaptation raises DeepSeek-V4 Flash from 82.7 to 88.1% under standard timeouts and to 89.1% on an 88-task common core. Extending only the remaining latency-sensitive task matches the reported 88.8% GPT-5.6 Sol max result. Final-score API usage is about $15 versus $574.68 for the GPT reference; total DeepSeek expenditure is $52.22. On BusinessBench, family-specific runbooks built on development sets yield held-out gains of 0.55 macro and 1.34 micro points; two mechanism-matched families improve by 10.04 points. Concrete rules generalize when tasks share execution structure, while the control methodology applies broadly. StateM turns selected postmortem findings into persistent, executable preconditions and practices, making learned controls explicit and enforceable through stateful controls. Code at github.com/henryqin1997/statem.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-09-23

The authors' main claim, as the classifier picked it out 0.32

“Concrete rules generalize when tasks share execution structure, while the control methodology applies broadly.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.05 · states limitation 0.37 · human participants 0.06 · controlled comparison 0.94 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.98 · mentions code or data availability 0.97

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Long-horizon agents can fail even when their underlying models can solve the constituent steps. background 0.98 0.12 0.05 asserted 0.77
They may lose track of mutable state, fail to reactivate lessons from earlier executions, skip known procedures, or stop prematurely. background 0.98 0.10 0.03 hedged possibility 1.00
We bet on harness scaling to improve the execution system around an agent without changing its model weights. method 0.56 0.22 0.14 speculative 0.82
We introduce StateM, an agent-native runtime that organizes execution around durable states, phase-local context, checked transitions, recoverable runbooks, and versioned procedural practices that agents and users can inspect together. contribution 0.85 0.85 0.11 asserted 1.00
On Terminal-Bench 2.1, StateM raises GPT-5.5 xhigh to 92.1\%, versus 83.1\% reference and GPT-5.6 Sol Ultra at 91.9\%. result 1.00 0.95 0.22 asserted 1.00
The runbook transfers unchanged to GPT-5.6. result 0.93 0.84 0.06 asserted 1.00
With GPT-5.6 Sol xhigh, StateM reaches 95.3\% raw accuracy across 445 trials and succeeds on all 89 tasks at least once. result 1.00 0.96 0.42 asserted 1.00
The frozen profile raises GPT-5.6 Luna from 76.7 to 85.4\%, above the 84.9\% Sol xhigh reference. result 1.00 0.94 0.08 asserted 1.00
Using the same runtime, runbook structure, and golden rules, less than \$38 of adaptation raises DeepSeek-V4 Flash from 82.7 to 88.1\% under standard timeouts and to 89.1\% on an 88-task common core. result 1.00 0.95 0.11 asserted 1.00
Extending only the remaining latency-sensitive task matches the reported 88.8\% GPT-5.6 Sol max result. result 1.00 0.86 0.07 asserted 1.00
Final-score API usage is about \$15 versus \$574.68 for the GPT reference; total DeepSeek expenditure is \$52.22. result 1.00 0.93 0.08 asserted 1.00
On BusinessBench, family-specific runbooks built on development sets yield held-out gains of 0.55 macro and 1.34 micro points; two mechanism-matched families improve by 10.04 points. result 1.00 0.93 0.06 asserted 1.00
Concrete rules generalize when tasks share execution structure, while the control methodology applies broadly. interpretation 0.98 0.70 0.26 asserted 1.00
StateM turns selected postmortem findings into persistent, executable preconditions and practices, making learned controls explicit and enforceable through stateful controls. method 0.46 0.84 0.27 asserted 1.00
Code at github.com/henryqin1997/statem. contribution 0.74 0.48 0.02 unclear 0.69

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.97

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 11,075 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

References (14)
  1. [1] Victor Barres, Honghua Dong, Soham Ray, Xujie Si, and Karthik Narasimhan. ^2 -bench: Evaluating conversational agents in a dual-control environment, 2025 on Pith
  2. [2] Xiang Deng, Jeff Da, Edwin Pan, Yannis Yiming He, Charles Ide, Kanak Garg, Niklas Lauffer, Andrew Park, Nitin Pasari, Chetan Rane, Karmini Sampath, Maya Krishnan, Srivatsa Kundurthy, Sean Hendryx, Zifan Wang, Vijay Bharadwaj, Jeff Holm, Raja Aluri, Chen Bo Calvin Zhang, Noah J... on Pith
  3. [3] Simon Dennis, Michael Diamond, Rivaan Patil, Kevin Shabahang, and Hao Guo. In-context prompting obsoletes agent orchestration for procedural tasks. arXiv preprint arXiv:2604.27891, 2026 on PithPith's reviewarXiv
  4. [4] Shangding Gu. From model scaling to system scaling: Scaling the harness in agentic ai. arXiv preprint arXiv:2605.26112, 2026 on PithPith's reviewarXiv
  5. [5] Jiahang Lin, Shichun Liu, Chengjun Pan, Lizhi Lin, Shihan Dou, Xuanjing Huang, Hang Yan, Zhenhua Han, and Tao Gui. Agentic harness engineering: Observability-driven automatic evolution of coding-agent harnesses. arXiv preprint arXiv:2604.25850, 2026 on PithPith's reviewarXiv
  6. [6] Shuyang Liu, Saman Dehghan, Jatin Ganhotra, Martin Hirzel, and Reyhaneh Jabbarvand. Evaluating plan compliance in autonomous programming agents, 2026 on Pith
  7. [7] Mike Merrill, Alexander Shaw, Nicholas Carlini, Boxuan Li, Harsh Raj, Ivan Bercovich, Lin Shi, Jeong Shin, Thomas Walshe, E Kelly Buchanan, et al. Terminal-bench: Benchmarking agents on hard, realistic tasks in command line interfaces. In International Conference on Learning R... on Pith
  8. [8] Jialin Wang and Zhihua Duan. Agent ai with langgraph: A modular framework for enhancing machine translation using large language models, 2024 on Pith
  9. [9] Yiran Wu, Tianwei Yue, Shaokun Zhang, Chi Wang, and Qingyun Wu. Stateflow: Enhancing llm task-solving through state-driven workflows, 2024 on Pith
  10. [10] Tianshi Xu, Huifeng Wen, and Meng Li. Adapting the interface, not the model: Runtime harness adaptation for deterministic llm agents. arXiv preprint arXiv:2605.22166, 2026 on PithPith's reviewarXiv
  11. [11] Chenyang Yang, Xinran Zhao, Tongshuang Wu, and Christian Kästner. Better harnesses, smaller models: Building 90\ on Pith
  12. [12] Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. -bench: A benchmark for tool-agent-user interaction in real-world domains, 2024 on Pith
  13. [13] Hangfan Zhang, Shao Zhang, Kangcong Li, Chen Zhang, Yang Chen, Yiqun Zhang, Lei Bai, and Shuyue Hu. Self-harness: Harnesses that improve themselves. arXiv preprint arXiv:2606.09498, 2026 on PithPith's reviewarXiv
  14. [14] Jianing Zhu, Yeonju Ro, John Robertson, Kevin Wang, Junbo Li, Haris Vikalo, Aditya Akella, and Zhangyang Wang. Your agents are aging too: Agent lifespan engineering for deployed systems. arXiv preprint arXiv:2605.26302, 2026 on PithPith's reviewarXiv
Cited by 3 Pith papers
How the literature uses it

What this paper does with the arXiv papers it cites

Each citing sentence is taken from the citing paper's LaTeX source and typed by Pith's classifier. A label appears at probability 0.8 or more (0.95 for support), a contest at 0.7 or more; the quoted words are the citing authors'. How each label was checked · JSON

What it cites · 5 sentences without a confident label

Cites as background 3 papers

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment