Pith. sign in

REVIEW 3 major objections 5 minor 38 references

The paper claims that editing an agent's runtime harness can be trained as an online reinforcement-learning skill, lifting a frozen 9B agent's average success by 9.3 percentage points across three interactive benchmarks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A dedicated harness-editor policy trained with RL on the realized outcomes of executable patches raises frozen-agent success by 9.3 points across WebShop, ALFWorld, and DBBench.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection New and largely well-executed: the editor is post-trained by RL on realized patch outcomes, but the headline gains are same-batch, so the strong co-evolution claim needs prospective evidence. the 3 major comments →

arxiv 2608.02276 v1 pith:UUA3NBO2 submitted 2026-08-03 cs.AI

Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectories

Shuai Shao , Kangning Zhang , Qingyao Li , Shijian Wang , Hao Wang , Wenxiang Jiao , Yuan Lu , Yi Guo
show 2 more authors
Weiwen Liu Weinan Zhang
This is my paper
classification cs.AI
keywords agent harnessreinforcement learningGRPOfailure-conditioned editingruntime patchesself-evolving agentstool-using agentslifecycle hooks
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

An LLM-based agent's behavior is usually fixed after deployment even though it keeps accumulating interaction trajectories. Harness-R1 argues that those trajectories—especially failures—can be used to improve the executable harness that surrounds the model: the code that builds context, mediates tools, validates actions, and handles recovery. The paper post-trains a separate 9B 'harness engineer' with online reinforcement learning, using the realized success change of its executable patches as the reward, so edits are chosen for the task outcomes they actually produce rather than for looking plausible. On three interactive benchmarks, the trained engineer raises a frozen target agent's average success from 44.3% to 53.6%, and a target-specific engineer adds a further 5.0 points after the agent itself has been fine-tuned, pointing toward co-evolution of agent and harness. A reader should care because it converts harness design from hand-authored scaffolding into a learnable, outcome-grounded capability that transfers to unseen tasks and target models.

Core claim

The core claim: failure-conditioned, lifecycle-wide editing of an executable runtime can be learned, and outcome-grounded post-training of a dedicated editor is what makes it reliable. The engineer reads mined failure packets, emits executable hooks at four lifecycle points, and is rewarded by the full-batch reward difference after the frozen target reruns the same tasks. Cold-start SFT initializes the policy; GRPO refines it online. Results: frozen vanilla target rises from 44.3% to 53.6% average success; the editor improves twenty unseen target models and 1,270 held-out tasks from only ten failures, while larger prompted frontier editors are unstable under the same protocol.

What carries the argument

The central machinery is the harness engineer Hθ, a separate 9B policy that reads a failure packet and emits an executable overlay of code hooks. The overlay wraps the frozen target at four lifecycle points—episode init, pre-decision, pre-action mediation, post-feedback recovery—and is validated before installation. Reward is the transductive full-batch difference Δ_B(P) = mean(R^P − R^0) after the frozen target reruns the same tasks; invalid or inert patches get zero. GRPO normalizes rewards across groups of eight sampled patches and updates only the engineer, after cold-start SFT supplies the initial editing prior. This grounds harness editing in realized task outcomes rather than plausibi

Load-bearing premise

The load-bearing premise is that rewarding the engineer with success changes on the very batch of tasks whose failures were mined teaches a policy that generalizes; if those failure packets are unrepresentative of the broader failure distribution, or if the engineer overfits the batch's specific failure modes, the reported gains could be a training artifact rather than a general editing capability.

What would settle it

Run the trained engineer on a batch of tasks drawn from a distribution with novel failure modes not present in the training batches—for example, tasks that require tool sequences or recovery patterns the engineer never saw. If patches systematically fail to beat the no-patch baseline, or cause regressions on multiple seeds under a controlled same-batch rerun, the central claim that failure-conditioned editing generalizes would be falsified. More directly, repeat the paper's ten-failure held-out protocol with failure packets sampled from disjoint task families and check whether positive gains o

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A deployed agent can improve its own effectiveness without any weight update, by learning to edit the runtime that surrounds it; harness improvement becomes optimizable rather than hand-designed.
  • The harness engineer and the target agent can co-evolve: gains persist and even grow after direct fine-tuning of the target, implying alternating agent/harness updates is a viable route to continual improvement.
  • Sparse failure evidence can generalize: ten failures can yield a patch that improves 1,270 held-out tasks, so failure logging alone can drive broad harness fixes.
  • The editing policy transfers across model families and scales without per-target retraining, making a single trained engineer a reusable asset for many agents.
  • Outcome grounding beats model scale for this task: a 9B engineer trained on realized rerun outcomes outperforms much larger prompted frontier editors, whose edits are unstable under the same held-out protocol.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the training reward is transductive (same batch used for mining and evaluation), the headline gains could partly reflect overfitting to the observed failure modes; a natural extension is to add held-out regression or distribution-shift penalties, and the paper itself flags this as future work.
  • The four-hook interface effectively defines a small API for runtime intervention; if standardized across environments, patches could become portable artifacts, although the paper's experiments only show policy-level transfer, not patch reuse.
  • The one-round co-evolution result suggests a possible compounding loop—fine-tune the target, retrain the engineer, repeat—but convergence behavior over multiple rounds is untested and could oscillate or saturate.
  • Fixed strategies such as self-refinement and reflection are shown to be unreliable or harmful for some agents; this suggests that outcome-grounded adaptation should become a default component of agent deployment pipelines rather than a research nicety.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Harness-R1, a method that post-trains a dedicated 9B "harness engineer" to edit an existing executable agent harness from batches of target-agent failure trajectories. The engineer is initialized by supervised fine-tuning on teacher-proposed, validated patches and then trained with online GRPO using as reward the full-batch success difference on the same task batch whose failures were shown to the engineer; the target agent's weights remain frozen. Evaluations on WebShop, ALFWorld, and DBBench report an average success increase from 44.3% to 53.6% for a vanilla Qwen3.5-9B target, a further 5.0-point gain after direct target-agent SFT, transfer across 21 target models with a 7.06-point average gain on 20 unseen targets, and +8.9 points on 1,270 held-out tasks from a sparse 10-failure protocol. The paper also includes a lifecycle-position ablation and qualitative case studies.

Significance. If the empirical claims hold, Harness-R1 is a meaningful step beyond fixed prompt-based harness editors and beyond proposer-search methods that do not update the editor policy from realized outcomes. The paper has real strengths: task-level SFT/RL/test splits are disjoint; the target agent is frozen during engineer training; code and models are released; the held-out-task protocol is a genuinely prospective check; and the lifecycle ablation isolates where interventions matter. The authors also state their main limitation explicitly in §5, which is commendable. The contribution is incremental but relevant to self-evolving-agent research, and the experimental design is considerably more careful than is typical in this area.

major comments (3)
  1. [§3.1 Eq. (1), §4.2 Table 1, §4.4] The main evaluation is transductive: r(B,P)=Δ_B(P) rewards the engineer for improving exactly the batch B whose failures are packaged into s_B, and the Table 1 rows are measured under this same protocol. Nothing in the reward, and only a prompt-level instruction in Appendix A, prevents a patch from keying on predicates that match the observed failure modes. The §5 limitation acknowledges this, and §4.4 is the only prospective check. However, §4.4 uses only 10 failures per benchmark, one patch per benchmark, and the vanilla target; it does not exercise the target-specific engineer behind the +5.0 co-evolution result, nor does it test the full-batch protocol that defines Table 1. Please add a prospective evaluation with full-size failure packets drawn from one partition and success measured on a disjoint partition, for both the vanilla and target-specific settings, or explicitly qualify th
  2. [Table 1, Fig. 3, Table E.1] The core point estimates — +9.3 pp on the vanilla target, +5.0 pp after target SFT, and the 21-target transfer matrix — are based on single evaluation runs with no confidence intervals, significance tests, or multiple seeds. Some cross-target deltas are close to zero or negative in individual benchmark columns (e.g., Qwen2.5-72B on ALFWorld, −2.0), so the claim that the policy improves every target depends on small positive averages. At minimum, report standard errors or bootstrap confidence intervals for the key rows and repeat the target-specific engineer evaluation over at least three seeds.
  3. [§4.2 Table 1, §4.1, Appendix B] The 'Agent SFT + Harness-R1' row is presented as evidence of co-evolution, but the manuscript does not specify how the target-specific engineer was trained: are its failure packets generated by the SFT target, does it reuse the same SFT/RL splits, how many failure packets and GRPO updates were used, and are its hyperparameters the same as in Appendix B? Without this information, the +5.0 result cannot be reproduced and the co-evolution claim is not fully verifiable. Please add an appendix subsection describing the target-specific training protocol and data counts.
minor comments (5)
  1. [Figure 1] The caption says 'matched-baseline reward changes' but the prompt settings, model versions, and exact edit protocol for the frontier editors in Figure 1 are not described. Please add a short description or reference to Appendix A/B for these baselines.
  2. [Eq. (4)] The notation ℓ^{tr}_{k,t} and ℓ^{ro}_{k,t} appears in the text immediately after Eq. (4); define these symbols before or directly within the equation display to avoid ambiguity.
  3. [Table 1 and §4.1] The Reflection row is correctly identified as a success@2 protocol, but the table would be clearer if it were visually separated from the success@1 rows or labeled with a dagger referenced in the caption, since the prose explicitly says it 'is not ranked against single-episode methods.'
  4. [Appendix D] In Table D.1, the 'Train total' column is the sum of SFT train and RL train but excludes validation and test. Please label it 'SFT+RL train' to avoid implying it includes all data.
  5. [Appendix F] The held-out protocol uses three 'matched evidence seeds'; specify how the seeds define the 10-failure packets and confirm that the same 10 failures are shown to all engineers in a seed. This would strengthen the comparison with the frontier baselines.

Circularity Check

0 steps flagged

No significant circularity; the same-batch reward is explicitly acknowledged and a held-out generalization check is provided.

full rationale

The learning loop is deliberately transductive: Eq. (1) defines the engineer reward as the full-batch performance difference Δ_B(P) on the same task batch B whose failures are compacted into s_B. The paper says this "defines a same-batch, transductive objective" and, in Limitations, that it "ties the signal to the tasks used to mine failures." The main Table 1 gains (44.3%→53.6%, and 59.2%→64.2%) are measured under the same transductive protocol: the engineer reads the target's failures on the benchmark task set and success is then reported on that same task set (Appendix E: "every delta computed on a matched target-specific task set"). This means the headline deltas are within-distribution repairs, not held-out predictions, and the paper could have been clearer that the headline is a transductive evaluation. However, this is not a construction-equivalence or fitted-parameter circularity: the test-batch deltas were not used to update the engineer weights; the policy was trained on disjoint task splits (SFT/RL train vs. test, Appendix D), and the patch must still be executable and behaviorally effective to improve the rerun. The paper also provides a genuine held-out check (§4.4, Fig. 4a): 10 failures per benchmark produce one patch per benchmark that improves 1,270 unseen tasks by +8.9±1.5 pp, which is a prospective result independent of the training reward. No load-bearing self-citation is present: GRPO is cited to DeepSeekMath (external), and related-work self-citations are not used to justify the method's validity. There is no uniqueness theorem, no ansatz smuggled via citation, and no renaming of a known result. The sole caveat is that the target-specific co-evolution gain (+5.0 pp) is not covered by the held-out protocol, but that is an evidence-strength issue, not circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 5 axioms · 0 invented entities

No fitted numerical constants appear in the method; the central claim is empirical and rests on benchmark settings, the four-hook edit space, and the same-batch transfer assumption. Training hyperparameters (K=8, clip bounds, LR) are standard design choices rather than parameters fitted to the headline result.

axioms (5)
  • domain assumption The full-batch performance difference Δ_B(P) on the same tasks used to mine failures is a reliable learning signal for improving harness quality on unseen tasks.
    §3.1, Eq. (1) defines the reward transductively; §4.4 provides held-out evidence, but the assumption is load-bearing for the RL training loop.
  • domain assumption The four lifecycle hooks (episode init, pre-decision, pre-action, post-feedback) span the useful editable surface of the runtime; edits outside them are not learnable by this method.
    §3.1 and Appendix C define the patch interface; if a useful intervention requires changing tool definitions or retrieval internals, the method cannot express it.
  • domain assumption Target-agent rollouts at temperature 0 are stable enough that reward deltas reflect patch effects rather than stochasticity.
    Appendix B.4 sets target temperature to 0, but environments may be stochastic; Table 1 reports single reruns with no error bars.
  • standard math GRPO with K=8 and truncated importance weights yields a valid optimization signal for the editing policy.
    §3.2 Eqs. (3)–(4) follow Shao et al. (2024); treated as standard background.
  • domain assumption The benchmark test splits are independent of the training and validation task splits.
    Appendix D details the splits; no evidence of leakage, and central to avoiding circular evaluation.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectories." pith.science (2026). https://pith.science/paper/UUA3NBO2

@misc{pith2026260802276,
  author       = {Pith},
  title        = {Pith review of: Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectories},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UUA3NBO2}},
  note         = {Machine review of arXiv:2608.02276}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Agents built around large language models continually accumulate interaction trajectories during deployment, yet their behavior typically remains fixed. Beyond updating model weights, these trajectories can improve the agent harness that constructs context, mediates tools, validates actions, and recovers execution. We introduce Harness-R1, the first method, to our knowledge, that makes failure-conditioned, lifecycle-wide editing of an existing executable runtime a learned capability. It post-trains a dedicated harness engineer with online reinforcement learning so that its edits are optimized for the realized task success they produce, rather than proposed by a fixed editor. A separate 9B engineer converts batches of target-agent failures into validated executable patches; fresh same-batch reruns of the frozen target provide outcome rewards, so training updates only the engineer. Cold-start supervised fine-tuning initializes this editing policy, which is then trained online with group-relative policy optimization. Across WebShop, ALFWorld, and DBBench, Harness-R1 raises vanilla Qwen3.5-9B success from 44.3% to 53.6% (+9.3 percentage points). After direct target-agent fine-tuning, a target-specific engineer raises the average further from 59.2% to 64.2% (+5.0 points); because these gains hold both before and after fine-tuning the target, Harness-R1 points toward co-evolving the harness engineer and the target agent.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

38 extracted references · 31 linked inside Pith

  1. [3]

    URL https://arxiv.org/abs/2507

    doi: 10.48550/arXiv.2507.21046. URL https://arxiv.org/abs/2507. 21046. Google DeepMind. Gemini 3.5 Flash: Model card. https://deepmind.google/models/model-cards/gemini-3-5-flash/, May

    Pith/arXiv arXiv doi:10.48550/arxiv.2507.21046

  2. [4]

    Qingyan Guo, Rui Wang, Junliang Guo, Bei Li, Kaitao Song, Xu Tan, Guoqing Liu, Jiang Bian, and Yujiu Yang

    Accessed: 2026-07-28. Qingyan Guo, Rui Wang, Junliang Guo, Bei Li, Kaitao Song, Xu Tan, Guoqing Liu, Jiang Bian, and Yujiu Yang. Evoprompt: Connecting llms with evolutionary algorithms yields powerful prompt optimizers,

    2026

  3. [6]

    Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T

    URL https://arxiv.org/abs/2605.09998. Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. Dspy: Compiling declarative language model calls into self-improving pipelines,

    Pith/arXiv arXiv

  4. [7]

    Yoonho Lee, Roshen Nair, Qizheng Zhang, Kangwook Lee, Omar Khattab, and Chelsea Finn

    URL https://arxiv.org/abs/2310.03714. Yoonho Lee, Roshen Nair, Qizheng Zhang, Kangwook Lee, Omar Khattab, and Chelsea Finn. Meta-harness: End-to-end optimization of model harnesses,

    Pith/arXiv arXiv

  5. [8]

    Xiaoxi Li, Wenxiang Jiao, Jiarui Jin, Guanting Dong, Jiajie Jin, Yinuo Wang, Hao Wang, Yutao Zhu, Ji-Rong Wen, Yuan Lu, and Zhicheng Dou

    URL https://arxiv.org/abs/2603.28052. Xiaoxi Li, Wenxiang Jiao, Jiarui Jin, Guanting Dong, Jiajie Jin, Yinuo Wang, Hao Wang, Yutao Zhu, Ji-Rong Wen, Yuan Lu, and Zhicheng Dou. Deepagent: A general reasoning agent with scalable toolsets, 2026a. URL https://arxiv.org/abs/2510.21618. Yanzhou Li, Yiran Zhang, Xiaoyu Zhang, Xiaoxia Liu, and Yang Liu. Codeskill...

    Pith/arXiv arXiv

  6. [10]

    Jiahang Lin, Shichun Liu, Chengjun Pan, Lizhi Lin, Shihan Dou, Zhiheng Xi, Xuanjing Huang, Hang Yan, Zhenhua Han, Tao Gui, and Yu-Gang Jiang

    URL https://arxiv.org/abs/2305.19118. Jiahang Lin, Shichun Liu, Chengjun Pan, Lizhi Lin, Shihan Dou, Zhiheng Xi, Xuanjing Huang, Hang Yan, Zhenhua Han, Tao Gui, and Yu-Gang Jiang. Agentic harness engineering: Observability-driven automatic evolution of coding-agent harnesses,

    Pith/arXiv arXiv

  7. [11]

    URL https://arxiv.org/abs/2604.25850. Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, and Jie Tang. Agentbench: Evaluating llms as agents,

    Pith/arXiv arXiv

  8. [12]

    Xinghua Lou, Miguel Lázaro-Gredilla, Antoine Dedieu, Carter Wendelken, Wolfgang Lehrach, and Kevin P

    URL https://arxiv.org/abs/2308.03688. Xinghua Lou, Miguel Lázaro-Gredilla, Antoine Dedieu, Carter Wendelken, Wolfgang Lehrach, and Kevin P. Murphy. Autoharness: improving llm agents by automatically synthesizing a code harness,

    Pith/arXiv arXiv

  9. [13]

    Zhengxi Lu, Zhiyuan Yao, Jinyang Wu, Chengcheng Han, Qi Gu, Xunliang Cai, Weiming Lu, Jun Xiao, Yueting Zhuang, and Yongliang Shen

    URL https://arxiv.org/abs/2603.03329. Zhengxi Lu, Zhiyuan Yao, Jinyang Wu, Chengcheng Han, Qi Gu, Xunliang Cai, Weiming Lu, Jun Xiao, Yueting Zhuang, and Yongliang Shen. Skill0: In-context agentic reinforcement learning for skill internalization,

    arXiv

  10. [14]

    Haochen Luo, Yi Huang, Sichun Luo, Fengyuan Liu, Lei Li, Zefa Hu, Junlan Feng, and Qi Liu

    URL https://arxiv.org/abs/2604.02268. Haochen Luo, Yi Huang, Sichun Luo, Fengyuan Liu, Lei Li, Zefa Hu, Junlan Feng, and Qi Liu. Harness-aware self-evolving: Co-evolving model weights, harness, and task solutions,

    Pith/arXiv arXiv

  11. [15]

    URL https://arxiv.org/abs/2607.03935. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhu- moye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterative refinement with self-feedback,

    Pith/arXiv arXiv

  12. [16]

    Moonshot AI

    URL https://arxiv.org/abs/2303.17651. Moonshot AI. Kimi K2.6: Advancing open-source coding. https://www.kimi.com/blog/kimi-k2-6, April

    Pith/arXiv arXiv

  13. [17]

    Fan Nie, Lan Feng, Haotian Ye, Weixin Liang, Pan Lu, Huaxiu Yao, Alexandre Alahi, and James Zou

    Accessed: 2026-07-28. Fan Nie, Lan Feng, Haotian Ye, Weixin Liang, Pan Lu, Huaxiu Yao, Alexandre Alahi, and James Zou. Weak-for-strong: Training weak meta-agent to harness strong executors,

    2026

  14. [18]

    URL https://arxiv.org/abs/2504.04785. OpenAI. Introducing GPT-5.5. https://openai.com/index/introducing-gpt-5-5/, April

    Pith/arXiv arXiv

  15. [19]

    Krista Opsahl-Ong, Michael J Ryan, Josh Purtell, David Broman, Christopher Potts, Matei Zaharia, and Omar Khattab

    Accessed: 2026-07-28. Krista Opsahl-Ong, Michael J Ryan, Josh Purtell, David Broman, Christopher Potts, Matei Zaharia, and Omar Khattab. Optimizing instructions and demonstrations for multi-stage language model programs,

    2026

  16. [20]

    gradient descent

    URL https://arxiv.org/abs/2406.11695. Reid Pryzant, Dan Iter, Jerry Li, Yin Tat Lee, Chenguang Zhu, and Michael Zeng. Automatic prompt optimization with "gradient descent" and beam search,

    Pith/arXiv arXiv

  17. [21]

    Qwen Team

    URL https://arxiv.org/abs/2305.03495. Qwen Team. Qwen3.5: Towards native multimodal agents. https://qwen.ai/blog?id=qwen3.5, February

    Pith/arXiv arXiv

  18. [22]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y

    Accessed: 2026-07-28. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models,

    2026

  19. [23]

    Yaorui Shi, Yuxin Chen, Zhengxi Lu, Yuchun Miao, Shugui Liu, Qi GU, Xunliang Cai, Xiang Wang, and An Zhang

    URL https: //arxiv.org/abs/2402.03300. Yaorui Shi, Yuxin Chen, Zhengxi Lu, Yuchun Miao, Shugui Liu, Qi GU, Xunliang Cai, Xiang Wang, and An Zhang. Skill1: Unified evolution of skill-augmented agents via reinforcement learning,

    Pith/arXiv arXiv

  20. [24]

    Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao

    URL https://arxiv.org/abs/2605.06130. Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning,

    Pith/arXiv arXiv

  21. [25]

    Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht

    URL https://arxiv.org/abs/2303.11366. Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. Alfworld: Aligning text and embodied environments for interactive learning,

    Pith/arXiv arXiv

  22. [27]

    Xiaoxuan Wang, Haixin Wang, Alexander Taylor, Jason Cong, Yizhou Sun, and Wei Wang

    URL https://arxiv.org/abs/2605.09359. Xiaoxuan Wang, Haixin Wang, Alexander Taylor, Jason Cong, Yizhou Sun, and Wei Wang. Harnessbridge: Learnable bidirectional controller for llm agent harness,

    Pith/arXiv arXiv

  23. [28]

    Rong Wu, Xiaoman Wang, Jianbiao Mei, Pinlong Cai, Daocheng Fu, Cheng Yang, Licheng Wen, Xuemeng Yang, Yufan Shen, Yuxin Wang, and Botian Shi

    URL https://arxiv.org/abs/2606.12882. Rong Wu, Xiaoman Wang, Jianbiao Mei, Pinlong Cai, Daocheng Fu, Cheng Yang, Licheng Wen, Xuemeng Yang, Yufan Shen, Yuxin Wang, and Botian Shi. EvolveR: Self-evolving LLM agents through an experience-driven lifecycle. InInternational Conference on Machine Learning,

    Pith/arXiv arXiv

  24. [29]

    URL https://arxiv.org/abs/2510.16079

    doi: 10.48550/arXiv.2510.16079. URL https://arxiv.org/abs/2510.16079. Peng Xia, Jianwen Chen, Hanyang Wang, Jiaqi Liu, Kaide Zeng, Yu Wang, Siwei Han, Yiyang Zhou, Xujiang Zhao, Haifeng Chen, Zeyu Zheng, Cihang Xie, and Huaxiu Yao. Skillrl: Evolving agents via recursive skill-augmented reinforcement learning,

    Pith/arXiv arXiv doi:10.48550/arxiv.2510.16079

  25. [30]

    Tianshi Xu, Huifeng Wen, and Meng Li

    URL https://arxiv.org/abs/2602.08234. Tianshi Xu, Huifeng Wen, and Meng Li. Adapting the interface, not the model: Runtime harness adaptation for deterministic llm agents,

    Pith/arXiv arXiv

  26. [31]

    11 Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V

    URL https://arxiv.org/abs/2605.22166. 11 Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, and Xinyun Chen. Large language models as optimizers,

    Pith/arXiv arXiv

  27. [32]

    Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan

    URL https://arxiv.org/abs/2309.03409. Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. Webshop: Towards scalable real-world web interaction with grounded language agents, 2023a. URL https://arxiv.org/abs/2207.01206. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and actin...

    Pith/arXiv arXiv

  28. [33]

    Hongzhuo Yu, Fei Zhu, Guo-Sen Xie, and Ling Shao

    URL https: //arxiv.org/abs/2607.05458. Hongzhuo Yu, Fei Zhu, Guo-Sen Xie, and Ling Shao. Self-consolidation for self-evolving agents,

    Pith/arXiv arXiv

  29. [34]

    Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Pan Lu, Zhi Huang, Carlos Guestrin, and James Zou

    URL https://arxiv.org/abs/ 2602.01966. Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Pan Lu, Zhi Huang, Carlos Guestrin, and James Zou. Optimizing generative ai by backpropagating language model feedback.Nature, 639(8055):609–616,

    arXiv

  30. [35]

    Hangfan Zhang, Shao Zhang, Kangcong Li, Chen Zhang, Yang Chen, Yiqun Zhang, Lei Bai, and Shuyue Hu

    Accessed: 2026-07-28. Hangfan Zhang, Shao Zhang, Kangcong Li, Chen Zhang, Yang Chen, Yiqun Zhang, Lei Bai, and Shuyue Hu. Self-harness: Harnesses that improve themselves, 2026a. URL https://arxiv.org/abs/2606.09498. Kangning Zhang, Wenxiang Jiao, Kounianhua Du, Yuan Lu, Weiwen Liu, Weinan Zhang, and Yong Yu. Looptool: Closing the data-training loop for ro...

    Pith/arXiv arXiv 2026

  31. [36]

    Mingda Zhang, Wenjin Liu, Tiesunlong Shen, Qika Lin, Rui Mao, Erik Cambria, Xiaoying Tang, and Haoran Luo

    URL https://arxiv.org/abs/2511.09148. Mingda Zhang, Wenjin Liu, Tiesunlong Shen, Qika Lin, Rui Mao, Erik Cambria, Xiaoying Tang, and Haoran Luo. Flowsteer: Towards agents designing agentic workflows via reinforced progressive canvas editing, 2026b. URL https://arxiv.org/abs/2602.01664. Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and...

    arXiv

  32. [37]

    Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba

    URL https://arxiv.org/abs/2308.10144. Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers,

    Pith/arXiv arXiv

  33. [38]

    <BENCHMARK_ID>

    URL https://arxiv.org/abs/2211.01910. 12 Appendix A Prompts This section presents the model-facing prompt template used to train and evaluate the harness engineer. Following the presentation style of prompt appendices, fixed instructions are shown separately from the per-example input. Angle-bracketed strings denote substituted fields rather than literal ...

    Pith/arXiv arXiv

  34. [2021]

    Yash Vishe, Rohan Surana, Xunyi Jiang, Zihan Huang, Xintong Li, Nikki Lijing Kuang, Tong Yu, Ryan A

    URL https://arxiv.org/abs/2010.03768. Yash Vishe, Rohan Surana, Xunyi Jiang, Zihan Huang, Xintong Li, Nikki Lijing Kuang, Tong Yu, Ryan A. Rossi, Jingbo Shang, Julian McAuley, and Junda Wu. Skill-r1: Agent skill evolution via reinforcement learning,

    Pith/arXiv arXiv 2010

  35. [2023]

    Huan-ang Gao et al

    URL https://arxiv.org/abs/2309.16797. Huan-ang Gao et al. A survey of self-evolving agents: What, when, how, and where to evolve on the path to artificial super intelligence.Transactions on Machine Learning Research,

    Pith/arXiv arXiv

  36. [2024]

    10 Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu

    URL https://arxiv.org/abs/2407.12821. 10 Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. Encouraging divergent thinking in large language models through multi-agent debate,

    Pith/arXiv arXiv

  37. [2025]

    Seth Karten, Joel Zhang, Tersoo Upaa Jr, Ruirong Feng, Wenzhe Li, Chengshuai Shi, Chi Jin, and Kiran Vodrahalli

    URL https://arxiv.org/abs/2309.08532. Seth Karten, Joel Zhang, Tersoo Upaa Jr, Ruirong Feng, Wenzhe Li, Chengshuai Shi, Chi Jin, and Kiran Vodrahalli. Continual harness: Online adaptation for self-improving foundation agents,

    Pith/arXiv arXiv

  38. [2026]

    URL https://arxiv.org/abs/2507. 19457. Mengzhuo Chen, Junjie Wang, Zhe Liu, Yawen Wang, Haiming Zheng, and Qing Wang. From failed trajectories to reliable llm agents: Diagnosing and repairing harness flaws, 2026a. URL https://arxiv.org/abs/2606.06324. Tingyang Chen, Shuo Lu, Kang Zhao, Weicheng Meng, Hanlin Teng, Tianhao Li, Chao Li, Xule Liu, Jian Liang,...

    Pith/arXiv arXiv

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.