Artificial Intelligence · arXiv:2610.11655 v1 · submitted 8 Oct 2026 · 29 references
Harness Evolution Hits a Ceiling: When Weight Training Should Begin
The paper says
“Improving a long-horizon LLM agent means evolving the harness around a frozen model or training its weights.”
more of the abstract
We let a self-evolving harness make the system stronger first, then cross seed and evolved harnesses with base and trained weights to learn which gains the trained model keeps and which still need the runtime. We show that the right lever can be read off the agent's failure composition: labelling failed trajectories by the first signal that fires separates process failures (blocked calls, loops, exhausted step budgets) from content failures (a delivered plan that is poor). Harness evolution repairs the former, the behaviour it instils can be trained into the weights, and content failures are what weight training is for. On DeepPlanning, a self-evolving harness loop lifts the held-out score of Qwen3.5-4B from 0.16 to 0.30 and of Qwen3.5-9B from 0.32 to 0.44; for 4B, held-out delivery rises from 55% to 90% while content failures are left for the weights. LoRA adapters trained on evolved-harness trajectories internalise the gain: under the original harness they add +0.13 on held-out tasks for both sizes; on 4B they stack with the harness to more than double the held-out score, and on 9B the adapter alone matches the full evolution line, cutting content failures from a quarter of trajectories to one in twenty. A placebo adapter trained on answer-shuffled trajectories falls below the base model. The loop transfers to WebArena-Lite (+0.09 on 117 unseen tasks), where the gain lives in what the model sees and adapters do not add to it. The result is a diagnose-then-intervene rule applied twice: read the failure composition to choose between harness and weights, then read what the accepted edits changed to decide which gains to train in. Scores are four-rollout means against fresh anchors, same-night except where marked, across eight models from six families and two benchmarks.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 10 Oct 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up. A carefully measured empirical paper that delivers a concrete, testable rule for when to evolve the harness vs. train the weights. The measurement rigour is a cut above most work in this area. This line is wrong
3 major objections · 8 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 29 references say.
The paper says, at Appendix I
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.73):
“All numbers in this paper use the default converter, whose repeat agreement with itself is 93.5% at the default temperature (96.8% at temperature 0), so every cell is converted the same way; because the bias depends on the multi-draft rate, which differs across configurations (Appendix F), we report that rate and read differences near 0.025 with care. 20 Preprint.”
Pith says: Objection 1 of 3 · major
The plan-to-JSON conversion bias (up to 0.025, varying with multi-draft rate) could affect quantitative comparisons, especially those near the 0.035 noise line. The authors acknowledge the bias and note that the converter is held constant, but they do not bound the potential differential impact across configurations.
more of objection 1
For example, the bare 4B model has a higher multi-draft rate (8.9%) than the evolved harness (1.0%), so the seed harness scores may be more inflated, making the measured gain conservative. A sensitivity analysis or explicit correction would strengthen the quantitative claims.
The paper says, at Section 3.1 / Section 7
The review did not quote this passage.
Pith says: Objection 2 of 3 · major
The failure taxonomy places F1 (evidence) as the first signal, and the Qwen3.6 thinking-on line shows harness evolution reduces F1 (Section H). However, the diagnostic rule in Section 7 does not mention F1; it directs to evolve the harness for process failures (F2, F3) and train weights for content failures (F4).
more of objection 2
The paper should clarify whether F1 is considered a process failure that the harness can address, or whether the rule should be extended to account for cases where F1 is dominant. The current presentation is ambiguous about the role of F1 in the decision tree.
The paper says, at Section 7
The review did not quote this passage.
Pith says: Objection 3 of 3 · major
The rule includes the threshold 'Delivered plans score below about 0.18: the harness lever paid nothing on the three such models.' This threshold is derived from only three models (MiniCPM5-2B, Spark-X2.5-4B, Ministral-3-8B), all small and non-Qwen. While stated as an observation, the rule is presented as a general guideline.
more of objection 3
A stronger qualification (e.g., 'in our experiments models with plan quality below 0.18 did not benefit') or a test on additional low-quality models would improve robustness. As written, the threshold may overclaim generality.
8 minor objections
The paper says, at Section 3.1
Pith says: minor objection 1 of 8
The term 'unbacked-citation ratio' is used without explicit definition. It should be defined, e.g., 'the fraction of factual claims in the plan that are not supported by a retrieved source.'
The paper says, at Table 1 / Appendix E
Pith says: minor objection 2 of 8
The caption for Table 1 states that the base-model cells under the evolved harness on held-out tasks were 'closing anchors from two nights earlier (4B: 2 rollouts).' This cross-night comparison violates the same-night principle and should be more prominently flagged in the main text, not only in the table note and Limitations.
show 6 more
The paper says, at Section 3.2
Pith says: minor objection 3 of 8
Proposition 1 bounds selection optimism, but the proof assumes each candidate reading is a single rollout (m=1). In the paper's actual loop, each candidate is evaluated with 4 rollouts. The text later adjusts s=σ/√4 for that case, but the proposition statement uses X_k as single-rollout readings. Clarify that the bound applies to per-reading noise and scale s accordingly for the multi-rollout case.
The paper says, at Appendix I
Pith says: minor objection 4 of 8
The sentence 'Eleven further converters tested on the five hardest answers against 23 manually established checkpoints did no better than 13 of 23, against 7-9 for the default' is slightly ambiguous. It would be clearer to state that the best alternative converter scored 13/23, and the default scored 7-9/23 on the same set.
The paper says, at Table 9 / Figure 6
Pith says: minor objection 5 of 8
The 4B LoRA qkvo+MLP adapter shows a reduction in F4 on dev (5.7%) but not on held-out (8.3%, comparable to base 7.6%). This asymmetry is not discussed. It may indicate overfitting or noise; a brief comment would help interpretation.
The paper says, at Section 2.2
Pith says: minor objection 6 of 8
The reference to 'Zhang et al. (2026c)' appears as 'Zhang et al.'; the standard form is 'Zhang et al.' or 'Zhang et al. (2026c)' is fine, but ensure consistent use of 'et al.' throughout.
The paper says, at Section 3.1
Pith says: minor objection 7 of 8
In the definition of F2, 'a harness intervention itself blocked the agent (a forced checkpoint unlock, or ten or more blocked tool calls) and the score is low.' The threshold 'ten or more blocked tool calls' is arbitrary; justify or make it configurable.
The paper says, at Appendix H
Pith says: minor objection 8 of 8
The description of the Qwen3.6 thinking-on line says 'the thinking-off line's closing harness' but the sentence structure is slightly garbled: 'Qwen3.6-35B-A3B was run with thinking off from the seed harness and then with thinking on from the thinking-off line's closing harness' – rephrase for clarity.
What would settle it
To further strengthen generalizability, reproduce the experiment on a third benchmark such as AgentBench with a different seed harness, applying the same failure taxonomy and rule. If the failure class movements deviate from predictions, the taxonomy may require architecture-specific calibration.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 10 Oct 2026
- Reader holds up
- Skeptic agrees
- Editor wrote the verdict above
- Referee minor revision: 3 major objections, 8 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
This paper is worth your time. The core contribution is a diagnose-then-intervene rule that reads the failure composition (F1–F4) of a long-horizon LLM agent and tells you whether to evolve the harness or fine-tune the weights. That rule is new, and the evidence for it is built from the ground up: noise-floor characterization, fresh anchors after every accept, same-night comparisons, placebo controls for weight training, and held-out validation. The paper also shows *why* a self-evolving harness stops at content failures (F4) – because the evidence package the proposer sees does not surface the scorer's failure composition – and demonstrates that the harness gain can be internalised by the weights when the gain changes *what the model does* but not when it changes *what the model sees* (e.g., extra context pages on WebArena-Lite). That last result is a clean empirical finding that gives practitioners a rule for deciding what to train in.
The soft spots are acknowledged by the authors and are proportional. The failure taxonomy requires process signals and a task-level scorer, which limits transfer to benchmarks that lack deterministic scoring. The conversion bias on DeepPlanning (about 0.025) is noted and all cells use the same converter, so comparisons hold, but absolute scores have a small systematic bump. A few cells cross nights (the held-out evolved-harness base for 4B, the WebArena-Lite adapter cells) and the authors flag them. The rule has been tested on two benchmarks and eight model families, but the strongest evidence comes from DeepPlanning; the WebArena-Lite case shows a different pattern (gain lives in what the model sees, and adapters don't add), which actually strengthens the rule by illustrating when it applies. There is no matched-budget test-time search baseline, but that's a limitation the authors state explicitly.
Who this is for: anyone building LLM agents and trying to decide which lever to pull will get a concrete measurement framework and a decision tree. The paper also sets a high bar for experimental hygiene in this field.
Recommendation: deserves a serious referee and should be accepted after addressing minor concerns about cross-night cells and perhaps expanding discussion of when the taxonomy does not apply.
The strongest objection
The paper's central claim is well-supported by controlled experiments, hold-out validation, and placebo controls. The failure taxonomy is deterministic and its mapping to the appropriate lever is validated by intervention. Limitations are explicitly stated and do not undermine the main conclusions. No critical flaw emerges from good-faith review.
I concur with the reader's assessment that the weakest assumption is the generality of the failure taxonomy. However, the paper provides initial transfer evidence (WebArena-Lite) and explicit limitations. The empirical chain is internally consistent and the measurements are careful. No single concern is load-bearing enough to alter the verdict. The rule is actionable and the paper's contributions stand.
The referee's report
If the claims hold, the paper supplies a principled, empirically grounded heuristic that tells developers which lever (harness vs. weights) to pull based on observable failure signals. The measurement protocol (noise floor, fresh anchors, same-night comparisons, placebo control) is a methodological contribution to the evaluation of agent interventions. The finding that gains from edits that change what the model writes can be trained into weights, while gains from edits that change what the model sees cannot, is practically actionable. The paper tests transfer across two benchmarks and multiple model families, lending generality to its rule.
Pith's reading of the paper (a machine's summary, not the authors')
This paper establishes a diagnostic rule for improving a long-horizon LLM agent: measure the failure composition of its trajectories, and if the failures are process problems (repeated calls, budget exhaustion, stuck steps) evolve the harness around the frozen model; if they are content problems (a delivered plan that is poor), fine-tune the weights. The rule is built on a four-category failure taxonomy (F1–F4) that is computed deterministically from process signals and the scorer, and it is tested by applying both levers on two model sizes and two benchmarks. A sympathetic reader would care because the rule saves expensive trial-and-error: it tells a developer which lever to pull before investing resources, and it also predicts whether a harness gain can later be transferred into the weights, which depends on whether the harness edit changed what the model does or what it sees.
What carries the argument. The paper's central machinery is a four-category failure taxonomy (F1–F4) that labels each trajectory by the first deterministic signal that fires: unbacked evidence (F1), harness friction blocking the agent (F2), execution control such as repeated calls or budget exhaustion (F3), or a delivered plan that scores below threshold with no process signal (F4). The taxonomy carries the argument because it separates failures that the harness can act on directly (F2, F3) from those that require weight training (F4), and this separation predicts the outcome of each lever.
Core claim. The central claim is that an agent's failure composition — specifically, whether failures are process-type (F2: harness friction, F3: execution control) or content-type (F4: planning capacity) — determines which improvement lever will work. Harness self-evolution removes process failures but leaves content failures untouched; weight training (via LoRA on evolved-harness trajectories) removes content failures. On Qwen3.5-4B and 9B, the paper shows that harness evolution lifts held-out scores by +0.133 and +0.119 respectively, and LoRA adapters trained on the resulting trajectories internalize the gain, adding +0.13 on held-out tasks under the original harness. On 4B the two levers stack; on 9B they substitute. The paper also shows that the loop's ceiling at F4 is due to its evidence package not surfacing the scorer's failure composition, and that when a harness gain comes from what the model sees (e.g., more context pages) rather than what it writes, it cannot be trained into the weights.
The premise it rests on. The load-bearing premise is that the F1–F4 taxonomy, computed deterministically from process signals and the scorer, correctly maps each failure to the lever that will fix it — specifically, that F2 and F3 are always fixable by the harness and F4 always requires weight training — and that this mapping holds across agent architectures, benchmarks, and model sizes beyond those tested.
If the paper is right (Pith's inferences, not the authors')
Agent developers can use the failure composition on a development set to decide whether to invest in harness evolution or weight fine-tuning first, avoiding wasted effort on the wrong lever.
The rule predicts that when delivery is saturated and F4 is dominant, weight training is the indicated next step; on Qwen3.5-9B, the adapter alone matched the full harness evolution line.
When a harness gain comes from edits that change what the model writes (tools, prompts, lessons), the gain can be trained into the weights; when it comes from what the model sees (context pages), it stays in the harness.
The paper's evidence that the loop stops at F4 because its evidence package does not surface the scorer's failure composition suggests that modifying the evidence package could unlock further gains.
The failure taxonomy and diagnose-then-intervene rule likely generalize to other long-horizon agent benchmarks with programmatic scoring, but the paper's test on only two benchmarks (DeepPlanning and WebArena-Lite) with one model family primarily leaves the generality unproven.
An immediate extension would be to surface the F4 signal (the scorer's per-check failure composition) into the evolution loop's evidence package and test whether the loop can then reduce F4 failures.
The size-dependent result (stacking on 4B, substitution on 9B) suggests that larger models may already have the capacity to internalize harness behaviours, while smaller models need both levers; this could inform when to use each approach for different-sized models.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Improving a long-horizon LLM agent means evolving the harness around a frozen model or training its weights. We let a self-evolving harness make the system stronger first, then cross seed and evolved harnesses with base and trained weights to learn which gains the trained model keeps and which still need the runtime. We show that the right lever can be read off the agent's failure composition: labelling failed trajectories by the first signal that fires separates process failures (blocked calls, loops, exhausted step budgets) from content failures (a delivered plan that is poor). Harness evolution repairs the former, the behaviour it instils can be trained into the weights, and content failures are what weight training is for. On DeepPlanning, a self-evolving harness loop lifts the held-out score of Qwen3.5-4B from 0.16 to 0.30 and of Qwen3.5-9B from 0.32 to 0.44; for 4B, held-out delivery rises from 55% to 90% while content failures are left for the weights. LoRA adapters trained on evolved-harness trajectories internalise the gain: under the original harness they add +0.13 on held-out tasks for both sizes; on 4B they stack with the harness to more than double the held-out score, and on 9B the adapter alone matches the full evolution line, cutting content failures from a quarter of trajectories to one in twenty. A placebo adapter trained on answer-shuffled trajectories falls below the base model. The loop transfers to WebArena-Lite (+0.09 on 117 unseen tasks), where the gain lives in what the model sees and adapters do not add to it. The result is a diagnose-then-intervene rule applied twice: read the failure composition to choose between harness and weights, then read what the accepted edits changed to decide which gains to train in. Scores are four-rollout means against fresh anchors, same-night except where marked, across eight models from six families and two benchmarks.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-09
The authors' main claim, as the classifier picked it out 0.72
“The result is a diagnose-then-intervene rule applied twice: read the failure composition to choose between harness and weights, then read what the accepted edits changed to decide which gains to train in.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.04 · states limitation 0.46 · human participants 0.02 · controlled comparison 0.96 · randomized assignment 0.03 · states quantitative result 0.99 · compares with prior results 0.90 · mentions code or data availability 0.04
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Improving a long-horizon LLM agent means evolving the harness around a frozen model or training its weights. | background 0.99 | 0.10 | 0.05 | asserted 0.96 |
| We let a self-evolving harness make the system stronger first, then cross seed and evolved harnesses with base and trained weights to learn which gains the trained model keeps and which still need the runtime. | method 0.87 | 0.92 | 0.08 | asserted 0.80 |
| We show that the right lever can be read off the agent's failure composition: labelling failed trajectories by the first signal that fires separates process failures (blocked calls, loops, exhausted step budgets) from content failures (a delivered plan that is poor). | method 0.85 | 0.90 | 0.12 | asserted 1.00 |
| Harness evolution repairs the former, the behaviour it instils can be trained into the weights, and content failures are what weight training is for. | interpretation 0.92 | 0.69 | 0.64 | asserted 1.00 |
| On DeepPlanning, a self-evolving harness loop lifts the held-out score of Qwen3.5-4B from 0.16 to 0.30 and of Qwen3.5-9B from 0.32 to 0.44; for 4B, held-out delivery rises from 55% to 90% while content failures are left for the weights. | result 1.00 | 0.96 | 0.13 | asserted 1.00 |
| LoRA adapters trained on evolved-harness trajectories internalise the gain: under the original harness they add +0.13 on held-out tasks for both sizes; on 4B they stack with the harness to more than double the held-out score, and on 9B the adapter alone matches the full evolution line, cutting content failures from a quarter of trajectories to one in twenty. | result 1.00 | 0.95 | 0.16 | asserted 1.00 |
| A placebo adapter trained on answer-shuffled trajectories falls below the base model. | result 1.00 | 0.92 | 0.03 | asserted 1.00 |
| The loop transfers to WebArena-Lite (+0.09 on 117 unseen tasks), where the gain lives in what the model sees and adapters do not add to it. | result 0.98 | 0.94 | 0.07 | asserted 1.00 |
| The result is a diagnose-then-intervene rule applied twice: read the failure composition to choose between harness and weights, then read what the accepted edits changed to decide which gains to train in. | method 0.52 | 0.88 | 0.70 | asserted 1.00 |
| Scores are four-rollout means against fresh anchors, same-night except where marked, across eight models from six families and two benchmarks. | method 1.00 | 0.90 | 0.03 | asserted 0.90 |
What kind of paper the classifier reports this is: method or tool 0.59 · evidence from observation or experiment 0.97
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-09: 1 request, 7,957 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (7)
Figures are the authors', reproduced from the paper.
References (29)
- [4] Darshan Deshpande, Varun Gangal, Hersh Mehta, Jitin Krishnan, Anand Kannappan, and Rebecca Qian. TRAIL: Trace reasoning and agentic issue localization.arXiv preprint arXiv:2505.08638, on PithPith's reviewarXiv
- [8] Haechan Kim, Yoonho Lee, Gisang Lee, Chelsea Finn, and Kangwook Lee. WHALE: A simple recipe for joint harness-weight optimization.arXiv preprint arXiv:2609.00196, 2026a. Kyungmin Kim, Youngbin Choi, Seoyeon Lee, Suhyeon Jun, Dongwoo Kim, and Sangdon Park. The interplay of harn... on PithPith's reviewarXiv
- [9] arXiv:2308.03688. Chang Ma, Junlei Zhang, Zhihao Zhu, Cheng Yang, Yujiu Yang, Yaohui Jin, Zhenzhong Lan, Ling- peng Kong, and Junxian He. Agentboard: An analytical evaluation board of multi-turn LLM agents. InNeurIPS, on PithPith's reviewarXiv
- [10] arXiv:2401.13178. Evan Miller. Adding error bars to evals: A statistical approach to language model evaluations.arXiv preprint arXiv:2411.00640, on PithPith's reviewarXiv
- [11] Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez. Memgpt: Towards LLMs as operating systems.arXiv preprint arXiv:2310.08560, on PithPith's reviewarXiv
- [12] Zehan Qi, Xiao Liu, Iat Long Iong, Hanyu Lai, Xueqiao Sun, Wenyi Zhao, Yu Yang, Xinyue Yang, Jiadai Sun, Shuntian Yao, Tianjie Zhang, Wei Xu, Jie Tang, and Yuxiao Dong. WebRL: Train- ing LLM web agents via self-evolving online curriculum reinforcement learning.arXiv preprint a... on PithPith's reviewarXiv
- [13] Salman Rahman, Yubin Kim, Mihir Parmar, A. Ali Heydari, Genglin Liu, Simon A. Lee, Weizhi Zhang, Arian Hosseini, Ahmed A. Metwally, Yuzhe Yang, Baharan Mirzasoleiman, Xin Liu, Pavel Izmailov, Saadia Gabriel, Mark Malhotra, Shwetak Patel, Daniel McDuff, and Hamid Palangi. Locat... on PithPith's reviewarXiv
- [14] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, on PithPith's reviewarXiv
- [16] Hui-Ze Tan, Xiao-Wen Yang, Hao Chen, Jie-Jing Shao, Yi Wen, Yuteng Shen, Weihong Luo, Xiku Du, Lan-Zhe Guo, and Yu-Feng Li. Hindsight credit assignment for long-horizon LLM agents. arXiv preprint arXiv:2603.08754, on PitharXiv
- [17] Su Wang, Pin Qian, Yifan Lin, Jingzhou Xu, Yihang Chen, Xiaochong Jiang, Lifei Liu, and Haoran Yu. Phantom guardrails: When self-improving agent harnesses fix failures that never happened. arXiv preprint arXiv:2607.13083, 2026a. Yike Wang, Huaisheng Zhu, Zhengyu Hu, Yige Yuan,... on PithPith's reviewarXiv
- [18] Peng Xia, Rujun Han, Zifeng Wang, Yanfei Chen, Yufan Zhuang, Yoonho Lee, Chengsong Huang, Han Yu, Zhongying CuiZhu, Yifei Ming, Huaxiu Yao, Burak Gokturk, Tomas Pfister, and Chen- Yu Lee. RRSI: Regularized recursive self-improvement of agent harnesses.arXiv preprint arXiv:2609.24972, on PithPith's reviewarXiv
- [19] Yang Xu, Jiefu Zhang, Haixiang Sun, Zihan Zhou, Tianyu Cao, and Vaneet Aggarwal. Towards reliable LLM evaluation: Correcting the winner’s curse in adaptive benchmarking.arXiv preprint arXiv:2605.05973, on PithPith's reviewarXiv
- [20] Jiangze Yan, Yi Shen, Wenjing Zhang, Jieyun Huang, Zhaoxiang Liu, Ning Wang, Kai Wang, and Shiguo Lian. HiMPO: Hindsight-informed memory policy optimization for less-entangled credit in long-horizon agents.arXiv preprint arXiv:2606.16285, on PithPith's reviewarXiv
- [21] Haoran Ye, Yuxing Lu, Haonan Dong, Zhaochen Su, and Guojie Song. Harness-zero: Harness distillation via agent-as-harness.arXiv preprint arXiv:2609.24974, on PithPith's reviewarXiv
- [22] 11 Preprint. Under review. Ping Yu, Jing Xu, Jason Weston, and Ilia Kulikov. Distilling system 2 into system 1.arXiv preprint arXiv:2407.06023, on PithPith's reviewarXiv
- [23] Zhou Yu, Bin Bi, Shiva Kumar Pentyala, Shubham Mehrotra, Sougata Chaudhuri, Shilpa Bhagavath, Zeyuan Chen, Ran Xu, Phil Mui, James Zhu, and Sitaram Asur. Co-evolving harnesses and models: On-policy correction helps weaker models catch up where imitation fails.arXiv preprint ar... on PithPith's reviewarXiv
- [24] Ruiqing Yue, Yu Cui, Zhuoyu Sun, Sicheng Pan, Xianhong Xue, Tingyu Li, Ting Li, Wenzhuo Zhu, Yi Chen, Yifei Liu, Baohan Huang, Zhe Cui, Haibin Zhang, and Cong Zuo. Ecdysis: Efficient and effective training of runtime harnesses for LLM agents.arXiv preprint arXiv:2609.11677, on PithPith's reviewarXiv
- [25] arXiv:2203.14465. Aohan Zeng, Mingdao Liu, Rui Lu, Bowen Wang, Xiao Liu, Yuxiao Dong, and Jie Tang. Agent- Tuning: Enabling generalized agent abilities for LLMs.arXiv preprint arXiv:2310.12823, on PithPith's reviewarXiv
- [26] Jiayi Zhang, Jinyu Xiang, Zhaoyang Yu, Fengwei Teng, Xionghui Chen, Jiaqi Chen, Mingchen Zhuge, Xin Cheng, Sirui Hong, Jinlin Wang, Bingnan Zheng, Bang Liu, Yuyu Luo, and Chenglin Wu. AFlow: Automating agentic workflow generation.arXiv preprint arXiv:2410.10762, on PithPith's reviewarXiv
- [27] Luan Zhang, Ruochen Zhou, Dandan Song, Zhengyu Chen, Yuhang Tian, Jun Yang, Huipeng Ma, Chenhao Li, Guangyuan Feng, Xudong Li, Yizhou Jin, and Yan Xu. HarnessCompass: Guiding automatic harness evolution toward generalizable and effective agent harnesses.arXiv preprint arXiv:26... on PithPith's reviewarXiv
show 9 more
- [28] arXiv:2307.13854. Kunlun Zhu, Zijia Liu, Bingxuan Li, Muxin Tian, Yingxuan Yang, Jiaxun Zhang, Pengrui Han, Qipeng Xie, Fuyang Cui, Weijia Zhang, Xiaoteng Ma, Xiaodong Yu, Gowtham Ramesh, Jialian Wu, Zicheng Liu, Pan Lu, James Zou, and Jiaxuan You. Where LLM agents fail and ho... on PithPith's reviewarXiv
- [29] use a larger second textbook (198 trajectories over 42 tasks for 4B, 200 over 44 for 9B, one epoch) collected the same way; full- parameter training uses learning rate2×10−6 with 5% warm-up and cosine decay, fully sharded, and a higher learning rate without warm-up diverged fo... on Pith
- [2013] Mert Cemri, Melissa Z. Pan, Shuyi Yang, Lakshya A. Agrawal, Bhavya Chopra, Rishabh Ti- wari, Kurt Keutzer, Aditya Parameswaran, Dan Klein, Kannan Ramchandran, Matei Zaharia, Joseph E. Gonzalez, and Ion Stoica. Why do multi-agent LLM systems fail?arXiv preprint arXiv:2503.13657, on PithPith's reviewarXiv
- [2017] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathe- matical reasoning in open language models.arXiv preprint arXiv:2402.03300, on PithPith's reviewarXiv
- [2022] arXiv:2106.09685. Shengran Hu, Cong Lu, and Jeff Clune. Automated design of agentic systems.arXiv preprint arXiv:2408.08435, on PithPith's reviewarXiv
- [2023] arXiv:2305.14314. Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), on PithPith's reviewarXiv
- [2024] Sayash Kapoor, Benedikt Stroebl, Zachary S. Siegel, Nitya Nadgir, and Arvind Narayanan. AI agents that matter.arXiv preprint arXiv:2407.01502, on PithPith's reviewarXiv
- [2025] Baian Chen, Chang Shu, Ehsan Shareghi, Nigel Collier, Karthik Narasimhan, and Shunyu Yao. FireAct: Toward language agent fine-tuning.arXiv preprint arXiv:2310.05915, on PithPith's reviewarXiv
- [2026] Zhengyu Chen, Teng Xiao, Huaisheng Zhu, Yige Yuan, Luan Zhang, and Jingang Wang. Co-harness: Co-evolving harnesses and model weights for LLM agents.arXiv preprint arXiv:2607.22688, on PithPith's reviewarXiv
Cited by 4 Pith papers
Reviewed papers on Pith that cite this one. The full citation record
- Knowledge in the Harness, Reliability in the Weights: Teacher-Free Co-Evolution of an LLM Agent cs.CL · Nov 2026
- RSI-PDEbench: A Benchmark for Self-Improving Agents in PDE Solving cs.AI · Nov 2026
- Flex-RSI: Benchmarking Recursive Self-Improvement of Frontier Agents cs.AI · Nov 2026
- RSI-Opt: Recursive Self-Improvement Optimizer Agents For Model Training cs.AI · Oct 2026
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment