Computation and Language · arXiv:2608.20169 v2 · submitted 20 Aug 2026 · 14 references
Task-CoEvolve: Efficient Harness Optimization via Adaptive Validation Task Selection
The paper says
“We present a novel approach to efficient LLM harness optimization through adaptive validation task selection. Harness optimization iteratively rewrites the harness code based on validation performance, enabling substantial performance gains without updating the underlying model weights.”
more of the abstract
Existing approaches, however, evaluate a fixed validation set in full at every iteration, incurring substantial evaluation costs even on tasks that become less discriminative as the harness evolves. We propose Task-CoEvolve, which co-evolves the validation tasks with the harness by addressing two challenges: selecting informative tasks and estimating full-set performance from partial evaluations. Task-CoEvolve builds on the observation that tasks on which candidate harnesses disagree are more informative for distinguishing among them than tasks that are consistently solved or failed. It uses variance-weighted sampling based on past outcomes to focus evaluation on tasks near the capability frontier, with the sampling distribution adapting as the harness evolves. It then estimates full-set scores from the sampled tasks by accounting for their sampling probabilities, enabling consistent comparisons across iterations despite evaluating different subsets. Experiments on online text classification and Terminal-Bench 2.1 show that Task-CoEvolve consistently outperforms subset-based baselines and matches the final performance of full-set search while reducing the number of evaluations during optimization by 80%. Code will be released at https://github.com/Agent4Science-UTokyo/Task-CoEvolve.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 27 Aug 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. A genuinely useful and honest method paper on adaptive validation sampling for harness optimization, with a real but addressable soft spot in the estimator-choice rule. This line is wrong
2 major objections · 4 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 14 references say.
The paper says, at §3.3, Appendix A, Table C, Appendix F
The review did not quote this passage.
Pith says: Objection 1 of 2 · major
The choice between the Hajek estimator (Eq. 3) and the anchored difference estimator (Eq. 4) is made per benchmark after inspecting Phase-0 evaluations and task-set structure, but no deterministic decision rule is specified. Appendix A shows this choice is load-bearing: on text classification, replacing the Hajek estimator with the difference estimator lowers mean test accuracy by 3.3 points at ρ=20% …
more of objection 1
… and 3.6 points at ρ=7% (Table A), and on Terminal-Bench the Hajek estimator produces an unstable estimate of 85.9% for a raw 33.3% score (Section A.2). Since these are exactly the two benchmarks in the headline tables, the reported 'matches or outperforms full search' results are conditional on a post hoc modeling choice that is not validated out-of-sample; Appendix F further indicates that LLMs were consulted specifically when choosing between estimators. Please specify a fully pre-registered rule for selecting the estimator from Phase-0 statistics, or evaluate both estimators under a fixed rule on both benchmarks and report the resulting performance.
The paper says, at §4.2, Table 1, §5.3
The review did not quote this passage.
Pith says: Objection 2 of 2 · major
The claim that Task-CoEvolve 'outperforms' Meta-Harness at ρ=20% rests on 49.3±0.8 versus 48.6±0.8 over three runs; the 0.7-point difference is smaller than the reported standard deviations and no paired test or per-run comparison is given. Table 6 also shows that at ρ=20% the \Hat{S}-max winner ranks 12/60 by true full-validation score, and at ρ=7% the rank correlation between \Hat{S} and true score is only 0.13.
more of objection 2
The paper should report paired differences with standard errors, increase the number of independent runs, or downgrade the text-classification claim from 'outperforms full-set search' to 'statistically indistinguishable from full-set search at this budget.'
4 minor objections
The paper says, at §3.3
Pith says: minor objection 1 of 4
The statement that the Hajek estimator suits a task set that 'splits into several pools, each sampled and estimated separately and then averaged' is not formalized in the main text; please specify how pools are defined (e.g., per dataset) and how per-pool estimates are combined to form \Hat{S}(h).
The paper says, at Table 3 caption
Pith says: minor objection 2 of 4
The parenthetical 'Trials: 1 or 2 trials fail before counts' is unclear; please state explicitly how failed or skipped sandbox trials are counted in the trial totals, token totals, and time measurements.
show 2 more
The paper says, at Eq. (2), Table D
Pith says: minor objection 3 of 4
The hyperparameters ℓ and λ control the exploration--exploitation trade-off in task selection, but no sensitivity analysis is reported; please add a small sensitivity study or state that the main conclusions are stable across reasonable values of these parameters.
The paper says, at Abstract, §4.5
Pith says: minor objection 4 of 4
The '80%' reduction refers to the number of task evaluations, while 'search cost' elsewhere refers to input tokens (67--80%) and wall-clock time (about 50%); please use consistent terminology so that the cost-reduction claims are not conflated across metrics.
What would settle it
Pre-specify an explicit threshold rule for estimator selection using only Phase-0 statistics, e.g., 'if the pool-level mean success rate lies in [0.3, 0.7] and more than 60% of tasks are at p_bar in {0,1}, use Eq. 4; otherwise use Eq. 3'. Then apply this fixed rule on a held-out benchmark pair (a new text-classification pool and a new Terminal-Bench-like pool) and rerun the protocol at rho=20%. If the rule picks the estimator that leads to the lower final test accuracy on either held-out benchmark, or if the 20%-budget advantage over full search disappears, the headline claim is not robust. A cheaper variant: leave-one-benchmark-out cross-validation over Law, S2D, USPTO, and Terminal-Bench, fitting the rule on the other three and testing on the held-out one; report whether the selected estimator matches the Appendix A better estimator in every fold.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 27 Aug 2026
- Reader holds up, with conditions
- Skeptic agrees
- Editor wrote the verdict above
- Referee major revision: 2 major objections, 4 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
Here's the short version: this is a legitimately useful method paper. Task-CoEvolve attacks a concrete cost—full validation-set evaluation at every harness-optimization iteration—and does it with a clean combination of variance-weighted task sampling and design-based full-set estimation. The problem framing is new as far as I know, and the paper is unusually transparent about where it works and where it doesn't.
What I like: the ablations isolate each component (resampling, estimation, variance-weighted selection all contribute), and Appendix A doesn't hide the fact that the estimator choice matters. The pseudocode for the discovered harnesses is a nice touch. The cost table (Table 3) is honest about the fact that Task-CoEvolve spends more tokens per trial than random sampling because it selects long, discriminative tasks.
Where I'd push back: the headline 'outperforms full-set search at 20% budget' is a 0.7-point gap with overlapping standard deviations on three runs. That's not a claim I'd put in the abstract. The estimator choice is the bigger issue: the rule in Section 3.3 is qualitative and applied per benchmark after seeing Phase-0 data. Appendix A shows a 3.3-point penalty for the wrong choice, which is five times the headline margin. This is a real soft spot, but not a fatal one—the rule is sensible and the sensitivity is documented. The fix is straightforward: pre-specify the rule or cross-validate it, and add error bars for Terminal-Bench, which currently has none.
Bottom line: the central idea is sound, the experiments are plausible, and the paper is a genuine step forward for automated harness optimization. It deserves a serious referee. I'd accept for review and ask for the estimator rule to be pre-specified or validated out-of-sample, plus variance estimates on Terminal-Bench. I'd also suggest softening the outperform claim to 'comparable within noise.' Who's it for? Anyone working on harness optimization, sample-efficient evaluation, or LLM self-improvement pipelines. I'd probably cite it if the code checks out.
The strongest objection
The central claim that Task-CoEvolve matches or beats full-set search at a 20% budget depends on selecting the right estimator. Section 3.3 chooses between the Hajek estimator (Eq. 3) and the anchored difference estimator (Eq. 4) 'guided by the Phase-0 evaluations and the type of task set', but no pre-specified decision rule is given. Appendix A shows the wrong choice is costly: on text classification, using Eq. 4 instead of Eq. 3 lowers the mean test accuracy by 3.3 points at rho=20% (Table A); on Terminal-Bench, using Eq. 3 instead of Eq. 4 produced an unstable 85.9% estimate for a raw 33.3% score at rho=10% (Section A.2). These are exactly the two benchmarks used in the headline tables, so the qualitative rule is applied to the same data on which success is declared, with no out-of-sample validation of the rule itself. The concern is not that the estimators are internally wrong; it is that the method as specified is under-determined, and the reported headline depends on a post hoc modeling choice. This is compounded by the strongest outperforms-full-set claim resting on three runs with overlapping standard deviations (49.3 +/- 0.8 vs 48.6 +/- 0.8), so the headline margin is smaller than the Appendix A estimator-selection penalty.
The reader's weakest assumption matches my reading: the estimator choice is the point where the method's success is least secured. The paper is otherwise coherent and the components are standard, with a sensible ablation (Table 4) and an honest estimator-swap appendix. But the headline quantitative claims are exactly what the estimator choice controls, and the appendix shows the penalty for the wrong choice exceeds the reported advantage over full search. The outperforms-full-set claim also rests on three runs with overlapping standard deviations, and Table 6 shows that at rho=20% the selected candidate is only the 12th best of 60 by true full-validation score, which further tempers the strongest claim. These are addressable: pre-specify the rule, cross-validate it, release code, and report error bars on Terminal-Bench. That is precisely the condition of a conditional acceptance; no part of the argument is internally inconsistent enough to reject.
The referee's report
If the reported results are robust, the paper addresses a real and relatively unexplored source of cost in harness optimization: the per-candidate evaluation budget. The proposed combination of variance-weighted task selection with design-based full-set estimation is coherent, and the paper is unusually transparent in reporting estimator swaps (Appendix A), tie-breaking sensitivity (Appendix B), and cost breakdowns (Table 3). These are genuine strengths. However, the headline claims currently rest on a per-benchmark estimator choice made after inspecting Phase-0 data, and on three-run averages with overlapping error bars. The central quantitative claims are therefore not yet supported at the level the abstract states.
Pith's reading of the paper (a machine's summary, not the authors')
The paper argues that automated harness optimization, which iteratively rewrites the code surrounding a fixed LLM, can be made far cheaper by co-evolving the validation tasks along with the harness. Instead of evaluating every validation task at every iteration, Task-CoEvolve samples a small subset that favors tasks where recent candidate harnesses disagree, and estimates the full-set score from that subset with sampling-aware estimators. On online text classification and Terminal-Bench 2.1 it claims to match the final performance of full-set search while using roughly 80% fewer evaluations, and to beat full-set search at a 20% budget on text classification. A sympathetic reader would care because evaluation cost, not candidate generation, is often the bottleneck in harness search, especially for long-horizon agent benchmarks.
What carries the argument. The load-bearing device is a two-step loop. First, variance-weighted selection sets each task's sampling weight to $w_t = \max\{\bar{p}_t(1-\bar{p}_t), \ell_t\} + \lambda/\sqrt{n_t}$, where $\bar{p}_t$ is the observed success rate and $n_t$ the observation count; this concentrates sampling on tasks near the 50% success frontier, where candidate harnesses disagree. Second, full-set estimation corrects for the non-uniform inclusion probabilities $\pi_t$ (estimated by Monte Carlo) using either the H´ajek estimator $\hat{S}(h) = \frac{\sum_{t\in S_k} x_t(h)/\pi_t}{\sum_{t\in S_k} 1/\pi_t}$ or the anchored difference estimator $\hat{S}(h) = \frac{1}{N}\sum_{t\in\mathcal{T}} \bar{p}_t + \frac{1}{N}\sum_{t\in S_k} (x_t(h)-\bar{p}_t)/\pi_t$, chosen per benchmark by a Phase-0 rule. The two components together let scores from different subsets be compared on a common scale while the budget tracks the capability frontier.
Core claim. Task-CoEvolve's central claim is that the validation set used to guide harness evolution should itself evolve. Tasks on which candidate harnesses disagree, quantified by the Bernoulli variance of their historical success rates, are the ones that discriminate among candidates, while tasks everyone solves or nobody solves still consume budget without signal. By sampling tasks with weight proportional to that variance and estimating the full-set score with inclusion-probability weighting, the method keeps successive candidate comparisons on a common scale. The paper reports that on three online text-classification datasets this reaches 49.3% held-out accuracy at a 20% evaluation budget, above the 48.6% of full-set search, and on Terminal-Bench 2.1 it lands within one task of full-set search while cutting token cost by 67–80%.
The premise it rests on. The method's success depends on the choice between the two full-set estimators being made correctly for the task pool before results are seen; the paper supplies a qualitative Phase-0 rule for that choice, but does not validate the rule out-of-sample.
If the paper is right (Pith's inferences, not the authors')
Full validation-set evaluation is not a necessary ingredient of effective harness search: at 20% budget the adaptive method matches full-set search on Terminal-Bench 2.1 and beats it on text classification.
The main cost reduction is in evaluation: 80% fewer task executions and 67–80% fewer input tokens, with wall-clock time roughly halved once the fixed meta-agent proposal cost is included.
Resampling the subset alone, without score estimation, recovers most but not all of the gain; the ablation shows that sampling-aware estimation and variance-weighted selection each add further accuracy, so comparability across iterations is a distinct requirement.
The task pool's discriminative structure changes as the harness improves (unsolved tasks become solvable, easy tasks saturate), so a static validation subset is a mismatched instrument for a moving capability frontier.
The qualitative estimator-choice rule (Hajek vs anchored difference) is selected per benchmark from Phase-0 outcomes; until a pre-specified decision procedure is validated out-of-sample, a reader should treat the 80%-cost claim as specific to the two benchmark families studied.
The same variance-weighted sampling plus design-based estimation recipe transfers naturally to other iterative agent-optimization loops, such as best-of-N candidate selection or test-time compute search, where the same tension between subset informativeness and cross-iteration comparability arises.
The paper's own 7%-budget result (Spearman 0.13 between estimated and true scores, yet the selected candidate still ranks 10/60) suggests that final selection is more robust than the score estimates; a natural extension is per-candidate adaptive sample sizes, which the paper lists as future work.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
We present a novel approach to efficient LLM harness optimization through adaptive validation task selection. Harness optimization iteratively rewrites the harness code based on validation performance, enabling substantial performance gains without updating the underlying model weights. Existing approaches, however, evaluate a fixed validation set in full at every iteration, incurring substantial evaluation costs even on tasks that become less discriminative as the harness evolves. We propose Task-CoEvolve, which co-evolves the validation tasks with the harness by addressing two challenges: selecting informative tasks and estimating full-set performance from partial evaluations. Task-CoEvolve builds on the observation that tasks on which candidate harnesses disagree are more informative for distinguishing among them than tasks that are consistently solved or failed. It uses variance-weighted sampling based on past outcomes to focus evaluation on tasks near the capability frontier, with the sampling distribution adapting as the harness evolves. It then estimates full-set scores from the sampled tasks by accounting for their sampling probabilities, enabling consistent comparisons across iterations despite evaluating different subsets. Experiments on online text classification and Terminal-Bench 2.1 show that Task-CoEvolve consistently outperforms subset-based baselines and matches the final performance of full-set search while reducing the number of evaluations during optimization by 80%. Code will be released at https://github.com/Agent4Science-UTokyo/Task-CoEvolve.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-09-23
The authors' main claim, as the classifier picked it out 1.00
“Experiments on online text classification and Terminal-Bench 2.1 show that Task-CoEvolve consistently outperforms subset-based baselines and matches the final performance of full-set search while reducing the number of evaluations during optimization by 80%.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.20 · states limitation 0.10 · human participants 0.03 · controlled comparison 0.94 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.97 · mentions code or data availability 0.97
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v2). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| We present a novel approach to efficient LLM harness optimization through adaptive validation task selection. | contribution 1.00 | 0.86 | 0.11 | asserted 0.99 |
| Harness optimization iteratively rewrites the harness code based on validation performance, enabling substantial performance gains without updating the underlying model weights. | background 0.70 | 0.27 | 0.09 | asserted 1.00 |
| Existing approaches, however, evaluate a fixed validation set in full at every iteration, incurring substantial evaluation costs even on tasks that become less discriminative as the harness evolves. | gap 0.61 | 0.06 | 0.03 | asserted 1.00 |
| We propose $\textbf{Task-CoEvolve}$, which co-evolves the validation tasks with the harness by addressing two challenges: selecting informative tasks and estimating full-set performance from partial evaluations. | contribution 0.99 | 0.85 | 0.11 | asserted 0.99 |
| Task-CoEvolve builds on the observation that tasks on which candidate harnesses disagree are more informative for distinguishing among them than tasks that are consistently solved or failed. | method 0.84 | 0.63 | 0.05 | asserted 0.99 |
| It uses variance-weighted sampling based on past outcomes to focus evaluation on tasks near the capability frontier, with the sampling distribution adapting as the harness evolves. | method 0.99 | 0.89 | 0.04 | asserted 1.00 |
| It then estimates full-set scores from the sampled tasks by accounting for their sampling probabilities, enabling consistent comparisons across iterations despite evaluating different subsets. | method 1.00 | 0.90 | 0.05 | asserted 1.00 |
| Experiments on online text classification and Terminal-Bench 2.1 show that Task-CoEvolve consistently outperforms subset-based baselines and matches the final performance of full-set search while reducing the number of evaluations during optimization by 80%. | result 1.00 | 0.97 | 0.79 | asserted 1.00 |
| Code will be released at https://github.com/Agent4Science-UTokyo/Task-CoEvolve. | future work 0.87 | 0.30 | 0.02 | asserted 0.94 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.99
What was examined, and what was not
Sources read
- arXiv listing abstract v2
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 6,952 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (2)
Figures are the authors', reproduced from the paper.
References (14)
- [1] Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al. gpt-oss-120b & gpt-oss-20b model card.arXiv preprint arXiv:2508.10925, on PithPith's reviewarXiv
- [4] Accessed: 2026-08-15. Zhiwei Fei, Xiaoyu Shen, Dawei Zhu, Fengzhe Zhou, Zhuo Han, Alan Huang, Songyang Zhang, Kai Chen, Zhixin Yin, Zongwen Shen, Jidong Ge, and Vincent Ng. LawBench: Benchmarking legal knowledge of large language models. InEMNLP, pp. 7933–7962, on Pith
- [6] Jiahang Lin, Shichun Liu, Chengjun Pan, Lizhi Lin, Shihan Dou, Zhiheng Xi, Xuanjing Huang, Hang Yan, Zhenhua Han, Tao Gui, and Yu-Gang Jiang. Agentic harness engi- neering: Observability-driven automatic evolution of coding-agent harnesses.arXiv preprint arXiv:2604.25850, on PithPith's reviewarXiv
- [8] Jun Nie, Yonggang Zhang, Jun Song, Qianshu Cai, Dahai Yu, Yike Guo, Xinmei Tian, and Bo Han. TTHE: Test-time harness evolution.arXiv preprint arXiv:2607.08124, on PithPith's reviewarXiv
- [9] Accessed: 2026-08-04. 11 Task-CoEvolve: Efficient Harness Optimization via Adaptive Validation Task Selection Qwen Team. Qwen3.6-35B-A3B: Agentic coding power, now open to all, April on Pith
- [11] Yike Wang, Huaisheng Zhu, Zhengyu Hu, Yige Yuan, Zhengyu Chen, Shakti Senthil, Hannaneh Hajishirzi, Yulia Tsvetkov, Pradeep Dasigi, and Teng Xiao. Rethinking the evaluation of harness evolution for agents. InCOLM 2026 The 2nd Workshop on Lifelong Agents: Learning, Aligning, an... on Pith
- [12] URLhttps://lilianweng.github.io/posts/2026-07-04-harness/. Ac- cessed: 2026-08-04. Yang Yang, Zining Zhong, Jindong Li, Jiemin Wu, Kaishen Yuan, Wenshuo Chen, Menglin Yang, and Yutao Yue. Turboevolve: Towards fast and robust llm-driven program evolution.arXiv preprint arXiv:26... on PithPith's reviewarXiv
- [13] Jiayi Zhang, Yongfeng Gu, Jianhao Ruan, Maojia Song, Yiran Peng, Zhiguang Han, Jinyu Xiang, Zhitao Wang, Caiyin Yang, Yixi Ouyang, Bang Liu, Chenglin Wu, and Yuyu Luo. Harnessing agentic evolution.arXiv preprint arXiv:2605.13821, 2026a. Luan Zhang, Ruochen Zhou, Dandan Song, Z... on PithPith's reviewarXiv
- [14] If a new candidate also solves this task, its residual becomesx t(h)−¯pt = 0, so the task does not affect the estimate. B SENSITIVITY TO THETIE-BREAKRULE Throughout the paper, when multiple candidates have the same selection score, we select the candi- date from the earliest i... on Pith
- [2009] Lirong Che, Yuzhe Yang, Peiwen Lin, Chuang Wang, Xueqian Wang, and Jian Su. Demoevolve: Overcoming sparse feedback in agentic harness evolution with demonstrations.arXiv preprint arXiv:2605.24539, on PithPith's reviewarXiv
- [2022] Muxin Tian, Zhe Wang, Blair Yang, Zhenwei Tang, Kunlun Zhu, Honghua Dong, Hanchen Li, Xinni Xie, Guangjing Wang, and Jiaxuan You. Swe-bench mobile: Can large language model agents develop industry-level mobile applications?arXiv preprint arXiv:2602.09540, on PithPith's reviewarXiv
- [2023] Accessed: 2026-07-22. Hanghui Guo, Weijie Shi, Zhangze Chen, Shengxiang Xu, Yishu Wang, Yimei Zhang, Wangze Ni, Jia Zhu, and Shimin Di. Drevo: Distilling recalibrated historical experience for harness self- evolution.arXiv preprint arXiv:2607.26722, on PithPith's reviewarXiv
- [2024] Mike A. Merrill, Alexander G. Shaw, Nicholas Carlini, Boxuan Li, Harsh Raj, Ivan Bercovich, Lin Shi, Jeong Yeon Shin, Thomas Walshe, E. Kelly Buchanan, Junhong Shen, Guanghao Ye, Haowei Lin, Jason Poulos, Maoyu Wang, Marianna Nezhurina, Jenia Jitsev, Di Lu, Orfeas Menis Mas- t... on PithPith's reviewarXiv
- [2026] URLhttps://www.anthropic.com/news/ claude-opus-4-6. Accessed: 2026-08-01. Yoshua Bengio, J´erˆome Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In ICML, pp. 41–48, on Pith
Cited by 4 Pith papers
Reviewed papers on Pith that cite this one. OpenAlex reports about 7 citations worldwide. The full citation record
- The RSI Tax: When Recursive Self-Improvement Is Not Worth the Cost cs.LG · Nov 2026
- HERA: Harness-Environment Co-Evolution for Reliable Agentic Abstention cs.AI · Oct 2026
- Scaling Trajectories for Complex Tasks through Recursive Self-Rewrite cs.AI · Oct 2026
- ActiveSaddler: Automated Curriculum Learning for Agent Harness Optimization cs.AI · Oct 2026
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites · 5 sentences without a confident label
Uses the method or tool of 1 paper
-
Following [ref] , we use [ref] with temperature $0$ as the classifier LLM, and automatically optimize its harness.
gpt-oss-120b & gpt-oss-20b Model Card · Setup on Online Text Classification · 1.00
Cites as background 7 papers
-
Recently, automated harness optimization has begun to attract considerable attention, where a meta-level agent iteratively rewrites the harness code, evaluates the resulting candidates on a benchmark, and retains changes that improve performance [ref] .
DREvo: Distilling Recalibrated Historical Experience for Harness Self-Evolution · Introduction · 0.82
-
Recently, automated harness optimization has begun to attract considerable attention, where a meta-level agent iteratively rewrites the harness code, evaluates the resulting candidates on a benchmark, and retains changes that improve performance [ref] .
Harnessing Agentic Evolution · Introduction · 0.82
-
Recently, automated harness optimization has begun to attract considerable attention, where a meta-level agent iteratively rewrites the harness code, evaluates the resulting candidates on a benchmark, and retains changes that improve performance [ref] .
Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses · Introduction · 0.82
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment