Artificial Intelligence · arXiv:2610.00906 v1 · submitted 1 Oct 2026 · 44 references
ActiveSaddler: Automated Curriculum Learning for Agent Harness Optimization
The paper says
“Automated harness optimization can substantially improve LLM agents by iteratively updating their prompts, tool interfaces, and control logic from execution feedback.”
more of the abstract
However, existing methods primarily optimize how the harness is updated while largely fixing which training scenarios generate the feedback that drives those updates. As the harness evolves, the scenarios most useful for further optimization can change, suggesting that the training curriculum itself should adapt alongside the harness. We formulate this missing dimension of harness optimization as an automated curriculum learning problem and introduce ActiveSaddler. ActiveSaddler models the evolving curriculum as a non-stationary bandit with dynamically instantiated optimization targets. It abstracts recurring failures into reusable failure-pattern arms, estimates the potential learning progress from further targeting each pattern, and adaptively balances revisiting known weaknesses with exploring unseen scenarios for new ones. Optimization outcomes continually update both the set of discovered failure patterns and their priorities, allowing the curriculum to co-evolve with the harness. Experiments on GAIA2 and Terminal-Bench 2.0 show that ActiveSaddler consistently discovers stronger harnesses, improving test Pass@1 by 4.4 and 7.5 percentage points over the same harness optimizer using a scenario order fixed before optimization, respectively. Ablations further show that these gains depend on dynamically constructing optimization targets, estimating their evolving utility, and balancing continued optimization with new failure discovery. Together, these results establish automated curriculum learning as a new crucial optimization dimension for harness optimization.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 2 Oct 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. Genuine method contribution with solid GAIA2 evidence; the Terminal-Bench 7.5pp headline is a 3-task swing from a single run and needs a second run or a qualified claim. This line is wrong
2 major objections · 6 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 44 references say.
The paper says, at Table 1 / Section 5.2 / Appendix C
The review did not quote this passage.
Pith says: Objection 1 of 2 · major
The Terminal-Bench 2.0 headline of +7.5 pp (72.5 to 80.0) is a 3-task change on a 40-task test set, and the manuscript reports a second independent optimization run only for GAIA2 (Appendix C). Every TB2 ablation row the mechanism attribution rests on — w/o Failure-Pattern arms (69.2/73.3), w/o Arm Prioritizer (72.5), w/o Exploration Controller (71.7) — likewise comes from a single optimization trajectory.
more of objection 1
The GAIA2 two-run data (AutoSaddler 55.4/56.6, ActiveSaddler 59.8/61.6) place run-to-run spread at roughly 1–2 pp over 300 tasks; by binomial scaling to n=40, the per-run standard error of a Pass@1 estimate is near 7–8 pp, so the TB2 delta and each single-run ablation row are inside the optimization-stochasticity noise the paper itself documents. The ± values in Table 1 are standard deviations over three test-time executions of one optimized harness, not over optimization runs, so they do not quantify this uncertainty. Before the 'consistent' cross-benchmark claim (abstract, [b7], [b9]) can stand, I ask for replicated TB2 runs for the headline and the three key ablation rows, with the number of runs stated explicitly per cell; alternatively, the TB2 claims should be explicitly downgraded and marked as preliminary single-run results.
The paper says, at Eq. (8) / Appendix K / Section 5.3 RQ2
The review did not quote this passage.
Pith says: Objection 2 of 2 · major
The mechanism attribution 'gains depend on ... estimating their evolving utility' presumes that the LLM score phi_t(a) in Eq. (8) is a faithful estimate of expected population-performance change from selecting arm a, but no calibration of phi_t against realized per-arm development-set improvement is reported.
more of objection 2
The score is a hand-chosen equal-weight mean — the Arm Scoring Prompt fixes (severity + fixability + breadth + (1 - side_effect)) / 4 — and the Exploration Controller's PULL/DRAW call is explicitly made 'with no fixed formula or threshold' (Appendix K). Appendix D's EMA (56.7) and UCB-AIR (55.6) comparisons are a genuine strength: they show the LLM components beat plausible mechanical alternatives, which rules out the weak version of the concern. What remains open is whether the specific four-axis weighting and the unthresholded exploration judgment are load-bearing design choices rather than implementation details. I ask the authors to either report a sensitivity check on the axis weights with at least one run, or state plainly that the mechanism attribution is contingent on un-calibrated LLM judgments that were not validated against measured per-arm improvement.
6 minor objections
The paper says, at Appendix K
Pith says: minor objection 1 of 6
The extraction prompt states that 'a single patch should be able to fix one pattern', i.e., it assumes harness weaknesses decouple into independently fixable atomic arms; composition via multi-tagging is described in the same prompt, so the assumption is partially operationalized, but non-atomic weaknesses that no single patch fixes are not analyzed — a sentence in the limitations section would help.
The paper says, at Table 1
Pith says: minor objection 2 of 6
AutoSaddler on Terminal-Bench 2.0 reports a standard deviation of 0.0 across three test-time executions; please confirm this is accurate and not a reporting artifact, since it contrasts with the several-point spreads of adjacent rows.
show 4 more
The paper says, at Reproducibility Statement / Abstract
Pith says: minor objection 3 of 6
The paper says code 'will be available' at a project URL rather than shipping an artifact; given the detailed reproducibility statement, a hosted repository link or artifact DOI in the current version would strengthen the verification story.
The paper says, at Figure 4
Pith says: minor objection 4 of 6
The in-panel transcripts of the exploration rationales are dense and hard to read at print size; moving the PULL/DRAW decision rationales into a table would make the adaptive-exploration argument easier to follow.
The paper says, at Section 4 / Appendix K
Pith says: minor objection 5 of 6
Terminology is inconsistent: phi_t(a) is introduced as the 'learning-progress score' in Eq. (8) but is called the 'cardinal learning-progress score' in the Arm Scoring Prompt; unify the naming.
The paper says, at Section 5.1
Pith says: minor objection 6 of 6
The fixed difficulty curricula are constructed from 'five evaluations of the base harness on the training set'; Appendix F correctly counts this cost toward the budget, but stating this explicitly in Section 5.1 would avoid the impression that these evaluations are free.
What would settle it
Replicate the complete ActiveSaddler-vs-AutoSaddler comparison on Terminal-Bench 2.0 under the identical protocol and 490-rollout budget for at least 3 (ideally 5) independent optimization runs, running the three attribution ablations (w/o Failure-Pattern Arms, w/o Arm Prioritizer, w/o Exploration Controller) across the same runs. Report mean ± std of test Pass@1 over runs, the per-task delta, and an effect size with confidence interval (or a paired test over run outcomes) for the 7.5 pp gap versus the fixed shuffled AutoSaddler baseline. If the TB2 delta does not replicate at the reported magnitude, or the ablation rows collapse under run variance, the headline claim and the Table 1 mechanism attribution need revision.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 2 Oct 2026
- Reader holds up, with conditions
- Skeptic partly agrees
- Editor wrote the verdict above
- Referee major revision: 2 major objections, 6 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
Quick read: the adaptive-curriculum idea is real, the GAIA2 evidence carries it, and the paper has one genuine soft spot — the Terminal-Bench 2.0 headline is a 3-task improvement from a single optimization run.
What's actually new: prior adapters (Task-CoEvolve, HarnessLens) select tasks on the evaluation side after candidates are proposed. This paper adapts which training scenarios generate the feedback for each harness update, using dynamically instantiated failure-pattern arms, adaptive arm prioritization, and an adaptive explore/exploit decision. That is a real distinction, and the design is clean. The paper is also honest about scope: the optimizer is held fixed, and Appendix L explicitly disclaims safety, security, and reward-hacking readiness. I don't hold that against them.
The empirical package is well above average. Two independent GAIA2 runs (59.8/61.6 vs 55.4/56.6 on a 300-task test set), a port to a second optimizer (GEPA, +3.0pp), non-LLM baselines (EMA scoring, UCB-AIR) that underperform the LLM modules, a cost analysis, and an ablation ladder whose ordering is internally consistent. Circularity burden is low: the comparison is against fixed-order baselines on external benchmarks, not fitted parameters. The case studies showing an arm retargeted after a failed first patch make the mechanism story plausible.
The soft spot is TB2. The 7.5pp delta is 3 tasks on a 40-task test set, from a single run, and every TB2 ablation row is also single-run. The paper's own GAIA2 data put run-to-run spread at ~1–2pp on 300 tasks; scaled to 40 tasks that becomes several pp of noise, so the TB2 number is closer to 2 sigma than to a confirmed effect. The paper discloses the 40-task size, which I credit, but quoting 7.5pp as a headline overstates what one run on 40 tasks supports. The mechanism attribution does not collapse — GAIA2 independently supports it — but the TB2 number needs a second run or a qualified write-up.
Secondary concern: the curriculum decisions are LLM judgments (equal-weight four-axis scores, an exploration call with explicitly no formula) with no calibration check against population performance. Partly mitigated by the EMA/UCB-AIR ablations, which show the LLM modules beat mechanical alternatives, but a calibration analysis would tighten it.
Who it's for: anyone working on offline harness or prompt optimization for LLM agents. This deserves a serious referee. My recommendation: send to review, and require a TB2 replication run and either calibration evidence for the scoring or a pruned headline claim.
The strongest objection
The central claim requires the 4.4/7.5 pp deltas to be real and attributable to the adaptive curriculum. The GAIA2 part is well supported: Appendix C reports two independent runs (ActiveSaddler 59.8/61.6 vs AutoSaddler 55.4/56.6) on a 300-task test set, and the ablation ladder (uniform scoring 55.3, EMA 56.7, UCB-AIR 55.6, LLM 59.8) plus the GEPA port (Appendix E, +3.0 pp) and cost analysis (Appendix F) are genuine strengths. The Terminal-Bench 2.0 part is not comparably supported. TB2 has 40 test tasks (Table 1), so the 7.5 pp headline delta (72.5→80.0, [b7], [b134]) is a 3-task change from a single optimization run, and every TB2 row the mechanism attribution rests on — w/o Failure-Pattern arms (69.2/73.3), w/o Arm Prioritizer (72.5), w/o Exploration Controller (71.7) — is also single-run. The paper's own GAIA2 two-run data place run-to-run spread at ~1.2 pp (AutoSaddler 55.4/56.6) and ~1.8 pp (ActiveSaddler 59.8/61.6) on 300 tasks; binomial scaling to 40 tasks grows variance ~2.7x, implying several-pp run noise, so the TB2 delta is only ~1.5-2 sigma. The reward signal does not shore this up: phi_t(a) in Eq. 8 is an equally weighted LLM judgment of severity/fixability/breadth/side-effect risk, and the Exploration Controller makes the PULL/DRAW call with explicitly 'no fixed formula or threshold' (Appendix K), with no reported calibration of either against actual per-arm development-set improvement. The authors disclose the 40-task test size and run GAIA2 twice only; the gap is that the larger claimed gain and the entire attribution table are unverified against run-to-run variance.
The paper's central claim has two parts: an empirical improvement (4.4 pp GAIA2, 7.5 pp TB2) and a mechanism attribution (failure-pattern arms, adaptive prioritization, adaptive exploration). The GAIA2 evidence is robust: two independent runs with a 300-task test set, and ablations placing the LLM curriculum above mechanical alternatives (EMA 56.7, UCB-AIR 55.6, uniform 55.3 vs LLM 59.8); the GEPA port (+3.0 pp) and the end-to-end cost analysis are supporting evidence, and Appendix L appropriately disclaims safety and security readiness, which I credit rather than penalize. The load-bearing weakness is confined to the Terminal-Bench 2.0 portion and to the single-run status of the attribution table: 40 test tasks, a 3-task headline delta, and all four TB2 ablation rows from one optimization run each. The paper's own two-run GAIA2 data (spread ~1.2-1.8 pp on 300 tasks) imply several-pp run noise when scaled to 40 tasks, making the 7.5 pp delta marginal. I distinguish my concern from the reader's weakest assumption (LLM calibration faithfulness). That concern is real but partially mitigated by the ablation ladder showing the LLM scores beat mechanical signals; the sharper gap is that the larger claimed gain and the entire attribution are unverified against run-to-run variance, which only replication can settle. This sharpens the condition on the existing CONDITIONAL verdict rather than changing its level, so I recommend UNCHANGED.
The referee's report
The GAIA2 evidence is strong: two independent optimization runs (Appendix C, 55.4/56.6 vs 59.8/61.6) on a 300-task test set of disjoint universes, a consistent ablation ladder (55.3, 55.3, 55.6, 56.7, 59.8), a cross-optimizer GEPA port, non-LLM alternative comparisons, and a careful end-to-end cost analysis showing that the curriculum's optimizer-side overhead is offset by better allocation of expensive task-agent rollouts. The appendices ship unusually complete detail: prompt templates for extraction, scoring, and exploration; a pattern CLI; per-arm discovery tables; and per-iteration pull-probability trajectories. These are genuine reproducibility assets. If the effect holds, the contribution is meaningful: it cleanly isolates the 'which feedback' dimension of harness optimization from 'how feedback is used' and shows it matters under a fixed rollout budget. The main weakness is that the Terminal-Bench 2.0 half of the headline rests on a single optimization run on a 40-task test set, leaving the cross-benchmark 'consistent' claim and the TB2 mechanism attributions within the run-noise range the paper itself documents on GAIA2.
Pith's reading of the paper (a machine's summary, not the authors')
This paper argues that in offline LLM-agent harness optimization, the training scenarios that generate the feedback for each harness update should adapt as the harness itself improves, rather than being fixed before optimization begins. Harness optimization iteratively patches prompts, tool interfaces, and runtime control logic from execution traces, but prior methods schedule those training scenarios in advance. The paper introduces ActiveSaddler, which models the evolving curriculum as a non-stationary bandit whose arms are failure-pattern weaknesses instantiated on the fly from failed executions, and at each step either revisits a known weakness or probes an unseen scenario to discover new ones. On GAIA2 and Terminal-Bench 2.0, the same AutoSaddler optimizer driven by ActiveSaddler's curriculum beats a fixed scenario order by 4.4 and 7.5 percentage points of test Pass@1, and ablations attribute the gains to failure-pattern arm granularity, adaptive arm prioritization, and adaptive exploration timing.
What carries the argument. The failure-pattern arm is the load-bearing abstraction: a harness weakness shared across failed trajectories, constructed by an extractor that first abstracts each failure into a candidate symptom without consulting existing arms, then normalizes candidates by linking them to an existing atomic pattern, composing several patterns, or instantiating a new arm. These arms form the arm set of a non-stationary bandit in which selection is a softmax over LLM-estimated learning-progress scores with temperature $\tau_{\text{sel}}$, and the Exploration Controller makes a binary draw-versus-pull call on whether the known arms still contain a worthwhile unresolved target. Because the arm pool, the scores, and the exploration decision are all recomputed from the latest optimization history, the curriculum co-evolves with the harness.
Core claim. The central claim is that scenario selection is a distinct, exploitable optimization dimension for harness optimization: under a fixed rollout budget, test Pass@1 rises from 55.4% to 59.8% on GAIA2 and from 72.5% to 80.0% on Terminal-Bench 2.0 compared with the same AutoSaddler optimizer using a scenario order fixed before optimization. The mechanism is a bandit over dynamically instantiated failure-pattern arms: recurring failures are abstracted through a two-stage extractor into reusable patterns, each arm is scored by an LLM estimate of learning progress $\phi_t(a)\in[0,1]$ based on severity, fixability, breadth, and inverted side-effect risk, and an exploration controller decides each iteration whether to pull a known arm or draw unseen scenarios. The paper further claims the gains decompose into three design principles, each validated by ablation: failure-pattern granularity rather than category- or scenario-level arms, adaptive priority rescoring as the harness evolves, and state-dependent exploration timing that repairs when unresolved weaknesses remain and explores when they do not.
The premise it rests on. The load-bearing premise is that the LLM's judgments are faithful: the four-axis learning-progress scores are treated as estimates of the true expected population-performance change from each arm, and the draw-versus-pull call, made with explicitly no fixed formula or threshold, picks the scenario batch that actually helps, with no calibration check against realized performance.
If the paper is right (Pith's inferences, not the authors')
Under a fixed rollout budget, adaptively selecting training scenarios yields stronger final harnesses than any fixed order: +4.4 percentage points on GAIA2 and +7.5 on Terminal-Bench 2.0 over AutoSaddler with a fixed order, and larger margins over difficulty-ordered curricula.
The curriculum benefit transfers to a different optimizer: applying the same scenario-selection policy to GEPA raises its average test Pass@1 from 54.2% to 57.2%.
Adaptive curricula improve end-to-end cost efficiency despite added optimizer overhead, reaching a given development accuracy at a fraction of the cost of fixed orders, for instance 58.5% at $298 on GAIA2 versus $1,360 for AutoSaddler.
Arm granularity matters: failure-pattern arms more often sample scenarios on which the harness still fails than category arms (too coarse) or scenario arms (too fragmented), by 16.5 and 35.3 points on GAIA2.
Curriculum decisions are state-dependent: the draw rate falls from 44% in the first half of GAIA2 optimization to 20% in the second half, and pulls occur with 2.0x to 6.9x as many unresolved arms as draws do.
The gains depend on all three components, since removing the failure-pattern arms, the arm prioritizer, or the exploration controller each costs several percentage points of test Pass@1.
A natural next experiment is an oracle substitution: replace the LLM arm scorer with the true realized arm-level performance gains and see how much of the gap over fixed order remains; if the oracle adds little, the curriculum structure, not scorer calibration, is doing the work, whereas a large gap would mean the method leans on subjective judgment that a worse scorer could erase.
Because the four-axis score is a hand-chosen equal-weight combination and the draw/pull call is explicitly made with no fixed formula or threshold, the method's resilience to weaker or cheaper LLM judges is untested; a smaller model or a lower reasoning setting could plausibly collapse the 4.4 and 7.5 point gains even though the bandit structure is sound.
The failure-pattern abstraction presumes weaknesses decouple into independently fixable atomic patterns; nothing in the method validates that premise, so the approach is a testable bet that harness defects do not interact in ways that make composed patterns degenerate into a single unresolved megatarget.
A portability prediction follows from the design: because the curriculum only chooses which scenarios feed a fixed optimizer, it should transfer to any offline harness optimizer that exposes per-batch failure traces; the GEPA result is initial support, and an optimizer with very different update semantics would settle whether the benefit is universal.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Automated harness optimization can substantially improve LLM agents by iteratively updating their prompts, tool interfaces, and control logic from execution feedback. However, existing methods primarily optimize how the harness is updated while largely fixing which training scenarios generate the feedback that drives those updates. As the harness evolves, the scenarios most useful for further optimization can change, suggesting that the training curriculum itself should adapt alongside the harness. We formulate this missing dimension of harness optimization as an automated curriculum learning problem and introduce ActiveSaddler. ActiveSaddler models the evolving curriculum as a non-stationary bandit with dynamically instantiated optimization targets. It abstracts recurring failures into reusable failure-pattern arms, estimates the potential learning progress from further targeting each pattern, and adaptively balances revisiting known weaknesses with exploring unseen scenarios for new ones. Optimization outcomes continually update both the set of discovered failure patterns and their priorities, allowing the curriculum to co-evolve with the harness. Experiments on GAIA2 and Terminal-Bench 2.0 show that ActiveSaddler consistently discovers stronger harnesses, improving test Pass@1 by 4.4 and 7.5 percentage points over the same harness optimizer using a scenario order fixed before optimization, respectively. Ablations further show that these gains depend on dynamically constructing optimization targets, estimating their evolving utility, and balancing continued optimization with new failure discovery. Together, these results establish automated curriculum learning as a new crucial optimization dimension for harness optimization.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-02
The authors' main claim, as the classifier picked it out 1.00
“Together, these results establish automated curriculum learning as a new crucial optimization dimension for harness optimization.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.20 · states limitation 0.11 · human participants 0.03 · controlled comparison 0.95 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.95 · mentions code or data availability 0.03
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Automated harness optimization can substantially improve LLM agents by iteratively updating their prompts, tool interfaces, and control logic from execution feedback. | background 1.00 | 0.09 | 0.04 | asserted 0.57 |
| However, existing methods primarily optimize how the harness is updated while largely fixing which training scenarios generate the feedback that drives those updates. | gap 0.86 | 0.07 | 0.03 | asserted 1.00 |
| As the harness evolves, the scenarios most useful for further optimization can change, suggesting that the training curriculum itself should adapt alongside the harness. | gap 0.70 | 0.10 | 0.06 | hedged possibility 0.99 |
| We formulate this missing dimension of harness optimization as an automated curriculum learning problem and introduce ActiveSaddler. | contribution 1.00 | 0.90 | 0.06 | asserted 0.98 |
| ActiveSaddler models the evolving curriculum as a non-stationary bandit with dynamically instantiated optimization targets. | method 0.96 | 0.88 | 0.04 | asserted 1.00 |
| It abstracts recurring failures into reusable failure-pattern arms, estimates the potential learning progress from further targeting each pattern, and adaptively balances revisiting known weaknesses with exploring unseen scenarios for new ones. | method 0.99 | 0.87 | 0.05 | asserted 1.00 |
| Optimization outcomes continually update both the set of discovered failure patterns and their priorities, allowing the curriculum to co-evolve with the harness. | method 0.97 | 0.79 | 0.10 | asserted 1.00 |
| Experiments on GAIA2 and Terminal-Bench 2.0 show that ActiveSaddler consistently discovers stronger harnesses, improving test Pass@1 by 4.4 and 7.5 percentage points over the same harness optimizer using a scenario order fixed before optimization, respectively. | result 1.00 | 0.96 | 0.24 | asserted 1.00 |
| Ablations further show that these gains depend on dynamically constructing optimization targets, estimating their evolving utility, and balancing continued optimization with new failure discovery. | result 0.74 | 0.95 | 0.08 | asserted 1.00 |
| Together, these results establish automated curriculum learning as a new crucial optimization dimension for harness optimization. | interpretation 1.00 | 0.81 | 0.90 | asserted 1.00 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 1.00
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-02: 1 request, 7,576 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (16)
Figures are the authors', reproduced from the paper.
References (44)
- [1] Pierre. Intrinsic Motivation Systems for Autonomous Mental Development , journal =. 2007 , url =. doi:10.1109/TEVC.2006.890271 , timestamp = on Pithdoi:10.1109/tevc.2006.890271
- [2] Manuel Lopes and Pierre. The strategic student approach for life-long exploration and learning , booktitle =. 2012 , url =. doi:10.1109/DEVLRN.2012.6400807 , timestamp = on Pithdoi:10.1109/devlrn.2012.6400807
- [3] Alex Graves and Marc G. Bellemare and Jacob Menick and R. Automated Curriculum Learning for Neural Networks , booktitle =. 2017 , url = on Pith
- [4] Yifan Yang and Ziyang Gong and Weiquan Huang and Qihao Yang and Ziwei Zhou and Zisu Huang and Yan Li and Xuemei Gao and Qi Dai and Bei Liu and Kai Qiu and Yuqing Yang and Dongdong Chen and Xue Yang and Chong Luo , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2605... on PithPith's reviewarXivdoi:10.48550/arxiv.2605.23904
- [5] Tambet Matiisen and Avital Oliver and Taco Cohen and John Schulman , title =. CoRR , volume =. 2017 , url =. 1707.00183 , timestamp = on PithPith's reviewarXiv
- [6] R. Teacher algorithms for curriculum learning of Deep. 3rd Annual Conference on Robot Learning, CoRL 2019, Osaka, Japan, October 30 - November 1, 2019, Proceedings , series =. 2019 , url = on Pith
- [7] Wenyi Wang and Piotr Piekos and Nanbo Li and Firas Laakom and Yimeng Chen and Mateusz Ostaszewski and Mingchen Zhuge and J. Huxley-G. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2510.21614 , eprinttype =. 2510.21614 , timestamp = on Pithdoi:10.48550/arxiv.2510.21614
- [8] Yizao Wang and Jean. Algorithms for Infinitely Many-Armed Bandits , booktitle =. 2008 , url = on Pith
- [9] Chenyu Zhou and Huacan Chai and Wenteng Chen and Zihan Guo and Rong Shan and Yuanyi Song and Tianyi Xu and Yingxuan Yang and Aofan Yu and Weiming Zhang and Congming Zheng and Jiachen Zhu and Zeyu Zheng and Zhuosheng Zhang and Xingyu Lou and Changwang Zhang and Zhihui Fu and Ju... on PithPith's reviewarXivdoi:10.48550/arxiv.2604.08224
- [10] Effective harnesses for long-running agents , author =. 2025 , howpublished = on Pith
- [11] Harness engineering: leveraging Codex in an agent-first world , author =. 2026 , howpublished = on Pith
- [12] Improving Deep Agents with harness engineering , author =. 2026 , howpublished = on Pith
- [13] Romain Froger and Pierre Andrews and Matteo Bettini and Amar Budhiraja and Ricardo Silveira Cabral and Virginie Do and Emilien Garreau and Jean. Gaia2: Benchmarking. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2602.11964 , eprinttype =. 2602.11964 , timestamp = on Pithdoi:10.48550/arxiv.2602.11964
- [14] Mike A. Merrill and Alexander Glenn Shaw and Nicholas Carlini and Boxuan Li and Harsh Raj and Ivan Bercovich and Lin Shi and Jeong Yeon Shin and Thomas Walshe and Estefany Kelly Buchanan and Junhong Shen and Guanghao Ye and Haowei Lin and Jason Poulos and Maoyu Wang and Marian... on PithPith's reviewarXivdoi:10.48550/arxiv.2601.11868
- [15] Yoonho Lee and Roshen Nair and Qizheng Zhang and Kangwook Lee and Omar Khattab and Chelsea Finn , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2603.28052 , eprinttype =. 2603.28052 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2603.28052
- [16] Sungho Park and Wonjoong Kim and Rongyuan Tan and Jue Zhang and Wook. AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Traces , journal =. 2026 , url =. doi:10.48550/ARXIV.2608.23041 , eprinttype =. 2608.23041 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2608.23041
- [17] Lakshya A. Agrawal and Shangyin Tan and Dilara Soylu and Noah Ziems and Rishi Khare and Krista Opsahl. The Fourteenth International Conference on Learning Representations,. 2026 , url = on Pith
- [18] Jeff Clune , title =. CoRR , volume =. 2019 , url =. 1905.10985 , timestamp = on PithPith's reviewarXiv
- [19] Rui Wang and Joel Lehman and Jeff Clune and Kenneth O. Stanley , title =. CoRR , volume =. 2019 , url =. 1901.01753 , timestamp = on PithPith's reviewarXiv
- [20] Felipe Petroski Such and Aditya Rawal and Joel Lehman and Kenneth O. Stanley and Jeffrey Clune , title =. Proceedings of the 37th International Conference on Machine Learning,. 2020 , url = on Pith
show 24 more
- [21] Richard S. Sutton and Michael H. Bowling and Patrick M. Pilarski , title =. CoRR , volume =. 2022 , url =. doi:10.48550/ARXIV.2208.11173 , eprinttype =. 2208.11173 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2208.11173
- [22] Yoshua Bengio and J. Curriculum learning , booktitle =. 2009 , url =. doi:10.1145/1553374.1553380 , timestamp = on Pithdoi:10.1145/1553374.1553380
- [23] Wojciech Zaremba and Ilya Sutskever , title =. CoRR , volume =. 2014 , url =. 1410.4615 , timestamp = on PithPith's reviewarXiv
- [24] Yuxin Wu and Yuandong Tian , title =. 5th International Conference on Learning Representations,. 2017 , url = on Pith
- [25] Jiahang Lin and Shichun Liu and Chengjun Pan and Lizhi Lin and Shihan Dou and Xuanjing Huang and Hang Yan and Zhenhua Han and Tao Gui , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2604.25850 , eprinttype =. 2604.25850 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2604.25850
- [26] Mengzhuo Chen and Junjie Wang and Zhe Liu and Yawen Wang and Qing Wang , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2606.06324 , eprinttype =. 2606.06324 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2606.06324
- [27] Hangfan Zhang and Shao Zhang and Kangcong Li and Chen Zhang and Yang Chen and Yiqun Zhang and Lei Bai and Shuyue Hu , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2606.09498 , eprinttype =. 2606.09498 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2606.09498
- [28] Biswa Sengupta and Jinhua Wang , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2604.20938 , eprinttype =. 2604.20938 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2604.20938
- [29] Hanghui Guo and Weijie Shi and Zhangze Chen and Shengxiang Xu and Yishu Wang and Yimei Zhang and Wangze Ni and Jia Zhu and Shimin Di , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2607.26722 , eprinttype =. 2607.26722 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2607.26722
- [30] Wenbo Pan and Shujie Liu and Chin. Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference , journal =. 2026 , url =. doi:10.48550/ARXIV.2606.05922 , eprinttype =. 2606.05922 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2606.05922
- [31] Luan Zhang and Ruochen Zhou and Dandan Song and Zhengyu Chen and Yuhang Tian and Jun Yang and Huipeng Ma and Chenhao Li and Guangyuan Feng and Xudong Li and Yizhou Jin and Yan Xu , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2608.01918 , eprinttype =. 2608.01918... on PithPith's reviewarXivdoi:10.48550/arxiv.2608.01918
- [32] Yue Huang and Wenjie Wang and Han Bao and Yuchen Ma and Xiaonan Luo and Yi Nian and Haomin Zhuang and Zheyuan Liu and Yue Zhao and Zhaoxiang Zhang , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2607.14159 , eprinttype =. 2607.14159 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2607.14159
- [33] Prannay Hebbar and Yogendra Manawat and Samuel Verboomen and Alesia Ivanova and Selvam Palanimalai and Kunal Bhatia and Vignesh Baskaran , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2605.27276 , eprinttype =. 2605.27276 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2605.27276
- [34] Zhengyu Chen and Teng Xiao and Huaisheng Zhu and Yige Yuan and Luan Zhang and Jingang Wang , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2607.22688 , eprinttype =. 2607.22688 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2607.22688
- [35] Seth Karten and Joel Zhang and Tersoo Upaa Jr and Ruirong Feng and Wenzhe Li and Chengshuai Shi and Chi Jin and Kiran Vodrahalli , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2605.09998 , eprinttype =. 2605.09998 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2605.09998
- [36] Tianxin Wei and Zhan Shi and Minhua Lin and Bing He and Zewen Liu and Yisi Sang and Yuanchen Bei and Xuying Ning and Jiaru Zou and Ting. Evo-Harness: Context-to-Harness Skill Compilation for Self-Evolving Agents , journal =. 2026 , url =. doi:10.48550/ARXIV.2608.15071 , eprint... on PithPith's reviewarXivdoi:10.48550/arxiv.2608.15071
- [37] Jun Nie and Yonggang Zhang and Jun Song and Qianshu Cai and Dahai Yu and Yike Guo and Xinmei Tian and Bo Han , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2607.08124 , eprinttype =. 2607.08124 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2607.08124
- [38] Minqi Jiang and Edward Grefenstette and Tim Rockt. Prioritized Level Replay , booktitle =. 2021 , url = on Pith
- [39] Shidong Yang and Ziyu Ma and Tongwen Huang and Yiming Hu and Yong Wang and Xiangxiang Chu , editor =. CoEvolve: Training. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),. 2026 , url =. doi:10.18653/V1/2026.ACL-LO... on Pithdoi:10.18653/v1/2026.acl-long.1055
- [40] Atsuyuki Miyai and Kiyoharu Aizawa and Toshihiko Yamasaki , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2608.20169 , eprinttype =. 2608.20169 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2608.20169
- [41] Jinghan Xu and Yikai Zhang and Aili Chen and Weiyuan Li and Jiaqing Liang and Deqing Yang , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2608.27311 , eprinttype =. 2608.27311 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2608.27311
- [42] GitHub Copilot SDK , author=. 2026 , howpublished= on Pith
- [43] WHALE: A Simple Recipe for Joint Harness-Weight Optimization , author=. arXiv preprint arXiv:2609.00196 , year= on PithPith's reviewarXiv
- [44] Terminus-KIRA: Boosting Frontier Model Performance on Terminal-Bench with Minimal Harness , author=. 2026 , howpublished= on Pith
Cited by 2 Pith papers
Reviewed papers on Pith that cite this one. The full citation record
- KnowMe Paper Radar: Lightweight Discovery of Promising Papers in the Age of Autoresearch cs.AI · Nov 2026
- Work Agents: Systems, Capabilities, and Optimization cs.LG · Oct 2026
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment