Pith.

Machine Learning · arXiv:2611.11342 v2 · submitted 8 Nov 2026

Agent Self-Improvement with Frozen Weights: The Added Value and Limits of Failure Traces

Arkid Mitra, Snehil Saluja

The paper says

“We evaluate whether an AI agent's harness, the notes around its model, can learn from the agent's failure traces beyond a matched search that never sees them. It can, but less than it looks.”

more of the abstract

We tested an open 9-billion-parameter model with fixed weights; a larger teacher model wrote its notes. On generated customer-service decisions, where both note writers were told the target rules, notes based on the agent's mistakes improved accuracy by 4.0 percentage points over notes from the same process with the mistakes withheld. That process alone, trying and keeping better-scoring notes, reached about 60% of the total improvement. In full customer-service conversations the same notes showed no demonstrated improvement. Separately, a strict screen of single lessons admitted two customer-service lessons, still awaiting tests on new data, and one tool-use lesson, written without seeing the agent's mistakes, which showed no demonstrated gain on new tool-use tasks. A neutral note with no procedure scored higher than the learned lessons (exploratory). A blind audit judged 146 of 240 tool-use test items defective.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 10 Nov 2026 · of version 1 of the paper; version 2 is newer · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up. Honest, well-controlled empirical study that quantifies how much of agent self-improvement comes from search vs. failure traces—modest claims, solid methods, worth taking seriously. This line is wrong

5 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the references say.

5 minor objections

The paper says, at Abstract

Pith says: minor objection 1 of 5

The phrase 'neutral note with no procedure' is initially unclear; consider specifying earlier that this is a fixed system note with no learning or search procedure, to avoid confusion with the no-evidence search.

The paper says, at Figure 1

Pith says: minor objection 2 of 5

In Figure 1(c), the pass^k curves for the learned arm reflect variation across different lesson libraries per trial, while the standard harness uses one prompt. The caption notes this but the figure itself could use a symbol or annotation to remind readers that the k>1 points are not directly comparable as a measure of agent consistency.

show 3 more

The paper says, at §2, Gate description

Pith says: minor objection 3 of 5

The description of the hierarchical Bayesian model and the three named priors is concise but could benefit from a brief justification of the priors' parameter choices (e.g., the sceptical prior's variance). This is minor because the registered document likely contains full details.

The paper says, at §3, BFCL results

Pith says: minor objection 4 of 5

When reporting the audited estimate for the gate winner (+3.9 [-0.4, +8.3]), the interval includes zero. The text says 'its interval includes zero' implicitly, but stating this explicitly would help the reader interpret the fragility of the result.

The paper says, at §4, Limitations

Pith says: minor objection 5 of 5

The limitation that the teacher replacement was registered before any result is mentioned but could be moved earlier (e.g., to §2) for readers who focus on the experimental design section. This is a minor organizational suggestion.

What would settle it

Re-run the τ2 decision-level comparison with an additional neutral-note control (a fixed non-procedural note) to test whether the trace lessons' advantage over the search is specific to learned content. If the neutral note also outperforms the search, the 4.0-point gain may partly reflect non-specific note effects.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 10 Nov 2026

  1. Reader holds up confidence high
  2. Skeptic agrees 32 s · 10,439 tokens read, 3,642 written
  3. Editor wrote the verdict above 11 s · 11,037 tokens read, 1,094 written
  4. Referee accept: 5 minor 17 s · 10,633 tokens read, 1,758 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

The paper does one thing that makes it worth your time: it runs a matched no-evidence search control that the agent self-improvement literature has been lacking. The headline result—failure traces add about +4 points over the control, but the control alone gets ~60% of the total gain—is clearly stated and honestly bounded. The tau2 result doesn't transfer to full conversations; the BFCL result is swallowed by benchmark defects; the neutral note beats the learned lessons descriptively. None of those are spun.

What's good: pre-registration with hash-chained commits, the gate procedure with per-context non-inferiority, the blind scorer audit reported alongside every BFCL estimate, and the explicit list of what they do not claim. The paper does not oversell. On tau2 decisions they note the writers were told the target rules, so it measures phrasing skill, not rule discovery. On BFCL they report the audit-adjusted intervals and let the primary stand. The discussion is unusually candid about selection noise and score defects.

Soft spots in proportion. The main one the authors own: the no-evidence control only partly isolates the effect of evidence from search because the search trajectories can differ. That's real, but they are transparent about it and the central claim (small added value from failure traces) does not require perfect isolation. The tau2 gate winners were not tested on held-out data and one restates a rule the control also wrote—this is acknowledged but means that part of the pipeline is unvalidated. The BFCL neutral note comparison was exploratory, so the observation that a fixed neutral note outperforms learned sets is suggestive but not confirmatory. The defect rate on BFCL (61%) is high enough that any claim on that benchmark should be read with caution; the paper handles this by reporting both raw and audited estimates and not changing the primary.

Who benefits: anyone working on harness evolution or agent self-improvement, especially people designing evaluations. The matched search control is a methodological contribution that should become standard practice. The results are a useful corrective to work that reports gains from trace-based learning without isolating the search component.

Recommendation: This paper deserves a serious referee. It is well-designed, honestly reported, and the central empirical contribution—the controlled quantification of search vs. evidence—is new and reproducible. I would accept it with minor comments (e.g., consider a neutral note control on tau2 decisions; clarify whether the 'matched' search could be made more tightly matched by tracking the search itself). No fatal weaknesses.

The strongest objection

The paper's central claim—that access to failure traces adds +4.0 points over a matched no-evidence search—is supported by the data, and the authors honestly discuss the limitations of the control. The main limitation (the control does not fully isolate the effect of evidence from search) is acknowledged and does not invalidate the conclusion. The study is well-designed with pre-registration, multiple controls, and blind audits. No internal inconsistency or critical flaw is apparent.

The paper honestly reports its limitations and the central claim is appropriately hedged. The reader's weakest assumption is correctly identified; I concur that it does not undermine acceptance. No new load-bearing concern emerges.

The referee's report

If the central claim holds—that access to failure traces provides a modest, domain-limited benefit over a matched search that never sees failures—the paper makes several methodological contributions: (i) a matched no-evidence control that isolates the effect of evidence (partially), (ii) a registered gate procedure with harm constraints, (iii) a blind scorer audit. These contributions strengthen the evidence base for what failure traces can and cannot do for small open models with frozen weights, and they provide reusable templates for controlled evaluation of agent self-improvement. The results also align with a growing sceptical literature on generalization from failure traces, and the high defect rate on BFCL raises important questions about benchmark reliability for tool-use evaluation.

Pith's reading of the paper (a machine's summary, not the authors')

The paper tests whether a teacher model can improve a frozen agent's notes by reading the agent's mistakes. On generated customer-service decisions, notes written from failures beat notes from a matched search that never saw failures by +4.0 percentage points. However, that search alone achieved about 60% of the total improvement over the standard harness, meaning the search process contributes most of the gain. In full conversations and on held-out tool-use tasks, the failure-derived notes showed no demonstrated improvement. The paper concludes that access to failure traces provides a small additional benefit, but the majority of the improvement comes from the process of searching for better notes, not from the evidence of failures.

What carries the argument. The matched no-evidence search: the same writer, prompt, loop, selection, budget, and development feedback as the trace-based search, but with the agent's episode contents replaced by a fixed sentence. This design isolates the effect of reading failure traces from the effect of the search process itself. The gate uses a hierarchical Bayesian model to screen single lessons for credible gain, per-context non-inferiority within a 1-point margin, and bounded harm. The scorer audit is a blind defect list frozen before outcomes were opened.

Core claim. The paper's central claim is that giving a teacher model access to the agent's failure traces yields a statistically significant but small improvement over a matched search that never sees failures. The improvement of +4.0 percentage points falls below the pre-registered worthwhile threshold of 5 points, and the matched search alone captures about 60% of the total gain over the standard harness. On full conversations and held-out BFCL tasks, no improvement from failure-derived notes is demonstrated, and a neutral note with no procedure scored higher than the learned lessons in a descriptive analysis. The paper also contributes a rigorous experimental protocol including a blind scorer audit and a gate for screening individual lessons, and finds that most BFCL test instances had clear scoring defects.

The premise it rests on. The control search only partly separates evidence from search because the actual search trajectories could differ, even though the budget and development feedback are matched.

If the paper is right (Pith's inferences, not the authors')

Most of the gain from learning from traces in this setup comes from the search process, not the content of failures.

Failure traces provide only a small incremental benefit on generated decisions and none on full conversations or held-out BFCL tasks.

Note effects may be non-specific: a neutral note outperformed both trace-derived and search-derived lessons in descriptive analyses.

Benchmark scorers need blind audits: over 60% of BFCL held-out instances had clear defects.

Screens on a single split do not certify lessons: the admitted lesson's gate gain did not replicate on held-out data.

The matched no-evidence search should become a standard baseline in harness evolution studies, as it separates search from evidence effects.

The neutral note's descriptive advantage suggests that any well-written, task-appropriate note may improve performance, not just those derived from agent traces.

The high defect rate in BFCL (60.8%) raises questions about the reliability of conclusions drawn from that leaderboard.

Future work could test whether the small gain from failure traces persists when the teacher is not given the target rules, isolating rule discovery from rule phrasing.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

We evaluate whether an AI agent's harness, the notes around its model, can learn from the agent's failure traces beyond a matched search that never sees them. It can, but less than it looks. We tested an open 9-billion-parameter model with fixed weights; a larger teacher model wrote its notes. On generated customer-service decisions, where both note writers were told the target rules, notes based on the agent's mistakes improved accuracy by 4.0 percentage points over notes from the same process with the mistakes withheld. That process alone, trying and keeping better-scoring notes, reached about 60% of the total improvement. In full customer-service conversations the same notes showed no demonstrated improvement. Separately, a strict screen of single lessons admitted two customer-service lessons, still awaiting tests on new data, and one tool-use lesson, written without seeing the agent's mistakes, which showed no demonstrated gain on new tool-use tasks. A neutral note with no procedure scored higher than the learned lessons (exploratory). A blind audit judged 146 of 240 tool-use test items defective.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-11-10

The authors' main claim, as the classifier picked it out 1.00

“It can, but less than it looks.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.04 · states limitation 0.97 · human participants 0.06 · controlled comparison 0.96 · randomized assignment 0.03 · states quantitative result 0.99 · compares with prior results 0.91 · mentions code or data availability 0.04

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
We evaluate whether an AI agent's harness, the notes around its model, can learn from the agent's failure traces beyond a matched search that never sees them. contribution 0.63 0.42 0.07 unclear 0.47
It can, but less than it looks. result 0.78 0.79 0.74 asserted 1.00
We tested an open 9-billion-parameter model with fixed weights; a larger teacher model wrote its notes. method 1.00 0.89 0.04 asserted 0.99
On generated customer-service decisions, where both note writers were told the target rules, notes based on the agent's mistakes improved accuracy by 4.0 percentage points over notes from the same process with the mistakes withheld. result 1.00 0.95 0.20 asserted 1.00
That process alone, trying and keeping better-scoring notes, reached about 60% of the total improvement. result 0.94 0.94 0.10 asserted 0.99
In full customer-service conversations the same notes showed no demonstrated improvement. result 0.90 0.93 0.14 asserted 0.99
Separately, a strict screen of single lessons admitted two customer-service lessons, still awaiting tests on new data, and one tool-use lesson, written without seeing the agent's mistakes, which showed no demonstrated gain on new tool-use tasks. result 0.84 0.92 0.06 asserted 0.97
A neutral note with no procedure scored higher than the learned lessons (exploratory). result 0.98 0.86 0.07 asserted 0.86
A blind audit judged 146 of 240 tool-use test items defective. result 0.90 0.82 0.04 asserted 1.00

What kind of paper the classifier reports this is: empirical study 1.00 · evidence from observation or experiment 0.99

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-10: 1 request, 6,784 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (1)
Figure 1 of the paper
Figure 1. (a) Decision-level (“Stage 1”) accuracy on 2,997 sealed 𝜏 2 retail cases: model only (B0), standard harness (B1), no-evidence search (“T+ search control”), trace lessons (“T+ learned library”; T+ is the teacher writer) and a person-written reference. Markers are per-task values for the address, variant and sequence tasks. (b) Wrong writes (write calls that are not exactly the correct one) for the same arms. (c) Official 𝜏 2 retail conversations (“Stage 2”; 114 tasks × 4 trials): pass^𝑘 for the standard harness and the trace lessons. pass^𝑘 is the chance that all 𝑘 trials of a task…

Figures are the authors', reproduced from the paper.

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment