Pith. sign in

REVIEW 3 major objections 3 minor 52 references

The evaluation metric itself can be the evolving object in a self-improving agent loop, and a metric evolved from ten labeled examples beats the bare LLM judge it contains on code generation.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 06:20 UTC pith:GNX4TTMF

load-bearing objection The metric-evolution result is solid, but the 'task score cannot certify a grader' claim is built on a confounded ablation that needs to be redone. the 3 major comments →

arxiv 2607.12790 v2 pith:GNX4TTMF submitted 2026-07-14 cs.AI cs.CLcs.MA

Who Grades the Grader? Co-Evolving Evaluation Metrics and Skills for Self-Improving LLM Agents

Xing Zhang , Guanghui Wang , Yanwei Cui , Ziyuan Li , Wei Qiu , Bing Zhu , Peiyang He This is my paper
classification cs.AI cs.CLcs.MA
keywords self-evolving agentsevaluation metricsLLM judgesevolutionary searchdrawback detectorsanchor setGoodhartco-evolution
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Self-improving agent loops normally assume a reliable evaluation metric already exists; this paper argues that in many real applications none does, and that the metric itself can be the evolving object. The proposed loop searches compositions of small typed drawback detectors under an evolutionary lifecycle, using only ten anchored labeled examples plus consensus over unlabeled outputs as its selection signal. The central claim is that this works: on code generation the evolved metric gains 0.21 agreement with hidden ground truth on a locked set that metric selection never reads (paired p=0.014), beating the bare LLM judge it contains. The authors further show that safety lives in the anchor discipline, not the detector lifecycle: remove the anchor guards and the metric collapses into a vacuous always-pass detector, while removing the lifecycle does not. They conclude that downstream task score cannot validate a self-evolved evaluator, and that an evolved metric suffices to drive a skill loop (Double Ratchet retains 88–110% of the lift that ground truth or a hand-written rubric would buy).

Core claim

The paper's load-bearing claim is that a grading function can be treated as a first-class evolving object in the sparse-anchor regime: no existing evaluator, only ten dev items with golden references and a locked test set that selection never touches. What evolves is an expression tree over typed drawback detectors — each detector checks one failure class (crash, print-instead-of-return, missing group-by, unhedged weak claim) — selected for recall-weighted agreement with the ten-item anchor and regularized by reliability-weighted consensus over unlabeled outputs. The measured result is that the evolved composition is valid: on MBPP+ it reaches 0.625 peak agreement with hidden unit tests vers

What carries the argument

The central object is a metric expression: a tree over atomic 'drawback detectors' (typed checks that each flag one failure class and otherwise abstain), composed with disjunction, conjunction, negation, and weighted votes. The machinery that carries the argument is the selection objective S(e) = A_dev(e) · A_train(e)^w − λC(e): recall-weighted agreement with the ten-item dev anchor (a missed drawback costs twice a false alarm) multiplied by reliability-weighted consensus over unlabeled train outputs (anchor-proven ops lead), minus a complexity penalty, behind a fail-closed gate that rejects candidates with no usable dev opinion and a validity gate that drops all-pass/all-fail/all-abstain ca

Load-bearing premise

The ten hand-picked dev items, soft-labeled by a same-family LLM teacher under a hand-designed protocol, are sufficient and unbiased enough that selection order by anchor agreement orders candidates by true validity — with teacher quality held fixed and anchor-robustness replayed only on the report task, a biased or noisy anchor would silently misdirect the search.

What would settle it

Run the metric loop on MBPP+ with a deliberately corrupted anchor — say, flipping fail/clean soft labels on three of the ten dev items — and check whether held-out agreement on the locked set still rises by +0.21 over the bare judge. If a corrupted anchor still yields the same gain, the anchor discipline is not doing the claimed safety work; if the gain vanishes, the method's dependence on ten golden labels is confirmed and its fragility to label noise is exposed.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A self-improving agent loop can bootstrap its own grader from ten labeled examples; the sparse-anchor regime is sufficient when selection is fail-closed and consensus-regularized.
  • Anchor discipline, not the detector lifecycle, is load-bearing for evaluator safety: the naive ablation collapses into a vacuous always-pass metric, the no-lifecycle ablation does not.
  • Downstream task score cannot validate a self-evolved evaluator: a worthlessly permissive metric trains skills just as well as the working one, so evaluator validity must be checked against a locked reference that metric selection never reads.
  • An evolved metric suffices inside a skill loop: Double Ratchet retains 88–110% of the lift that ground truth or a hand-written rubric buys, and solves held-out problems that the frozen solver never solves.
  • Inspectable metrics enable targeted repair: the rubric-gaming failure was fixed by one vocabulary-aware detector plus hints, with no loss of task score.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Inference: the detectability spectrum implies metric evolution is a moving target — each solver improvement retires mechanically checkable failures and leaves semantic ones, so the value of an evolved metric shifts from agreement toward legibility; triage and repair would remain valuable even where the metric barely beats chance.
  • Inference: because teacher quality is held fixed and anchor-robustness was replayed only on report generation, a direct test of how anchor quality degrades with a weaker teacher — or with the same subsampling replay on MBPP+ and Spider — would determine whether ten golden examples remain sufficient across the whole spectrum.
  • Inference: the Figure-3 dissociation is a proof-of-concept caution for any co-evolved judge or learned reward: if a degenerate reward trains a policy as well as the correct one, downstream performance alone cannot detect reward hacking; the field would need held-out references the judge never reads.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper proposes Double Ratchet, a system that co-evolves an evaluation metric with a self-improving skill loop. The metric is an expression tree over typed drawback detectors, selected by agreement with a ten-item anchored reference set and regularized by consensus on unlabeled outputs; the locked test set is never read by metric selection. The authors report a +0.21 gain in agreement with hidden ground truth on MBPP+, robustness of the gain to a paired test, and ablation results claiming that anchor guards, rather than detector lifecycle, carry the safety load, so that downstream task score cannot certify a self-evolved evaluator. In co-evolution, the evolved metric allegedly retains 88–110% of the lift of ground-truth or rubric-based grading across MBPP+, Spider 2.0-Snow, and report generation.

Significance. If the central claims hold, this is a valuable mechanism study: it demonstrates that an inspectable, composed metric can be evolved from ten labeled examples, with a genuine locked-set validity measurement, and it articulates a measurable dissociation between metric validity and downstream task score. The paper is unusually disciplined: metric selection never reads the locked set, all selection constants are listed in Appendix C, the code is released, item-level verdict counts are reported, and paired tests are given at both peak and final-round columns. The within-task Spider detectability contrast is a strong controlled design. However, the headline negative claim—that task score cannot certify a self-evolved evaluator—is currently confounded by the definition of the naive arm, and the paired significance test is not statistically valid as run. These issues are load-bearing and require revision.

major comments (3)
  1. [§Co-Evolution with the Skill Loop; §Problem Setup] The naive arm is defined as disabling 'fail-closed anchoring, the validity gate, and skill-loop rollback,' yet Problem Setup states that task scores 'remain comparable across arms because every arm rolls back identically against the same set.' These statements are inconsistent. Tables 3–4 and Figure 3 use the co naive arm's equal or higher task scores (e.g., MBPP+ peak 0.742 vs 0.717; Report 0.841 vs 0.812) to support the claim that downstream task score cannot validate an evaluator. That conclusion is not isolated: the naive arm's score could reflect the absence of rollback (retaining regressive skills) rather than the vacuous metric. Please rerun the naive arm with rollback enabled while disabling only the metric-level guards, or alternatively disable rollback in all arms, and report rollback counts for the naive arm.
  2. [§Results, 'The lift survives a paired test'] The exact McNemar test pools three seeds over a single 40-item locked set, so the 120 matched verdicts are not independent observations: the same 40 items appear under all three seeds. The reported p=0.014 is therefore anti-conservative and cannot support the claim that the +0.21 gain is 'beyond seed noise' in a task-level sense. The point estimate is not in question, but the significance claim is load-bearing. Reanalyze with items as clusters (clustered McNemar, item-stratified bootstrap, or a mixed model with item random effects), or state that the paired test only demonstrates within-set grader differences conditional on the chosen 40 items.
  3. [§Roles and models; §Limitations] The dev soft labels are produced by Claude Opus 4.7, the same model family used for the judge ops inside the metric, and for the report task the entire anchor depends on that teacher under a closed taxonomy. The manuscript acknowledges that teacher quality is held fixed and untested. Since the report task is the deployment-like case where no golden metric exists, the paper's 'anchor discipline' safety claim is not yet robust to teacher bias or label noise. Please add a teacher-strength or teacher-family sweep, or an explicit label-noise analysis showing that selection does not silently follow teacher errors. The MBPP+ execution-based validity result is unaffected by this concern.
minor comments (3)
  1. [Abstract] The phrase '+0.21 agreement with hidden ground truth' is ambiguous: the text in §Results shows this is the gain over the round-0 hand-authored seed metric (0.417 to 0.625), not the absolute agreement. State this explicitly.
  2. [Figure 3] The 'hatched bar' is referenced in the caption but is not directly labeled in the figure panel (b). Label the bars or add a legend so the reader can identify the co naive arm without rereading the caption.
  3. [Throughout] The spelling of 'naive' vs. 'naïve' is inconsistent (e.g., 'co naive' in tables and 'naive arm' in text). Use a single spelling consistently.

Circularity Check

0 steps flagged

No significant circularity: metric validity is measured against references selection never reads; the main caveat is a confound, not circularity.

full rationale

The load-bearing validity claim is genuinely out-of-sample. Problem Setup defines dev as 'the only supervised signal any loop reads' and test as 'locked against the metric... no part of metric evolution reads it', and Algorithm 1's Audit step is 'measurement only, never a training signal'. Selection (Eq. 2) uses only dev agreement and train consensus, so the MBPP+ +0.21 / p=0.014 and the Spider/report held-out agreements are measured against hidden unit tests, official execution comparison, or a locked reference that metric search never reads. The self-citations to Ratchet are declared component adoptions ('that published loop is adopted unchanged here as the skill-side component') and the skill loop is actually executed as the reference loop in the paper, so the self-citation is not load-bearing. The paper also flags its main anchor-quality limitations itself ('Teacher quality is held fixed, so how anchor quality degrades with a weaker teacher is untested'). The one serious problem is not circularity: the naive arm is defined in Co-Evolution as disabling 'fail-closed anchoring, the validity gate, and skill-loop rollback', while Problem Setup asserts 'every arm rolls back identically against the same set'; this confounds the task-score-dissociation evidence for 'task score cannot validate a self-evolved evaluator.' That is an experimental-control contradiction, not a prediction that reduces by construction to its inputs, so it does not raise the circularity score.

Axiom & Free-Parameter Ledger

8 free parameters · 7 axioms · 3 invented entities

The central claim rests on the ten-item dev anchor as a sufficient selection signal (with a same-family teacher and hand-set constants), the drawback-detector stance that a clean verdict means 'no known drawback,' the assumption that typed deterministic detectors resist solver-judge collusion, the unchanged Ratchet skill loop, the outer judge's reliability, and the disclosed statistical unit of the paired test. None of these is machine-verified; they are design assumptions with empirical support only in the paper's own ablations. The report task's locked reference is itself teacher-generated from the same model family, so its validity numbers are weaker as external evidence than MBPP+'s execution-based ones.

free parameters (8)
  • dev recall weights w_f=2, w_p=1 = w_f=2, w_p=1
    Hand-set in §The Metric Loop / Appendix C: recall-weighted dev agreement makes a missed drawback cost twice a false alarm. Headline locked-set results depend on this weighting.
  • reliability scale κ=10 = 10
    Appendix C: op consensus votes scaled by 1+10·max(0, m_o). Controls how strongly anchor-proven ops dominate the consensus regularizer; hand-set, not tuned per task.
  • consensus exponent w = 1.0 (MBPP+, Spider); 0.25 (report)
    Appendix C: A_train(e)^w enters Eq. 2 multiplicatively; task-specific hand-set value reflects 'op consensus is a weaker regularizer' for report generation.
  • complexity penalty = 0.001 per op beyond 3-leaf floor; +0.002 per LLM-calling op
    Appendix C: keeps selected expressions small and mostly deterministic; affects which compositions win selection.
  • birth gate thresholds = fire on ≥1/2 of cluster; clean on known-good outputs
    §The Metric Loop: admits 2–4 ops out of ~1000 synthesized (Table 1); the gate, not the search, does most pool curation.
  • report anchor pass threshold = pass iff ≤1 miss class under seven-class taxonomy
    Appendix B: frozen demo-delta soft label defines the report-task locked reference the metric is scored against.
  • ten dev items, five clear failures + five clean positives = 10 items (5/5)
    Problem Setup / Appendix B: deliberate sparse-anchor design choice; anchor-size replay (App. G) tests 4/6/8 items only on the report task.
  • co-evolution curriculum = metric phases 15/8/5/2 rounds between four 25-round skill phases
    Experimental Setup: fixed front-loaded curriculum; the lift-retention numbers depend on this interleaving.
axioms (7)
  • domain assumption Ten-item dev anchor with teacher soft labels is a sufficient, unbiased selection signal for metric validity on the broad distribution.
    Problem Setup: 'dev, tiny and anchored, ten items throughout... the only supervised signal any loop reads.' Teacher degradation is untested (Limitations), and anchor-robustness replay covers only the report task.
  • domain assumption Drawback-detector stance: 'given an output we can usually find drawbacks, so a clean verdict means no known drawback was found, not certified correctness.'
    Introduction / §Metrics as Compositions: underpins representing metrics as detector compositions; the Spider semantic-failure result (0.50 agreement) is the boundary where it fails.
  • domain assumption Mostly-deterministic typed detectors 'fail differently from the LLM being graded' and thereby resist shared-blind-spot collusion.
    Introduction: asserted as the safety rationale for the op-pool design; validity in the semantic regime is bounded by the judge ops inside the composition, which the paper concedes.
  • domain assumption Same-family teacher (Claude Opus 4.7) comparing outputs to golden dev references is not a circular self-grader.
    Experimental Setup / Roles: 'A same-family teacher is not a circular self-grader: it compares candidate outputs against golden references on dev rather than judging unaided, and never sees the locked anchor.' Weakens on Report, where the locked reference itself is teacher-generated.
  • domain assumption The skill loop adopted from Ratchet (Zhang et al. 2026b) behaves as published and is identical across arms.
    Introduction / Related Work: 'that published loop is adopted unchanged here as the skill-side component'; head-to-head comparisons assume the only difference between arms is the grading signal.
  • domain assumption The independent final judge (Claude Opus 4.8) reliably audits quality, and the audit can be fixed when it misreads the format contract.
    §The Goodhart Episode: the generic judge was convention-blind (win rate 0.122→0.126 across the repair); only a task-aware rubric written after reading the generic judge's rationales revealed the repair's effect. The audit therefore needs a correct task contract to be trustworthy.
  • domain assumption Paired McNemar over 3 seeds × 40 locked items (120 seed-item verdicts) licenses a claim about graders, not tasks.
    Results / Limitations: explicitly disclosed; 'its unit is the seed-item verdict, not an independent task.'
invented entities (3)
  • typed drawback detector ops (static/execution/judge cost tiers) with shadow vs active tiers independent evidence
    purpose: Atomic failure-class checks whose compositions form the evolved metric; shadow tier records verdicts without being selectable until anchored evidence appears.
    Falsifiable handles: the naive-arm collapse (all-pass detector) and the locked-set agreements test the design's safety claims; shadow/active distinction is observable in Table 1 and Fig. 6.
  • metric expression grammar over ops (Eq. 1) independent evidence
    purpose: Search space for the evaluator as a readable tree (disjunction/conjunction/negation/votes).
    Reproducible from expression strings plus the registered op pool; selected expressions are inspected verbatim (Fig. 5).
  • Double Ratchet co-evolution loop independent evidence
    purpose: Alternates metric evolution and the Ratchet skill loop with the evolved metric as the trainer.
    Empirical handles: lift-retention numbers (88–110%) and the Goodhart repair are falsifiable artifacts of the loop.

pith-pipeline@v1.3.0-alltime-deepseek · 21359 in / 32394 out tokens · 317427 ms · 2026-08-02T06:20:26.822585+00:00 · methodology

0 comments
read the original abstract

Self-evolving agent systems create, revise, and retire their own skills, but every such loop assumes a reliable evaluation metric already exists. In many real applications none does. We show the metric itself can be the evolving object: our loop searches compositions of small typed drawback detectors under a full evolutionary lifecycle, selecting for agreement with a ten-item anchored reference set and regularizing by consensus over unlabeled outputs. What evolves is the function that grades one output, never the fixed task sets it is scored on, and what comes out is an inspectable expression rather than an opaque judge. It is also valid: on code generation it gains 0.21 agreement with hidden ground truth on a locked set that metric selection never reads (paired $p=0.014$), beating the bare LLM judge it contains. Validity is where safety lives: removing the anchor guards collapses the metric into a vacuous always-pass detector while removing the detector lifecycle does not, inverting the lesson from skill evolution. That collapse warns this line of work that downstream task score cannot validate a self-evolved evaluator, since the collapsed metric trains skills just as well. Task score answers only sufficiency, and an evolved metric suffices: \emph{Double Ratchet}, co-evolving the metric with a lifecycle-managed skill loop, retains 88--110\% of the lift ground truth or a hand-written rubric buys, across MBPP+, Spider~2.0-Snow, and report generation. When evolved skills gamed the report rubric, an independent judge caught it and one added detector repaired it.

Figures

Figures reproduced from arXiv: 2607.12790 by Bing Zhu, Guanghui Wang, Peiyang He, Wei Qiu, Xing Zhang, Yanwei Cui, Ziyuan Li.

Figure 1
Figure 1. Figure 1: The evaluation metric is the evolving object; the task sets are fixed and never altered. Center: from a frozen solver, no metric, and ten anchored items (left), the metric loop synthesizes typed drawback detectors behind a birth gate, composes them into an expression selected by anchor agreement and consensus, and curates the pool by marginal contribution. What comes out is a readable expression, not a lea… view at source ↗
Figure 2
Figure 2. Figure 2: An evolved metric buys most of the lift ground truth would have, and what evolves stays readable. (a) Lift Double Ratchet retains as a fraction of the reference loop’s; 100% means the evolved metric bought everything ground truth would have. (b) Locked-set learning curves (mean ± std, three seeds): reference loop orange, Double Ratchet blue, frozen solver dotted. (c) Every distinct metric the MBPP+ loop se… view at source ↗
Figure 3
Figure 3. Figure 3: Metric validity and downstream task score dissociate, so task score cannot certify a self-evolved grader. The same three report-generation arms, measured two ways. (a) Metric validity on an absolute 0-to-1 scale against a locked reference no metric loop reads: the arms separate sharply, and the naive arm is vacuous rather than merely worse. (b) The task score of the skill loop each grader trained, on a zoo… view at source ↗
Figure 4
Figure 4. Figure 4: Whether an evolved metric transfers is governed by how mechanically detectable the solver’s failures are. Top: the three task families on the axis from mechanically checkable failures to purely semantic ones, each with its evolved metric’s peak held-out agreement. Those three numbers sit on three different references, so the placement is an interpretation, not a measurement. Middle, the measurement: a with… view at source ↗
Figure 5
Figure 5. Figure 5: The final evolved metric on each task, verbatim, with a plain reading above it. Each is a short drawback disjunction over typed detectors: nothing here is a learned scalar, and each leaf names the failure class it fires on. A synthesized op born, gated, promoted, and selected (Spider 2.0) Tracing one op through the whole lifecycle, on the metric-loop seed that ends at (any syn_forbids_select_star_when_colu… view at source ↗
Figure 6
Figure 6. Figure 6: One synthesized detector through the whole lifecycle: born from unlabeled outputs, admitted as a shadow, promoted only once it raised anchor agreement, then selected. A Spider 2.0 metric-loop seed; the aggregate counts are [PITH_FULL_IMAGE:figures/full_fig_p016_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The same verdict, reached two ways: the hidden tests say fail, and the evolved metric says fail while naming why. One MBPP+ attempt whose code runs cleanly and returns a number but computes the wrong quantity. Only the middle column is ground truth; the right column is what an inspectable composition adds on top of an equally correct verdict. Same lesson, two loops functionally matched skills, reference vs… view at source ↗
Figure 8
Figure 8. Figure 8: Trained on ground truth and trained on the evolved metric, the two loops independently learn the same task lesson. One functionally matched skill pair per task, condensed. Neither loop saw the other, so the convergence is the qualitative face of the lift-retention result in [PITH_FULL_IMAGE:figures/full_fig_p017_8.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

52 extracted references · 18 linked inside Pith

  1. [1]

    Amodei, D.; Olah, C.; Steinhardt, J.; Christiano, P.; Schulman, J.; and Man \'e , D. 2016. Concrete Problems in AI Safety. arXiv preprint arXiv:1606.06565

    Pith/arXiv arXiv 2016

  2. [2]

    Anthropic . 2026. New in Claude Managed Agents: Dreaming, Outcomes, and Multiagent Orchestration. https://claude.com/blog/new-in-claude-managed-agents. Product announcement, May 19, 2026

    2026

  3. [3]

    Austin, J.; Odena, A.; Nye, M.; Bosma, M.; Michalewski, H.; Dohan, D.; Jiang, E.; Cai, C.; Terry, M.; Le, Q.; and Sutton, C. 2021. Program Synthesis with Large Language Models. arXiv preprint arXiv:2108.07732

    Pith/arXiv arXiv 2021

  4. [4]

    Bai, Y.; Kadavath, S.; Kundu, S.; Askell, A.; Kernion, J.; Jones, A.; Chen, A.; Goldie, A.; Mirhoseini, A.; McKinnon, C.; et al. 2022. Constitutional AI : Harmlessness from AI Feedback. arXiv preprint arXiv:2212.08073

    Pith/arXiv arXiv 2022

  5. [5]

    H.; Chen, S.; Liu, Z.; Jiang, F.; and Wang, B

    Chen, G. H.; Chen, S.; Liu, Z.; Jiang, F.; and Wang, B. 2024 a . Humans or LLMs as the Judge? A Study on Judgement Bias. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP)

    2024

  6. [6]

    Chen, M.; Li, Y.; Yang, Y.; Yu, S.; Lin, B.; and He, X. 2024 b . AutoManual : Generating Instruction Manuals by LLM Agents via Interactive Environmental Learning. In Advances in Neural Information Processing Systems, volume 37

    2024

  7. [7]

    Chen, M.; Tworek, J.; Jun, H.; Yuan, Q.; Pinto, H. P. d. O.; Kaplan, J.; Edwards, H.; Burda, Y.; Joseph, N.; Brockman, G.; et al. 2021. Evaluating Large Language Models Trained on Code. arXiv preprint arXiv:2107.03374

    Pith/arXiv arXiv 2021

  8. [8]

    Y.; Cheung, S

    Chen, T. Y.; Cheung, S. C.; and Yiu, S. M. 1998. Metamorphic Testing: A New Approach for Generating Next Test Cases. Technical Report HKUST-CS98-01, Hong Kong University of Science and Technology

    1998

  9. [9]

    Deng, M.; Ramachandran, A.; Xu, C.; Hu, L.; Yao, Z.; Datta, A.; and Zhang, H. 2025. ReFoRCE : A Text-to- SQL Agent with Self-Refinement, Consensus Enforcement, and Column Exploration. arXiv preprint arXiv:2502.00675

    Pith/arXiv arXiv 2025

  10. [10]

    Dong, P.; Li, L.; Tang, Z.; Liu, X.; Pan, X.; Wang, Q.; and Chu, X. 2024 a . Pruner-Zero: Evolving Symbolic Pruning Metric from Scratch for Large Language Models. In Proceedings of the 41st International Conference on Machine Learning (ICML)

    2024

  11. [11]

    Dong, Y.; Ding, J.; Jiang, X.; Li, G.; Li, Z.; and Jin, Z. 2024 b . CodeScore : Evaluating Code Generation by Learning Code Execution. ACM Transactions on Software Engineering and Methodology

    2024

  12. [12]

    Dubois, Y.; Li, X.; Taori, R.; Zhang, T.; Gulrajani, I.; Ba, J.; Guestrin, C.; Liang, P.; and Hashimoto, T. B. 2023. AlpacaFarm : A Simulation Framework for Methods that Learn from Human Feedback. Advances in Neural Information Processing Systems, 36

    2023

  13. [13]

    H.; and Hutter, F

    Elsken, T.; Metzen, J. H.; and Hutter, F. 2019. Neural Architecture Search: A Survey. Journal of Machine Learning Research, 20(55): 1--21

    2019

  14. [14]

    Fernando, C.; Banarse, D.; Michalewski, H.; Osindero, S.; and Rockt \"a schel, T. 2024. Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution. In Proceedings of the 41st International Conference on Machine Learning (ICML)

    2024

  15. [15]

    Gao, L.; Schulman, J.; and Hilton, J. 2023. Scaling Laws for Reward Model Overoptimization. In Proceedings of the 40th International Conference on Machine Learning (ICML), 10835--10866

    2023

  16. [16]

    Goodhart, C. A. E. 1984. Problems of Monetary Management: The UK Experience. Monetary Theory and Practice: The UK Experience, 91--121

    1984

  17. [17]

    Gu, J.; Jiang, X.; Shi, Z.; Tan, H.; Zhai, X.; Xu, C.; Li, W.; Shen, Y.; Ma, S.; Liu, H.; Wang, S.; Zhang, K.; Wang, Y.; Gao, W.; Ni, L.; and Guo, J. 2024. A Survey on LLM -as-a-Judge. arXiv preprint arXiv:2411.15594

    Pith/arXiv arXiv 2024

  18. [18]

    Hu, S.; Lu, C.; and Clune, J. 2025. Automated Design of Agentic Systems. In International Conference on Learning Representations (ICLR)

    2025

  19. [19]

    T.; Moazam, H.; Miller, H.; Zaharia, M.; and Potts, C

    Khattab, O.; Singhvi, A.; Maheshwari, P.; Zhang, Z.; Santhanam, K.; Vardhamanan, S.; Haq, S.; Sharma, A.; Joshi, T. T.; Moazam, H.; Miller, H.; Zaharia, M.; and Potts, C. 2024. DSPy : Compiling Declarative Language Model Calls into Self-Improving Pipelines. In International Conference on Learning Representations (ICLR)

    2024

  20. [20]

    Lei, F.; Chen, J.; Ye, Y.; Cao, R.; Shin, D.; Su, H.; Suo, Z.; Gao, H.; Hu, W.; Yin, P.; Zhong, V.; Xiong, C.; Sun, R.; Liu, Q.; Wang, S.; and Yu, T. 2025. Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to- SQL Workflows. In International Conference on Learning Representations (ICLR)

    2025

  21. [21]

    Li, X.; Chen, W.; Liu, Y.; Zheng, S.; Chen, X.; He, Y.; Li, Y.; You, B.; Shen, H.; Sun, J.; et al. 2026. SkillsBench : Benchmarking how well agent skills work across diverse tasks. arXiv preprint arXiv:2602.12670

    Pith/arXiv arXiv 2026

  22. [22]

    S.; Wang, Y.; and Zhang, L

    Liu, J.; Xia, C. S.; Wang, Y.; and Zhang, L. 2023. Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation. In Advances in Neural Information Processing Systems, volume 36

    2023

  23. [23]

    Liu, P.; Zhang, G.; Wang, B.; Xu, H.; Liang, X.; Jiang, Y.; and Li, Z. 2021. Loss Function Discovery for Object Detection via Convergence-Simulation Driven Search. In International Conference on Learning Representations (ICLR)

    2021

  24. [24]

    Madaan, A.; Tandon, N.; Gupta, P.; Hallinan, S.; Gao, L.; Wiegreffe, S.; Alon, U.; Dziri, N.; Prabhumoye, S.; Yang, Y.; et al. 2023. Self-Refine: Iterative Refinement with Self-Feedback. Advances in Neural Information Processing Systems, 36

    2023

  25. [25]

    Y.; Harada, D.; and Russell, S

    Ng, A. Y.; Harada, D.; and Russell, S. 1999. Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping. In Proceedings of the 16th International Conference on Machine Learning (ICML)

    1999

  26. [26]

    Ni, J.; Liu, Y.; Liu, X.; Sun, Y.; Zhou, M.; Cheng, P.; Wang, D.; Jiang, X.; and Jiang, G. 2026. Trace2Skill : Parallel Inductive Skill Distillation for LLM Agents. arXiv preprint arXiv:2603.25158

    Pith/arXiv arXiv 2026

  27. [27]

    Z.; Shirobokov, S.; Kozlovskii, B.; Ruiz, F

    Novikov, A.; V \ u , N.; Eisenberger, M.; Dupont, E.; Huang, P.-S.; Wagner, A. Z.; Shirobokov, S.; Kozlovskii, B.; Ruiz, F. J. R.; Mehrabian, A.; Kumar, M. P.; See, A.; Chaudhuri, S.; Holland, G.; Davies, A.; Nowozin, S.; Kohli, P.; and Balog, M. 2025. AlphaEvolve : A Coding Agent for Scientific and Algorithmic Discovery. arXiv preprint arXiv:2506.13131

    Pith/arXiv arXiv 2025

  28. [28]

    OpenAI . 2026. Codex: Record & Replay. https://developers.openai.com/codex/record-and-replay. Developer documentation

    2026

  29. [29]

    Ouyang, L.; Wu, J.; Jiang, X.; Almeida, D.; Wainwright, C.; Mishkin, P.; Zhang, C.; Agarwal, S.; Slama, K.; Ray, A.; Schulman, J.; Hilton, J.; Kelton, F.; Miller, L.; Simens, M.; Askell, A.; Welinder, P.; Christiano, P.; Leike, J.; and Lowe, R. 2022. Training Language Models to Follow Instructions with Human Feedback. Advances in Neural Information Proces...

    2022

  30. [30]

    Pan, A.; Bhatia, K.; and Steinhardt, J. 2022. The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models. In International Conference on Learning Representations (ICLR)

    2022

  31. [31]

    S.; O'Brien, J

    Park, J. S.; O'Brien, J. C.; Cai, C. J.; Morris, M. R.; Liang, P.; and Bernstein, M. S. 2023. Generative Agents: Interactive Simulacra of Human Behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology

    2023

  32. [32]

    Real, E.; Liang, C.; So, D.; and Le, Q. 2020. AutoML-Zero : Evolving Machine Learning Algorithms From Scratch. In Proceedings of the 37th International Conference on Machine Learning (ICML), 8007--8019

    2020

  33. [33]

    Shinn, N.; Cassano, F.; Gopinath, A.; Narasimhan, K.; and Yao, S. 2023. Reflexion: Language Agents with Verbal Reinforcement Learning. In Advances in Neural Information Processing Systems, volume 36

    2023

  34. [34]

    Skalse, J.; Howe, N.; Krasheninnikov, D.; and Krueger, D. 2022. Defining and Characterizing Reward Gaming. Advances in Neural Information Processing Systems, 35: 9460--9471

    2022

  35. [35]

    Wang, G.; Xie, Y.; Jiang, Y.; Mandlekar, A.; Xiao, C.; Zhu, Y.; Fan, L.; and Anandkumar, A. 2024 a . Voyager: An Open-Ended Embodied Agent with Large Language Models. Transactions on Machine Learning Research

    2024

  36. [36]

    Wang, J.; Ren, Y.; and Zhang, H. 2026. From Procedural Skills to Strategy Genes: Towards Experience-Driven Test-Time Evolution. arXiv preprint arXiv:2604.15097

    Pith/arXiv arXiv 2026

  37. [37]

    X.; Wei, Z.; and Wen, J

    Wang, L.; Ma, C.; Feng, X.; Zhang, Z.; Yang, H.; Zhang, J.; Chen, Z.; Tang, J.; Chen, X.; Lin, Y.; Zhao, W. X.; Wei, Z.; and Wen, J. 2024 b . A Comprehensive Survey of LLM -Based Autonomous Agents. Frontiers of Computer Science, 18(6)

    2024

  38. [38]

    Z.; Mao, J.; Fried, D.; and Neubig, G

    Wang, Z. Z.; Mao, J.; Fried, D.; and Neubig, G. 2024 c . Agent Workflow Memory. arXiv preprint arXiv:2409.07429

    Pith/arXiv arXiv 2024

  39. [39]

    Wu, R.; Wang, X.; Mei, J.; Cai, P.; Fu, D.; Yang, C.; Wen, L.; Yang, X.; Shen, Y.; Wang, Y.; et al. 2025. Self-Evolving LLM Agents through an Experience-Driven Lifecycle. arXiv preprint arXiv:2510.16079

    Pith/arXiv arXiv 2025

  40. [40]

    Xia, P.; Chen, J.; Wang, H.; Liu, J.; Zeng, K.; Wang, Y.; Han, S.; Zhou, Y.; Zhao, X.; Chen, H.; et al. 2026. SkillRL : Evolving Agents via Recursive Skill-Augmented Reinforcement Learning. arXiv preprint arXiv:2602.08234

    Pith/arXiv arXiv 2026

  41. [41]

    Yang, B.; Xia, Y.; Sun, W.; and Liu, Y. 2025. Hallucination Detection for LLM -based Text-to- SQL Generation via Two-Stage Metamorphic Testing. arXiv preprint arXiv:2512.22250

    arXiv 2025

  42. [42]

    V.; Zhou, D.; and Chen, X

    Yang, C.; Wang, X.; Lu, Y.; Liu, H.; Le, Q. V.; Zhou, D.; and Chen, X. 2024. Large Language Models as Optimizers. In International Conference on Learning Representations

    2024

  43. [43]

    Yang, Y.; Li, J.; Pan, Q.; Zhan, B.; Cai, Y.; Du, L.; Zhou, J.; Chen, K.; Chen, Q.; Li, X.; et al. 2026. AutoSkill : Experience-Driven Lifelong Learning via Skill Self-Evolution. arXiv preprint arXiv:2603.01145

    arXiv 2026

  44. [44]

    Y.; Cho, K.; Li, X.; Sukhbaatar, S.; Xu, J.; and Weston, J

    Yuan, W.; Pang, R. Y.; Cho, K.; Li, X.; Sukhbaatar, S.; Xu, J.; and Weston, J. 2024. Self-Rewarding Language Models. In Proceedings of the 41st International Conference on Machine Learning (ICML)

    2024

  45. [45]

    Yuksekgonul, M.; Bianchi, F.; Boen, J.; Liu, S.; Huang, Z.; Guestrin, C.; and Zou, J. 2024. TextGrad : Automatic ``Differentiation'' via Text. arXiv preprint arXiv:2406.07496

    Pith/arXiv arXiv 2024

  46. [46]

    Zhang, J.; Qu, C.; Fang, J.; et al. 2025. A Survey of Self-Evolving Agents: On Path to Artificial Super Intelligence. arXiv preprint arXiv:2507.21046

    Pith/arXiv arXiv 2025

  47. [47]

    Zhang, X.; Cui, Y.; Wang, G.; Li, Z.; Qiu, W.; Zhu, B.; and He, P. 2026 a . Library Drift: Diagnosing and Fixing a Silent Failure Mode in Self-Evolving LLM Skill Libraries. In ICML 2026 Workshop on Failure Modes in Agentic AI

    2026

  48. [48]

    Zhang, X.; Cui, Y.; Wang, G.; Li, Z.; Qiu, W.; Zhu, B.; and He, P. 2026 b . Ratchet: A Minimal Hygiene Recipe for Self-Evolving LLM Agents. arXiv preprint arXiv:2605.22148

    Pith/arXiv arXiv 2026

  49. [49]

    Zhang, X.; Wang, G.; Cui, Y.; Qiu, W.; Li, Z.; Zhu, B.; and He, P. 2026 c . Experience Compression Spectrum: Unifying Memory, Skills, and Rules in LLM Agents. arXiv preprint arXiv:2604.15877

    Pith/arXiv arXiv 2026

  50. [50]

    Zhao, A.; Huang, D.; Xu, Q.; Lin, M.; Liu, Y.-J.; and Huang, G. 2024. ExpeL : LLM Agents Are Experiential Learners. In Proceedings of the AAAI Conference on Artificial Intelligence

    2024

  51. [51]

    P.; Zhang, H.; Gonzalez, J

    Zheng, L.; Chiang, W.-L.; Sheng, Y.; Zhuang, S.; Wu, Z.; Zhuang, Y.; Lin, Z.; Li, Z.; Li, D.; Xing, E. P.; Zhang, H.; Gonzalez, J. E.; and Stoica, I. 2023. Judging LLM -as-a-Judge with MT -Bench and Chatbot Arena. Advances in Neural Information Processing Systems, 36

    2023

  52. [52]

    Zhuge, M.; Zhao, C.; Ashley, D.; Wang, W.; Khizbullin, D.; Xiong, Y.; Liu, Z.; Chang, E.; Krishnamoorthi, R.; Tian, Y.; Shi, Y.; Chandra, V.; and Schmidhuber, J. 2024. Agent-as-a-Judge: Evaluate Agents with Agents. arXiv preprint arXiv:2410.10934

    Pith/arXiv arXiv 2024