Pith. sign in

REVIEW 3 major objections 5 minor 201 references

Self-improvement can be driven primarily by a curated knowledge base rather than by the agent itself.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-01 12:17 UTC pith:HRBEWU5X

load-bearing objection A plausible, transparent, and genuinely different mechanism for self-improvement via curated knowledge, but the main-loop causal claim is not yet supported — the held-out transfer results are the cleanest evidence in the paper. the 3 major comments →

arxiv 2607.19592 v1 pith:HRBEWU5X submitted 2026-07-21 cs.AI cs.CLcs.LGcs.MA

Knowledge-Centric Self-Improvement

Xuefei Julie Wang , Lauren Hyoseo Yoon , Chengrui Qu , Amanda Zichang Wang , Atharva Sehgal , Eric Mazumdar , Yisong Yue This is my paper
classification cs.AI cs.CLcs.LGcs.MA
keywords knowledge-centric self-improvementknowledge base curationmulti-agent forumsknowledge distillationdisposable agentszero-shot transferLLM generalizationevidence-grounded claims
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that the object of self-improvement in an AI system can be a shared, curated knowledge base instead of the agent. Its protocol keeps agents generic, stateless, and disposable: each reads distilled guidance, attempts one task, and writes evidence-grounded claims into task-level and cross-task forums, which are then distilled back into the knowledge base. Because every agent starts from a fresh context, any gain across generations is attributed to the knowledge they received. Across abstract-reasoning, coding, and terminal benchmarks, the paper reports higher solve rates and lower dollar costs than agent-centric self-improvement baselines, and it shows the distilled knowledge transfers to held-out tasks and across LLM families. This would move the bottleneck in autonomous reasoning from agent architecture to the quality and structure of the curated information agents consume.

Core claim

The paper's central claim is that a system can improve at tasks without ever improving its agents, and that the persistent improving object should be a curated knowledge base. Agents are kept generic, stateless, and disposable: each one starts with a clean context, receives a distilled knowledge bundle, attempts a task, and contributes evidence back. A three-stage protocol consisting of task-level forum discussion, cross-task forum debate, and distillation converts individual attempts into scoped, evidence-grounded guidance. On the evaluated benchmarks, the paper reports that this protocol outperforms agent-centric self-improvement baselines in solve rate while using less money, that the dis

What carries the argument

The central object is the curated knowledge base shaped by a three-stage protocol: task-level forums, where agents post evidence-grounded claims about what worked and failed; cross-task forums, where claims must take explicit agree, disagree, or synthesize stances toward cited posts and survive only when grounded in concrete evidence; and distillation, which compresses the surviving claims into typed, actionable bundles—'when X, do Y'—that the next generation of fresh agents consumes. The mechanism isolates knowledge as the only changing variable: every agent is re-instantiated with fresh context and no private memory, so improvement across generations is attributed to the knowledge base.

Load-bearing premise

The central claim rests on the assumption that every generation-over-generation gain comes from the knowledge base, because agents are re-instantiated with fresh context; the paper includes no control that retries unsolved tasks for the same number of generations without seed knowledge, so stochastic retrying alone could account for part of the improvement.

What would settle it

Run the same 10-generation loop with a no-knowledge control: fresh agents with identical prompts, tools, and retry budget but an empty or sham seed knowledge base. If this control closes the gap with the knowledge-seeded solve rates on any benchmark, the claimed attribution to curated knowledge fails. On the transfer side, replacing the frozen distilled bundle with task-irrelevant text should recover the no-knowledge baseline if the bundle is the true carrier of value.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X LinkedIn Reddit HN

If this is right

  • Self-improvement no longer requires a persistent, specialized agent; generic disposable agents suffice, shifting the bottleneck from agent architecture to the curated knowledge they consume.
  • Solve-rate gains and lower dollar costs on the evaluated benchmarks imply self-improvement can be made cheaper to operate and easier to maintain.
  • Because distilled knowledge transfers across LLM families, knowledge produced by one model can be consumed by another, making the improvement portable.
  • Frozen knowledge improves zero-shot performance on held-out tasks, so the curated asset has standalone value after the run ends.
  • Disagreement in forums is preserved as evidence rather than resolved, which may prevent premature convergence on wrong hypotheses.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A direct extension of the paper's design is a no-knowledge retry control that re-attempts unsolved tasks for the same number of generations with empty seed knowledge; the paper does not report one, so the size of the pure retrying component is untested.
  • If the knowledge-centric paradigm scales, the cost of self-improvement shifts from inference-time agent complexity to offline curation, so knowledge can be amortized across many tasks and models.
  • The protocol's practice of preserving unresolved disagreements as falsified-versus-untried hypotheses suggests a general anti-premature-consensus mechanism for multi-agent systems.
  • A natural extension, not studied in the paper, is to fine-tune a base model on the distilled bundles, turning the reusable knowledge artifact into a direct weight update.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes knowledge-centric self-improvement: instead of improving the agent, the system maintains a persistent, curated knowledge base that fresh, stateless agents read from and write to. A three-stage protocol (task-level forums, cross-task forums, and distillation) converts agent attempts into typed, evidence-grounded bundles. The authors evaluate this protocol on ARC-AGI-1/2, Polyglot, SWE-bench Pro, and Terminal-Bench 2, comparing against agent-centric self-improvement baselines (DGM, HyperAgents, Meta-Harness) and prompt-optimization baselines (GEPA, OpenEvolve). They report higher or competitive solve rates at lower dollar cost, cross-LLM effectiveness (Haiku 4.5 and GPT-5.4-mini), and positive transfer of a frozen knowledge asset to held-out tasks and to a different LLM family. The central claim is that progress in self-improving agentic systems can be driven primarily by the curated persistent knowledge rather than by the agent design.

Significance. If the central claim is sustained, the paper makes a useful conceptual and practical contribution: it identifies a design axis (what persists) that is complementary to the usual agent-centric optimization, and it provides a cheap, inspectable, and transferable improvement artifact. The paper is unusually transparent: it ships detailed protocol schemas (Appendix E), full task maps (Appendix J), cost accounting (Appendix G), baseline provenance/deviation documentation (Appendix H), and worked examples of disagreement-driven curation (Appendix C). The held-out transfer experiment is a genuine out-of-sample test with a frozen knowledge asset and disjoint task splits, and the cross-LLM transfer is a concrete falsifiable prediction. These strengths are real. However, the evidence currently underdetermines the central attribution claim because the main self-improvement loop lacks a no-knowledge control, and the baseline comparisons are against modified forks with information-parity gates rather than the original published systems.

major comments (3)
  1. [§3, §4.1, Table 1] The paper's central attribution claim—"any performance gain across generations must come from the accumulated and distilled knowledge" (§3)—is not established by the reported experiments. Section 4.1 runs 10 generations on a 50-task pool, but there is no control condition that repeats the same fresh-agent retry schedule without any knowledge base. Because solved tasks are removed and unsolved tasks are retried by new stochastic agents, cumulative chance success across 10 attempts can be substantial; the paper's own Table 4 shows single-generation no-knowledge solve rates of 13–23% on held-out ARC-AGI-1 tasks. A no-knowledge generation-loop control (same number of generations, same per-task budget, no forums, no distillation, no knowledge injection) on the same 50-task pool is necessary to separate retrying/stochastic effects from knowledge-driven gains. The held-out transfer experiment (
  2. [§4.1, Appendix H] Table 1a compares against DGM and HyperAgents, but the reported baseline numbers come from forks with substantial modifications: LLM substitution, information-parity gates that remove gold answers and hidden tests from the self-improvement loop, egress isolation, and newly authored benchmark adapters. The claim that "all baselines are rerun under our evaluation protocol" is transparent, but the claim in the abstract and Section 4.1 that the method "outperforms agent-centric baselines" should be qualified as outperforming the authors' modified, information-parity-gated re-runs, not the original published systems. Whether the parity gates preserve the "core optimization algorithm" is an assumption, not a demonstrated fact; for example, DGM's diagnosis loop is described in Appendix H as no longer seeing gold patches and hidden tests, which may be integral to its original self-modification m
  3. [§4.4, Appendix L, Appendix J] The transfer evidence is weakened by post-hoc design choices that are disclosed but not tested. First, the task-conditioned adapter's item bounds were relaxed after observing benchmark variation (Appendix L: "we relaxed the constraints and allowed the agent to dynamically determine how much knowledge to transfer"), making the adapter a tuned component rather than a fixed property of the frozen knowledge asset. Second, the transfer evaluation set was selected from tasks that both recipient LLMs fail at seed 0 (Appendix J), which inflates headroom and makes the no-knowledge baseline low by construction; this is disclosed, but it means the transfer numbers are conditional on a selection procedure that depends on the recipients' no-knowledge abilities. Third, for Polyglot the transfer split is task-identifier-disjoint but not exercise-disjoint: 7 of 20 evaluation tasks are the same Exercism
minor comments (5)
  1. [§1] Typographical: "in thisknowledge-centric self-improvementparadigm" is missing spaces around "knowledge-centric" and "paradigm."
  2. [Table 1/Table 3] The table header "OURShaiku 4.5" and "OURSgpt 5.4−mini" are visually cramped; please add spacing or a line break for readability.
  3. [Figure 3b] The example labels contain apparent typos ("Count-Dataset-T okens", "T ask-level Insights"). Please proofread figure text.
  4. [§4.4] The caption for Table 4 says "20 held-out tasks" but the selection procedure in Appendix J says tasks were chosen from the intersection of both recipients' failure sets; please make the selection criterion explicit in the main text so readers do not interpret the held-out set as a random sample of all held-out tasks.
  5. [§4.2] The comparison with GEPA and OpenEvolve uses matched dollar budgets but different iteration counts; it would be helpful to state explicitly that the prompt-optimization baselines were not extended to other benchmarks for budget reasons, and to note that this comparison is therefore narrower than the agent-centric comparison.

Circularity Check

0 steps flagged

No significant circularity: benchmark comparisons and held-out transfer are externally grounded; the causal-attribution gap is a missing control, not a self-referential derivation.

full rationale

The solve-rate claims in Table 1 rest on external benchmark outcomes (official ARC exact-match scoring, SWE-bench Pro harness, Polyglot tests, Terminal-Bench 2 verifier), and the reported costs cover all loop phases, so the headline comparison is not a rescaling of the authors' own definitions. The held-out transfer experiment (Section 4.4) is a genuine out-of-sample test: the generation-10 asset is frozen and consumed zero-shot by fresh recipients on disjoint 20-task splits with no recipient-side forum or distillation, and the comparison is against a no-knowledge condition on the same tasks. No parameter is fitted to a subset and then reported as a prediction of a closely related quantity. The paper contains no load-bearing self-citation and no imported uniqueness theorem; its references are independent prior systems, and no argument's force comes from an author-overlapping citation. Two caveats should be weighed as validity concerns rather than circularity. First, Section 3 asserts 'any performance gain across generations must come from the accumulated and distilled knowledge they receive' merely from re-instantiating agents; without a no-knowledge generation control that retries unsolved tasks for the same number of generations, cumulative stochastic retrying is a confound, but this is an omitted control rather than an input-output identity. Second, Appendix J admits the transfer tasks were selected from a seed-0 no-knowledge failure intersection ('the no-knowledge baseline solve rates are low by construction') and that 7 of 20 Polyglot transfer tasks are the same Exercism exercise in a different language; Section L similarly notes the transfer adapter was relaxed after observing benchmark variation. These limit the strength of the transfer claims but do not make the improvement definitionally equal to the knowledge base's own content. I therefore find no significant circularity.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 0 invented entities

The central claim rests on protocol hyperparameters, a causal attribution assumption, and the validity of modified baseline forks. No new physical or mathematical entities are introduced; the knowledge base, forums, and bundles are software artifacts.

free parameters (5)
  • generations = 10
    Hand-set improvement budget; final solve rates and costs are reported at this stopping point; no ablation shows sensitivity to this choice.
  • forum_rounds = 1 per-task forum round; 2 cross-task forum rounds per generation
    Chosen protocol structure; no ablation varying the number of rounds.
  • task_conditioned_adapter_bounds = 0-3 items per list field
    Transfer adapter limits; Section L says constraints were relaxed after observing benchmark variation, so bounds are post-hoc.
  • transfer_eval_set = 20 tasks that both recipient LLMs failed at seed-0
    Selection rule in Section 4.4/Appendix J deliberately keeps hard tasks; acknowledged by authors and controlled by using the same set for the no-knowledge baseline, but it affects absolute solve rates.
  • baseline_fork_timeouts = 3600s per task vs 1800s in main runs
    Budget-parity choice in Appendix H; changes the cost and success profile of the rerun baselines.
axioms (6)
  • domain assumption LLM agents produce consistent enough behavior across fresh contexts that generation-over-generation gains can be attributed to the knowledge base.
    Section 3/4.1 assumes 'any performance gain across generations must come from accumulated knowledge'; no no-knowledge generation loop is run to test this.
  • ad hoc to paper The information-parity gates and LLM substitutions in the baseline forks preserve the core optimization algorithms of DGM and HyperAgents.
    Appendix H documents modified forks; if the gates cripple baseline capability, the comparison 'OURS vs baselines' is not a comparison to the original systems.
  • domain assumption Held-out transfer tasks are representative of the self-improvement task distribution.
    Appendix J samples from disjoint pools, but Polyglot transfer includes 7 tasks that reuse the same exercise as a donor task in another language; ARC is disjoint.
  • domain assumption Benchmark exact-match and verification protocols are accepted measures of task success.
    Section 4.1 relies on ARC official exact-match scoring, SWE-bench Pro harness, Polyglot test commands, and Terminal-Bench verification.
  • domain assumption Removing solved tasks from the active pool does not bias comparisons against baselines.
    Section 4.1 removes solved tasks to save tokens; it is not stated whether the rerun baselines use the same removal rule, so cost/solve comparisons may not be apples-to-apples.
  • ad hoc to paper Forum and distillation prompts elicit evidence-grounded, actionable knowledge rather than generic advice.
    The protocol's effectiveness depends on this; the only evidence is worked examples in Appendix C, not a controlled comparison of prompt variants.

reviewed 2026-08-01 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Knowledge-Centric Self-Improvement." pith.science (2026). https://pith.science/paper/HRBEWU5X

@misc{pith2026260719592,
  author       = {Pith},
  title        = {Pith review of: Knowledge-Centric Self-Improvement},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HRBEWU5X}},
  note         = {Machine review of arXiv:2607.19592}
}
Share X LinkedIn Reddit HN
read the original abstract

Self-improving AI systems typically treat the agent as the object that improves, by optimizing prompts, workflows, harnesses, or even the agent's own code. This agent-centric view can make improvements expensive to maintain and difficult to transfer, because gains become tied to a particular agent design, task distribution, or adaptation run. We study a complementary paradigm: knowledge-centric self-improvement, in which agents remain generic and disposable while the persistent object is a curated knowledge base that agents can leverage for future tasks. We conduct controlled case studies to operationalize this idea via a simple protocol. Agents attempt one task, then contribute evidence-grounded insights to a shared knowledge base via task-level and cross-task forums, followed by knowledge distillation. Because self-improvement is contained in the knowledge rather than the agent, improvement can be more inspectable, transferable, and portable. Across abstract reasoning, coding, and terminal benchmarks, this protocol improves solve rates while reducing dollar cost relative to agent-centric baselines. The resulting distilled knowledge also transfers to held-out tasks and across LLM families, indicating that the improvement is not merely an LLM- or run-specific behavior. These results support a new view of self-improving agentic systems: progress can be driven primarily by the curated persistent knowledge. Code is available at https://github.com/recursive-knowledge/KSI.

Figures

Figures reproduced from arXiv: 2607.19592 by Amanda Zichang Wang, Atharva Sehgal, Chengrui Qu, Eric Mazumdar, Lauren Hyoseo Yoon, Xuefei Julie Wang, Yisong Yue.

Figure 1
Figure 1. Figure 1: From agent-centric to knowledge-centric self-improvement. (Left) Conventional self￾improving agentic systems treat the agent as the persistent substrate, evolving prompts, workflows, or agent code. (Middle) We invert this design. Agents are kept generic and disposable, while the persistent improving substrate is a shared knowledge base, curated through a simple protocol to generate and utilize distilled in… view at source ↗
Figure 2
Figure 2. Figure 2: Knowledge curation protocol consisting of three stages. Task-level forum (agents propose task-level posts summarizing local outcomes and evi￾dence on the current task), Cross-task forum (agents discuss transferable pat￾terns and review peer posts with support￾ing or challenging evidence), and Distil￾lation (forum posts are consolidated into typed task-level and cross-task bundles written back to the shared… view at source ↗
Figure 3
Figure 3. Figure 3: Examples of knowledge curation on two task families. Local task evidence is first organized [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Simplified attempt runtime lifecycle. Solid arrows are forward calls, dashed arrows are [PITH_FULL_IMAGE:figures/full_fig_p014_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: In-depth overview and example of knowledge curation: full curation framework and actual [PITH_FULL_IMAGE:figures/full_fig_p015_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: In-depth overview and example of knowledge curation: full curation framework and actual [PITH_FULL_IMAGE:figures/full_fig_p016_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

201 extracted references · 30 linked inside Pith

  1. [1]

    L. A. Agrawal, S. Tan, D. Soylu, N. Ziems, R. Khare, K. Opsahl-Ong, A. Singhvi, H. Shandilya, M. J. Ryan, M. Jiang, C. Potts, K. Sen, A. G. Dimakis, I. Stoica, D. Klein, M. Zaharia, and O. Khattab. Gepa: Reflective prompt evolution can outperform reinforcement learning.arXiv preprint arXiv:2507.19457, 2025

    Pith/arXiv arXiv 2025

  2. [2]

    Akiba, M

    T. Akiba, M. Shing, Y . Tang, Q. Sun, and D. Ha. Evolutionary Optimization of Model Merging Recipes.Nature Machine Intelligence, 7(2):195–204, 2025. ISSN 2522-5839. doi: 10.1038/s42256-024-00975-8

    doi:10.1038/s42256-024-00975-8 2025

  3. [3]

    Assumpção, D

    H. Assumpção, D. Ferreira, L. Campos, and F. Murai. CodeEvolve: An open source evolutionary coding agent for algorithmic discovery and optimization.arXiv preprint arXiv:2510.14150, 2025

    Pith/arXiv arXiv 2025

  4. [4]

    goose: An open source, extensible ai agent, 2025

    Block, Inc. goose: An open source, extensible ai agent, 2025. URL https://github.com/ block/goose

    2025

  5. [5]

    Borgeaud, A

    S. Borgeaud, A. Mensch, J. Hoffmann, T. Cai, E. Rutherford, K. Millican, G. van den Driessche, J.-B. Lespiau, B. Damoc, A. Clark, D. de Las Casas, A. Guy, J. Menick, R. Ring, T. Hennigan, S. Huang, L. Maggiore, C. Jones, A. Cassirer, A. Brock, M. Paganini, G. Irving, O. Vinyals, S. Osindero, K. Simonyan, J. W. Rae, E. Elsen, and L. Sifre. Improving langua...

    Pith/arXiv arXiv 2022

  6. [6]

    Chhikara, D

    P. Chhikara, D. Khant, S. Aryan, T. Singh, and D. Yadav. Mem0: Building production-ready ai agents with scalable long-term memory.arXiv preprint arXiv:2504.19413, 2025

    Pith/arXiv arXiv 2025

  7. [7]

    F. Chollet. On the measure of intelligence.arXiv preprint arXiv:1911.01547, 2019

    Pith/arXiv arXiv 1911

  8. [8]

    W. L. de Oliveira, M. Bobokhonov, M. Caorsi, A. Podestà, G. Beltramo, L. Crosato, M. Bonotto, F. Cecchetto, H. Espic, D. T. Salajan, S. Taga, L. Pana, and J. Carthy. Arc-agi-2 technical report. arXiv preprint arXiv:2603.06590, 2026

    arXiv 2026

  9. [9]

    X. Deng, J. Da, E. Pan, Y . Y . He, C. Ide, K. Garg, N. Lauffer, A. Park, N. Pasari, C. Rane, K. Sampath, M. Krishnan, S. Kundurthy, S. Hendryx, Z. Wang, V . Bharadwaj, J. Holm, R. Aluri, C. B. C. Zhang, N. Jacobson, B. Liu, and B. Kenstler. Swe-bench pro: Can ai agents solve long-horizon software engineering tasks?arXiv preprint arXiv:2509.16941, 2025

    Pith/arXiv arXiv 2025

  10. [10]

    R. Fang, Y . Liang, X. Wang, J. Wu, S. Qiao, P. Xie, F. Huang, H. Chen, and N. Zhang. Memp: Exploring agent procedural memory.arXiv preprint arXiv:2508.06433, 2025

    Pith/arXiv arXiv 2025

  11. [11]

    Fernando, D

    C. Fernando, D. Banarse, H. Michalewski, S. Osindero, and T. Rocktäschel. Promptbreeder: Self-referential self-improvement via prompt evolution. InProceedings of the 41st International Conference on Machine Learning, volume 235 ofICML’24, pages 13481–13544. JMLR.org, 2024

    2024

  12. [12]

    Fu, D.-K

    Y . Fu, D.-K. Kim, J. Kim, S. Sohn, L. Logeswaran, K. Bae, and H. Lee. Autoguide: Automated generation and selection of context-aware guidelines for large language model agents.arXiv preprint arXiv:2403.08978, 2024

    Pith/arXiv arXiv 2024

  13. [13]

    Gauthier

    P. Gauthier. o1 tops aider’s new polyglot leaderboard.https://aider.chat/2024/12/21/ polyglot.html, Dec. 2024. Accessed: 2026-01-28

    2024

  14. [14]

    S. Hu, C. Lu, and J. Clune. Automated design of agentic systems.arXiv preprint arXiv:2408.08435, 2025

    Pith/arXiv arXiv 2025

  15. [15]

    Terminus-KIRA: Boosting frontier model performance on terminal-bench with minimal harness, 2026

    KRAFTON AI and Ludo Robotics. Terminus-KIRA: Boosting frontier model performance on terminal-bench with minimal harness, 2026. URL https://github.com/krafton-ai/ KIRA

    2026

  16. [16]

    Y . Lee, R. Nair, Q. Zhang, K. Lee, O. Khattab, and C. Finn. Meta-harness: End-to-end optimization of model harnesses.arXiv preprint arXiv:2603.28052, 2026. 11

    Pith/arXiv arXiv 2026

  17. [17]

    Lewis, E

    P. Lewis, E. Perez, A. Piktus, F. Petroni, V . Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. tau Yih, T. Rocktäschel, S. Riedel, and D. Kiela. Retrieval-augmented generation for knowledge- intensive nlp tasks.arXiv preprint arXiv:2005.11401, 2020

    Pith/arXiv arXiv 2005

  18. [18]

    Maharana, D.-H

    A. Maharana, D.-H. Lee, S. Tulyakov, M. Bansal, F. Barbieri, and Y . Fang. Evaluating very long-term conversational memory of llm agents. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers), pages 13851–13870. Association for Computational Linguistics, 2024

    2024

  19. [19]

    M. A. Merrill, A. G. Shaw, N. Carlini, B. Li, H. Raj, I. Bercovich, L. Shi, J. Y . Shin, T. Walshe, E. K. Buchanan, J. Shen, G. Ye, H. Lin, J. Poulos, M. Wang, M. Nezhurina, J. Jitsev, D. Lu, O. M. Mastromichalakis, Z. Xu, Z. Chen, Y . Liu, R. Zhang, L. L. Chen, A. Kashyap, J.-L. Uslu, J. Li, J. Wu, M. Yan, S. Bian, V . Sharma, K. Sun, S. Dillmann, A. Ana...

    Pith/arXiv arXiv 2026

  20. [20]

    Novikov, N

    A. Novikov, N. V˜u, M. Eisenberger, E. Dupont, P.-S. Huang, A. Z. Wagner, S. Shirobokov, B. Kozlovskii, F. J. R. Ruiz, A. Mehrabian, M. P. Kumar, A. See, S. Chaudhuri, G. Holland, A. Davies, S. Nowozin, P. Kohli, and M. Balog. Alphaevolve: A coding agent for scientific and algorithmic discovery.arXiv preprint arXiv:2506.13131, 2025

    Pith/arXiv arXiv 2025

  21. [21]

    Packer, S

    C. Packer, S. Wooders, K. Lin, V . Fang, S. G. Patil, I. Stoica, and J. E. Gonzalez. Memgpt: Towards llms as operating systems.arXiv preprint arXiv:2310.08560, 2024

    Pith/arXiv arXiv 2024

  22. [22]

    Y . Pu, T. Lin, and H. Chen. PiFlow: Principle-Aware Scientific Discovery with Multi-Agent Collaboration.arXiv preprint arXiv:2505.15047, May 2025

    arXiv 2025

  23. [23]

    A. Sharma. Openevolve: an open-source evolutionary coding agent, 2025. URL https: //github.com/algorithmicsuperintelligence/openevolve

    2025

  24. [24]

    X. Tang, T. Qin, T. Peng, Z. Zhou, D. Shao, T. Du, X. Wei, P. Xia, F. Wu, H. Zhu, G. Zhang, J. Liu, X. Wang, S. Hong, C. Wu, H. Cheng, C. Wang, and W. Zhou. Agent kb: Leveraging cross-domain experience for agentic problem solving.arXiv preprint arXiv:2507.06229, 2025

    arXiv 2025

  25. [25]

    G. Wang, Y . Xie, Y . Jiang, A. Mandlekar, C. Xiao, Y . Zhu, L. Fan, and A. Anandkumar. V oyager: An Open-Ended Embodied Agent with Large Language Models.arXiv preprint arXiv:2305.16291, Oct. 2023

    Pith/arXiv arXiv 2023

  26. [26]

    J. Wang, Z. Hu, and L. Bing. Evolving Prompts In-Context: An Open-ended, Self-replicating Perspective. InF orty-Second International Conference on Machine Learning, June 2025

    2025

  27. [27]

    W. Wang, P. Pi˛ ekos, L. Nanbo, F. Laakom, Y . Chen, M. Ostaszewski, M. Zhuge, and J. Schmid- huber. Huxley-gödel machine: Human-level coding agent development by an approximation of the optimal self-improving machine.arXiv preprint arXiv:2510.21614, 2025

    arXiv 2025

  28. [28]

    X. Wang, B. Li, Y . Song, F. F. Xu, X. Tang, M. Zhuge, J. Pan, Y . Song, B. Li, J. Singh, H. H. Tran, F. Li, R. Ma, M. Zheng, B. Qian, Y . Shao, N. Muennighoff, Y . Zhang, B. Hui, J. Lin, R. Brennan, H. Peng, H. Ji, and G. Neubig. Openhands: An open platform for ai software developers as generalist agents.arXiv preprint arXiv:2407.16741, 2025

    Pith/arXiv arXiv 2025

  29. [29]

    Z. Z. Wang, J. Mao, D. Fried, and G. Neubig. Agent workflow memory.arXiv preprint arXiv:2409.07429, 2024

    Pith/arXiv arXiv 2024

  30. [30]

    D. Wu, H. Wang, W. Yu, Y . Zhang, K.-W. Chang, and D. Yu. Longmemeval: Benchmarking chat assistants on long-term interactive memory.arXiv preprint arXiv:2410.10813, 2024

    Pith/arXiv arXiv 2024

  31. [31]

    R. Wu, X. Wang, J. Mei, P. Cai, D. Fu, C. Yang, L. Wen, X. Yang, Y . Shen, Y . Wang, and B. Shi. Evolver: Self-evolving llm agents through an experience-driven lifecycle.arXiv preprint arXiv:2510.16079, 2026. 12

    Pith/arXiv arXiv 2026

  32. [32]

    Z. Wu, C. Han, Z. Ding, Z. Weng, Z. Liu, S. Yao, T. Yu, and L. Kong. OS-Copilot: Towards Generalist Computer Agents with Self-Improvement, Feb. 2024

    2024

  33. [33]

    C. S. Xia, Z. Wang, Y . Yang, Y . Wei, and L. Zhang. Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly?, Nov. 2025

    2025

  34. [34]

    W. Xu, Z. Liang, K. Mei, H. Gao, J. Tan, and Y . Zhang. A-mem: Agentic memory for llm agents.arXiv preprint arXiv:2502.12110, 2025

    Pith/arXiv arXiv 2025

  35. [35]

    J. Yang, C. E. Jimenez, A. Wettig, K. Lieret, S. Yao, K. R. Narasimhan, and O. Press. SWE- agent: Agent-computer interfaces enable automated software engineering. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  36. [36]

    L. Yang, Z. Yu, T. Zhang, S. Cao, M. Xu, W. Zhang, J. E. Gonzalez, and B. Cui. Buffer of thoughts: Thought-augmented reasoning with large language models.arXiv preprint arXiv:2406.04271, 2024

    Pith/arXiv arXiv 2024

  37. [37]

    H. Ye, X. He, V . Arak, H. Dong, and G. Song. Meta Context Engineering via Agentic Skill Evolution.arXiv preprint arXiv:2601.21557, Feb. 2026

    arXiv 2026

  38. [38]

    X. Yin, X. Wang, L. Pan, L. Lin, X. Wan, and W. Y . Wang. Gödel Agent: A Self-Referential Agent Framework for Recursively Self-Improvement. In W. Che, J. Nabende, E. Shutova, and M. T. Pilehvar, editors,Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers), pages 27890–27913, Vienna, Austria, July

  39. [39]

    Zelikman, Y

    E. Zelikman, Y . Wu, J. Mu, and N. Goodman. STaR: Bootstrapping Reasoning With Reasoning. InAdvances in Neural Information Processing Systems, volume 35, pages 15476–15488, Oct. 2022

    2022

  40. [40]

    Zhang, M

    G. Zhang, M. Fu, G. Wan, M. Yu, K. Wang, and S. Yan. G-memory: Tracing hierarchical memory for multi-agent systems.arXiv preprint arXiv:2506.07398, 2025

    Pith/arXiv arXiv 2025

  41. [41]

    Zhang, H

    G. Zhang, H. Ren, C. Zhan, Z. Zhou, J. Wang, H. Zhu, W. Zhou, and S. Yan. MemEvolve: Meta-Evolution of Agent Memory Systems.arXiv preprint arXiv:2512.18746, Dec. 2025

    Pith/arXiv arXiv 2025

  42. [42]

    Zhang, S

    J. Zhang, S. Hu, C. Lu, R. Lange, and J. Clune. Darwin godel machine: Open-ended evolution of self-improving agents.arXiv preprint arXiv:2505.22954, 2025

    Pith/arXiv arXiv 2025

  43. [43]

    Zhang, B

    J. Zhang, B. Zhao, W. Yang, J. Foerster, J. Clune, M. Jiang, S. Devlin, and T. Shavrina. Hyperagents.arXiv preprint arXiv:2603.19461, 2026

    arXiv 2026

  44. [44]

    Zhang, C

    Q. Zhang, C. Hu, S. Upasani, B. Ma, F. Hong, V . Kamanuru, J. Rainton, C. Wu, M. Ji, H. Li, U. Thakker, J. Zou, and K. Olukotun. Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models.arXiv preprint arXiv:2510.04618, Mar. 2026

    Pith/arXiv arXiv 2026

  45. [45]

    Zhang, J

    S. Zhang, J. Wang, R. Zhou, J. Liao, Y . Feng, Z. Li, Y . Zheng, W. Zhang, Y . Wen, Z. Li, F. Xiong, Y . Qi, B. Tang, and M. Wen. Memrl: Self-evolving agents via runtime reinforcement learning on episodic memory.arXiv preprint arXiv:2601.03192, 2026

    Pith/arXiv arXiv 2026

  46. [46]

    Zhang, P

    Y . Zhang, P. Ye, X. Yang, S. Feng, S. Zhang, L. Bai, W. Ouyang, and S. Hu. Nature-Inspired Population-Based Evolution of Large Language Models.arXiv preprint arXiv:2503.01155, Mar. 2025

    Pith/arXiv arXiv 2025

  47. [47]

    A. Zhao, D. Huang, Q. Xu, M. Lin, Y .-J. Liu, and G. Huang. Expel: Llm agents are experiential learners.arXiv preprint arXiv:2308.10144, 2024

    Pith/arXiv arXiv 2024

  48. [48]

    Zheng, M

    B. Zheng, M. Y . Fatemi, X. Jin, Z. Z. Wang, A. Gandhi, Y . Song, Y . Gu, J. Srinivasa, G. Liu, G. Neubig, and Y . Su. Skillweaver: Web agents can self-improve by discovering and honing skills.arXiv preprint arXiv:2504.07079, 2025

    Pith/arXiv arXiv 2025

  49. [49]

    Zhong, L

    W. Zhong, L. Guo, Q. Gao, H. Ye, and Y . Wang. Memorybank: Enhancing large language models with long-term memory.arXiv preprint arXiv:2305.10250, 2023

    Pith/arXiv arXiv 2023

  50. [50]

    adaptive rejection sampling

    M. Zhuge, W. Wang, L. Kirsch, F. Faccio, D. Khizbullin, and J. Schmidhuber. GPTSwarm: Language Agents as Optimizable Graphs. InF orty-First International Conference on Machine Learning, June 2024. 13 Appendix Contents A. Attempt Runtime Lifecycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...

    2024

  51. [52]

    Read each training pair; note dimensions, palette, objects, and what changed input -> output

  52. [53]

    Within your first few turns, write a first-pass guess to BOTH attempt_1.txt and attempt_2.txt (a copy of the test input is fine as a placeholder) so the workspace always contains a submission

  53. [54]

    Form one transformation rule that explains every training pair and verify it pair-by-pair before applying

  54. [55]

    Apply the rule to the test input and overwrite attempt_1.txt and attempt_2.txt with the refined answer (submit two structurally different hypotheses if you can construct them; otherwise submit the same grid into both files)

  55. [56]

    Read ‘TASK.md‘ in the active task workspace

    Validate before exit: python3 /workspace/task/workspace/validate_prediction.py \\ /workspace/task/workspace/attempt_1.txt \\ /workspace/task/workspace/attempt_2.txt """ Listing 7: Illustrative ARC prompt excerpt; abstracted from _build_arc_no_mcp_execution_promptinprompts/__init__.py. SWE-bench Pro.For swebench_pro, the prompt frames each task as reposito...

  56. [57]

    25d487eb

    83302e8f 26. 25d487eb

  57. [58]

    ea32f347

    f76d97a5 27. ea32f347

  58. [59]

    1fad071e

    8efcae92 28. 1fad071e

  59. [60]

    dc0a314f

    1190e5a7 29. dc0a314f

  60. [61]

    6ecd11f4

    57aa92db 30. 6ecd11f4

  61. [62]

    9ecd008a

    a85d4709 31. 9ecd008a

  62. [63]

    b8825c91

    a416b8f3 32. b8825c91

  63. [64]

    25d8a9c8

    890034e9 33. 25d8a9c8

  64. [65]

    776ffc46 27

    694f12f3 34. 776ffc46 27

  65. [66]

    9172f3a0

    a1570a43 35. 9172f3a0

  66. [67]

    6cf79266

    780d0b14 36. 6cf79266

  67. [68]

    caa06a1f

    bdad9b1f 37. caa06a1f

  68. [69]

    d22278a0

    4938f0c2 38. d22278a0

  69. [70]

    4612dd53

    a740d043 39. 4612dd53

  70. [71]

    b6afb2da

    3428a4f5 40. b6afb2da

  71. [72]

    94f9d214

    6455b5f5 41. 94f9d214

  72. [73]

    aabf363d

    f35d900a 42. aabf363d

  73. [74]

    5ad4f10b

    23b5c85d 43. 5ad4f10b

  74. [75]

    1e0a9b12

    ce22a75a 44. 1e0a9b12

  75. [76]

    b548a754

    5582e5ca 45. b548a754

  76. [77]

    05f2a901

    af902bf9 46. 05f2a901

  77. [78]

    239be575

    e48d4e1a 47. 239be575

  78. [79]

    e8593010

    c8f0f002 48. e8593010

  79. [80]

    8731374e

    36d67576 49. 8731374e

  80. [81]

    e509e548 ARC-AGI-1 Knowledge-Transfer Evaluation Tasks

    6b9890af 50. e509e548 ARC-AGI-1 Knowledge-Transfer Evaluation Tasks

Showing first 80 references.

This paper was first reviewed by deepseek-v4-flash on August 1, 2026.