Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

EvoLib claims a language model can learn during inference by building an evolving, weighted library of reusable skills and insights, improving across math, code, and agentic tasks without weight updates or ground-truth labels.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 14:01 UTC pith:ZYBOZ7AD

load-bearing objection EvoLib is a clearly specified test-time learning method with real, consistent gains on static benchmarks, but the headline claim is scoped by a transductive protocol and an unvalidated self-score assumption. the 3 major comments →

arxiv 2605.14477 v2 pith:ZYBOZ7AD submitted 2026-05-14 cs.LG

Test-Time Learning with an Evolving Library

Weijia Xu , Alessandro Sordoni , Chandan Singh , Zelalem Gero , Michel Galley , Xingdi Yuan , Jianfeng Gao This is my paper
classification cs.LG
keywords test-time learningevolving libraryabstraction inductionself-supervised credit assignmentinformation gainmodular skillsreflective insightscontinual learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

EvoLib proposes that a language model can learn during inference without changing its weights and without ground-truth answers, by building a library of reusable knowledge from its own solutions. The library stores two kinds of abstractions—modular skills and reflective insights—and each entry carries a weight based on how much it improves current solutions and how much it helps create useful future abstractions. The paper reports that this evolving, shared library outperforms established test-time scaling and test-time learning baselines on math, code, and multi-turn agentic benchmarks, while using less token budget in the low-compute regime. If the claim holds, API-only and black-box models can self-improve on a stream of tasks, and a modest base model can close or reverse the gap to much larger models.

Core claim

The central claim is that test-time improvement can be driven by accumulating and evolving knowledge abstractions instead of adapting parameters or storing raw experience. Beginning with an empty library, the agent samples skills and insights relevant to each new problem, generates and self-scores solutions, extracts new abstractions, consolidates similar entries, and propagates credit both to the new abstraction and to the older abstractions that helped produce it. The paper reports consistent accuracy gains over spending more compute per problem and over prior memory-based test-time learning, with particularly large gains when the compute budget is tight.

What carries the argument

The evolving library K, a weighted collection of two abstraction types: modular skills (functions, sub-problems, or sub-task workflows) and reflective insights (natural-language lessons about errors and corrections). The weighting mechanism is the core engine: Information Gain (IG) measures how much better the model solves a task when a given abstraction appears, while Future Information Gain (FutureIG) propagates credit backward to the abstractions that enabled the creation of useful new ones. Consolidation merges similar abstractions by embedding similarity plus an LLM call, turning task-specific entries into general reusable ones and controlling memory growth.

Load-bearing premise

The load-bearing premise is that a model's self-assigned score for its own solution is closer to true quality than to noise; if self-evaluation is no better than guessing, the IG and FutureIG weights average noise and the library has no real learning signal.

What would settle it

Measure the correlation between the model's self-evaluation S^Φ and ground-truth correctness on a held-out set from the same benchmarks. If the correlation is near zero or negative—or if EvoLib's gains vanish when self-evaluation scores are replaced by random scores in otherwise identical runs—then the credit-assignment mechanism is not learning from signal.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A black-box model can improve on a stream of tasks using only its own judgments, with no parameter access and no labeled examples.
  • Sharing knowledge across problem instances is more token-efficient than spending the same compute independently on each problem.
  • Consolidating abstractions keeps the library from growing linearly and yields increasingly general skills that transfer across problems.
  • Learning over interleaved, randomly ordered tasks is less dependent on curriculum order than methods that update memory linearly.
  • A weaker base model augmented with the library can exceed the reported performance of stronger, larger models on a code benchmark.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to seed the library with a few human-written skills; the paper starts from empty, so seeding could separate the value of evolution from the value of the initial content.
  • Because the library is external to the model, skills learned in one deployment could in principle be shared across agents, yielding collective knowledge transfer the paper does not evaluate.
  • The IG/FutureIG weighting can be viewed as an online credit-assignment policy; comparing it with simple recency or frequency heuristics would isolate how much of the gain comes from the information-theoretic scores versus mere reuse.
  • Treating consolidation as online clustering suggests a concrete hypothesis: controlling library growth, rather than the specific merge prompt, is what sustains performance at scale.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces EvoLib, a test-time learning framework for black-box LLMs that maintains a library of modular skills and reflective insights extracted from the model's own inference trajectories. The library is updated through a weighting mechanism based on Information Gain (IG) and Future Information Gain (FutureIG), both estimated from the model's self-evaluated solution scores S^Φ. The method requires no parameter updates and no ground-truth labels. Experiments on mathematical reasoning (HMMT), code generation (BigCodeBench Hard, LiveCodeBench Hard), and multi-turn agentic tasks (ScienceWorld, PDDL) report consistent gains over test-time scaling and test-time learning baselines, together with ablations and cost-performance curves.

Significance. If the results are valid, EvoLib is a meaningful contribution: it provides a parameter-free, label-free way to transfer knowledge across instances at test time, with a clear algorithmic formulation and convincing ablations on the role of consolidation and weighting. The paper also ships pseudocode, detailed hyperparameters, and claims code release, which supports reproducibility. However, the central empirical claim is currently conditional on two unresolved issues: the evaluation protocol is transductive (learning and testing on the same static problem pools), and the entire credit-assignment mechanism relies on the model's self-evaluation S^Φ without any calibration evidence. Both issues are acknowledged in spirit by the authors' own limitation statement, but they are not addressed experimentally. The significance would be high if the authors can demonstrate held-out generalization and validate or replace the self-evaluation signal.

major comments (3)
  1. [Section 4.1, Table 1] The static benchmark protocol is transductive: the agent iterates over the whole task pool and the final best solutions are evaluated on the same problems used to build the library. This means the headline gains over test-time scaling baselines may reflect memorization or task-specific overfitting rather than reusable knowledge. The per-instance ablation in Fig. 3(a) does not resolve this, because the per-instance variant still repeatedly solves the same test tasks. Please add a held-out evaluation, e.g., build the library on one set of tasks and evaluate on a disjoint set, or report math/code results in the continual setting where each task is attempted only once (as is done for agentic tasks). Without such evidence, the abstract's claim of 'improves substantially ... without ground-truth feedback' is not established as a generalization claim.
  2. [Section 3.2.2, Eq. (7), Algorithm 1, Section 5] All credit assignment — IG, FutureIG, and best-solution selection in Algorithm 1 lines 9–14 — depends on the model's self-evaluated score S^Φ, but the paper provides no evidence that S^Φ is more signal than noise. The Limitations paragraph explicitly concedes that the framework assumes self-judgment is useful. This is a load-bearing issue, not a peripheral caveat: Eq. (7) takes the maximum IG over tasks, so systematic positive bias in self-evaluation can dominate library weights. The 'quality evolution' plots in Fig. B.4 are self-referential because IG/FutureIG are computed from S^Φ and do not measure external solution quality. Please include a calibration analysis of S^Φ against ground-truth correctness (e.g., correlation, AUC, or accuracy of self-judgment) for each benchmark, or ablate with an oracle score to show that the mechanism works when the signal is reliable.
  3. [Table 1, Appendix A.4] Table 1 reports averages over three runs with no standard deviations or confidence intervals; the asterisks for significance are not sufficient. With small datasets (HMMT: 93, BigCodeBench: 148, LiveCodeBench: 80), three runs give limited statistical power, and the paired t-test details are not specified: what is the unit of pairing, and which pairs are tested? Please report per-benchmark mean ± std or CIs, and clarify the test procedure. This is necessary to assess whether differences such as 77.4 vs 74.2 on HMMT or 57.4 vs 55.9 on ScienceWorld are reliable.
minor comments (4)
  1. [Appendix A.2] The paper states GPT-4o is run with temperature 0 and top_p 0.5 on BigCodeBench. With temperature 0, sampling is typically deterministic; please clarify how multiple candidate solutions are generated in the K-trials loop, or specify the actual sampling temperature.
  2. [Section 3.2.2, Eq. (7)] The weighting formula is a heuristic combination of a max over tasks and an expectation over FutureIG. Please clarify whether this choice was motivated analytically or selected empirically, and whether the relative weight τ is tuned per benchmark.
  3. [Algorithm 1, line 19] IG is set to 0 for insights while Eq. (7) uses τ=0 for insights. This is redundant; clarify whether insights ever receive direct IG, and if not, explain the rationale in the main text.
  4. [Figure B.4] The y-axis labels 'Average Top-100 IG' and 'Average Top-100 Future IG' are fine, but the surrounding text refers to 'quality of abstractions'. Since these metrics are computed from self-scores, please relabel as self-estimated information gain to avoid implying external quality.

Circularity Check

1 steps flagged

Headline benchmark results are externally evaluated; only the internal 'quality evolution' evidence in Fig. B.4 is self-referential.

specific steps
  1. self definitional [Appendix B, 'Evolution of abstraction quality' (Fig. B.4)]
    "Both the average IG and average Future IG among the top entries increase as the number of iterations grows. The increase in IG indicates that the library progressively contains abstractions that directly improve solution quality."

    The plotted quantities are not externally measured solution quality: IG and FutureIG are defined in Eqs. (3) and (6) as log-ratios of the model's self-estimated score S^Phi, and the 'top entries' are ranked by w(z|K)=τ max_x IG(z|x,K)+E_x FutureIG(z|x,K) from Eq. (7). Thus the upward trend in average top-100 IG/FutureIG is partly a selection artifact of optimizing the same self-scored objective used to sample and rank abstractions. The text interprets this internal self-scores as evidence that abstractions 'directly improve solution quality,' but the figure does not independently validate true quality. This is a minor internal-support circularity and does not affect the externally graded headline results.

full rationale

EvoLib's central empirical claims are evaluated against external ground truth: HMMT accuracy uses the official MathArena grading scripts, BigCodeBench and LiveCodeBench use true/hidden test cases, and ScienceWorld/PDDL use AgentBoard success/progress metrics. Those comparisons are not forced by the method's definitions and are therefore not circular. The IG/FutureIG weighting mechanism is estimated from the same model's self-evaluated score S^Phi, and the paper itself concedes in Section 5 that 'the current EVOLIB framework assumes that the model's own judgment of a solution provides more useful signals than noise for self-evaluation.' That is an unvalidated assumption about signal quality rather than a derivation that makes the benchmark outcome equal to an input. The only notable circular step is Fig. B.4, where 'quality of the most useful abstractions' is measured by average IG/FutureIG among top entries; since those quantities are defined through S^Phi and used directly in the weighting/sampling scheme, the reported increase is partly a self-scoring/selection artifact rather than independent evidence of abstraction quality. This does not infect the main benchmark comparisons, and the self-citations in the related work (e.g., [4], [27]) are not load-bearing for the central claims. Overall, no significant circularity in the headline derivation.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

EvoLib is an empirical method; the central claim rests on plausible but unproven assumptions about self-evaluation quality, embedding-based retrieval, and LLM summarization fidelity. Its free parameters (τ, thresholds, K) are set by hand rather than fit or swept, so they are not the main risk; the main risk is the self-scoring assumption.

free parameters (4)
  • τ (tau) = 1 for skills, 0 for insights
    Controls trade-off between immediate (IG) and future (FutureIG) utility in Eq. (7); chosen by hand, not derived or swept.
  • consolidation similarity threshold = 0.8
    Embedding cosine similarity threshold for deciding whether a new abstraction merges with an existing one (Section A.4); set by hand.
  • sampling similarity threshold = 0.2 (agentic), 0 (other benchmarks)
    Threshold for filtering library entries before weighted sampling (Section A.4); set by hand.
  • trials K = 3 (code), 10 (math), 1 (agentic)
    Number of sampled solutions per task for score estimation and solution generation; budget choices, not optimized.
axioms (4)
  • domain assumption Self-scoring signal S^Φ is positively correlated with true solution quality
    Used in Eqs. (1)-(6) and Algorithm 1 lines 8-13; the paper's own Limitations section flags this as an assumption.
  • domain assumption The LLM extracts and consolidates abstractions faithfully without harmful loss
    Section 3.2.1 and 3.3; if summarization corrupts skills, library quality drops.
  • domain assumption Embedding similarity is a valid proxy for functional similarity of abstractions and task relevance
    Used for sampling filter and consolidation retrieval (Section 3.2 and A.4).
  • domain assumption Synthetic test cases generated by the model predict performance on held-out true tests
    Section 3.2.1; this underlies the self-score for coding.

pith-pipeline@v1.3.0-alltime-deepseek · 18771 in / 15251 out tokens · 122375 ms · 2026-08-02T14:01:02.808550+00:00 · methodology

0 comments
read the original abstract

We introduce EvoLib, a test-time learning framework that enables large language models to accumulate, reuse, and evolve knowledge across problem instances without parameter updates or external supervision. Instead of adapting model parameters, our approach maintains a shared library of knowledge abstractions, including modular skills and reflective insights, automatically extracted from the model's own inference trajectories. To support continual improvement, we introduce a principled weighting and consolidation mechanism that jointly optimizes for immediate utility and long-term value. This allows simple, instance-specific abstractions to evolve into more general and reusable ones over time. Across challenging benchmarks in mathematical reasoning, code generation, and multi-turn agentic environments, EvoLib improves substantially over the top test-time scaling and learning methods without ground-truth feedback.

Figures

Figures reproduced from arXiv: 2605.14477 by Alessandro Sordoni, Chandan Singh, Jianfeng Gao, Michel Galley, Weijia Xu, Xingdi Yuan, Zelalem Gero.

Figure 1
Figure 1. Figure 1: Overview of the EVOLIB algorithm. EVOLIB performs test-time learning by repeatedly: (i) solving tasks using sampled abstractions, (ii) extracting new abstractions, consolidating them into the library, and propagating credit to both new and previously used abstractions via Information Gain (IG) and Future IG. reflective insights from model-generated programs or trajectories together with feedback [25–29]. F… view at source ↗
Figure 2
Figure 2. Figure 2: Cost–performance curves comparing EVOLIB with competitive baselines on (a) Big￾CodeBench and (b) LiveCodeBench. Each curve plots performance (y-axis) as the test-time compute cost (x-axis) increases for each method. gains indicate that learning and sharing knowledge across problem instances yields consistent improvements over TTS. Compared to TTL baselines, EVOLIB consistently outperforms DC, the best TTL … view at source ↗
Figure 3
Figure 3. Figure 3: Ablation study on BigCodeBench. (a) Comparison of variants using different abstraction [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. AgentStream: How Well Do Self-Evolving LLM Agents Perform Under Streaming Tasks?

    cs.AI 2026-07 conditional novelty 6.0

    Self-evolving LLM agents improve under streaming evaluation only when the base model is strong enough; gains are largest in single-domain streams, and no method dominates across models.

Reference graph

Works this paper leans on

45 extracted references · 15 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. InThe Eleventh International Conference on Learning Representations, 2023

    2023

  2. [2]

    Large language models are better reasoners with self-verification

    Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Shengping Liu, Bin Sun, Kang Liu, and Jun Zhao. Large language models are better reasoners with self-verification. In Houda Bouamor, Juan Pino, and Kalika Bali, editors,Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2550–2575, Singapore, December 2023. Association for Compu...

    2023

  3. [3]

    Scaling llm test-time compute optimally can be more effective than scaling model parameters.arXiv preprint arXiv:2408.03314, 2024

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters.arXiv preprint arXiv:2408.03314, 2024

    Pith/arXiv arXiv 2024

  4. [4]

    Test-time recursive thinking: Self-improvement without external feedback.arXiv preprint arXiv:2602.03094, 2026

    Yufan Zhuang, Chandan Singh, Liyuan Liu, Yelong Shen, Dinghuai Zhang, Jingbo Shang, Jianfeng Gao, and Weizhu Chen. Test-time recursive thinking: Self-improvement without external feedback.arXiv preprint arXiv:2602.03094, 2026

    arXiv 2026

  5. [5]

    s1: Simple test-time scaling

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Proceedings of the 2025 Conference on Empirical Methods in Natural Lan...

    2025

  6. [6]

    Recursive self-aggregation unlocks deep thinking in large language models.arXiv preprint arXiv:2509.26626, 2025

    Siddarth Venkatraman, Vineet Jain, Sarthak Mittal, Vedant Shah, Johan Obando-Ceron, Yoshua Bengio, Brian R Bartoldson, Bhavya Kailkhura, Guillaume Lajoie, Glen Berseth, et al. Recursive self-aggregation unlocks deep thinking in large language models.arXiv preprint arXiv:2509.26626, 2025

    arXiv 2025

  7. [7]

    Memory-assisted prompt editing to improve GPT-3 after deployment

    Aman Madaan, Niket Tandon, Peter Clark, and Yiming Yang. Memory-assisted prompt editing to improve GPT-3 after deployment. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang, editors,Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2833–2861, Abu Dhabi, United Arab Emirates, December 2022. Association for Compu...

    2022

  8. [8]

    Thought-retriever: Don’t just retrieve raw data, retrieve thoughts, 2024

    Tao Feng, Pengrui Han, Guanyu Lin, Ge Liu, and Jiaxuan You. Thought-retriever: Don’t just retrieve raw data, retrieve thoughts, 2024

    2024

  9. [9]

    Evo-memory: Benchmarking llm agent test-time learning with self-evolving memory.arXiv preprint arXiv:2511.20857, 2025

    Tianxin Wei, Noveen Sachdeva, Benjamin Coleman, Zhankui He, Yuanchen Bei, Xuying Ning, Mengting Ai, Yunzhe Li, Jingrui He, Ed H Chi, et al. Evo-memory: Benchmarking llm agent test-time learning with self-evolving memory.arXiv preprint arXiv:2511.20857, 2025

    Pith/arXiv arXiv 2025

  10. [10]

    Test-time learning for large language models

    Jinwu Hu, Zitian Zhang, Guohao Chen, Xutao Wen, Chao Shuai, Wei Luo, Bin Xiao, Yuanqing Li, and Mingkui Tan. Test-time learning for large language models. In Aarti Singh, Maryam Fazel, Daniel Hsu, Simon Lacoste-Julien, Felix Berkenkamp, Tegan Maharaj, Kiri Wagstaff, and Jerry Zhu, editors, Proceedings of the 42nd International Conference on Machine Learni...

    2025

  11. [11]

    The surprising effectiveness of test-time training for few-shot learning

    Ekin Akyürek, Mehul Damani, Adam Zweiger, Linlu Qiu, Han Guo, Jyothish Pari, Yoon Kim, and Jacob Andreas. The surprising effectiveness of test-time training for few-shot learning. In Aarti Singh, Maryam Fazel, Daniel Hsu, Simon Lacoste-Julien, Felix Berkenkamp, Tegan Maharaj, Kiri Wagstaff, and Jerry Zhu, editors,Proceedings of the 42nd International Conf...

    2025

  12. [12]

    Learning to discover at test time.arXiv preprint arXiv:2601.16175, 2026

    Mert Yuksekgonul, Daniel Koceja, Xinhao Li, Federico Bianchi, Jed McCaleb, Xiaolong Wang, Jan Kautz, Yejin Choi, James Zou, Carlos Guestrin, et al. Learning to discover at test time.arXiv preprint arXiv:2601.16175, 2026. 10

    Pith/arXiv arXiv 2026

  13. [13]

    Learning to (learn at test time): Rnns with expressive hidden states

    Yu Sun, Xinhao Li, Karan Dalal, Jiarui Xu, Arjun Vikram, Genghan Zhang, Yann Dubois, Xinlei Chen, Xiaolong Wang, Sanmi Koyejo, et al. Learning to (learn at test time): Rnns with expressive hidden states. arXiv preprint arXiv:2407.04620, 2024

    Pith/arXiv arXiv 2024

  14. [14]

    Gradmem: Learning to write context into memory with test-time gradient descent.arXiv preprint arXiv:2603.13875, 2026

    Yuri Kuratov, Matvey Kairov, Aydar Bulatov, Ivan Rodkin, and Mikhail Burtsev. Gradmem: Learning to write context into memory with test-time gradient descent.arXiv preprint arXiv:2603.13875, 2026

    Pith/arXiv arXiv 2026

  15. [15]

    How to grow a mind: Statistics, structure, and abstraction.science, 331(6022):1279–1285, 2011

    Joshua B Tenenbaum, Charles Kemp, Thomas L Griffiths, and Noah D Goodman. How to grow a mind: Statistics, structure, and abstraction.science, 331(6022):1279–1285, 2011

    2011

  16. [16]

    Cognitive skill acquisition.Annual review of psychology, 47(1):513–539, 1996

    Kurt VanLehn. Cognitive skill acquisition.Annual review of psychology, 47(1):513–539, 1996

    1996

  17. [17]

    On the measure of intelligence.arXiv preprint arXiv:1911.01547, 2019

    François Chollet. On the measure of intelligence.arXiv preprint arXiv:1911.01547, 2019

    Pith/arXiv arXiv 1911

  18. [18]

    Walt: Web agents that learn tools.arXiv preprint arXiv:2510.01524, 2025

    Viraj Prabhu, Yutong Dai, Matthew Fernandez, Jing Gu, Krithika Ramakrishnan, Yanqi Luo, Silvio Savarese, Caiming Xiong, Junnan Li, Zeyuan Chen, et al. Walt: Web agents that learn tools.arXiv preprint arXiv:2510.01524, 2025

    arXiv 2025

  19. [19]

    Skill0: In-context agentic reinforcement learning for skill internalization.arXiv preprint arXiv:2604.02268, 2026

    Zhengxi Lu, Zhiyuan Yao, Jinyang Wu, Chengcheng Han, Qi Gu, Xunliang Cai, Weiming Lu, Jun Xiao, Yueting Zhuang, and Yongliang Shen. Skill0: In-context agentic reinforcement learning for skill internalization.arXiv preprint arXiv:2604.02268, 2026

    Pith/arXiv arXiv 2026

  20. [20]

    ReGAL: Refactoring programs to discover generalizable abstractions

    Elias Stengel-Eskin, Archiki Prasad, and Mohit Bansal. ReGAL: Refactoring programs to discover generalizable abstractions. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors,Proceedings of the 41st International Conference on Machine Learning, volume 235 ofProceedings of Ma...

    2024

  21. [21]

    Inducing programmatic skills for agentic tasks

    Zora Zhiruo Wang, Apurva Gandhi, Graham Neubig, and Daniel Fried. Inducing programmatic skills for agentic tasks. InSecond Conference on Language Modeling, 2025

    2025

  22. [22]

    Reuseit: Synthesizing reusable ai agent workflows for web automation

    Yimeng Liu, Misha Sra, Jeevana Priya Inala, and Chenglong Wang. Reuseit: Synthesizing reusable ai agent workflows for web automation. InProceedings of the 31st International Conference on Intelligent User Interfaces, pages 885–908, 2026

    2026

  23. [23]

    Autorefine: From trajectories to reusable expertise for continual llm agent refinement.arXiv preprint arXiv:2601.22758, 2026

    Libin Qiu, Zhirong Gao, Junfu Chen, Yuhang Ye, Weizhi Huang, Xiaobo Xue, Wenkai Qiu, and Shuo Tang. Autorefine: From trajectories to reusable expertise for continual llm agent refinement.arXiv preprint arXiv:2601.22758, 2026

    arXiv 2026

  24. [24]

    Arcmemo: Abstract reasoning composition with lifelong llm memory.arXiv preprint arXiv:2509.04439, 2025

    Matthew Ho, Chen Si, Zhaoxiang Feng, Fangxu Yu, Yichi Yang, Zhijian Liu, Zhiting Hu, and Lianhui Qin. Arcmemo: Abstract reasoning composition with lifelong llm memory.arXiv preprint arXiv:2509.04439, 2025

    arXiv 2025

  25. [25]

    Expel: Llm agents are experiential learners.Proceedings of the AAAI Conference on Artificial Intelligence, 38(17):19632– 19642, Mar

    Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. Expel: Llm agents are experiential learners.Proceedings of the AAAI Conference on Artificial Intelligence, 38(17):19632– 19642, Mar. 2024

    2024

  26. [26]

    Evolver: Self-evolving llm agents through an experience-driven lifecycle.arXiv preprint arXiv:2510.16079, 2025

    Rong Wu, Xiaoman Wang, Jianbiao Mei, Pinlong Cai, Daocheng Fu, Cheng Yang, Licheng Wen, Xuemeng Yang, Yufan Shen, Yuxin Wang, et al. Evolver: Self-evolving llm agents through an experience-driven lifecycle.arXiv preprint arXiv:2510.16079, 2025

    Pith/arXiv arXiv 2025

  27. [27]

    Evolving programmatic skill networks.arXiv preprint arXiv:2601.03509, 2026

    Haochen Shi, Xingdi Yuan, and Bang Liu. Evolving programmatic skill networks.arXiv preprint arXiv:2601.03509, 2026

    Pith/arXiv arXiv 2026

  28. [28]

    Rlad: Training llms to discover abstractions for solving reasoning problems.arXiv preprint arXiv:2510.02263, 2025

    Yuxiao Qu, Anikait Singh, Yoonho Lee, Amrith Setlur, Ruslan Salakhutdinov, Chelsea Finn, and Aviral Kumar. Rlad: Training llms to discover abstractions for solving reasoning problems.arXiv preprint arXiv:2510.02263, 2025

    arXiv 2025

  29. [29]

    Hybrid-gym: Training coding agents to generalize across tasks.arXiv preprint arXiv:2602.16819, 2026

    Yiqing Xie, Emmy Liu, Gaokai Zhang, Nachiket Kotalwar, Shubham Gandhi, Sathwik Acharya, Xingyao Wang, Carolyn Rose, Graham Neubig, and Daniel Fried. Hybrid-gym: Training coding agents to generalize across tasks.arXiv preprint arXiv:2602.16819, 2026

    arXiv 2026

  30. [30]

    The ai scientist: Towards fully automated open-ended scientific discovery.arXiv preprint arXiv:2408.06292, 2024

    Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. The ai scientist: Towards fully automated open-ended scientific discovery.arXiv preprint arXiv:2408.06292, 2024

    Pith/arXiv arXiv 2024

  31. [31]

    Alphaevolve: A coding agent for scientific and algorithmic discovery.arXiv preprint arXiv:2506.13131, 2025

    Alexander Novikov, Ngân V˜u, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco JR Ruiz, Abbas Mehrabian, et al. Alphaevolve: A coding agent for scientific and algorithmic discovery.arXiv preprint arXiv:2506.13131, 2025. 11

    Pith/arXiv arXiv 2025

  32. [32]

    Thetaevolve: Test-time learning on open problems.arXiv preprint arXiv:2511.23473, 2025

    Yiping Wang, Shao-Rong Su, Zhiyuan Zeng, Eva Xu, Liliang Ren, Xinyu Yang, Zeyi Huang, Xuehai He, Luyao Ma, Baolin Peng, et al. Thetaevolve: Test-time learning on open problems.arXiv preprint arXiv:2511.23473, 2025

    Pith/arXiv arXiv 2025

  33. [33]

    Skillclaw: Let skills evolve collectively with agentic evolver.arXiv preprint arXiv:2604.08377, 2026

    Ziyu Ma, Shidong Yang, Yuxiang Ji, Xucong Wang, Yong Wang, Yiming Hu, Tongwen Huang, and Xiangx- iang Chu. Skillclaw: Let skills evolve collectively with agentic evolver.arXiv preprint arXiv:2604.08377, 2026

    Pith/arXiv arXiv 2026

  34. [34]

    Evoskill: Automated skill discovery for multi-agent systems.arXiv preprint arXiv:2603.02766, 2026

    Salaheddin Alzubi, Noah Provenzano, Jaydon Bingham, Weiyuan Chen, and Tu Vu. Evoskill: Automated skill discovery for multi-agent systems.arXiv preprint arXiv:2603.02766, 2026

    Pith/arXiv arXiv 2026

  35. [35]

    Agent workflow memory

    Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, and Graham Neubig. Agent workflow memory. In Forty-second International Conference on Machine Learning, 2025

    2025

  36. [36]

    Dynamic cheatsheet: Test-time learning with adaptive memory

    Mirac Suzgun, Mert Yuksekgonul, Federico Bianchi, Dan Jurafsky, and James Zou. Dynamic cheatsheet: Test-time learning with adaptive memory. InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7080–7106, 2026

    2026

  37. [37]

    Reasoningbank: Scaling agent self-evolving with reasoning memory

    Siru Ouyang, Jun Yan, I-Hung Hsu, Yanfei Chen, Ke Jiang, Zifeng Wang, Rujun Han, Long T Le, Samira Daruki, Xiangru Tang, Vishy Tirumalashetty, George Lee, Mahsan Rofouei, Hangfei Lin, Jiawei Han, Chen-Yu Lee, and Tomas Pfister. Reasoningbank: Scaling agent self-evolving with reasoning memory. In The Fourteenth International Conference on Learning Represen...

    2026

  38. [38]

    Judging llm-as-a- judge with mt-bench and chatbot arena

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E Gonzalez, and Ion Stoica. Judging llm-as-a- judge with mt-bench and chatbot arena. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors,Advances in Neural Information Processing Sy...

    2023

  39. [39]

    Bigcodebench: Benchmarking code generation with diverse function calls and complex instructions

    Terry Yue Zhuo, Vu Minh Chien, Jenny Chim, Han Hu, Wenhao Yu, Ratnadira Widyasari, Imam Nur Bani Yusuf, Haolan Zhan, Junda He, Indraneil Paul, Simon Brunner, Chen GONG, James Hoang, Armel Randy Zebaze, Xiaoheng Hong, Wen-Ding Li, Jean Kaddour, Ming Xu, Zhihan Zhang, Prateek Yadav, Naman Jain, Alex Gu, Zhoujun Cheng, Jiawei Liu, Qian Liu, Zijian Wang, Davi...

    2025

  40. [40]

    Livecodebench: Holistic and contamination free evaluation of large language models for code

    Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar- Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  41. [41]

    Ruoyao Wang, Peter Jansen, Marc-Alexandre Côté, and Prithviraj Ammanabrolu. ScienceWorld: Is your agent smarter than a 5th grader? In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang, editors,Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11279–11298, Abu Dhabi, United Arab Emirates, December 2022. Association...

    2022

  42. [42]

    The 2014 international planning competition: Progress and trends.Ai Magazine, 36(3):90–98, 2015

    Mauro Vallati, Lukas Chrpa, Marek Grze´s, Thomas Leo McCluskey, Mark Roberts, Scott Sanner, et al. The 2014 international planning competition: Progress and trends.Ai Magazine, 36(3):90–98, 2015

    2014

  43. [43]

    Agentboard: An analytical evaluation board of multi-turn llm agents

    Chang Ma, Junlei Zhang, Zhihao Zhu, Cheng Yang, Yujiu Yang, Yaohui Jin, Zhenzhong Lan, Lingpeng Kong, and Junxian He. Agentboard: An analytical evaluation board of multi-turn llm agents. In A. Glober- son, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors,Advances in Neural Information Processing Systems, volume 37, pages 74325–...

    2024

  44. [44]

    Matharena: Evaluating llms on uncontaminated math competitions.Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmark, 2025

    Mislav Balunovi´c, Jasper Dekoninck, Ivo Petrov, Nikola Jovanovi ´c, and Martin Vechev. Matharena: Evaluating llms on uncontaminated math competitions.Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmark, 2025

    2025

  45. [45]

    Limitations

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024. 12 Table A.2: Summary of datasets used in evaluation. Domain Dataset # Instances Notes Math Reasoning HMMT Feb 2025 30 Competitive math problems HMMT Nov ...

    Pith/arXiv arXiv 2024