Computation and Language · arXiv:2610.19951 v1 · submitted 14 Oct 2026 · 73 references
AtomSearch: Evidence-Grounded Tree Search for Agent Skill Optimization
The paper says
“Agent skills adapt frozen models through editable external instructions. Revising several rules at once makes changes difficult to inspect, while judging each local edit only by its immediate score can discard useful intermediate states.”
more of the abstract
We introduce AtomSearch, which keeps revision actions local while using subsequent skill states to guide search. Contrastive atomic editing links each primitive rule edit to failed and successful executions, a behavioral hypothesis, and an intended scope. Skill-State Tree Search (STS) evaluates short edit compositions and selectively retains them as executable states that can be revisited and extended. Outcome memory supplies later proposals with observed repair and regression statistics. We evaluate AtomSearch across four tasks and five target models. With deepseek-v4-flash, it exceeds SkillOpt on three of four tasks, including a 9.68-percentage-point gain on LiveMathematicianBench, and SkillOpt's final skills contain 2.3-19.7 times as many characters. In 67% of completed searches, the best validation state passes through an ancestor that was not highest-scoring at its depth. These results show how local rule edits can support the exploration of useful revision paths and yield compact skills, with additional offline evaluations used to explore alternative states.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 15 Oct 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. A useful engineering contribution that combines atomic edits and MCTS for skill optimization, but the key 67% statistic is less robust than presented due to small validation panels and no uncertainty quantification. This line is wrong
2 major objections · 5 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 73 references say.
The paper says, at Section 4.4, Table 4, and the paper's own limitation b203
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.74):
“Repeated evaluation of fixed checkpoints would be needed to quantify their uncertainty and separate selection effects from differences in the learned skills. 15”
Pith says: Objection 1 of 2 · major
The 67% statistic (20 out of 30 completed searches) is presented as evidence that 'the best validation state passes through an ancestor that was not highest-scoring at its depth' and is used to motivate the tree search over greedy approaches.
more of objection 1
However, the validation panels used to compute state scores are small: 105 items for NDR, 40 for SpreadsheetBench, 18 for LiveMathematicianBench, and 200 for SearchQA (Table 4). As the paper itself acknowledges (b203), 'finite validation panels and stochastic execution limit the precision of small score differences.' The difference of 2.9 points on NDR (Figure 4) corresponds to roughly 3 items out of 105; stochastic execution could easily flip such a ranking. The paper does not provide any uncertainty quantification (e.g., bootstrapping, confidence intervals) for the 67% figure, so it is unclear how many of these 'non-myopic paths' would persist under repeated evaluation on the same validation panel. This is a load-bearing issue because the non-myopic path claim is the main empirical justification for the STS component. The authors should estimate the uncertainty of the 67% statistic—for example by repeatedly resampling the validation set or by re-running the search with different noise seeds—and report a range or confidence interval. If the statistic is not robust, the paper's central motivation for the tree search is weakened.
The paper says, at Section 4.3 and Appendix C (Table 2, validation-to-test gain ratio)
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.82):
“These are point estimates, not significance claims.”
Pith says: Objection 2 of 2 · major
Table 2 reports that AtomSearch achieves a median gain ratio (Δtest/Δval) of 124% versus 82% for SkillOpt, and the paper uses this to argue that AtomSearch generalizes better from validation to test. The authors appropriately note in Appendix C that these ratios are computed from point estimates, that initial scores can differ between methods, and that a small denominator can inflate the ratio (b329–b331, b335).
more of objection 2
However, the main text (Section 4.3) presents the ratio as a descriptive comparison without reiterating these caveats, and no uncertainty is attached to the 124% vs. 82% difference. Because the ratio comparisons are not the primary result and the paper explicitly states they are descriptive, this is a moderate concern. I recommend adding a brief statement in Section 4.3 that references the Appendix caveats, or moving the ratio analysis entirely to Appendix C to avoid over-interpretation.
5 minor objections
The paper says, at Section 4.4
Pith says: minor objection 1 of 5
The sentence 'In 20 of 30 completed searches (67%)...' should clarify what constitutes a 'completed search.' From context, it appears to mean searches that reached the maximum number of attempts (20 or 30, Table 5), but this could be stated explicitly. Also, it would be helpful to show how the 30 searches are distributed across the different tasks and models, given that validation panel sizes vary substantially.
The paper says, at Section 3, Equation (1) and surrounding text
Pith says: minor objection 2 of 5
The navigation set D_nav is defined, but the paper later uses both 'validation set' and 'navigation set' interchangeably. For clarity, choose one term (e.g., 'validation set') throughout, or explicitly state that they are the same.
show 3 more
The paper says, at Figure 3 caption
Pith says: minor objection 3 of 5
The caption says 'Both paths include a score decrease before reaching their maxima.' Consider adding a note about the SpreadsheetBench path's specific dip (validation accuracy falls from 52.50 to 49.50) to help readers interpret the non-monotonicity.
The paper says, at Table 3
Pith says: minor objection 4 of 5
The table shows that contrastive atomic editing alone yields 75.71% test accuracy with 2,352 characters, while the full method yields 78.21% with 2,936 characters. The incremental gain from tree search and outcome memory is small (2.5 points) and comes from one setting. The paper acknowledges this limitation (b204), but the main text could more prominently note that the search and memory components provide only modest improvements in this ablation.
The paper says, at Section 5, discussion of overfitting
Pith says: minor objection 5 of 5
The paper cites Cawley & Talbot (2010) on overfitting, which is appropriate. However, the paper does not discuss the specific risk of overfitting to the validation set over multiple tree expansions when the validation panel is small (e.g., 40 items for SpreadsheetBench). Adding a brief note about how the number of candidate comparisons interacts with validation set size would strengthen the limitation statement.
What would settle it
For the 20 AtomSearch searches on NDR (105-item validation panel), bootstrap the validation scores of all visited states with 1000 resamples. For each search, compute the proportion of resamples in which the best final validation state has at least one ancestor that was not the highest-scoring state at its depth. If the median proportion across searches is significantly below 67% (e.g., <50%), then the observed statistic is consistent with noise rather than a systematic non-myopic effect.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 15 Oct 2026
- Reader holds up, with conditions
- Skeptic partly agrees
- Editor wrote the verdict above
- Referee major revision: 2 major objections, 5 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
Here's my take on AtomSearch. The paper does something genuinely useful: it decouples revision granularity from valuation horizon by combining contrastive atomic edits with a tree search over full skill states. The idea that a locally weak edit can enable later improvements is well-motivated, and the empirical demonstration that such paths occur (67% of searches) is interesting if true. The compact skills and the validation-to-test ratio analysis (124% vs 82%) are also notable.
The honest and direct limitation statements (Section 5) deserve credit. The authors say outright these are point estimates, that validation panels are finite, that the component ablation covers one task, and that the 67% figure may reflect noise. That is refreshing.
Now the soft spots. The 67% statistic is the headline evidence for non-myopic search, but it sits on shaky ground. NDR validation panel has 105 items; a 2.9-point gap is about 3 items. With stochastic execution and no bootstrapping, the statistic could be substantially inflated by ranking noise. The authors acknowledge the concern in prose but do not quantify it. The stress-test note is right: without confidence intervals, we have no way to know how many of those 'non-myopic' paths would replicate on repeated evaluation.
Second, the component ablation (Table 3) covers only one task-target setting. The increments from search and memory are tiny (1.4 and 1.0 points) and could be noise. Extending ablation across tasks is essential before claiming generalizability.
Third, code and data are not released, and NDR is an in-house dataset. That limits reproducibility, though the public benchmarks help.
Who is this for? Researchers working on LLM agent customization and prompt optimization. It is a solid engineering paper with a clear method and honest reporting, but the central claim about non-myopic paths needs stronger evidence before it becomes a settled result.
For peer review: I would send this to review, because the method is novel and the problem is relevant. But I would flag the need for uncertainty quantification, multi-task ablation, and open-source code as conditions for acceptance.
The strongest objection
The central claim that non-greedy search over skill edits yields better final skills relies heavily on the observation that in 67% of completed searches, the best validation state has an ancestor that was not highest-scoring at its depth (Section 4.4). The paper presents this as evidence that locally weaker states lead to better outcomes, supported by the example in Figure 4 where node 4 (2.9 points below its parent) eventually leads to the best state. However, the validation panels for these searches are small (e.g., 105 items for NDR, 40 for SpreadsheetBench, Table 4). With such panels, a difference of 2.9 points on NDR corresponds to roughly 3 items; a few items’ stochastic execution can easily flip the ordering. The paper acknowledges (Section 5, b203) that 'finite validation panels and stochastic execution limit the precision of small score differences,' but it does not quantify the uncertainty of the state scores used to define the 67% statistic. Without bootstrapping or confidence intervals, it is unclear how many of the observed 'non-myopic' paths would persist under repeated evaluation on the same panel. The statistic could be substantially inflated by ranking noise. Consequently, the strongest empirical support for the claim that local weak states systematically lead to better skills is less robust than presented.
The reader's weakest assumption identified the representativeness of the validation set and the risk of overfitting, which is related but not identical. The load-bearing concern here is more specific: the 67% statistic, which is central to the strongest claim (that non-greedy search discovers better skills via weak intermediates), may be inflated by noise because the validation panels are small and score differences are not quantified with uncertainty. The paper acknowledges the limitation but still uses the statistic as key evidence. The concrete test would directly assess the robustness of that statistic. The CONDITIONAL verdict already reflects caution, so no change is needed, but if the test shows instability, the strongest claim would require significant qualification. Since no new evidence is presented to overturn the verdict, UNCHANGED is appropriate.
The referee's report
If the reported improvements are robust, AtomSearch provides a principled way to decouple the granularity of skill edits from the evaluation horizon, addressing a known weakness of local greedy methods. The contrastive atomic editing interface is well-motivated and makes each revision traceable; the tree search over full skill states is a natural extension of MCTS to the skill-optimization setting. The concrete finding that locally weaker intermediate states can lead to better final skills (illustrated in Figure 4) is potentially important for the design of future agent skill optimizers, but this claim rests heavily on a single aggregate statistic whose uncertainty is not quantified. The empirical evaluation covers a reasonable range of tasks and models, and the paper honestly reports limitations, including small validation panels and the fact that the component ablation covers only one setting. The compactness of the resulting skills is an interesting emergent property that the paper does not explicitly optimize for.
Pith's reading of the paper (a machine's summary, not the authors')
AtomSearch proposes a method for optimizing agent skills (external instructions for frozen models) by keeping each revision action as a single local edit, but using a tree search to evaluate short compositions of edits and retain promising states. The core claim is that this decoupling of edit granularity from evaluation horizon allows the search to discover useful revision paths that a greedy optimizer would discard. The paper demonstrates that in 67% of completed searches, the best final validation state had an ancestor that was not the highest-scoring at its depth, confirming that locally weaker states can lead to better overall skills. Across four tasks and five target models, AtomSearch often outperforms the baseline SkillOpt while producing significantly shorter skills.
What carries the argument. Skill-State Tree Search (STS) is the central mechanism. It uses a Monte Carlo Tree Search backbone with progressive widening and UCT selection, where each node is a complete executable skill. The action space is contrastively grounded primitive edits (add, rewrite, delete), each justified by a specification linking it to evidence. STS decouples the local edit from its valuation by allowing short deterministic rollouts (up to two additional edits) to test whether a primitive edit becomes more valuable after further composition. If the rollout yields a higher score than the primitive child alone, the composition is kept as a persistent state. Outcome memory records repair/regression statistics per edit direction and feeds them to later proposal rounds without affecting UCT values.
Core claim. AtomSearch establishes that contrastive atomic editing—where each primitive rule edit is linked to failed and successful executions, a behavioral hypothesis, and intended scope—can serve as the action unit in a tree search over complete executable skills. Skill-State Tree Search (STS) applies these edits locally, evaluates short rollouts of up to two further edits, and retains compositions only if they score strictly higher than the primitive child. The key result is that 67% of the best validation states in completed searches trace back to an ancestor that was not the highest-scoring state at its depth, showing that non-greedy exploration over edit combinations can find stronger skills than local greedy optimization. This demonstrates that the horizon over which a revision's value is judged can be longer than the revision itself, and that keeping intermediate states that are not locally optimal can pay off.
The premise it rests on. The method assumes that the held-out validation set is large enough and representative enough to reliably distinguish skill quality, and that the contrastive mining step produces edit candidates that are genuinely useful for improving the skill; if the validation set is small or noisy, the tree search may overfit, as the paper acknowledges.
If the paper is right (Pith's inferences, not the authors')
If AtomSearch's method is adopted, agent skill optimization can become more sample-efficient in terms of final skill quality per retained edit, as the paper shows shorter skills achieving higher or comparable scores.
The decoupling of local edits from long-term valuation suggests that other iterative improvement tasks (e.g., prompt optimization, code repair) could benefit from tree search over atomic revisions rather than greedy acceptance.
The 67% finding implies that validation-set-driven greedy optimization may systematically miss better solutions that require temporarily worse intermediate states, challenging common practice.
The compact final skills, attained without explicit length penalty, indicate that tree search over local edits can automatically prune redundant rules, reducing inference cost and complexity.
The method's reliance on a held-out validation set for scoring introduces a risk of overfitting, especially given small validation panels (e.g., 18 items for LiveMathematicianBench) and stochastic model outputs, which the paper acknowledges but does not quantify.
AtomSearch's tree search increases offline compute (optimizer calls 16–45 per run). This trade-off is worthwhile when the resulting skill is reused many times, but for one-off tasks the search cost may outweigh the benefit.
The contrastive atomic editing step itself requires a method for mining behavioral hypotheses from failed/successful executions; the paper does not deeply analyze failures of this mining step, which may limit applicability to tasks where failure patterns are not easily separable.
The component ablation covers only one task-target setting (SpreadsheetBench+deepseek-v4-flash), so the marginal contributions of STS and outcome memory are not yet broadly validated.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Agent skills adapt frozen models through editable external instructions. Revising several rules at once makes changes difficult to inspect, while judging each local edit only by its immediate score can discard useful intermediate states. We introduce AtomSearch, which keeps revision actions local while using subsequent skill states to guide search. Contrastive atomic editing links each primitive rule edit to failed and successful executions, a behavioral hypothesis, and an intended scope. Skill-State Tree Search (STS) evaluates short edit compositions and selectively retains them as executable states that can be revisited and extended. Outcome memory supplies later proposals with observed repair and regression statistics. We evaluate AtomSearch across four tasks and five target models. With deepseek-v4-flash, it exceeds SkillOpt on three of four tasks, including a 9.68-percentage-point gain on LiveMathematicianBench, and SkillOpt's final skills contain 2.3-19.7 times as many characters. In 67% of completed searches, the best validation state passes through an ancestor that was not highest-scoring at its depth. These results show how local rule edits can support the exploration of useful revision paths and yield compact skills, with additional offline evaluations used to explore alternative states.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-15
The authors' main claim, as the classifier picked it out 0.99
“These results show how local rule edits can support the exploration of useful revision paths and yield compact skills, with additional offline evaluations used to explore alternative states.”
How the main claim is stated: asserted 0.99
What the abstract itself states, as probabilities: claims first 0.05 · states limitation 0.42 · human participants 0.05 · controlled comparison 0.83 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.98 · mentions code or data availability 0.04
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Agent skills adapt frozen models through editable external instructions. | background 1.00 | 0.08 | 0.04 | asserted 1.00 |
| Revising several rules at once makes changes difficult to inspect, while judging each local edit only by its immediate score can discard useful intermediate states. | background 0.68 | 0.14 | 0.03 | asserted 0.94 |
| We introduce AtomSearch, which keeps revision actions local while using subsequent skill states to guide search. | contribution 0.91 | 0.81 | 0.13 | asserted 0.99 |
| Contrastive atomic editing links each primitive rule edit to failed and successful executions, a behavioral hypothesis, and an intended scope. | method 0.94 | 0.79 | 0.04 | asserted 0.99 |
| Skill-State Tree Search (STS) evaluates short edit compositions and selectively retains them as executable states that can be revisited and extended. | method 0.98 | 0.84 | 0.07 | asserted 1.00 |
| Outcome memory supplies later proposals with observed repair and regression statistics. | method 0.99 | 0.77 | 0.03 | asserted 1.00 |
| We evaluate AtomSearch across four tasks and five target models. | method 0.67 | 0.94 | 0.03 | asserted 0.93 |
| With deepseek-v4-flash, it exceeds SkillOpt on three of four tasks, including a 9.68-percentage-point gain on LiveMathematicianBench, and SkillOpt's final skills contain 2.3-19.7 times as many characters. | result 1.00 | 0.95 | 0.26 | asserted 1.00 |
| In 67% of completed searches, the best validation state passes through an ancestor that was not highest-scoring at its depth. | result 0.99 | 0.95 | 0.09 | asserted 1.00 |
| These results show how local rule edits can support the exploration of useful revision paths and yield compact skills, with additional offline evaluations used to explore alternative states. | interpretation 1.00 | 0.86 | 0.79 | asserted 0.99 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.96
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-15: 1 request, 7,451 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (4)
Figures are the authors', reproduced from the paper.
References (73)
- [1] Yifan Yang and Ziyang Gong and Weiquan Huang and Qihao Yang and Ziwei Zhou and Zisu Huang and Yan Li and Xuemei Gao and Qi Dai and Bei Liu and Kai Qiu and Yuqing Yang and Dongdong Chen and Xue Yang and Chong Luo , booktitle =. 2026 , url = on Pith
- [2] Progressive Agent Skill Generation via Reinforcement Learning , author =. 2026 , eprint = on Pith
- [4] Contrastive Reflection for Iterative Prompt Optimization , author =. 2026 , eprint = on Pith
- [5] Xinyuan Wang and Chenxi Li and Zhen Wang and Fan Bai and Haotian Luo and Jiayou Zhang and Nebojsa Jojic and Eric P. Xing and Zhiting Hu , booktitle =. 2024 , url = on Pith
- [6] Jiayi Zhang and Jinyu Xiang and Zhaoyang Yu and Fengwei Teng and Xiong-Hui Chen and Jiaqi Chen and Mingchen Zhuge and Xin Cheng and Sirui Hong and Jinlin Wang and Bingnan Zheng and Bang Liu and Yuyu Luo and Chenglin Wu , booktitle =. 2025 , url = on Pith
- [7] Bilevel Optimization of Agent Skills via Monte Carlo Tree Search , author =. 2026 , eprint = on Pith
- [9] Bandit Based Monte-Carlo Planning , author =. Machine Learning: ECML 2006 , pages =. 2006 , publisher = on Pith
- [10] A Survey of Monte Carlo Tree Search Methods , author =. IEEE Transactions on Computational Intelligence and AI in Games , volume =. 2012 , doi = on Pith
- [11] Barry Zhang and Keith Lazuka and Mahesh Murag , title =. 2025 , month = on Pith
- [12] Zheng, Zhi and Xie, Zhuoliang and Wang, Zhenkun and Hooi, Bryan , booktitle =. Monte Carlo Tree Search for Comprehensive Exploration in. 2025 , publisher = on Pith
- [13] Shinn, Noah and Cassano, Federico and Gopinath, Ashwin and Narasimhan, Karthik and Yao, Shunyu , booktitle =. 2023 , url = on Pith
- [14] Zhao, Andrew and Huang, Daniel and Xu, Quentin and Lin, Matthieu and Liu, Yong-Jin and Huang, Gao , journal =. 2024 , doi = on Pith
- [15] Large Language Models Are Human-Level Prompt Engineers , author =. International Conference on Learning Representations , year = on Pith
- [16] Automatic Prompt Optimization with ``Gradient Descent'' and Beam Search , author =. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages =. 2023 , doi = on Pith
- [17] Large Language Models as Optimizers , author =. International Conference on Learning Representations , year = on Pith
- [18] Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers , author =. International Conference on Learning Representations , year = on Pith
- [19] Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution , author =. Proceedings of the 41st International Conference on Machine Learning , volume =. 2024 , publisher = on Pith
- [21] Khattab, Omar and Singhvi, Arnav and Maheshwari, Paridhi and Zhang, Zhiyuan and Santhanam, Keshav and Vardhamanan, Sri and Haq, Saiful and Sharma, Ashutosh and Joshi, Thomas and Moazam, Hanna and Miller, Heather and Zaharia, Matei and Potts, Christopher , booktitle =. 2024 , url = on Pith
- [22] Self-Refine: Iterative Refinement with Self-Feedback , author =. Advances in Neural Information Processing Systems , volume =. 2023 , url = on Pith
- [23] Gou, Zhibin and Shao, Zhihong and Gong, Yeyun and Shen, Yelong and Yang, Yujiu and Duan, Nan and Chen, Weizhu , booktitle =. 2024 , url = on Pith
show 53 more
- [26] Ma, Zeyao and Zhang, Bohan and Zhang, Jing and Yu, Jifan and Zhang, Xiaokang and Zhang, Xiaohan and Luo, Sijia and Wang, Xi and Tang, Jie , booktitle =. 2024 , doi = on Pith
- [29] On Over-Fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation , author =. Journal of Machine Learning Research , volume =. 2010 , url = on Pith
- [30] Agrawal, Lakshya A. and Tan, Shangyin and Soylu, Dilara and Ziems, Noah and Khare, Rishi and Opsahl-Ong, Krista and Singhvi, Arnav and Shandilya, Herumb and Ryan, Michael J. and Jiang, Meng and Potts, Christopher and Sen, Koushik and Dimakis, Alexandros G. and Stoica, Ion and ... on Pith
- [31] Agent Workflow Memory , author =. Proceedings of the 42nd International Conference on Machine Learning , pages =. 2025 , volume = on Pith
- [32] Wu, Shirley and Zhao, Shiyu and Huang, Qian and Huang, Kexin and Yasunaga, Michihiro and Cao, Kaidi and Ioannidis, Vassilis N. and Subbian, Karthik and Leskovec, Jure and Zou, James , booktitle =. 2024 , url = on Pith
- [33] Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models , author =. International Conference on Learning Representations , year = on Pith
- [35] Automated Design of Agentic Systems , author =. 2025 , booktitle = on Pith
- [36] Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models , author =. Proceedings of the 41st International Conference on Machine Learning , year = on Pith
- [37] Tree of Thoughts: Deliberate Problem Solving with Large Language Models , author =. Advances in Neural Information Processing Systems , volume =. 2023 , url = on Pith
- [38] Voyager: An Open-Ended Embodied Agent with Large Language Models , author =. 2023 , eprint = on Pith
- [39] Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory , author =. 2025 , eprint = on Pith
- [43] Lakshya A. Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J. Ryan, Meng Jiang, Christopher Potts, Koushik Sen, Alexandros G. Dimakis, Ion Stoica, Dan Klein, Matei Zaharia, and Omar Khattab. GEPA : Refle... on Pith
- [44] Cameron B. Browne, Edward Powley, Daniel Whitehouse, Simon M. Lucas, Peter I. Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, and Simon Colton. A survey of monte carlo tree search methods. IEEE Transactions on Computational Intelligence and AI... on Pithdoi:10.1109/tciaig.2012.2186810
- [45] Gavin C. Cawley and Nicola L. C. Talbot. On over-fitting in model selection and subsequent selection bias in performance evaluation. Journal of Machine Learning Research, 11: 0 2079--2107, 2010. URL https://jmlr.org/papers/v11/cawley10a.html on Pith
- [46] Matthew Dunn, Levent Sagun, Mike Higgins, V. Ugur Guney, Volkan Cirik, and Kyunghyun Cho. SearchQA : A new q&a dataset augmented with context from a search engine, 2017. URL https://arxiv.org/abs/1704.05179 on PithPith's reviewarXiv
- [47] Yuru Feng, Yaoqi Chen, Beidi Zhao, Qianxi Zhang, Xinjiang Wang, Jianan Lu, Zhirui Wang, Shusen Xu, Zengzhong Li, and Qi Chen. SkillHEX : Improving agent skills via hypothesis-driven autonomous exploration and exploitation, 2026. URL https://arxiv.org/abs/2608.05628 on PithPith's reviewarXiv
- [48] Chrisantha Fernando, Dylan Sunil Banarse, Henryk Michalewski, Simon Osindero, and Tim Rockt \"a schel. Promptbreeder: Self-referential self-improvement via prompt evolution. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of M... on Pith
- [49] Yudong Gao, Zongjie Li, Yuanyuan Yuan, Zimo Ji, Pingchuan Ma, and Shuai Wang. SkillReducer : Optimizing LLM agent skills for token efficiency, 2026. URL https://arxiv.org/abs/2603.29919 on PithPith's reviewarXiv
- [50] Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. CRITIC : Large language models can self-correct with tool-interactive critiquing. In International Conference on Learning Representations, 2024. URL https://arxiv.org/abs/2305.11738 on PithPith's reviewarXiv
- [51] Qingyan Guo, Rui Wang, Junliang Guo, Bei Li, Kaitao Song, Xu Tan, Guoqing Liu, Jiang Bian, and Yujiu Yang. Connecting large language models with evolutionary algorithms yields powerful prompt optimizers. In International Conference on Learning Representations, 2024. URL https:... on PithPith's reviewarXiv
- [52] Linyang He, Qiyao Yu, Hanze Dong, Baohao Liao, Xinxing Xu, Micah Goldblum, Jiang Bian, and Nima Mesgarani. LiveMathematicianBench : A live benchmark for mathematician-level reasoning with proof sketches, 2026. URL https://arxiv.org/abs/2604.01754 on PithPith's reviewarXiv
- [53] Shengran Hu, Cong Lu, and Jeff Clune. Automated design of agentic systems. In International Conference on Learning Representations, 2025. URL https://arxiv.org/abs/2408.08435 on PithPith's reviewarXiv
- [54] Chenyi Huang, Haoting Zhang, Jingxu Xu, Zeyu Zheng, and Yunduan Lin. Bilevel optimization of agent skills via monte carlo tree search, 2026. URL https://arxiv.org/abs/2604.15709 on PithPith's reviewarXiv
- [55] Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. DSPy : Compiling declarative language model calls into state-of-the... on PithPith's reviewarXiv
- [56] Levente Kocsis and Csaba Szepesv \'a ri. Bandit based monte-carlo planning. In Machine Learning: ECML 2006, pp.\ 282--293. Springer, 2006 on Pith
- [57] Derek Koh, Jinghui Mo, Benjamin H. Le, Jiening Zhan, Baofen Zheng, Kevin Bevis, Nathaniel C. Owen, Lauren Elizabeth Charney, Wenqiong Liu, and Jingwei Wu. Contrastive reflection for iterative prompt optimization, 2026. URL https://arxiv.org/abs/2606.30840 on PithPith's reviewarXiv
- [58] Yuxuan Liu, Zhaochen Su, Lingyun Xie, Yuhao Zhang, Qing Zong, Jiahe Guo, Zhongwei Xie, Yiyan Ji, Yauwai Yim, Hongyu Luo, Xiyu Ren, Chenyu Ruan, Haoran Li, and Yangqiu Song. SkillRevise : Improving llm-authored agent skills via trace-conditioned skill revision, 2026. URL https:... on PithPith's reviewarXiv
- [59] Zeyao Ma, Bohan Zhang, Jing Zhang, Jifan Yu, Xiaokang Zhang, Xiaohan Zhang, Sijia Luo, Xi Wang, and Jie Tang. SpreadsheetBench : Towards challenging real world spreadsheet manipulation. In Advances in Neural Information Processing Systems, volume 37, 2024. doi:10.52202/079017-... on PithPith's reviewarXiv
- [60] Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterativ... on PithPith's reviewarXiv
- [61] Krista Opsahl-Ong, Michael J. Ryan, Josh Purtell, David Broman, Christopher Potts, Matei Zaharia, and Omar Khattab. Optimizing instructions and demonstrations for multi-stage language model programs. In Proceedings of the 2024 Conference on Empirical Methods in Natural Languag... on Pithdoi:10.18653/v1/2024.emnlp-main.525
- [62] Savvas Petridis, Ben Wedin, Ann Yuan, James Wexler, and Nithum Thain. ConstitutionalExperts : Training a mixture of principle-based prompts. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp.\ 574--582, Bang... on Pithdoi:10.18653/v1/2024.acl-short.52
- [63] Reid Pryzant, Dan Iter, Jerry Li, Yin Lee, Chenguang Zhu, and Michael Zeng. Automatic prompt optimization with ``gradient descent'' and beam search. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp.\ 7957--7968, 2023. doi:10.18653/v... on Pithdoi:10.18653/v1/2023.emnlp-main.494
- [64] Junhao Shen, Zhanqiu Zhang, Yiwen Guo, and Hong Cheng. Progressive agent skill generation via reinforcement learning, 2026. URL https://arxiv.org/abs/2608.01678 on PithPith's reviewarXiv
- [65] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion : Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, volume 36, 2023. URL https://proceedings.neurips.cc/paper_files/paper/2023/h... on Pith
- [66] Mirac Suzgun, Mert Yuksekgonul, Federico Bianchi, Dan Jurafsky, and James Zou. Dynamic cheatsheet: Test-time learning with adaptive memory, 2025. URL https://arxiv.org/abs/2504.07952 on PithPith's reviewarXiv
- [67] Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models, 2023. URL https://arxiv.org/abs/2305.16291 on PithPith's reviewarXiv
- [68] Xinyuan Wang, Chenxi Li, Zhen Wang, Fan Bai, Haotian Luo, Jiayou Zhang, Nebojsa Jojic, Eric P. Xing, and Zhiting Hu. PromptAgent : Strategic planning with language models enables expert-level prompt optimization. In International Conference on Learning Representations, 2024. U... on Pith
- [69] Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, and Graham Neubig. Agent workflow memory. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pp.\ 63897--63911. PMLR, 2025. URL https://proceedings.mlr.pr... on Pith
- [70] Shirley Wu, Shiyu Zhao, Qian Huang, Kexin Huang, Michihiro Yasunaga, Kaidi Cao, Vassilis N. Ioannidis, Karthik Subbian, Jure Leskovec, and James Zou. AvaTaR : Optimizing LLM agents for tool usage via contrastive reasoning. In Advances in Neural Information Processing Systems, ... on Pith
- [71] Siwei Wu, Jincheng Ren, Yizhi Li, Haau-Sing Li, Chengran Yang, Yuxuan Zhang, Weicheng Gu, Jian Yang, Riza Batista-Navarro, Chuanyi Zhang, Xianglong Liu, Ming Zhou, Bryan Dai, and Chenghua Lin. ModularRSI : Modular and generalizable recursive harness self-improvement, 2026. URL... on PithPith's reviewarXiv
- [72] Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, and Xinyun Chen. Large language models as optimizers. In International Conference on Learning Representations, 2024. URL https://arxiv.org/abs/2309.03409 on PithPith's reviewarXiv
- [73] Yifan Yang, Ziyang Gong, Weiquan Huang, Qihao Yang, Ziwei Zhou, Zisu Huang, Yan Li, Xuemei Gao, Qi Dai, Bei Liu, Kai Qiu, Yuqing Yang, Dongdong Chen, Xue Yang, and Chong Luo. SkillOpt : Executive strategy for self-evolving agent skills. In Advances in Neural Information Proces... on PithPith's reviewarXiv
- [74] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In Advances in Neural Information Processing Systems, volume 36, 2023. URL https://arxiv.org/abs/2305.10601 on PithPith's reviewarXiv
- [75] Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Zhi Huang, Carlos Guestrin, and James Zou. TextGrad : Automatic ``differentiation'' via text, 2024. URL https://arxiv.org/abs/2406.07496 on PithPith's reviewarXiv
- [76] Eric Zelikman, Eliana Lorch, Lester Mackey, and Adam Tauman Kalai. Self-taught optimizer ( STOP ): Recursively self-improving code generation, 2023. URL https://arxiv.org/abs/2310.02304 on PithPith's reviewarXiv
- [77] Barry Zhang, Keith Lazuka, and Mahesh Murag. Equipping agents for the real world with agent skills. Anthropic Engineering, October 2025 a . URL https://www.anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills. Published October 16, 2025 on Pith
- [78] Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune. Darwin G \"o del machine: Open-ended evolution of self-improving agents, 2025 b . URL https://arxiv.org/abs/2505.22954 on PithPith's reviewarXiv
- [79] Jiayi Zhang, Jinyu Xiang, Zhaoyang Yu, Fengwei Teng, Xiong-Hui Chen, Jiaqi Chen, Mingchen Zhuge, Xin Cheng, Sirui Hong, Jinlin Wang, Bingnan Zheng, Bang Liu, Yuyu Luo, and Chenglin Wu. AFlow : Automating agentic workflow generation. In The Thirteenth International Conference o... on Pith
- [80] Qizheng Zhang, Changran Hu, Shubhangi Upasani, Boyuan Ma, Fenglu Hong, Vamsidhar Kamanuru, Jay Rainton, Chen Wu, Mengmeng Ji, Hanchen Li, Urmish Thakker, James Zou, and Kunle Olukotun. Agentic context engineering: Evolving contexts for self-improving language models. In Intern... on PithPith's reviewarXiv
- [81] Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. ExpeL : LLM agents are experiential learners. Proceedings of the AAAI Conference on Artificial Intelligence, 38 0 (17): 0 19632--19642, 2024. doi:10.1609/aaai.v38i17.29936. URL https://ojs.aaai.o... on Pithdoi:10.1609/aaai.v38i17.29936
- [82] Zhi Zheng, Zhuoliang Xie, Zhenkun Wang, and Bryan Hooi. Monte carlo tree search for comprehensive exploration in LLM -based automatic heuristic design. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Resear... on Pith
- [83] Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman, Haohan Wang, and Yu-Xiong Wang. Language agent tree search unifies reasoning, acting, and planning in language models. In Proceedings of the 41st International Conference on Machine Learning, 2024. URL https://arxiv.org/abs/2310.04406 on PithPith's reviewarXiv
- [84] Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers. In International Conference on Learning Representations, 2023. URL https://arxiv.org/abs/2211.01910 on PithPith's reviewarXiv
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment