Artificial Intelligence · arXiv:2610.32026 v1 · submitted 23 Oct 2026 · 35 references
RSI-Opt: Recursive Self-Improvement Optimizer Agents For Model Training
The paper says
“Conventional neural-network training typically follows a predefined optimization recipe, motivating a LLM agent that monitors training dynamics and adjusts optimizer settings as training progresses, but a fixed agent policy does not accumulate experience across runs.”
more of the abstract
We introduce RSI-Opt, a plug-and-play Recursive Self-Improvement Optimizer Agents which evolves the knowledge that guides this process: a prompt, a playbook of procedures, a library of tested optimizer code, and a decision cadence. A meta-agent rewrites this policy using evidence from earlier runs, and an archive retains evaluated policies for further evolution. Across 5 generations on three training tasks, the meta-agents develop reusable procedures and code and conduct their own CPU proxy experiments. RSI-Opt substantially improves optimizer-agent performance, with representative raw-space summary z-scores increasing from + 10.48 to + 14.33 on the evolution suite with Opus 5.5 and from −0.36 to + 3.11 on held-out tasks with GPT-5.5 through cross-model policy transfer. On the Burgers PDE, it reduces mean relative L2 error by approximately 2.7-fold with Opus 5.5 and 8.8-fold with GPT-6.1 over the fixed-prompt agent baseline, although PDE transfer varies across LLMs and tasks. Controlled comparisons show that the evolved policy combines reusable optimization knowledge with adaptation to each task: initial decisions capture most observed gains on seen tasks, while continued adaptation improves outcomes on unseen tasks. These findings demonstrate how procedures and executable tools accumulated across training runs can improve optimizer-agent performance and support transfer to scientific machine learning.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 26 Oct 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up. A clever framework for evolving optimizer-agent policies across runs, but the empirical evaluation is too narrow to support the broad claims of substantial improvement. This line is wrong
3 major objections · 5 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 35 references say.
The paper says, at Abstract; §5.1; §5.4
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.83):
“Broader task coverage, additional independent evolution runs, and a separate policy-validation suite would strengthen claims about generalization and the reliability of the search procedure.”
Pith says: Objection 1 of 3 · major
The abstract's central claim that 'RSI-Opt substantially improves optimizer-agent performance' is stronger than the evidence from a single evolution trajectory. Section 5.1 reports one run of 21 policies over five generations, and §5.4 shows that held-out ML transfer peaks after generation 1 and then declines as policies specialize.
more of objection 1
Table 3 further shows that for Opus 5.5 the aggregate held-out score is lower for RSI-Opt (+4.87) than for the single-agent baseline (+5.22). The authors acknowledge the narrow base and the need for independent evolution runs in §6, so I do not question the honesty of the reporting; the issue is that the headline is not conditioned on the tested conditions and model/task. Please rephrase the abstract and introduction to say that RSI-Opt can substantially improve optimizer-agent performance in the tested settings, with selective transfer, rather than presenting the representative examples as a uniform improvement.
The paper says, at §3.3; Table 3
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.60):
“These are descriptive measurements, not evidence that each component caused the performance change.”
Pith says: Objection 2 of 3 · major
The headline quantitative differences are expressed as z-scores normalized by each task's static seed standard deviation. Because the static standard deviations are very small for GPT-34M/FineWeb (0.008) and Flow UNet/MNIST (0.0004), these tasks dominate the training-suite aggregate, and the paper itself notes that the normalization is 'not a test of statistical significance.'
more of objection 2
The abstract's 'substantially improves' nevertheless relies on these descriptive z units. Since per-seed raw outcomes are available, please add paired seed-level effect sizes or bootstrap intervals for the main baseline-versus-RSI-Opt group comparisons, or state explicitly in the abstract that the headline numbers are descriptive normalized aggregates and not statistical evidence. The raw metrics in Table 2 support the qualitative conclusions, so this is a reporting issue rather than a flaw in the experimental design.
The paper says, at §5.1; Section A
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.92):
“Since archive summaries can transmit information between lineages, the direct-access audit alone does not certify the absence of indirect exposure.”
Pith says: Objection 3 of 3 · major
The held-out claims are the primary evidence against overfitting, but Section A states that the audit cannot rule out indirect exposure through archive summaries: meta-agents can read scores and changelogs from other archive branches, and the paper acknowledges that a complete provenance audit would need to track whether the excluded lineage's summaries remained visible after exclusion.
more of objection 3
I accept the authors' statement that no direct reads of held-out material were found beyond the one reported case, and I am not treating indirect propagation as established. However, this caveat is load-bearing for the transfer results and should be stated at the point where held-out results are first summarized in §5.3, rather than appearing only in the appendix. A one-sentence cross-reference or caveat in the results section would suffice.
5 minor objections
The paper says, at §4
Pith says: minor objection 1 of 5
The first sentence of Section 4 is garbled: 'We compare baseline single-agent baseline that studies a coding agent...' appears to be missing punctuation and repeated words; please rewrite it.
The paper says, at §3.3
Pith says: minor objection 2 of 5
The parent-selection distribution contains a temperature parameter λ, but no numerical value or sensitivity analysis is reported. Please specify the value used; if λ was not tuned, say so.
show 3 more
The paper says, at §3.3, §4
Pith says: minor objection 3 of 5
The term 'eligible parent' is used but not precisely defined. Please state that eligibility excludes the exposed lineage and its descendant, and clarify whether any other criterion (e.g., validation success or score threshold) applies.
The paper says, at §5.5
Pith says: minor objection 4 of 5
The first-change-frozen condition replays the first modified updater from a full-agent run; that first change may have been produced with knowledge that later decisions would follow. The one-decision condition partly addresses this, but the text should explicitly note this conditioning when interpreting the two controls.
The paper says, at §3.3
Pith says: minor objection 5 of 5
The term 'z-score' could be confused with a standard normal test statistic; consider renaming it 'normalized score' or adding a sentence in the caption of Table 3 that these are descriptive normalizations relative to static seed spread.
What would settle it
Replicate the evolution with an expanded training suite that includes all three current held-out ML tasks (ViT/CIFAR-100, LSTM/WikiText-2, ResNet-18/Noisy CIFAR-10) as training tasks. Reserve the PDE tasks and a new set of ML tasks (e.g., different architecture or dataset) for held-out evaluation. Compare fresh-seed results on the original held-out tasks. If the improvement pattern (large gains for GPT-5.5 and GPT-6.1, mixed for Opus 5.5) replicates, the overfitting concern is reduced. If improvements vanish or reverse, the original findings are specific to the three-task suite.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 26 Oct 2026
- Reader holds up
- Skeptic agrees
- Editor wrote the verdict above
- Referee minor revision: 3 major objections, 5 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
This paper introduces RSI-Opt, a recursive self-improvement framework that evolves an optimizer agent's policy (prompt, playbook, code library, cadence) across training runs. The core idea is sensible and the implementation is clean: separate the within-run agent from the cross-run meta-agent, archive policies, and transfer procedures. The paper deserves credit for its transparent reporting—it documents a known held-out data exposure, runs audits, and clearly states its limitations. The controlled comparisons (frozen first change, one-decision, ablated components) are a genuine strength, and the PDE transfer experiments add useful scope.
That said, the central empirical claim—that RSI-Opt substantially improves optimizer-agent performance—rests on a very narrow base. Evolution runs on three ML tasks with only two seeds per policy. The paper's own results show that held-out ML performance peaks early, and for Opus 5.5 the aggregate held-out summary score is actually lower than the baseline (+4.87 vs +5.22). The stress-test concern about overfitting to a small training suite is well-founded; the two-seed evaluation during evolution amplifies selection risk, and the archive can indirectly propagate information across lineages. The paper acknowledges these issues, but acknowledgment doesn't magically widen the evidence.
The method is not conceptually flawed—it's an incremental but useful advance in automated ML. The question is whether the experiments as presented warrant the level of generality claimed. For now, the results are suggestive but not conclusive. A reader interested in LLM-based optimization will find the architecture and controls worth studying, but anyone citing the 'substantial improvement' claim should do so with the narrow conditions attached.
I'd send this to peer review—the problem is important and the methodology is well-structured. But I'd ask for either scaled-up experiments (more tasks, more seeds, independent evolution runs) or a significant toning-down of the claims. Right now the paper oversells its empirical support.
The strongest objection
The paper's core evidence for improvement comes from evolution on just three training tasks (ResNet-18/CIFAR-10, GPT-34M/FineWeb, Flow UNet/MNIST) with only two seeds per policy. The paper acknowledges that held-out ML performance peaks early and later generations specialize to this small suite. For Opus 5.5, the aggregate held-out summary score for RSI-Opt is +4.87 versus the single-agent baseline's +5.22, meaning the 'substantial improvement' is not uniform and depends critically on the model and task. The two-seed evaluation during evolution amplifies selection risk; although fresh seeds are used, the tasks themselves are unchanged. The ablation studies show the evolved library template can harm held-out ML performance. Thus the central claim that RSI-Opt substantially improves optimizer-agent performance rests on a narrow empirical base that does not guarantee generalizable knowledge beyond the tested conditions.
The reader correctly identified the weakest assumption as overfitting to the small training suite and potential information leakage. This stress test confirms that this is the single most load-bearing concern. However, the paper is fully transparent about this limitation—it acknowledges specialization, the small suite, the early held-out peak, and does not overclaim universal improvement. The reader's ACCEPT verdict remains appropriate because the paper's claims are honestly scoped and supported for the tested conditions. The concrete test would strengthen the findings but is not required to justify the current verdict.
The referee's report
If taken at face value, the paper makes a useful empirical contribution to agentic optimizer search: it introduces an explicit, inspectable policy representation for optimizer agents, connects it to archive-based policy evolution, and tests transfer across tasks and LLMs. The experimental design is considerably more careful than is typical for this area: final evaluation uses fresh seeds that do not guide selection, top candidates are re-evaluated before the final choice, held-out tasks are reserved, PDE transfer is tested, and the authors provide controls that separate the contribution of reusable code, the first optimizer change, and continued adaptation. The paper is also unusually candid about its limitations: it acknowledges that the evolution suite is small, that held-out ML transfer peaks early, that z-score aggregates are descriptive and not significance tests, that the audit cannot fully rule out archive-mediated indirect exposure, and that no matched-budget inheritance control was run. These disclosures are a genuine strength.
Pith's reading of the paper (a machine's summary, not the authors')
This paper introduces RSI-Opt, a framework that lets an LLM-powered optimizer agent accumulate and refine its own policy (prompt, playbook, code library, and decision cadence) across multiple training runs. An outer meta-agent rewrites the policy using evidence from previous runs, and an archive stores evaluated policies for further evolution. The authors find that the evolved policy substantially improves the training-suite performance (e.g., z-scores rising from +10.48 to +14.33 with Opus 5.5) and can transfer to held-out ML and PDE tasks when paired with a different LLM, though the transfer is selective and can degrade if the policy overfits to the narrow evolution suite. The work demonstrates that accumulating reusable optimization knowledge is feasible and beneficial, but it also exposes the risk of specialization when the evolution set is small.
What carries the argument. The recursive self-improvement loop: a meta-agent samples a parent policy and run evidence from an archive, rewrites the policy (prompt, playbook, code library, cadence), validates it, evaluates the child on a training suite, and adds the child to the archive. The inner-loop agent uses the evolved policy to monitor training dynamics and edit optimizer and diagnostic programs at scheduled decision points. The archive, following the Darwin Gödel Machine approach, enables exploration and inheritance without a strict improvement filter.
Core claim. The paper shows that recursively evolving an optimizer agent's policy—its prompt, playbook of procedures, library of optimizer code, and decision cadence—across training runs improves performance beyond a fixed-prompt agent. On three training-suite tasks, the best archive policy achieves a raw-space summary z-score of +14.33 versus +10.48 for the baseline. When transferred unchanged to a different LLM (GPT-5.5), the same policy raises the held-out aggregate score from -0.36 to +3.11. However, the transfer is not uniform: Opus 5.5 improves four of six held-out tasks but has a lower aggregate held-out score, and LSTM loss worsens for both GPT models. The evolved policy combines reusable modules (Muon on convolutional layers, parameter-role checks, learning-rate schedules) with task-conditioned online adaptation, but initial decisions capture most of the seen-task benefit, while later decisions improve unseen-task outcomes for both baseline and evolved agents.
The premise it rests on. The three training tasks and two seeds per policy during evolution are sufficient to produce a policy that generalizes to unseened tasks.
If the paper is right (Pith's inferences, not the authors')
Optimizer agents can be made to accumulate and reuse knowledge across runs, potentially reducing the cost of re-discovering effective optimization strategies for new tasks.
The evolved policy is inspectable and transferable across different LLMs, enabling cross-model portability of optimization expertise.
The method reveals a trade-off: a small training suite leads to specialization, so future applications should use broader evolution sets or separate validation suites to maintain generalization.
The framework can be extended beyond optimizer agents to other types of coding agents that interact with training loops, such as hyperparameter tuners or architecture searchers.
The selective transfer and early peak in held-out ML performance suggest that a validation-based early stopping criterion could be derived from the archive's held-out trajectories without additional evaluation cost.
If the evolution suite were expanded to include tasks with diverse objectives and data distributions, specialization would likely decrease, and the policy might generalize better—this is a natural next experiment.
The reliance on a single LLM (Opus 5.5) for meta-agent and in-run agent during evolution raises the question of whether using a weaker or stronger model as meta-agent would change the knowledge quality or specialization dynamics.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Conventional neural-network training typically follows a predefined optimization recipe, motivating a LLM agent that monitors training dynamics and adjusts optimizer settings as training progresses, but a fixed agent policy does not accumulate experience across runs. We introduce RSI-Opt, a plug-and-play Recursive Self-Improvement Optimizer Agents which evolves the knowledge that guides this process: a prompt, a playbook of procedures, a library of tested optimizer code, and a decision cadence. A meta-agent rewrites this policy using evidence from earlier runs, and an archive retains evaluated policies for further evolution. Across 5 generations on three training tasks, the meta-agents develop reusable procedures and code and conduct their own CPU proxy experiments. RSI-Opt substantially improves optimizer-agent performance, with representative raw-space summary z-scores increasing from + 10.48 to + 14.33 on the evolution suite with Opus 5.5 and from −0.36 to + 3.11 on held-out tasks with GPT-5.5 through cross-model policy transfer. On the Burgers PDE, it reduces mean relative L2 error by approximately 2.7-fold with Opus 5.5 and 8.8-fold with GPT-6.1 over the fixed-prompt agent baseline, although PDE transfer varies across LLMs and tasks. Controlled comparisons show that the evolved policy combines reusable optimization knowledge with adaptation to each task: initial decisions capture most observed gains on seen tasks, while continued adaptation improves outcomes on unseen tasks. These findings demonstrate how procedures and executable tools accumulated across training runs can improve optimizer-agent performance and support transfer to scientific machine learning.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-26
The authors' main claim, as the classifier picked it out 0.99
“These findings demonstrate how procedures and executable tools accumulated across training runs can improve optimizer-agent performance and support transfer to scientific machine learning.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.08 · states limitation 0.93 · human participants 0.03 · controlled comparison 0.97 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.97 · mentions code or data availability 0.06
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Conventional neural-network training typically follows a predefined optimization recipe, motivating a LLM agent that monitors training dynamics and adjusts optimizer settings as training progresses, but a fixed agent policy does not accumulate experience across runs. | background 0.63 | 0.08 | 0.03 | asserted 1.00 |
| We introduce RSI-Opt, a plug-and-play Recursive Self-Improvement Optimizer Agents which evolves the knowledge that guides this process: a prompt, a playbook of procedures, a library of tested optimizer code, and a decision cadence. | contribution 0.97 | 0.87 | 0.06 | asserted 1.00 |
| A meta-agent rewrites this policy using evidence from earlier runs, and an archive retains evaluated policies for further evolution. | method 0.96 | 0.74 | 0.07 | asserted 0.99 |
| Across 5 generations on three training tasks, the meta-agents develop reusable procedures and code and conduct their own CPU proxy experiments. | method 0.71 | 0.93 | 0.06 | asserted 1.00 |
| RSI-Opt substantially improves optimizer-agent performance, with representative raw-space summary $z$-scores increasing from $+10.48$ to $+14.33$ on the evolution suite with Opus 5.5 and from $-0.36$ to $+3.11$ on held-out tasks with GPT-5.5 through cross-model policy transfer. | result 1.00 | 0.95 | 0.30 | asserted 1.00 |
| On the Burgers PDE, it reduces mean relative $L_2$ error by approximately 2.7-fold with Opus 5.5 and 8.8-fold with GPT-6.1 over the fixed-prompt agent baseline, although PDE transfer varies across LLMs and tasks. | result 1.00 | 0.95 | 0.13 | asserted 1.00 |
| Controlled comparisons show that the evolved policy combines reusable optimization knowledge with adaptation to each task: initial decisions capture most observed gains on seen tasks, while continued adaptation improves outcomes on unseen tasks. | interpretation 0.84 | 0.94 | 0.23 | asserted 1.00 |
| These findings demonstrate how procedures and executable tools accumulated across training runs can improve optimizer-agent performance and support transfer to scientific machine learning. | interpretation 1.00 | 0.91 | 0.85 | asserted 1.00 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.61
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-26: 1 request, 6,569 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (3)
Figures are the authors', reproduced from the paper.
References (35)
- [1] Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W. Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando de Freitas. Learning to learn by gradient descent by gradient descent. InAdvances in Neural Information Processing Systems, 2016 on Pith
- [2] Xingjian Bai, Jia-Hao Liang, Jayden Teoh, and Vincent Sitzmann. Reactive optimizers. https://xingjianbai.com/reactive-optimizers/, October 2026. OPAL software:https: //github.com/xingjian-bai/opal on Pith
- [3] Lang Cao, Binghang Lu, Yuhao Shen, and Yue Guo. Medrouter: Demystifying knowledge differences across medical llms for routing-based reasoning.arXiv preprint arXiv:2609.33119, 2026 on PithPith's reviewarXiv
- [4] Xiangning Chen, Chen Liang, Da Huang, Esteban Real, Kaiyuan Wang, Yao Liu, Hieu Pham, Xuanyi Dong, Thang Luong, Cho-Jui Hsieh, Yifeng Lu, and Quoc V. Le. Symbolic discovery of optimization algorithms.arXiv preprint arXiv:2302.06675, 2023 on PithPith's reviewarXiv
- [5] Zheyuan Deng, Binghang Lu, Hanqi Feng, Shirley Huang, Dianzhuo Wang, Yuanda Xu, Zhiwei Zhang, Yige Sun, Changhong Mou, Runyu Zhang, et al. Contramem: Learning self-evolving procedural memory from contrasting multi-model trajectories.arXiv preprint arXiv:2608.22533, 2026 on PithPith's reviewarXiv
- [6] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at... on Pith
- [7] Chrisantha Fernando, Dylan Banarse, Henryk Michalewski, Simon Osindero, and Tim Rock- täschel. Promptbreeder: Self-referential self-improvement via prompt evolution.arXiv preprint arXiv:2309.16797, 2023 on PithPith's reviewarXiv
- [8] Vineet Gupta, Tomer Koren, and Yoram Singer. Shampoo: Preconditioned stochastic tensor optimization. InInternational Conference on Machine Learning, 2018 on Pith
- [9] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InIEEE Conference on Computer Vision and Pattern Recognition, 2016 on Pith
- [10] Shengran Hu, Cong Lu, and Jeff Clune. Automated design of agentic systems.arXiv preprint arXiv:2408.08435, 2024 on PithPith's reviewarXiv
- [11] Max Jaderberg, Valentin Dalibard, Simon Osindero, Wojciech M. Czarnecki, Jeff Donahue, Ali Razavi, Oriol Vinyals, Tim Green, Iain Dunning, Karen Simonyan, Chrisantha Fernando, and Koray Kavukcuoglu. Population based training of neural networks.arXiv preprint arXiv:1711.09846, 2017 on PithPith's reviewarXiv
- [12] Keller Jordan, Yuchen Jin, Vlado Boza, Jiacheng You, Franz Cesista, Laker Newhouse, and Jeremy Bernstein. Muon: An optimizer for hidden layers in neural networks. https:// kellerjordan.github.io/posts/muon/, 2024 on Pith
- [13] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015 on Pith
- [14] Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009 on Pith
- [15] Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. InInternational Conference on Learning Representations, 2023 on Pith
- [16] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InInternational Conference on Learning Representations, 2019 on Pith
- [17] Binghang Lu, Zheyuan Deng, Runyu Zhang, Bing Hu, Yunhan Zhao, Yuan Tian, Changhong Mou, Guang Lin, and Xiaomin Li. Muon-ogd: Muon-based spectral orthogonal gradient projection for llm continual learning.arXiv preprint arXiv:2605.08949, 2026 on PithPith's reviewarXiv
- [18] Binghang Lu, Runyu Zhang, Changhong Mou, Na Li, and Guang Lin. Adamflip: Adaptive momentum feedback linearization optimization for hard constrained pinn training.arXiv preprint arXiv:2605.08408, 2026 on PithPith's reviewarXiv
- [19] Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. InInternational Conference on Learning Representations, 2017 on Pith
- [20] Luke Metz, James Harrison, C. Daniel Freeman, Amil Merchant, Lucas Beyer, James Bradbury, Naman Agrawal, Ben Poole, Igor Mordatch, Adam Roberts, and Jascha Sohl-Dickstein. VeLO: Training versatile learned optimizers by scaling up.arXiv preprint arXiv:2211.09760, 2022 on PithPith's reviewarXiv
show 15 more
- [21] Alexander Novikov, Ngân V˜ u, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco J. R. Ruiz, Abbas Mehrabian, et al. AlphaEvolve: A coding agent for scientific and algorithmic discovery.arXiv preprint arXiv:25... on PithPith's reviewarXiv
- [22] Guofei Pang, Lu Lu, and George Em Karniadakis. fPINNs: Fractional physics-informed neural networks.SIAM Journal on Scientific Computing, 41(4):A2603–A2626, 2019 on Pith
- [23] Guilherme Penedo, Hynek Kydlíček, Loubna Ben Allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, and Thomas Wolf. The FineWeb datasets: Decanting the web for the finest text data at scale.arXiv preprint arXiv:2406.17557, 2024 on PithPith's reviewarXiv
- [24] Maziar Raissi, Paris Perdikaris, and George Em Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.Journal of Computational Physics, 378:686–707, 2019 on Pith
- [25] Pratik Rathore, Weimu Lei, Zachary Frangella, Lu Lu, and Madeleine Udell. Challenges in training PINNs: A loss landscape perspective. InInternational Conference on Machine Learning, 2024 on Pith
- [26] Jürgen Schmidhuber. Gödel machines: Fully self-referential optimal universal self-improvers. InArtificial General Intelligence, pages 199–226. Springer, 2007 on Pith
- [27] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Re- flexion: Language agents with verbal reinforcement learning. InAdvances in Neural Information Processing Systems, 2023 on Pith
- [28] Yuan Tian, Bing Hu, Hao Wang, Binghang Lu, and Fang Wu. Harness evolution hits a ceiling: When weight training should begin.arXiv preprint arXiv:2610.11655, 2026 on PithPith's reviewarXiv
- [29] Yuan Tian, Bing Hu, Fang Wu, Xiaomin Li, Binghang Lu, and Neil Zhenqiang Gong. When think-with-image meets safety: What determines multimodal jailbreak robustness?arXiv preprint arXiv:2605.27932, 2026 on PithPith's reviewarXiv
- [30] Nikhil Vyas, Depen Morwani, Rosie Zhao, Itai Shapira, David Brandfonbrener, Lucas Janson, and Sham Kakade. SOAP: Improving and stabilizing Shampoo using Adam.arXiv preprint arXiv:2409.11321, 2024 on PithPith's reviewarXiv
- [31] Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291, 2023 on PithPith's reviewarXiv
- [32] Sifan Wang, Yujun Teng, and Paris Perdikaris. Understanding and mitigating gradient flow pathologies in physics-informed neural networks.SIAM Journal on Scientific Computing, 43 (5):A3055–A3081, 2021 on Pith
- [33] Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, and Xinyun Chen. Large language models as optimizers.arXiv preprint arXiv:2309.03409, 2023 on PithPith's reviewarXiv
- [34] Eric Zelikman, Eliana Lorch, Lester Mackey, and Adam Tauman Kalai. Self-taught optimizer (STOP): Recursively self-improving code generation.arXiv preprint arXiv:2310.02304, 2023 on PithPith's reviewarXiv
- [35] Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune. Darwin Gödel machine: Open-ended evolution of self-improving agents.arXiv preprint arXiv:2505.22954, 2025. 18 A Appendix The harness records every agent tool call. The audit scans 528 in-run transcripts (18,895 t... on PithPith's reviewarXiv
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment