REVIEW 4 major objections 5 minor 42 references
Persistent Recursive Worlds Enable Autonomous Software Evolution
T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read A project-centered agent architecture formed a 249k-line C compiler, continued development across a foundation-model swap, and ported MESA modules to Rust with 1.55-6.87x measured speedups.
desk verdict A credible, unusually honest large-scale capability report whose headline causal claim — persistent project rather than persistent agent — the experiments do not actually support. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
Work is split by recursion: a root agent delegates to a sub-folder, which can delegate deeper. Only accepted changes move the project forward, so rejected work leaves no trace. The authors ran three tests. First, a fresh repository with no compiler code grew into a 249,000-line Rust-based C compiler over five days, using about 1,000 short agent episodes and $44 of model-token charges; the compiler passed external tests including c-testsuite, most of the LLVM test programs, and randomly generated Csmith programs. Second, a compiler built by the GLM model was continued by fresh agents from the same checkpoint, once with the same model and once with a different model, and both branches kept passing tests. Third, 13 MESA stellar-physics modules totaling about 139,000 Fortran lines were re-written as Rust crates; six numerical workloads agreed with the Fortran versions to within 1e-9 or exactly, and the Rust versions ran 1.55 to 6.87 times faster in the reported setup.
The paper is careful about what it does not show. There is no comparison against a flat, non-recursive organization and no test of whether the extra context files matter more than the code itself.
Extended reading notes
Core claim
The load-bearing assertion is the Abstract's 'long-horizon software development can be organized around a persistent project rather than a persistent agent', sharpened in Section 5.1: 'the lifetime of the development process can exceed the lifetime of the agents acting within it.' If correct, Genesis shows that finite-lived agents, repeatedly re-instantiated from an accepted version and path, can form a 248,989-line compiler, continue it across a model change, and re-implement 139k Fortran lines of MESA in Rust while preserving tested numerical behaviour.
Load-bearing premise
The premise that the persistent project records (version history, CONTEXT.md, constraints, validation results) and path-scoped recursive delegation, not the foundation model's intrinsic knowledge or the detailed human specification, are what enable the long-horizon outcomes. This is untested: Section 12.3 defines the required experiment (Eq. 9) and states it was not run, and Section 5.2 concedes recursion's causal superiority is not established. If the model alone plus the human task contract suffices, the title's 'enable' overstates the organization's causal role.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces EvoX Genesis, an agentic software-engineering system whose central idea is to make the software project persistent while keeping individual coding agents finite-lived. A local world is defined as w=(v,p), an accepted project version v together with a repository-relative path p; finite-lived agents propose changes in that world, recursive delegation moves work to more specific paths without advancing the version, and only parent-accepted changes advance the persistent history. The paper reports three single-run evaluations: formation of a Rust-based C compiler from an implementation-empty repository (248,989 tracked physical lines, 1,019 archived episodes, US$44.38 in token charges, passing the reported c-testsuite, most LLVM cases, and Csmith runs); continuation of a separate GLM-generated compiler after switching the foundation model to DeepSeek V4 Flash and after repeated agent turnover; and redevelopment of 13 MESA modules from Fortran to Rust with six numerical workloads showing checksum agreement and median speedups of 1.55x to 6.87x. The paper's central claim, stated in the Abstract and Section 5.1, is that long-horizon software development can be organized around a persistent project rather than a persistent agent.
Significance. If the central claim is accepted, the result is significant: it offers a concrete alternative to persistent-memory and persistent-manager architectures, and it suggests that the repository record, not the agent identity, is the right locus of continuity for long-horizon software work. The paper has genuine strengths that should be credited. The measurement discipline is unusually careful: limitation tables S21 and S22 state what each experiment supports and what else could explain it, archive coverage and missing records are reported rather than hidden, incompatible test denominators are kept separate, and the timing analysis includes the conservative 1.23x 40-run burn result alongside the more favorable six-workload table. The formal model in Section 7 is definitional, and validation is external (c-testsuite, Csmith, LZ4, SQLite, and independent checksum workloads) with no fitted parameters used to define the target results. The problem is that the load-bearing causal claim is not tested. All three demonstrations are single runs, no ablation varies the persistent non-code records, recursion, or agent persistence, and the paper's own decisive experiment in Eq.
major comments (4)
- [Section 5.2 and Section 12.3, Eq. (9)] The title and Abstract assert that persistent recursive worlds enable autonomous software evolution, and Section 5.1 sharpens this to a claim that the lifetime of the development process can exceed the lifetime of the agents acting within it. The paper's own decisive test of whether the non-code project records matter, Eq. (9), requires two versions with identical executable code but different non-code records, and Section 12.3 states explicitly that this experiment was not run. No other ablation varies recursion, record keeping, or agent persistence, and Section 4.1 reports one run per setting. The observed outcomes are therefore compatible with the alternative that a capable foundation model given the detailed compiler blueprint, a Git repository, and the human task contract would reach similar results without Genesis's context and history gating. Because this is the paper's central contribution, the claim should either be backed by the planned ablations or reframed as a capability/existence demonstration, with the title and Abstract adjusted accordingly.
- [Section 4.3, Tables S10 and S13] The continuation study cannot support the claim that the saved project records, rather than the inherited code and the model, enabled continuation. The two continuation branches start from the same commit 37216cfa254a, but they use different retained LLVM test manifests (1,445/1,448 for GLM versus 1,820/1,820 for DeepSeek), different token budgets (543.6M versus 902.8M input tokens), different wall-clock budgets (21.99 h versus 17.10 h), different agent counts, and different archive coverage. The paper honestly reports these differences, but it still presents the result as evidence for project-centered persistence. Without a code-only control or a no-records control, the continuation is also compatible with the model plus the inherited compiler being sufficient. The conclusion should be limited to the observed fact that two models each extended the same starting repository, not to a statement about which persistent components were necessary.
- [Section 4.2 and Table S4] The Abstract's phrase 'rather than a persistent agent' is not directly supported by the formation run. Table S4 shows that the optimization-phase root session spanned 99.497 h of the 123.402 h total, so a single root manager episode is active for about 80 percent of the run. The experiment therefore demonstrates one long-lived root manager with transient children and a single re-instantiation at the phase boundary, not repeated replacement of the top-level agent. The paper should report root-session durations in the main text and temper the 'persistent project rather than persistent agent' framing, or it should provide a run in which the root itself is repeatedly re-instantiated.
- [Section 4.1 and Section 9.2] The formation result is a single trajectory, and the paper does not report any measure of run-to-run variability. Given that the title claims enablement, a single successful run is weak evidence that the organization, rather than a favorable trajectory or the model's prior knowledge, is responsible for the outcome. Section 9.2 additionally notes that the task supplied substantial high-level design and testing constraints and that the run 'does not test architecture-free formation'; this should be reflected in the Abstract, where 'from scratch' is currently too strong. At minimum, the paper should state clearly that the formation result is an existence proof under a detailed human specification and that no repeated-mechanism test was performed.
minor comments (5)
- [Section 4.1] The heading 'Continuity' should be 'Continuation' to match Section 4.3 and the terminology used throughout the rest of the paper.
- [Abstract and Section 4.2] The Abstract says 'complete c-testsuite' and 'about 250k tracked lines', while the body reports the 'complete reported c-testsuite set' and a repository total that includes comments, blank lines, documentation, and tests. Recommend using the same qualified wording in the Abstract to avoid overstating conformance and implementation size.
- [Section 4.4 and Tables S18-S19] The Abstract and Section 4.4 report median speedups of 1.55x to 6.87x, but the separate 40-run burn-proxy check gives a more conservative median ratio of 1.23x, and the paper itself cautions that the six-workload ratios are host- and harness-specific. The conservative burn result should appear in the main text alongside the six-workload figures so that readers are not left with the more favorable number alone.
- [Section 12.3, Eq. (9)] The notation P(Y | (v_A,p), u, B) and the definition of Dev(v) are informal. Since this equation is the paper's proposed decisive experiment, a short explanation of what Y is, what the probability space is, and how Dev(v) would be operationalized would make the criterion testable.
- [Section 9.6 and Table S6] The paper carefully distinguishes 'LLVM-compatible' from reuse of LLVM code, which is good. Minor wording: Table S6's 'LLVM test suite 32/36 (88.9%)' could be misinterpreted as a conformance score; consider renaming the row to 'LLVM SingleSource cases evaluated' for consistency with the continuation tables.
Circularity Check
No circularity: the central results rest on external validation targets, and the only same-group citation is contextual rather than load-bearing.
full rationale
The paper's central claims are supported by externally defined validation targets rather than by definitional or fitted quantities. The compiler outcome is checked against c-testsuite, LLVM test programs, Csmith, LZ4 and SQLite (Section 9.6, Table S6), and the MESA redevelopment is checked against reference Fortran behaviour and independent checksum workloads (Section 11.4, Table S18). No parameter is fitted to a subset of data and then renamed as a prediction; the reported costs, line counts, episode counts and test pass rates are observed records, not quantities derived from the formal model. The formal model in Section 7 is a definitional vocabulary for accepted versions, paths, finite-lived agents and accepted events; it does not by itself generate the empirical outcomes, so no claimed result reduces to the model's definitions. The only same-group citation, EvoGit (Huang et al., 2025), appears in related work as background context and is never used as evidence for the observed runs. The paper explicitly disclaims the stronger causal reading: Section 5.2 states that the observations 'do not establish that recursion is causally superior to flat or alternative organizations,' and Section 12.3 presents the decisive Dev-records control (Eq. 9) and then states, 'We did not run this experiment in the present study.' These are honest evidential limitations about causal attribution, not circular reductions; the absence of a persistent-agent control makes the 'rather than a persistent agent' conclusion underdetermined, but underdetermination is a correctness-risk concern, not a circularity concern. No equation in the paper is identical to its input by construction, and no self-citation carries a load-bearing argument, so the circularity score is 0.
Assumptions & free parameters
free parameters (4)
- Maximum delegation depth =
8
- Retry limit =
15
- Context-compression threshold =
150,000 tokens
- Turn limits =
2,048 root; 128 delegated
assumptions (4)
- domain assumption Source-control history plus CONTEXT.md, constraints, and validation records fully capture the state that later agents inherit.
- domain assumption Parent agents can reliably judge acceptance using tests, constraints, and integration evidence.
- domain assumption Fresh agents instantiated from (v, p) can make progress within bounded turns, meaning the foundation model's parametric knowledge plus project state suffices.
- standard math Standard definitions of sets, functions, and transitions (Eqs. 1-8) are used without further proof.
invented entities (1)
-
Local software world w=(v, p)
Cite this review
Pith. "Pith review of Persistent Recursive Worlds Enable Autonomous Software Evolution." pith.science (2026). https://pith.science/paper/JXDKLMFT
@misc{pith2026260810450,
author = {Pith},
title = {Pith review of: Persistent Recursive Worlds Enable Autonomous Software Evolution},
year = {2026},
howpublished = {\url{https://pith.science/paper/JXDKLMFT}},
note = {Machine review of arXiv:2608.10450}
}
abstract
Complex software systems develop over timescales that exceed the lifespan of any individual coding agent. Most agentic software systems preserve continuity through persistent sessions, memories, managers or shared context. We introduce EvoX Genesis (hereafter, Genesis), which instead makes the software project persistent while allowing local agents to remain finite-lived. Genesis represents software as a persistent recursive world: each local world is situated by an accepted version and a repository path, finite-lived agents propose local changes, recursive delegation moves work across paths, and only accepted consequences advance the persistent version history. We evaluate this organization across formation, continuation and redevelopment. Starting from a repository with no compiler implementation, Genesis used DeepSeek V4 Flash to build a Rust-based C compiler with about 250k tracked lines; the run lasted over 120 hours, archived over 1,000 agent episodes and incurred only US$44 in model-token charges. The compiler passed the complete c-testsuite and most LLVM and Csmith tests. In a separate compiler world generated with GLM 5.2, development continued after repeated agent replacement while retaining full test performance. Genesis also reimplemented 13 MESA modules with over 100k Fortran lines as a Rust workspace with nearly 90k Rust lines; across six numerical workloads, it achieved median speedups of 1.55--6.87x. These results show that long-horizon software development can be organized around a persistent project rather than a persistent agent.
Figures
Reference graph
Works this paper leans on
-
[1]
Jimenez and John Yang and Alexander Wettig and Shunyu Yao and Kexin Pei and Ofir Press and Karthik Narasimhan , title =
Carlos E. Jimenez and John Yang and Alexander Wettig and Shunyu Yao and Kexin Pei and Ofir Press and Karthik Narasimhan , title =. International Conference on Learning Representations , year =
-
[2]
John Yang and Carlos E. Jimenez and Alexander Wettig and Kilian Lieret and Shunyu Yao and Karthik Narasimhan and Ofir Press , title =. Advances in Neural Information Processing Systems , volume =. 2024 , doi =
work page 2024
-
[3]
arXiv preprint arXiv:2506.02049 , year =
Beichen Huang and Ran Cheng and Kay Chen Tan , title =. arXiv preprint arXiv:2506.02049 , year =. doi:10.48550/arXiv.2506.02049 , eprint =
-
[4]
IEEE Transactions on Evolutionary Computation , volume =
Beichen Huang and Ran Cheng and Zhuozhao Li and Yaochu Jin and Kay Chen Tan , title =. IEEE Transactions on Evolutionary Computation , volume =. 2025 , doi =
work page 2025
-
[5]
Adam J. Riesselman and John B. Ingraham and Debora S. Marks , title =. Nature Methods , volume =. 2018 , doi =
work page 2018
-
[6]
Nature Computational Science , volume =
Chi Chen and Shyue Ping Ong , title =. Nature Computational Science , volume =. 2022 , doi =
work page 2022
-
[7]
Tsz Wai Ko and Bowen Deng and Marcel Nassar and Luis Barroso-Luque and Runze Liu and Ji Qi and Atul C. Thakur and Adesh Rohan Mishra and Elliott Liu and Gerbrand Ceder and Santiago Miret and Shyue Ping Ong , title =. npj Computational Materials , volume =. 2025 , doi =
work page 2025
-
[8]
Theesfeld and Kevin Yao and Kathleen M
Jian Zhou and Chandra L. Theesfeld and Kevin Yao and Kathleen M. Chen and Aaron K. Wong and Olga G. Troyanskaya , title =. Nature Genetics , volume =. 2018 , doi =
work page 2018
Show all 42 references
-
[9]
Haberl and Christopher Churas and Lucas Tindall and Daniela Boassa and Sonya Phan and Eric A
Matthias G. Haberl and Christopher Churas and Lucas Tindall and Daniela Boassa and Sonya Phan and Eric A. Bushong and Matthew Madany and Rajgopal Akay and Thomas J. Deerinck and Steven T. Peltier and Mark H. Ellisman , title =. Nature Methods , volume =. 2018 , doi =
2018
-
[10]
Parnas , title =
David L. Parnas , title =. Communications of the ACM , volume =. 1972 , doi =
1972
-
[11]
Xu and Xiangru Tang and Mingchen Zhuge and others , title =
Xingyao Wang and Boxuan Li and Yufan Song and Frank F. Xu and Xiangru Tang and Mingchen Zhuge and others , title =. International Conference on Learning Representations , year =
-
[12]
Minh V. T. Thai and Tue Le and Dung Nguyen Manh and Huy Phan Nhat and Nghi D. Q. Bui , title =. arXiv preprint arXiv:2512.18470 , year =
-
[13]
arXiv preprint arXiv:2605.15846 , year =
Xinbo Xu and Ruihan Yang and Haiyang Shen and Wendong Xu and Bofei Gao and Ruoyu Wu and others , title =. arXiv preprint arXiv:2605.15846 , year =
-
[14]
Lyu , title =
Shuzheng Gao and Wenhao Zeng and Zhaojian Yu and Jianqiao Wangni and Chaozheng Wang and Kai Cai and Shilin He and Michael R. Lyu , title =. arXiv preprint arXiv:2606.28434 , year =
- [15]
-
[16]
International Conference on Learning Representations , year =
Sirui Hong and Mingchen Zhuge and Jonathan Chen and Xiawu Zheng and Yuheng Cheng and Jinlin Wang and Ceyao Zhang and Zili Wang and Steven Ka Shing Yau and Zijuan Lin and Liyang Zhou and Chenyu Ran and Lingfeng Xiao and Chenglin Wu and J. International Conference on Learning Re...
-
[17]
Lau and Thomas Pasquier and Merc
Ana Trisovic and Matthew K. Lau and Thomas Pasquier and Merc. A large-scale study on research code quality and execution , journal =. 2022 , doi =
2022
-
[18]
Chue Hong and Daniel S
Michelle Barker and Neil P. Chue Hong and Daniel S. Katz and Anna-Lena Lamprecht and Carlos Martinez-Ortiz and others , title =. Scientific Data , volume =. 2022 , doi =
2022
-
[19]
Nature Reviews Methods Primers , volume =
David Moreau and Kristina Wiebels and Carl Boettiger , title =. Nature Reviews Methods Primers , volume =. 2023 , doi =
2023
-
[20]
Nature Computational Science , volume =
Gabrielle O'Brien , title =. Nature Computational Science , volume =. 2025 , doi =
2025
-
[21]
Eser Ayg. An. Nature , volume =. 2026 , doi =
2026
-
[22]
Artificial Life , volume =
Guy Theraulaz and Eric Bonabeau , title =. Artificial Life , volume =. 1999 , doi =
1999
-
[23]
2006 , doi =
Hartmut Ehrig and Karsten Ehrig and Ulrike Prange and Gabriele Taentzer , title =. 2006 , doi =
2006
-
[24]
Lehman , title =
Meir M. Lehman , title =. Proceedings of the IEEE , volume =. 1980 , doi =
1980
-
[25]
arXiv preprint arXiv:2603.24755 , year =
Gabriel Orlanski and Devjeet Roy and Alexander Yun and Changho Shin and Alex Gu and Albert Ge and Dyah Adila and Nicholas Roberts and Frederic Sala and Aws Albarghouthi , title =. arXiv preprint arXiv:2603.24755 , year =
-
[26]
K. N. Ajay Shastry and Ganesh Senrayan and Shrey Satapara and Pranoy Panda and Chaitanya Devaguptapu , title =. arXiv preprint arXiv:2604.03035 , year =
-
[27]
arXiv preprint arXiv:2603.17104 , year =
Lu Yan and Xuan Chen and Xiangyu Zhang , title =. arXiv preprint arXiv:2603.17104 , year =
-
[28]
Advances in Neural Information Processing Systems , volume =
Noah Shinn and Federico Cassano and Ashwin Gopinath and Karthik Narasimhan and Shunyu Yao , title =. Advances in Neural Information Processing Systems , volume =
-
[29]
Transactions on Machine Learning Research , year =
Guanzhi Wang and Yuqi Xie and Yunfan Jiang and Ajay Mandlekar and Chaowei Xiao and Yuke Zhu and Linxi Fan and Anima Anandkumar , title =. Transactions on Machine Learning Research , year =
-
[30]
Bieman , title =
Upulee Kanewala and James M. Bieman , title =. Information and Software Technology , volume =. 2014 , doi =
2014
-
[31]
Weyuker , title =
Elaine J. Weyuker , title =. The Computer Journal , volume =. 1982 , doi =
1982
-
[32]
Tsong Yueh Chen and Fei-Ching Kuo and Huai Liu and Pak-Lok Poon and Dave Towey and T. H. Tse and Zhi Quan Zhou , title =. ACM Computing Surveys , volume =. 2018 , doi =
2018
-
[33]
Future Generation Computer Systems , volume =
Luc Moreau and Ben Clifford and Juliana Freire and Joe Futrelle and Yolanda Gil and Paul Groth and Natalia Kwasnikowska and Simon Miles and Paolo Missier and Jim Myers and Beth Plale and Yogesh Simmhan and Eric Stephan and Jan Van den Bussche , title =. Future Generation Compu...
2011
-
[34]
Silva and Steven P
Juliana Freire and Claudio T. Silva and Steven P. Callahan and Emanuele Santos and Carlos E. Scheidegger and Huy T. Vo , title =. Provenance and Annotation of Data: International Provenance and Annotation Workshop , series =. 2006 , doi =
2006
-
[35]
Mathematical Software -- ICMS 2020 , series =
Roberto Di Cosmo , title =. Mathematical Software -- ICMS 2020 , series =. 2020 , doi =
2020
-
[36]
Smith and Daniel S
Arfon M. Smith and Daniel S. Katz and Kyle E. Niemeyer and. Software Citation Principles , journal =. 2016 , doi =
2016
-
[37]
ReproZip: Computational Reproducibility With Ease , booktitle =
Fernando Chirigati and R. ReproZip: Computational Reproducibility With Ease , booktitle =. 2016 , doi =
2016
-
[38]
The Astrophysical Journal Supplement Series , volume =
Bill Paxton and Lars Bildsten and Aaron Dotter and Falk Herwig and Pierre Lesaffre and Frank Timmes , title =. The Astrophysical Journal Supplement Series , volume =. 2011 , doi =. 1009.1622 , archiveprefix =
2011 arXiv
-
[39]
arXiv preprint arXiv:2512.12730 , year =
Jingzhe Ding and Shengda Long and Changxin Pu and others , title =. arXiv preprint arXiv:2512.12730 , year =. doi:10.48550/arXiv.2512.12730 , eprint =
- [40]
-
[41]
Nature , volume =
Bernardino Romera-Paredes and Mohammadamin Barekatain and Alexander Novikov and others , title =. Nature , volume =. 2024 , doi =
2024
- [42]
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.