{"id":"2f813651-b9a9-4b07-a937-b1640a7ab133","arxiv_id":"2608.10450","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A project-centered agent architecture formed a 249k-line C compiler, continued development across a foundation-model swap, and ported MESA modules to Rust with 1.55-6.87x measured speedups.","lead":"This paper presents EvoX Genesis, which keeps the software project itself persistent while spawning fresh, short-lived coding agents for each task. It demonstrates the architecture by autonomously building a C compiler, continuing a compiler after swapping the underlying AI model, and porting physics modules from Fortran to Rust.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Causal role of the persistent project is untested: no persistent-agent or no-records control, and the formation run's optimization-phase root manager was active for about 99.5 of 123.4 hours, so 'rather than a persistent agent' is not established.","rationale":"I agree with the reader's weakest assumption. The paper is unusually honest: Section 5.2, Table S21, and Section 12.3 already flag the missing causal decomposition, and the SI reports failures, skipped tests, archive gaps, and the one-line count discrepancy. The capability demonstrations themselves are credible in the sense that they are concrete, externally validated (c-testsuite, Csmith, numerical checksums), and bounded in their claims. But the title's 'enable' and the abstract's 'can be organized around a persistent project rather than a persistent agent' assert a contrast that no experiment directly tests. A sufficiency reading of 'can be organized' is supported; a causal or comparative reading is not. The stress-test adds one concrete mechanism-level observation: the formation run's root sessions lasted the entire run, so the '1,000 finite-lived episodes' coexist with a long-lived coordinating root manager; only at the two phase boundaries, and in the continuation stages, is agency actually re-instantiated from the project state. This sharpens, rather than replaces, the reader's concern. The concrete test above would settle the matter by directly varying Dev records and adding a persistent-agent arm. Until then, CONDITIONAL is the right verdict, so I recommend UNCHANGED.","tokens_in":24440,"tokens_out":8810,"duration_ms":90496,"concrete_test":"Implement Eq. 9 with a matched control. Fork the completed GLM compiler at commit 37216cfa254a into two repositories with byte-identical executable code; keep CONTEXT.md, constraints, and handoff history in one and strip all accepted non-code Dev records in the other. Give both the same continuation objective, model, and token budget, and compare downstream test pass rate and task completion. Add a third arm with a single persistent root agent that carries private state across the whole task under the same spec and budget, to test the 'rather than a persistent agent' contrast. If the stripped-records arm matches the records arm, the persistent project's non-code records are not the active enabler; if the persistent-agent arm matches or exceeds Genesis, the central organizational contrast is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that persistent-project organization, rather than a persistent agent, enables long-horizon development is underdetermined by the reported experiments. All three demonstrations are single runs, and none varies the persistent non-code records independently of the executable code, the foundation model, or the human task contract. Section 5.2 concedes recursion's causal superiority is not established, and the paper's own decisive experiment, Eq. 9 in Section 12.3 (identical executable code with different Dev records), is explicitly stated as not run. The continuation study shares a starting commit but uses one run per arm, unequal token and wall budgets, and distinct LLVM test manifests (Table S13), so it cannot separate 'the saved project records enabled continuation' from 'the model plus inherited code enabled continuation.' A further internal tension: in formation, two sequential root sessions spanned the full 123.4 hours, with the optimization-phase root manager active for about 99.5 hours; the run therefore shows re-instantiation across phase boundaries, not that short-lived agents alone coordinate the work. Combined with the acknowledged absence of a complete human-action audit log (Sections 4.1 and 9.9), the results are compatible with the alternative that a capable model, given the detailed compiler blueprint and a Git repository, would reach similar outcomes without Genesis's Context/history gating. Because the title asserts enablement, the causal role of the persistent recursive world is the load-bearing unverified premise.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces EvoX Genesis, an agentic software-engineering system whose central idea is to make the software project persistent while keeping individual coding agents finite-lived. A local world is defined as w=(v,p), an accepted project version v together with a repository-relative path p; finite-lived agents propose changes in that world, recursive delegation moves work to more specific paths without advancing the version, and only parent-accepted changes advance the persistent history. The paper reports three single-run evaluations: formation of a Rust-based C compiler from an implementation-empty repository (248,989 tracked physical lines, 1,019 archived episodes, US$44.38 in token charges, passing the reported c-testsuite, most LLVM cases, and Csmith runs); continuation of a separate GLM-generated compiler after switching the foundation model to DeepSeek V4 Flash and after repeated agent turnover; and redevelopment of 13 MESA modules from Fortran to Rust with six numerical workloads showing checksum agreement and median speedups of 1.55x to 6.87x. The paper's central claim, stated in the Abstract and Section 5.1, is that long-horizon software development can be organized around a persistent project rather than a persistent agent.","tokens_in":24711,"tokens_out":6285,"duration_ms":62874,"significance":"If the central claim is accepted, the result is significant: it offers a concrete alternative to persistent-memory and persistent-manager architectures, and it suggests that the repository record, not the agent identity, is the right locus of continuity for long-horizon software work. The paper has genuine strengths that should be credited. The measurement discipline is unusually careful: limitation tables S21 and S22 state what each experiment supports and what else could explain it, archive coverage and missing records are reported rather than hidden, incompatible test denominators are kept separate, and the timing analysis includes the conservative 1.23x 40-run burn result alongside the more favorable six-workload table. The formal model in Section 7 is definitional, and validation is external (c-testsuite, Csmith, LZ4, SQLite, and independent checksum workloads) with no fitted parameters used to define the target results. The problem is that the load-bearing causal claim is not tested. All three demonstrations are single runs, no ablation varies the persistent non-code records, recursion, or agent persistence, and the paper's own decisive experiment in Eq.","major_comments":[{"comment":"The title and Abstract assert that persistent recursive worlds enable autonomous software evolution, and Section 5.1 sharpens this to a claim that the lifetime of the development process can exceed the lifetime of the agents acting within it. The paper's own decisive test of whether the non-code project records matter, Eq. (9), requires two versions with identical executable code but different non-code records, and Section 12.3 states explicitly that this experiment was not run. No other ablation varies recursion, record keeping, or agent persistence, and Section 4.1 reports one run per setting. The observed outcomes are therefore compatible with the alternative that a capable foundation model given the detailed compiler blueprint, a Git repository, and the human task contract would reach similar results without Genesis's context and history gating. Because this is the paper's central contribution, the claim should either be backed by the planned ablations or reframed as a capability/existence demonstration, with the title and Abstract adjusted accordingly.","section":"Section 5.2 and Section 12.3, Eq. (9)"},{"comment":"The continuation study cannot support the claim that the saved project records, rather than the inherited code and the model, enabled continuation. The two continuation branches start from the same commit 37216cfa254a, but they use different retained LLVM test manifests (1,445/1,448 for GLM versus 1,820/1,820 for DeepSeek), different token budgets (543.6M versus 902.8M input tokens), different wall-clock budgets (21.99 h versus 17.10 h), different agent counts, and different archive coverage. The paper honestly reports these differences, but it still presents the result as evidence for project-centered persistence. Without a code-only control or a no-records control, the continuation is also compatible with the model plus the inherited compiler being sufficient. The conclusion should be limited to the observed fact that two models each extended the same starting repository, not to a statement about which persistent components were necessary.","section":"Section 4.3, Tables S10 and S13"},{"comment":"The Abstract's phrase 'rather than a persistent agent' is not directly supported by the formation run. Table S4 shows that the optimization-phase root session spanned 99.497 h of the 123.402 h total, so a single root manager episode is active for about 80 percent of the run. The experiment therefore demonstrates one long-lived root manager with transient children and a single re-instantiation at the phase boundary, not repeated replacement of the top-level agent. The paper should report root-session durations in the main text and temper the 'persistent project rather than persistent agent' framing, or it should provide a run in which the root itself is repeatedly re-instantiated.","section":"Section 4.2 and Table S4"},{"comment":"The formation result is a single trajectory, and the paper does not report any measure of run-to-run variability. Given that the title claims enablement, a single successful run is weak evidence that the organization, rather than a favorable trajectory or the model's prior knowledge, is responsible for the outcome. Section 9.2 additionally notes that the task supplied substantial high-level design and testing constraints and that the run 'does not test architecture-free formation'; this should be reflected in the Abstract, where 'from scratch' is currently too strong. At minimum, the paper should state clearly that the formation result is an existence proof under a detailed human specification and that no repeated-mechanism test was performed.","section":"Section 4.1 and Section 9.2"}],"minor_comments":[{"comment":"The heading 'Continuity' should be 'Continuation' to match Section 4.3 and the terminology used throughout the rest of the paper.","section":"Section 4.1"},{"comment":"The Abstract says 'complete c-testsuite' and 'about 250k tracked lines', while the body reports the 'complete reported c-testsuite set' and a repository total that includes comments, blank lines, documentation, and tests. Recommend using the same qualified wording in the Abstract to avoid overstating conformance and implementation size.","section":"Abstract and Section 4.2"},{"comment":"The Abstract and Section 4.4 report median speedups of 1.55x to 6.87x, but the separate 40-run burn-proxy check gives a more conservative median ratio of 1.23x, and the paper itself cautions that the six-workload ratios are host- and harness-specific. The conservative burn result should appear in the main text alongside the six-workload figures so that readers are not left with the more favorable number alone.","section":"Section 4.4 and Tables S18-S19"},{"comment":"The notation P(Y | (v_A,p), u, B) and the definition of Dev(v) are informal. Since this equation is the paper's proposed decisive experiment, a short explanation of what Y is, what the probability space is, and how Dev(v) would be operationalized would make the criterion testable.","section":"Section 12.3, Eq. (9)"},{"comment":"The paper carefully distinguishes 'LLVM-compatible' from reuse of LLVM code, which is good. Minor wording: Table S6's 'LLVM test suite 32/36 (88.9%)' could be misinterpreted as a conformance score; consider renaming the row to 'LLVM SingleSource cases evaluated' for consistency with the continuation tables.","section":"Section 9.6 and Table S6"}],"recommendation":"major_revision","confidential_remarks":"The paper is unusually honest and the demonstrations are technically impressive, but the title and Abstract assert a causal claim that the experiments do not test. I recommend major revision with the option of reframing the contribution as a capability/existence proof rather than an enablement claim; if the authors either soften the claims appropriately or add the ablations described in Section 12.3, the paper would be suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the empirical work is real and worth engaging with: a 249k-line C compiler formed from an implementation-empty repository under a detailed human spec, continuation of a completed compiler after a foundation-model swap, and a 139k-line Fortran MESA module chain re-implemented in Rust with bit-exact or ~1e-9-level numerical agreement. Validation is external — c-testsuite, Csmith, SQLite, LZ4, checksums — and no fitted parameters define the targets. Second, the headline claim, that long-horizon development is organized 'around a persistent project rather than a persistent agent,' is not established by these runs, and the authors concede as much. Section 5.2 says recursion's causal superiority is not shown; Section 12.3 gives the decisive experiment (Eq. 9: identical executable code, different non-code Dev records) and states it was not run.\n\nWhat the paper does well: it is unusually honest. Table S21 lists alternative explanations for each experiment; Table S22 logs failures, skipped seeds, missing parent records, and the one-line count discrepancy; the MESA timing section reports the noisy 0.82x interleaved run and the 1.23x 40-run median rather than cherry-picking. Archive coverage is quantified per stage. The citation pattern is fine; EvoGit is the authors' own prior work and is directly relevant, not padding. The formal (v, p) model is simple but adequate — it is a definitional framing, not a source of outcome claims.\n\nWhere it is soft: the causal core. There is no ablation of recursion, no fresh-versus-persistent-agent control, and no experiment varying Dev records while holding code fixed. The continuation branches are single runs with different LLVM test manifests and unequal token/wall budgets, so they document that continuation happened, not what enabled it. The stress-test note about the root manager is fair and lands: in formation, the optimization-phase root session ran about 99.5 of 123.4 hours, and each continuation branch's root manager spans the whole branch. What was actually observed is a long-lived root agent coordinating mostly finite-lived workers — not the absence of a persistent agent.\n\nNone of this undercuts the capability results. As an observed-run report, the paper holds up. But the title's 'enable' overstates: the runs are consistent with a strong model plus a Git repository doing most of the work.\n\nWho this is for: researchers building long-horizon coding agents, and anyone evaluating reimplementation of scientific software. It deserves a serious referee. I would send it out, with a major-revision request to either run the Section 12.3 ablations or reframe title and abstract as a capability demonstration. I'd bring it to reading group and would cite it as a capability result, not as evidence for the necessity claim.","headline":"A credible, unusually honest large-scale capability report whose headline causal claim — persistent project rather than persistent agent — the experiments do not actually support.","tokens_in":25287,"tokens_out":4488,"would_cite":true,"duration_ms":41724,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":null,"created_at":"2026-08-15T14:20:37.891555+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":null,"supporting_citations":[],"review_version":1}