{"id":"7d743ca3-3811-4d2e-8d45-621a9abc3679","arxiv_id":"2608.09380","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"OpenLoopEvolve improves long-horizon agent performance by versioning and evolving the entire control loop as a policy asset, with online and offline modes that beat a fixed initial loop policy on YC-Bench.","lead":"This paper introduces OpenLoopEvolve, a framework that treats an AI agent's control loop (observe, plan, act, verify, recover, stop, budget) as a versioned, reusable policy asset and improves it with candidate proposals and paired evaluation. On the simulated business benchmark YC-Bench, both online and offline evolution modes improved final funds, task success, survival, and drawdown relative to a fixed initial policy.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Offline mode may be evaluated on the same seeds that generated its archived traces; without a documented train/evaluation split, the headline gains could be in-sample fitting rather than policy improvement.","rationale":"The reader identified the thin empirical basis of Table II: three seeds, no confidence intervals, and unequal main-task call counts. That is a legitimate concern about statistical strength. My review focuses on a different but more consequential internal-validity threat: potential overlap between the archived traces / paired evaluations used by offline evolution and the exact seeds and initial state used for final evaluation. If such overlap exists, the offline mode has privileged access to the evaluation trajectory, and the measured gains do not isolate the effect of policy quality even with more seeds or significance tests. This concern is load-bearing because OLE-offline, the stronger result (974,627.52 final funds, 3/3 survival), is exactly the mode that consumes archived traces before evaluation. The paper's lack of any stated data split or limitation makes this unresolved. The concrete test is a single, feasible repository inspection plus a held-out re-run. I keep the reader's CONDITIONAL verdict because the concern does not prove fraud or a wrong result; it shows that acceptance should be conditioned on documenting and verifying the offline data protocol. I mark agreement as partial because my concern overlaps with the reader's general point that Table II may not isolate the Loop Policy effect, but the specific mechanism I identify is data leakage rather than stochastic variation or call-count confounding.","tokens_in":17270,"tokens_out":6216,"duration_ms":67023,"concrete_test":"Inspect the released repository's offline experiment configuration to determine the source of Doff and the runs used for candidate paired evaluation. List which seeds, runs, and initial states contribute traces to evidence construction and to candidate selection. If seeds 1/2/3 or runs from the same initial state appear in Doff or in the selection evaluations, re-run OLE-offline with archived traces constructed from held-out seeds (e.g., seeds 4/5/6) while keeping the final evaluation seeds fixed at 1/2/3, and report final funds, task success rate, and annual survival. If the advantage over Fixed-pi0 persists under this disjoint split, the leakage concern is settled; if it disappears, the reported offline gains are attributable to in-sample fitting.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that OLE-offline's improvement over Fixed-pi0 reflects a reusable Loop Policy, not a policy selected using outcome information from the exact evaluation runs. Section V.A states that all settings use official seeds 1/2/3 and start from the same initial state, and Section IV.C constructs offline candidates from archived traces Doff and selects a successor via paired evaluation. The paper never specifies whether Doff or the paired evaluations used for selection overlap with the final evaluation seeds. If those archived traces include runs from seeds 1/2/3 from the same initial state, then OLE-offline's 974,627.52 mean final funds and 3/3 survival could be achieved by tailoring the Loop Policy to the specific trajectory, which would invalidate the comparison with Fixed-pi0. The robust release gate in Eq. (10) does not address this because it checks task-contract compliance, not out-of-sample generalization. The online mode is less vulnerable since it starts from pi0 and receives feedback only during the evaluated run, but the paper's central claim covers both modes. This is a possible leakage rather than a proven flaw, yet the manuscript contains no data-split description or limitation statement to rule it out, so the evidence for the offline claim is currently incomplete.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces OpenLoopEvolve (OLE), a framework for externalizing and evolving 'Loop Policies' for long-horizon LLM-agent tasks. It defines a Loop Policy as an eight-component control specification (observation, planning, memory, action, verification, recovery, stopping, budget), packages it with applicability conditions and provenance in a versioned 'Bundle', and proposes online and offline evolution modes that share a Champion–Challenger paired evaluation and a robust release gate. Both modes propose candidate policies via an LLM, evaluate them in paired runs against the current Champion, and release a successor only if it passes the gate; the online mode also includes degradation monitoring and rollback. The experimental section evaluates the framework on YC-Bench with DeepSeek-V4-flash on three official seeds, reporting that OLE-online and OLE-offline improve mean final funds, task success rate, annual survival, and drawdown relative to a fixed initial Loop Policy.","tokens_in":17520,"tokens_out":7154,"duration_ms":66789,"significance":"If the reported results are robust, OLE would offer a useful contribution to the growing area of agent self-evolution by providing a concrete asset representation and release-governance mechanism for loop-level control experience. The paper's strengths include a clear separation between candidate generation, evaluation, release, and activation; formal definitions of traces, evidence, and policy assets; and the release of code on GitHub. However, the empirical support for the headline improvements is currently incomplete, as detailed in the major comments. With appropriate additional experiments and reporting, this line of work could be valuable to the community.","major_comments":[{"comment":"The most serious concern is that OLE-offline may be evaluated on the very runs used to construct its archived traces. Section V.A states that all settings use official seeds 1/2/3 and start from the same initial state, but Section IV.C does not specify whether Doff—the archived traces used for offline evolution—includes runs from those same seeds and that initial state. If the archived traces overlap with the evaluation runs, then the paired evaluation in Eq. (9) and the selection in Algorithm 2 use outcome information from the exact evaluation trajectories, and the Table II gains for OLE-offline would reflect in-sample fitting rather than a reusable policy. The release gate in Eq. (10) does not rule this out because it checks task-contract compliance (benefit, evidence quality, tail risk, resource cost), not out-of-sample generalization. Please specify the composition of Doff and, if necessary, evaluate on held-out seeds or runs.","section":"V.A, IV.C, Eq. (9)"},{"comment":"The statistical support for the main empirical claim is thin: all aggregate results are means over three seeds, with no standard deviations, confidence intervals, or significance tests. Annual survival is a coarse 1/3 vs 2/3 vs 3/3, and the shaded regions in Fig. 3 show cross-seed ranges that are not sufficient to establish that the mean differences (e.g., 878,601.79 vs 974,627.52 vs 365,976.30) are reliable. Please report per-seed results, confidence intervals, and paired statistical tests (e.g., bootstrap or permutation tests) for the key metrics.","section":"Table II"},{"comment":"The comparison of OLE against Fixed-π0 in Tables II and III(a) confounds policy quality with the amount of compute: OLE-online and OLE-offline make 1575 and 1907 main-task calls against 799 for Fixed-π0, and their mean survival durations are 294 and 365 days versus 207.78 days. The higher final funds and success rates could therefore result from the larger number of task calls (i.e., longer survival) rather than from the evolution mechanism. To support the claim that evolution itself improves the policy, please add an ablation that holds the number of calls or the resource budget fixed across the compared settings—for example, a Fixed-π0 variant with a matched or increased budget, or a budget-capped OLE variant.","section":"Table III(a)"},{"comment":"The 'verifiable' claim is not fully substantiated because the release gate is left abstract: the forms of the four constraint functions Γ_b, Γ_e, Γ_t, Γ_c and the threshold configuration Θ are never specified, and the experiments do not report how many candidates passed or failed each constraint. Since the abstract and title emphasize verifiability, please provide the concrete gate implementation and a summary of gate outcomes (e.g., for the 12 online candidate attempts and the K offline generations), including for each candidate which constraints were satisfied or violated.","section":"IV.A, Eq. (10)"},{"comment":"The online evolution mechanism depends on the update condition Trig and the degradation monitor CANARYMONITOR, but neither is specified and no rollback events are reported. Without describing what Trig checks (feedback amount, period, or task stage) and the exact degradation condition used for rollback, it is difficult to assess whether the online mode's improvement comes from the proposed release/rollback governance or simply from repeatedly prompting the LLM with recent feedback. Please provide these details and a log of the online update decisions (candidate, gate result, release, rollback).","section":"IV.B, Algorithm 1"}],"minor_comments":[{"comment":"There are several typos and LaTeX artifacts, e.g., 'difficult' (Abstract), 'official' (Section V.A), and 'Y ao' in references [1] and [5]; these should be corrected.","section":"Throughout"},{"comment":"The symbol Π is used both for the Loop Policy space in Eq. (6) and as the product operator in Eq. (10), which is confusing; please use a different symbol (e.g., ⋀) for the conjunction over constraints.","section":"Eq. (10) and Eq. (6)"},{"comment":"The statement that after early termination terminal funds are carried forward 'only to retain a fixed aggregation denominator' is unusual; please clarify the justification and note that the shaded region is a cross-seed range, not a confidence band, so readers are not misled.","section":"Fig. 3 caption"},{"comment":"The eight Loop Policy components are given only as high-level YAML descriptions; for reproducibility, please provide formal domains or a more concrete example instantiated for YC-Bench.","section":"III.C, Fig. 2"},{"comment":"The term 'official medium configuration' for the deepseek-v4-flash model is not defined; please specify the exact inference settings (temperature, top-p, max tokens, etc.) used in the experiments.","section":"V.A"},{"comment":"The claim of being 'the first to apply Champion–Challenger release mechanism to both offline and online self-evolution of externalized Loop Policies' is a strong novelty claim; please soften it or provide a more thorough comparison with related work to make the claim verifiable.","section":"II.C"},{"comment":"The gate thresholds Θ and offline search hyperparameters (number of generations, candidate count, mutation rates) are not listed; please include them in the experimental setup or appendix for reproducibility.","section":"IV–V"}],"recommendation":"major_revision","confidential_remarks":"The framework contribution is potentially interesting, but the current empirical validation is not yet sufficient for a journal publication. The most pressing issue is the possible in-sample selection in OLE-offline; the authors must specify the composition of Doff and provide a holdout evaluation. The three-seed statistics and the compute confound also need to be addressed with additional analyses. If the authors can fix these issues, the paper may become acceptable; I do not see grounds for rejection based on the conceptual framework itself."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The headline: this is a coherent systems paper with a genuinely useful new object—the externalized, versioned Loop Policy—and a release-governance mechanism that ties self-improvement to evidence. The formal parts hold up. The empirical case does not yet support the title's claim to 'verifiable self-evolution,' and there is a possible leakage in the offline evaluation that the authors need to address.\n\nWhat's new: OLE separates the full agent loop (observation, planning, memory, action, verification, recovery, stopping, budget) into a portable Bundle with version lineage, and applies a Champion–Challenger paired evaluation and a four-criterion gate (benefit, evidence quality, tail risk, resource cost) before any policy is released. The online mode adds activation-at-task-boundary and rollback. That is a real contribution, and the definitions in Section III are clear enough to implement. The paper gets credit for reporting that Fixed-π0 is 29% worse than the native baseline, for tracking token usage, and for linking to code. The evolved policies do survive longer and use fewer tokens per call.\n\nThe soft spots. The evidence is thin: one benchmark, three seeds, no confidence intervals, no significance tests, no ablation separating policy quality from the additional compute. The bigger issue is offline score leakage. Section V.A says all settings use official seeds 1/2/3 and start from the same initial state, and Algorithm 2 selects the offline successor from archived traces Doff. The paper never states whether those archived traces include the exact seeds used in the final evaluation. If they do, the offline policy was selected on outcome information from the same runs being scored, so the 974k final funds and 3/3 survival would not demonstrate reusable policy improvement. This is a possible flaw rather than a proven one, but the absence of a data-split description or limitation statement makes the offline result incomplete.\n\nThe online mode is less vulnerable because it starts from π0 and only learns from feedback during the evaluated run, but the 12 update attempts mean it also gets substantially more main-task calls, so even the online comparison is confounded.\n\nBottom line: the framework is worth engaging, and the formalization is a solid base. But the paper needs a clear statement of where Doff comes from, a train/evaluation split, and at minimum per-seed results plus an ablation that controls for compute. The citation pattern is fine; the issue is data provenance, not self-citation. I'd send it to review—the contribution is original and the authors can fix the empirical holes.","headline":"A well-specified framework for externalized loop policies with a real contribution in asset/release machinery, but the empirical evidence is thin and the offline mode needs a documented data split before the headline gains can be believed.","tokens_in":18059,"tokens_out":4015,"would_cite":false,"duration_ms":40846,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"OpenLoopEvolve claims that externalizing an agent's full execution loop as a versioned, gated Loop Policy raises long-horizon performance, increasing mean final funds on YC-Bench by 140% and 166% over a fixed policy.","keywords":["Loop Policy","self-evolution","long-horizon tasks","large language model agents","Champion-Challenger","policy asset versioning","release gate","YC-Bench"],"falsifier":"Rerun YC-Bench with the same seeds while matching the number of main-task calls across fixed and evolved policies, or compute paired confidence intervals across many seeded environments; if the fund differences shrink to within seed noise, the central claim fails.","tokens_in":17060,"feed_emoji":"🤖","tokens_out":7917,"duration_ms":69126,"temperature":0.7,"pith_summary":"The paper's central claim is that the complete control logic of a long-horizon agent—how it observes, plans, uses memory, acts, verifies, recovers, stops, and budgets—can be lifted out of the host program and treated as a single versioned asset, called a Loop Policy, that accumulates experience across runs. If that claim holds, the same machinery that governs software releases—candidate generation, paired evaluation against a current best version, a release gate, and rollback—can govern agent behavior, so recurring failures in a year-long task become reusable evidence rather than lost traces. On YC-Bench, a simulated year of business management, both an online mode that evolves from live feedback and an offline mode that searches archived traces improve on keeping the initial policy fixed: mean final funds rise from about 366,000 to 879,000 and 975,000 simulated units, task success from 73.9% to 87.9% and 91.8%, and annual survival from one of three seeds to two and three of three. The paper frames this as evidence that loop-control experience is portable, comparable, and reusable.","feed_headline":"Evolving the agent's loop lifts long-horizon returns 140-166%","feed_subtitle":"Versioned, gated loop policies raise task success from 73.9% to 91.8% and survival from 1/3 to 3/3 seeds.","key_machinery":"The load-bearing object is the Bundle, a versioned asset pairing a Loop Policy with applicability conditions and parent evidence. The Loop Policy's eight components define the state-update and action-decision functions $f_\\pi$ and $h_\\pi$ in the loop dynamics $x_{t+1}=f_\\pi(x_t,o_t,a_t,e_t)$. The mechanism that makes evolution verifiable is paired Champion-Challenger evaluation with the relative change score of Eq. (9), followed by the robust release gate of Eq. (10), whose four constraints require demonstrated benefit, valid evidence, acceptable tail risk, and resource compliance before any candidate can touch the execution path.","core_discovery":"The discovery is that an agent's execution loop can be made into an external, governable asset: OLE formalizes the Loop Policy $\\pi = \\langle \\pi_{\\mathrm{obs}}, \\pi_{\\mathrm{plan}}, \\pi_{\\mathrm{mem}}, \\pi_{\\mathrm{act}}, \\pi_{\\mathrm{ver}}, \\pi_{\\mathrm{rec}}, \\pi_{\\mathrm{stop}}, \\pi_{\\mathrm{bud}}\\rangle$ and packages it with applicability conditions and provenance into a Bundle $B_v = \\langle \\pi_v, \\chi_v, \\ell_v\\rangle$ with stable version indices and a lineage. Traces from runs are distilled into evolution evidence $\\mathcal{E} = \\Phi(\\mathcal{D})$, each item pointing at a component to change and carrying a measured direction. An LLM proposes candidate Bundles, and each candidate is compared with the current Champion in paired runs under shared conditions using the relative change $r_{i,j} = \\Delta_{i,j}/\\max(|q(\\pi_c;g_i,z_{i,j})|,\\epsilon)$; only candidates passing a four-part gate—benefit, evidence quality, tail risk, resource cost—are released. Online releases activate at the next task boundary and can be rolled back on degradation, while offline evolution runs a multi-generation population search. The YC-Bench results show both modes outperform a fixed initial Loop Policy on final funds, task success, annual survival, and drawdown.","pith_inferences":["A natural extension is to apply the same asset-and-gate protocol to other long-horizon interactive benchmarks, since the gate's constraints are not specific to YC-Bench.","If the release gate were made stricter or weaker, the framework predicts a direct trade-off between evolution speed and tail safety; this is testable without changing the Loop Policy representation.","The three-seed evaluation suggests that a confidence-interval-based gate, rather than point estimates, could be integrated into the release criterion itself.","Treating loop policies as versioned assets also creates an audit trail connecting each release to the trace evidence that motivated it, which could matter for deployment governance."],"forward_implications":["If correct, Loop Policies become versioned assets that can be accumulated, diffed, reused, and rolled back across tasks and agent hosts.","The shared evidence-candidate-evaluation-release chain means online live feedback and offline archived traces can feed the same policy lineage.","On YC-Bench, evolved policies improve final funds, task success, and annual survival while reducing maximum drawdown, showing loop governance is a measurable performance lever.","The token accounting shows evolution has an explicit cost (29.82M and 24.02M tokens for online and offline modes) that buys longer survival and lower per-call usage at this scale."],"supporting_citations":[{"why":"Supplies the YC-Bench simulated-year business environment and official seeds used to measure the claimed gains.","marker":"[6]"},{"why":"Introduces the Loop Engineering view of external loop design that motivates treating loops as reusable objects.","marker":"[16]"},{"why":"Provides the evidence-gating principle that underpins requiring verifiable evidence before a loop change is released.","marker":"[17]"},{"why":"Supplies the Champion-Challenger paired-comparison paradigm used for candidate evaluation.","marker":"[18]"},{"why":"Extends Champion-Challenger comparison to online AutoML, informing the online evolution mode.","marker":"[19]"},{"why":"The large language model used for all agent and policy-proposal calls in the experiments.","marker":"[20]"}],"fun_headline_variants":["Loop policies as governable assets lift long-horizon agent returns","Versioned loop policies boost task success to 91.8%","Self-evolving loop policies raise funds, survival, success","OpenLoopEvolve: turning execution loops into reusable policy assets","Gated loop policies: agent control becomes a versioned asset"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The empirical case rests on three official seeds on YC-Bench with no confidence intervals or significance tests, so the large measured gains could in principle come from seed randomness or from the evolved policies simply making more model calls, rather than from policy quality.","fun_headline_variants_meta":{"raw":{"variants":["Loop policies as governable assets lift long-horizon agent returns","Versioned loop policies boost task success to 91.8%","Self-evolving loop policies raise funds, survival, success","OpenLoopEvolve: turning execution loops into reusable policy assets","Gated loop policies: agent control becomes a versioned asset"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000547,"raw_usage":{"total_tokens":2704,"prompt_tokens":1125,"completion_tokens":1579,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":741,"completion_tokens_details":{"reasoning_tokens":1492}},"tokens_in":741,"tokens_out":1579,"duration_ms":9074,"temperature":1.0,"reasoning_tokens":1492,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:23:06.837685+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun YC-Bench with the same seeds while matching the number of main-task calls across fixed and evolved policies, or compute paired confidence intervals across many seeded environments; if the fund differences shrink to within seed noise, the central claim fails.","supporting_citations":[{"cited_title":"Proof-or-Stop: Don't Trust the Agent, Trust the Evidence -- Loop Engineering for Verifiable Evidence-Gated Lifecycle Control","cited_arxiv_id":"2607.14890","evidence_quote":"Provides the evidence-gating principle that underpins requiring verifiable evidence before a loop change is released."},{"cited_title":"Champion-challenger based predictive model selection,","cited_arxiv_id":null,"evidence_quote":"Supplies the Champion-Challenger paired-comparison paradigm used for candidate evaluation."},{"cited_title":"ChaCha for online AutoML,","cited_arxiv_id":null,"evidence_quote":"Extends Champion-Challenger comparison to online AutoML, informing the online evolution mode."}],"review_version":1}