{"id":"b7baa55e-9469-4d80-a90c-3f12531eb214","arxiv_id":"2608.03392","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A systematic survey that defines self-evolving coding agents, organizes systems into a five-category object-centered taxonomy, and analyzes when and on what evidence these agents evolve.","lead":"This survey maps the emerging field of self-evolving coding agents, where LLM-based coding assistants update their memory, skills, tools, models, or workflows from prior coding attempts. It proposes an object-centered taxonomy and identifies executable feedback as the distinctive enabler and risk for such agents.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The definition of 'self-evolving' is inconsistent with the task-time category: within-task adaptation is admitted, so the boundary against conventional agents is underdetermined.","rationale":"The reader's weakest assumption—executable feedback reliability—is a genuine open problem, and the paper itself flags it in Section 6. However, that concern targets the viability of the field rather than the coherence of the survey's central taxonomic claim. A survey can legitimately define and organize a category even when the field's evidence is imperfect; reliability is a challenge, not a refutation of the framework. By contrast, the task-time/persistence inconsistency directly undermines the claim that self-evolving coding agents are a distinct category with a workable boundary. Section 2.3 requires persistent adaptation, while Section 4.1 admits within-task, local adaptation and includes systems that change tools, workflows, and topologies only for the current task. This makes the definition either too broad—encompassing ordinary test-retry coding agents—or internally inconsistent with the taxonomy's own examples. The proposed concrete check would settle this by testing whether the artifacts evolved during task-time are actually carried across tasks. The reader's CONDITIONAL verdict remains appropriate because the paper is a useful survey whose central framing needs sharpening; this concern does not warrant rejection but does support the conditional status.","tokens_in":22531,"tokens_out":4686,"duration_ms":53777,"concrete_test":"For each Table 2 row with Timing='Task-time' (Live-SWE-Agent, SEMAG, SEW, EvoMAC, AgentConductor), inspect the source code or paper to determine whether the artifact changed during evolution—a created tool, modified workflow, communication DAG, or collaboration network—is retained and reused after the current task ends, or whether it is recreated from scratch for each new task. If none of these artifacts persists across tasks, then under the Section 2.3 'persistent adaptation' criterion these systems are not self-evolving, and the taxonomy's boundary fails on its own examples. If some artifacts do persist, the definition can be tightened to exclude simple within-task retry loops and the boundary can be restored.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The reader's feedback-reliability concern is real but secondary. The most load-bearing weakness is internal: the paper's definition and its own temporal taxonomy disagree about what counts as self-evolution. Section 2.3 defines a self-evolving coding agent as an agent that updates its behavior or internal components based on previous coding attempts and software-specific feedback, and distinguishes it from a conventional coding agent by 'turning these interactions into sources of persistent adaptation.' Section 4.1, however, defines task-time evolution as change occurring 'while the agent is still solving the current coding task,' citing Live-SWE-Agent creating tools during issue solving and SEMAG, AgentConductor, and SEW adapting workflows/topologies per task, and explicitly says such adaptations are 'often local to the current task.' If within-task adaptation is sufficient, then any coding agent that retries after test failures or reorganizes its plan is self-evolving, and the claimed boundary against 'conventional coding agents' dissolves. If persistence across tasks is required, then several systems placed in the task-time category are misclassified. Either way, the central taxonomic claim is underdetermined by the paper's own criteria. The survey needs an operational persistence test: which artifacts survive the task, and do they condition later tasks?","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper is a survey of \"self-evolving coding agents,\" defined as agentic software-engineering systems that update their own behavior or internal components using previous coding attempts and software-specific feedback. It proposes an object-centered taxonomy with five categories (agent framework, memory, skills/tools, model, workflow/topology), complemented by temporal dimensions (task-time, post-task, stage-wise) and evidence dimensions (outcome, environmental, trajectory-derived). The survey maps roughly thirty systems onto this taxonomy, reviews benchmarks and evaluation metrics, and discusses open challenges such as feedback reliability, benchmark overfitting, safety, and generalization. The central contribution is the taxonomy itself, framed as a foundation for designing and comparing self-evolving coding agents.","tokens_in":22799,"tokens_out":5913,"duration_ms":73970,"significance":"If the taxonomy is accepted, it provides a useful organizing framework for a rapidly growing but fragmented literature. The paper is broad in coverage, identifies concrete software-specific feedback signals, and draws attention to important open problems, especially the risk that unreliable executable feedback may undermine self-evolution. The survey's value is primarily conceptual rather than empirical; it does not introduce new algorithms or results. Its main strength is the systematic grouping of heterogeneous systems, and its main weakness is that the boundary of the central concept is not yet operationally precise enough to make the taxonomy's inclusion/exclusion decisions reproducible.","major_comments":[{"comment":"The core definition of self-evolving coding agents is internally inconsistent with the task-time evolution category. Section 2.3 distinguishes self-evolving agents from conventional ones by \"turning these interactions into sources of persistent adaptation,\" but Section 4.1 defines task-time evolution as change occurring \"while the agent is still solving the current coding task,\" including \"an immediate change in the current patch, tool use, or workflow,\" and later states that such adaptations are \"often local to the current task.\" If changes to the current patch count as self-evolution, then any coding agent that retries after a test failure is self-evolving, collapsing the boundary that the survey claims to draw. If, instead, persistence across tasks is required, then the task-time examples (Live-SWE-Agent, SEMAG, SEW, EvoMAC, AgentConductor) are either misclassified or need to show tha","section":"Section 2.3 vs Section 4.1"},{"comment":"The boundary between model self-evolution and ordinary SWE-oriented model optimization is not applied consistently. Section 3.4 states that \"ordinary post-training becomes self-evolution only when software-specific experience is fed back into the model-side components that govern later agent behavior,\" and later restricts the category further to signals \"closed around the agent's own evolving attempts.\" However, Agent-RLVR is included in Table 2 as a model self-evolution system even though Section 3.4 describes it as receiving \"guidance and environment rewards\" and using \"guided reattempts\" to update the policy. If the guidance is not generated by the agent's own attempts, the \"closed loop\" criterion is not met. The criterion needs an operational definition of whose attempts generate the signal, otherwise the inclusion/exclusion of systems such as Agent-RLVR, SWE-RL, and SWE-Gym is subje","section":"Section 3.4 and Table 2"}],"minor_comments":[{"comment":"The reference \"Anonymous. Mendel Gödel machine..., 2026\" and its use in Table 2 and Figure 2 as \"Mendel GM [2026]\" violate normal scholarly practice for a survey. An anonymous, unpublished manuscript cannot be independently verified by readers. The authors should either replace it with a citable, attributed source or remove it from the taxonomy.","section":"References / Table 2"},{"comment":"The phrase \"immediate change in the current patch\" in the definition of task-time evolution should be removed or qualified, since changing the current patch is exactly what a conventional coding agent does after a failed test. Clarifying this wording would help address the major concern above.","section":"Section 4.1"},{"comment":"The discussion of evaluation metrics mentions Pass@k, solve rate, and repair rate, but does not define Pass@k or explain how it is computed in this context. A brief definition or citation would improve clarity for readers outside the code-generation subfield.","section":"Section 5.2"},{"comment":"Table 1's row for \"General self-evolving agents\" lists \"Prompts, memory, tools, policies, or architectures\" as what changes, but the text in Section 2.2 also includes \"workflows\" and \"multi-agent co-evolution.\" Making the table consistent with the text would avoid confusion.","section":"Section 2.3 / Table 1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript includes the authors' own systems (EvoRepair, EvoMAC) among the representative examples, and the \"Anonymous, 2026\" Mendel Gödel Machine citation appears likely to be a hidden self-citation. This is not itself grounds for rejection, but I would urge the editor to enforce disclosure norms for anonymous references in a published survey. The taxonomy is a genuinely useful contribution, but the central definitional boundary needs to be fixed before the paper can serve as a reliable reference for the field."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a serious, useful survey. The object-centered taxonomy—framework, memory, skills/tools, model, workflow/topology—plus the timing and evidence dimensions is a genuine contribution. It gave me a clean way to organize a messy literature, and the tables plus the GitHub repository make it practically usable. The prose is clear, and the challenges section is honest about feedback reliability, benchmark overfitting, and safety. The authors have read widely and grouped systems sensibly. For a survey, that is most of the job, and they do it well.\n\nThe soft spots are real but not fatal. The biggest one is internal: Section 2.3 defines self-evolving coding agents by turning interactions into \"sources of persistent adaptation,\" but Section 4.1 defines task-time evolution as within-task change that is \"often local to the current task.\" If local within-task adaptation counts, then any agent that retries after a test failure or adjusts its plan qualifies, and the boundary against conventional coding agents dissolves. If persistence is required, then several systems placed in the task-time category—Live-SWE-Agent, SEMAG, SEW, AgentConductor—are misclassified. The paper needs an operational persistence test: which artifacts survive the task and condition later tasks? That would resolve the boundary.\n\nThe inclusion/exclusion decisions are also hand-applied. Excluding SWE-RL, SWE-Gym, R2E-Gym, and SWE-RM as \"model optimization\" or \"infrastructure\" may be defensible, but the criteria are not systematic enough for a survey that claims to map the field. And the \"Anonymous, 2026\" citation is a genuine problem: it is almost certainly a self-citation in a line of work the authors are already part of (EvoRepair, EvoMAC, and the Gödel-machine lineage), and readers cannot assess overlap. That should be de-anonymized or dropped before publication.\n\nThe feedback-reliability concern the reader raised is real, and the paper acknowledges it in Section 6, but it is also load-bearing: the whole category rests on executable feedback being trustworthy. The survey could have pressed harder on which surveyed systems actually validate that their evolution signal is reliable rather than just noting the risk.\n\nBottom line: this is a well-executed survey that will be valuable to anyone entering the area. It deserves peer review, but with major revision: tighten the definition, add a persistence criterion, and clean up the citation. I would not desk-reject it.","headline":"A useful, well-organized survey with a genuine taxonomy, but its own definition of self-evolution is at odds with the task-time category, and the anonymous citation is a real problem.","tokens_in":23285,"tokens_out":1957,"would_cite":true,"duration_ms":25776,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper establishes self-evolving coding agents as a distinct research category and organizes them by what evolves: framework, memory, skills and tools, model, or workflow and topology, with timing and evidence as orthogonal dimensions.","keywords":["self-evolving coding agents","LLM agents","software engineering","executable feedback","agent taxonomy","memory","skills and tools","workflow evolution"],"falsifier":"Run the same base coding agent in two modes on a set of public repository issues: a static baseline and a version that evolves by retaining memory and skills from its attempts. Then evaluate both on a fresh, contamination-free set of issues from the same repositories. If the evolved version does not outperform the static baseline on the fresh issues despite matching or beating it on the evolution-time issues, the premise that executable feedback grounds generalizable self-improvement is falsified.","tokens_in":1686,"feed_emoji":"🤖","tokens_out":2633,"duration_ms":100185,"temperature":0.7,"pith_summary":"The paper argues that coding agents should be understood as self-evolving systems: after solving coding tasks, they can update their own framework, memory, skills, tools, model behavior, or multi-agent workflow using the software-specific feedback produced along the way. It proposes a definition and a taxonomy that organizes the emerging literature into five object-centered categories, plus two orthogonal dimensions: when evolution happens and what evidence drives it. The motivation is that software engineering is unusually rich in executable feedback, such as tests, compiler errors, CI logs, and runtime traces, which makes it a natural domain for agents to learn from their own attempts. At the same time, the paper argues this feedback is double-edged: the same signals that enable evolution can be incomplete or misleading, so trustworthy evolution requires validation and selective retention. A sympathetic reader takes away both a clarifying map of the field and a research agenda centered on reliable adaptation.","feed_headline":"Five parts of a coding agent can self-evolve from its own fixes","feed_subtitle":"New survey maps how coding agents learn from tests, logs, and past repairs, and why the feedback can mislead them.","key_machinery":"The key machinery is the object-centered taxonomy. It classifies self-evolving coding agents by the primary artifact being updated: agent framework self-evolution (the agent rewrites or searches over versions of its own scaffold), memory self-evolution (experience banks, repository memory, repair experience), skill and tool self-evolution (distilling trajectories into reusable procedures or creating tools during a task), model self-evolution (policy updates from self-play, coder-verifier co-evolution, adversarial tests), and workflow and topology self-evolution (changing multi-agent collaboration, prompts, or communication structures). Two orthogonal axes complete the framework: evolving tim","core_discovery":"The paper's central claim is that 'self-evolving coding agent' names a distinct kind of system: an agentic software engineering system that updates its behavior or internal components based on previous coding attempts and software-specific feedback. The survey organizes this family by the object that evolves: agent framework, memory, skills and tools, model-side components, or workflow and topology, with timing (task-time, post-task, stage-wise) and evidence type (outcome, environmental, trajectory-derived) as complementary axes. It argues that repository-level context and executable feedback make software engineering a natural domain for agent self-evolution, while feedback unreliability, b","pith_inferences":["The authors treat feedback unreliability as a challenge, but the taxonomy implies something stronger: when benchmark outcomes are the evolution evidence, benchmark overfitting is not an evaluation artifact, it is the objective being optimized. A design principle follows: evolution benchmarks need contamination-resistant validation by construction.","The five categories are not mutually exclusive, and the compound case is left unexplored. A testable prediction is that combining memory (what to attend to) with skills (how to act) compounds gains on repository-level tasks more than either alone.","Holding the base model fixed and varying only the evolving object would separate model capability from agent adaptation. The framework suggests task-time tool and skill evolution helps short tasks, while stage-wise workflow evolution matters more for long-horizon repository work."],"forward_implications":["SWE-bench-style repository issue resolution becomes the canonical test bed, since repository context and executable validation are what make self-evolution observable and measurable.","Evaluation must move beyond one-shot pass rates to include cost, runtime, token use, contamination resistance, and transfer to held-out repositories or tasks.","Each evolution category needs its own safeguards: scaffold rewrites need validation and rollback, memory needs filtering and abstraction, skills need cross-repository generalization checks, and model updates need verifier calibration.","The taxonomy gives system designers a checklist: decide what evolves, when it evolves, and what evidence authorizes the change; systems that evolve several objects should state which one is primary.","The definition draws a line between model post-training on SWE data and genuine self-evolution: the training signal must be closed around the agent's own attempts and executable outcomes."],"supporting_citations":[{"why":"Provides the general self-evolving agent framing (what, when, how, where) from which this survey separates coding agents.","marker":"[Gao et al., 2026]"},{"why":"Companion general survey of self-evolving AI agents that motivates the need for a software-engineering-specific synthesis.","marker":"[Fang et al., 2025]"},{"why":"Main exemplar of framework self-evolution, an agent that edits and validates its own scaffold on coding benchmarks.","marker":"[Robeyns et al., 2025]"},{"why":"Exemplar of archive-based framework evolution, maintaining and selecting among coding-agent variants.","marker":"[Zhang et al., 2025b]"},{"why":"Main exemplar of task-time evolution and tool creation during repository issue solving.","marker":"[Xia et al., 2025]"},{"why":"Exemplar of memory self-evolution, constructing an experience bank from issue-resolution trajectories.","marker":"[Chen et al., 2026]"},{"why":"Exemplar of skill self-evolution, distilling coding trajectories into a reusable skill bank.","marker":"[Li et al., 2026]"},{"why":"Exemplar of workflow and topology self-evolution in automated code generation.","marker":"[Liu et al., 2025]"},{"why":"Defines repository-level issue resolution, the canonical task setting that makes self-evolution measurable.","marker":"[Jimenez et al., 2023]"}],"fun_headline_variants":["Survey: coding agents learn from their own fixes","How coding agents self-evolve from past fixes","Coding agents evolve, but feedback can mislead","Self-evolving coding agents: what changes and why"],"cache_read_input_tokens":25216,"weakest_assumption_plain":"The load-bearing premise is that executable software feedback, such as tests, compiler diagnostics, CI logs, generated tests, and reward models, is trustworthy enough that incorporating it into memory, skills, workflows, or model weights makes an agent better rather than more confidently wrong; the paper itself concedes these signals are imperfect and may carry biases and blind spots.","fun_headline_variants_meta":{"raw":{"variants":["Survey: coding agents learn from their own fixes","How coding agents self-evolve from past fixes","Coding agents evolve, but feedback can mislead","Self-evolving coding agents: what changes and why"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000879,"raw_usage":{"total_tokens":3660,"prompt_tokens":790,"completion_tokens":2870,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":534,"completion_tokens_details":{"reasoning_tokens":2821}},"tokens_in":534,"tokens_out":2870,"duration_ms":20807,"temperature":1.0,"reasoning_tokens":2821,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T19:40:59.458635+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same base coding agent in two modes on a set of public repository issues: a static baseline and a version that evolves by retaining memory and skills from its attempts. Then evaluate both on a fresh, contamination-free set of issues from the same repositories. If the evolved version does not outperform the static baseline on the fresh issues despite matching or beating it on the evolution-time issues, the premise that executable feedback grounds generalizable self-improvement is falsified.","supporting_citations":[],"review_version":1}