{"id":"28251da2-96a1-40d6-9a40-383933ee86b9","arxiv_id":"2611.15268","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A Gemma 4 12B agent evolves its own rules and weights from deployment logs, improving pass@1 by 12-19 points on τ2-bench retail and airline tasks without a stronger teacher.","lead":"A 12-billion-parameter AI model improves its own customer-service performance by writing rules for itself and then training on its own conversations, all without help from a stronger teacher. The method closes most of the performance gap to frontier models on a standard benchmark, suggesting that smaller open models can be self-sufficient in private deployments.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"No significant objection identified.","rationale":"The paper presents a well-engineered system with clear experimental design, honest reporting, and strong empirical results. The reader's weakest assumption—that the generated validation set V is a faithful proxy for official tasks—is the most consequential assumption, but the paper provides substantial evidence for it: high correlation (r=0.98), matched knowledge-point coverage, and within-domain agreement within 2.1 points. The limitations, including the single base model, two domains, two rounds, and non-standard simulator, are explicitly acknowledged in the Limitations section. The internal comparisons are valid, and the ablation study convincingly shows the complementary roles of harness rules and trained weights. I agree with the reader's ACCEPT verdict; no additional concern rises to the level of changing it. The suggested concrete test—including rejected candidates—would strengthen the proxy evidence further but is not required to uphold the current claims.","tokens_in":26784,"tokens_out":3743,"duration_ms":29500,"concrete_test":"Recompute the correlation between V and official gains including the two rejected harness candidates (from the retry attempts in Table 14 footnote). If the correlation remains above 0.9, the proxy assumption is further validated; if it drops below 0.8, the gate decisions may rely on V-specific patterns that do not transfer identically.","verdict_should_be":"UNCHANGED","load_bearing_attack":"After a thorough review, I find that the central claim—that a 12B model can co-evolve harness and weights from its own logs under programmatic checks and noise-calibrated gates—is well supported by the evidence. The weakest assumption identified by the reader (the validation set V being a faithful proxy for official tasks) is defended with a high correlation (r=0.98) between gains on V and official tasks, and the authors acknowledge the use of a non-standard simulator and the custom setting. The limitations are openly stated, and the ablation experiments, retest simulations, and gate calibration provide internal validity. No single concern undermines the main conclusion; the paper is methodologically sound and honestly evaluated. I do not have a load-bearing concern that would change the verdict.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"The paper proposes a teacher-free co-evolution loop in which a 12B LLM agent (Gemma 4 12B) improves both its harness (policy skills) and its weights (via GRPO) solely from its own deployment logs and reference outcomes, without a stronger teacher model. On the retail and airline domains of τ²-bench, after two rounds the agent gains 12.5 pass@1 and 18.4 pass@4 points in retail and 19.5 and 22.0 points in airline over its starting point, and exceeds the reference agent (same model with the full policy in its system prompt) by similar margins, closing most of the gap to three frontier models. A 2×2 ablation demonstrates that the evolved rules rescue tasks the agent never solved and carry policy knowledge that RL does not internalize, while the trained weights make already-solved tasks reliable. The system uses programmatic checks, self-review, noise-calibrated gates, and a generated validation set that correlates with official-task gains at r=0.98.","tokens_in":27006,"tokens_out":2393,"duration_ms":19602,"significance":"If the claimed results hold, this work shows that a small open model can self-improve in both instruction-following and behavioral consistency without any frontier model in the loop. This is directly relevant to privacy-sensitive customer-service deployments where sending data to a larger API is not an option. The methodology is carefully designed: training batches, validation sets, and official evaluation tasks are disjoint; all harness edits go through multiple automated checks; and the gate decisions are calibrated to measured dialogue noise. The authors openly acknowledge the limitations (single base model, two domains, two rounds, custom simulator), and the central claim is supported by held-out official results, a convincing ablation, and a high validation-to-official correlation.","major_comments":[],"minor_comments":[{"comment":"The definition of pass@k is given, but in Table 2 and the text it appears as \"pass1\" and \"pass4\". Consider using \"pass@1\" and \"pass@4\" consistently to match the notation in the metrics section and avoid ambiguity with column labels.","section":"§3.1","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The caption states \"The reference agent places the whole policy in its system prompt; all other rows use the skill-based harness.\" It would help to remind the reader that the reference agent is a strong baseline (same model, full policy in prompt) and is run under the same protocol, so the gains are not from a simple system-prompt difference.","section":"Table 2 caption","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The confidence intervals for the bin-wise means in Table 3 are not shown. Adding bootstrap intervals (as done in Table 11) would let readers assess the uncertainty behind statements such as \"rules alone reach 44% success\" and \"weights alone reach 25% success\" for the never-solved bin.","section":"§5.4, Table 3","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"accept"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"This paper shows something real: a 12B Gemma 4 agent improves on τ²-bench by writing its own policy rules and training its own LoRA adapter, with no stronger teacher. The gains are meaningful—12.5 pass@1 in retail, 19.5 in airline—and the ablation cleanly separates what rules do (rescue never-solved tasks) from what RL does (stabilize usually-solved ones). The engineering is careful: programmatic checks, self-review, noise-calibrated gates, and a generated validation set that correlates at r=0.98 with official-task gains.\n\nThe soft spots are the ones the authors already state. Two domains on one benchmark, two rounds, one model. The validation set is faithful but not identical to the real tasks; the loop relies on a non-standard user simulator. The conclusions are solid within that scope, but how well the method generalizes to other domains, models, or longer evolution remains open.\n\nWho this paper is for: anyone building private-deployment customer-service agents who wants a practical recipe for closed-loop improvement. It is also useful for researchers studying self-improving LLM agents as a concrete, well-documented case study.\n\nVerdict: this paper deserves a fair peer review. The limitations are honest and the claims are commensurate with the evidence. It is not a paradigm shift, but it is a competent piece of work that moves the needle on an applied problem.","headline":"A well-engineered demonstration that a 12B model can co-evolve its own harness and weights under programmatic checks, but the evidence is limited to two domains, two rounds, and one base model.","tokens_in":27547,"tokens_out":888,"duration_ms":8256},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"A 12B model co-evolves its own rules and weights from deployment logs without a teacher, closing the gap to frontier models on customer-service tasks.","keywords":["teacher-free co-evolution","LLM agents","harness evolution","reinforcement learning","GRPO","skill-based agents","τ²-bench","self-improvement"],"falsifier":"Run the same co-evolution loop on a different benchmark or domain with a similarly sized model and check whether the gains on a held-out test set match those on the self-generated validation set; a large discrepancy would indicate that the validation set is not representative.","tokens_in":28260,"feed_emoji":"🤖","tokens_out":3812,"duration_ms":29252,"temperature":0.7,"pith_summary":"This paper argues that a small open model can improve its performance on multi-turn customer-service tasks through a self-contained co-evolution loop: it diagnoses failed conversations, writes and tests new rules (the harness), then trains its weights with reinforcement learning on its own rollouts. The key insight is that the two layers play complementary roles—rules supply knowledge the model lacks, while training makes the model apply that knowledge consistently. On the τ²-bench retail and airline domains, a Gemma 4 12B agent starting from a skill-based system that trails frontier models by a wide margin closes most of the gap after two rounds, with gains that transfer to held-out official tasks.","feed_headline":"12B model self-improves with no teacher, rivals frontier agents","feed_subtitle":"By co-evolving its own rules and weights from deployment logs, a small model closes the gap to GPT-5.6 on customer-service tasks.","key_machinery":"The central mechanism is the teacher-free co-evolution loop where a Gemma 4 12B model serves as agent, editor, and trainee. The harness is organized as on-demand skills (split from the domain policy), and the editor diagnoses errors from evidence packs built from failed conversations. Edits are constrained by root-cause prescription, self-review, paired retesting, and a validation gate on a generated validation set. GRPO then trains a LoRA adapter on the agent's own rollouts under the new harness. The separation of policy into skills allows local edits and observable knowledge use, and the noise-calibrated gates compensate for dialogue variability.","core_discovery":"The central discovery is that a 12B model can act as both the agent and the editor, and that its own unreliable judgments can be compensated by programmatic checks and noise-calibrated gates. The evolved rules (harness) and the trained weights (via GRPO) serve different functions: rules rescue tasks the agent never solved and encode policy knowledge that reinforcement learning does not internalize when the rules are present in context, while the trained weights make already-solved tasks more reliable. The 2×2 ablation shows that each layer contributes independently and that their combination yields the largest gains, closing the gap to frontier models (GPT-5.6-sol, Claude Sonnet 5, GLM-5.2) on the benchmark.","pith_inferences":["The noise-calibrated gating framework could be adapted to other settings where a weak model must evaluate its own candidate improvements, providing a principled way to trade off false positives and false negatives.","Because the loops repair structural defects caused by the skill split—such as the baggage-allowance dependency in airline—this method may be especially valuable when a policy is naturally modular but creates cross-document dependencies.","The finding that rules rescue unsolved tasks while weights stabilize solved ones suggests a general decomposition for self-improving systems: first expand coverage via explicit knowledge, then consolidate via training.","Using the same base model as editor may set an upper bound on the complexity of rules it can write; larger or more capable editors could plausibly accelerate the loop or handle more intricate policy defects."],"forward_implications":["Small models can reach near-frontier performance on structured customer-service tasks without any teacher data, reducing privacy and infrastructure costs.","The complementary roles of harness rules and trained weights suggest that closed-loop systems should actively evolve both layers rather than relying on one alone.","The self-generated validation set serves as a reliable proxy for official task distributions, enabling version selection without accessing the benchmark's test set.","The co-evolution loop can be extended to other domains with verifiable outcomes, such as booking, scheduling, or compliance dialogues.","The fact that GRPO under the evolved rules fails to internalize rule knowledge confirms that outcome-based RL alone is insufficient for acquiring new policy facts; explicit rule editing is necessary."],"supporting_citations":[{"why":"Provides the τ²-bench evaluation environment and official tasks used to measure performance.","marker":"Barres et al., 2026"},{"why":"Introduces GRPO, the group-relative policy optimization algorithm used to train the agent's weights.","marker":"Shao et al., 2024"},{"why":"Proposes LoRA, the low-rank adaptation method used for efficient weight updates in the agent.","marker":"Hu et al., 2022"},{"why":"Defines the skill-loading protocol that organizes the policy into on-demand documents.","marker":"Agent Skills, 2025"},{"why":"Describes ExpeL, a precursor that distills insights from successes and failures via add/edit operations, informing the editor design.","marker":"Zhao et al., 2024"},{"why":"Releases the Gemma 4 12B model that serves as the base agent, editor, and training subject.","marker":"Gemma Team, 2026"}],"fun_headline_variants":["12B agent co-evolves rules and weights from deployment logs","Small model closes gap to frontier agents using own logs","Teacher-free: 12B learns rules and trains weights from failures","Co-evolution of skills and RL makes 12B rival GPT-5.6","Rules and RL together lift small model to frontier level"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The generated validation set V is a faithful proxy for the official τ²-bench tasks, so that gains measured on V transfer to the official task distribution.","fun_headline_variants_meta":{"raw":{"variants":["12B agent co-evolves rules and weights from deployment logs","Small model closes gap to frontier agents using own logs","Teacher-free: 12B learns rules and trains weights from failures","Co-evolution of skills and RL makes 12B rival GPT-5.6","Rules and RL together lift small model to frontier level"]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":0.000253,"raw_usage":{"cost":0.000253008,"is_byok":false,"cost_details":{"upstream_inference_cost":0.000253008,"upstream_inference_prompt_cost":0.000116648,"upstream_inference_completions_cost":0.00013636},"total_tokens":1525,"prompt_tokens":1038,"completion_tokens":487,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":256,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":397}},"tokens_in":1038,"tokens_out":487,"cost_source":"openrouter_usage","duration_ms":4568,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":397,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-11-13T20:16:00.062696+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":"Run the same co-evolution loop on a different benchmark or domain with a similarly sized model and check whether the gains on a held-out test set match those on the self-generated validation set; a large discrepancy would indicate that the validation set is not representative.","supporting_citations":[],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}