{"id":"5fecbf32-e461-4457-8580-b79c8895498f","arxiv_id":"2607.19829","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"An evolving attack-defense loop, DARWIN, achieves state-of-the-art jailbreak success rates on frontier LLMs/guardrails and trains a guardrail with 91.6% average unsafe recall while retaining ~100% benign pass rate.","lead":"DARWIN is a framework that continuously evolves jailbreak attacks and trains guardrails in a loop, reporting near-perfect attack success on several frontier LLMs and strong guardrail detection. It matters because it replaces static red-teaming with an adaptive attack-defense cycle, potentially changing how LLM safety is evaluated and defended.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Guardrail generalization claim lacks evidence of train/eval disjointness: §3.3 training set D is unspecified; Table 2 benchmarks may overlap D, inflating 91.6% recall.","rationale":"The paper presents a coherent evolutionary attack-defense framework, and the experimental design is extensive in coverage. However, the central empirical claims depend on a clear separation between training and evaluation data. The defense claim in particular — 91.6% average recall while maintaining 100% benign pass rate — is the kind of result that would be expected if the guardrail had memorized the test benchmarks during online adversarial training. The manuscript's silence on the composition of D (§3.3) and the absence of any statement that the 12 evaluation benchmarks are disjoint from D is a concrete, checkable omission. It is more load-bearing than the reader's specific concern about JailbreakHub appearing in both the strategy pool and the attack evaluation, because the guardrail claim rests entirely on generalization to held-out harmful prompts, and the training procedure directly optimizes on adversarial rewrites of whatever D contains. The attack-side concern about sandbox proxy representativeness is real but weaker: the adaptive feedback loop during attack execution can compensate for a weak sandbox filter, and the attack ASR numbers are on HarmBench/AdvBench, which are not obviously in the strategy pool. Therefore, I identify the guardrail train/eval disjointness as the primary risk. The paper should be accepted only conditionally on the authors providing the training data composition and demonstrating that the reported recall is not inflated by overlap. This is consistent with the reader's conditional verdict, but with a sharper condition. No code or data is provided; the absence of error bars further limits confidence, though the large reported margins make random variation an unlikely explanation for the main effect.","tokens_in":14029,"tokens_out":8480,"duration_ms":86979,"concrete_test":"Obtain the full composition of training set D used in §3.3 and compute its overlap (exact or near-duplicate after disguise) with the 12 harmful benchmarks in Table 2. Then recompute DARWIN-Guard's unsafe recall on the disjoint subset of each benchmark (i.e., prompts whose content appears in D are excluded). If the average recall on the disjoint subsets drops by more than a few percentage points from 91.6%, the result is inflated by training contamination. Alternatively, retrain DARWIN-Guard on a version of D that explicitly excludes all 12 evaluation benchmarks and rerun Table 2; the average should remain close to 91.6% to support the generalization claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central defense claim of DARWIN-Guard (91.6% average unsafe recall across 12 safety benchmarks, Table 2) is only meaningful if the evaluation benchmarks are not part of the training data. In §3.3, the training set D is introduced as 'a dataset containing harmful and benign queries' without specifying its source or composition. §4.1.2 lists the 12 harmful benchmarks used for evaluation, but the paper never states that D is disjoint from them. Since DARWIN-Guard is trained on adversarial rewrites of harmful prompts generated by DARWIN-Attack, and since several of those benchmarks (e.g., JailbreakHub, XSTest, HarmBench) are commonly used as harmful-prompt collections, the risk of direct overlap or near-duplicate content is high unless an explicit separation is enforced. The only held-out construction reported in §4.4 is a 'held-out attack set' of strategy families excluded from the evolving training process; it does not address benchmark separation for guardrail training. Consequently, the reported 91.6% average could reflect memorization of training prompts rather than generalization to unseen harmful inputs. Without disclosure of D or a retraining experiment that excludes the 12 benchmarks, the guardrail's headline result is not verifiable.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DARWIN, an evolutionary attack-defense framework for jailbreaking LLMs and training guardrails. DARWIN-Attack maintains an evolving strategy pool fed by external sources, genetic mutation, and feedback-driven composition, and selects/composes strategies using a Q-learning-inspired transition update. DARWIN-Guard performs online adversarial training on the adversarial rewrites produced by DARWIN-Attack, jointly on harmful and benign disguised queries. The paper claims state-of-the-art attack success rates on HarmBench and AdvBench against four frontier LLMs and two guardrails, and an average unsafe recall of 91.6% across 12 safety benchmarks while maintaining 100% pass rates on benign benchmarks. The core contribution is replacing static attack/defense evaluation with a closed-loop evolution process.","tokens_in":14413,"tokens_out":2813,"duration_ms":33875,"significance":"If the claims are substantiated, DARWIN would be a meaningful step toward dynamic safety evaluation: the idea of continuously expanding an attack-strategy pool and using the resulting adversarial samples to iteratively retrain a guardrail is timely and potentially impactful. The symmetric treatment of harmful and benign disguised queries to preserve utility is a sensible design. The paper also attempts a cross-stage evaluation with a held-out attack set, which is the right kind of evidence for generalization. However, the manuscript currently lacks the reproducibility artifacts and verification details needed to assess these contributions. The attack results are reported without variance or code, and the guardrail training/evaluation split is not shown to be disjoint. The benign pass-rate metric is saturated at ~100% for all baselines, so it does not discriminate over-refusal behavior. These gaps are material because the headline numbers cannot be independently checked and may reflect contamination or metric insensitivity rather than genuine capability.","major_comments":[{"comment":"The training set D for DARWIN-Guard is described only as 'a dataset containing harmful and benign queries' with no source or composition, and the evaluation benchmarks in Table 2 are never stated to be disjoint from D. Since several of those benchmarks (e.g., JailbreakHub, HarmBench, XSTest) are commonly used harmful-prompt collections, the 91.6% average unsafe recall could be inflated by overlap or near-duplicates between training and evaluation. This is load-bearing for the guardrail's generalization claim. The held-out attack set in §4.4 concerns strategy families, not benchmark separation. Please specify D, perform a retraining experiment excluding the 12 evaluation benchmarks, or otherwise demonstrate disjointness.","section":"§3.3, §4.1.2, Table 2"},{"comment":"The benign pass-rate results are non-discriminative: every model, including all baselines, scores 100% or near-100% on all eleven benign benchmarks. This ceiling effect means the claim that DARWIN-Guard 'avoids over-refusal' is not supported by the data. The experiment needs a harder benign distribution, e.g., benign prompts that resemble jailbreak syntax, difficult safe/unsafe boundary cases, or disguised benign queries, to show meaningful differences. As written, the reported 100.0% pass rate is indistinguishable from the trivial behavior of 'always safe' on this set.","section":"§4.3, Table 3"},{"comment":"Several components of DARWIN-Attack are underspecified, preventing reproduction and verification. The sandbox filter ASR_sb uses an unspecified 'local aligned LLM' and threshold τ_sb; the external strategy sources are not identified beyond 'public platforms' and 'broad external sources'; and the genetic operations (crossover, mutation) are not defined. Without these details, a reader cannot assess whether the reported ASR improvements come from the evolutionary process or from ingesting already-successful strategies, nor whether strategies from JailbreakHub (also an evaluation benchmark) leaked into the attack pool. Please disclose the sandbox model and τ_sb, list the external source categories, and describe the genetic operators.","section":"§3.2, §4.1.1"},{"comment":"All quantitative results are reported as point estimates with no variance. Given the stochastic nature of LLM generation and the small number of baseline repetitions, single-run numbers are not sufficient to support claims of state-of-the-art performance, especially where margins are 10–20 points. The cross-stage cross-play matrix in Figure 4 is described qualitatively in §4.4; a full numeric matrix and the exact held-out strategy families should be reported. Please provide multiple seeds with confidence intervals or at least run-to-run ranges, and specify the query budget for each method.","section":"§4.2, Table 1, Figure 4"}],"minor_comments":[{"comment":"The phrase 'nearly 100%' is used repeatedly; precise numbers would be more appropriate. Also, 'an 91.6%' in §1 should be 'a 91.6%'.","section":"Abstract"},{"comment":"Equation (5) is called 'Q-learning inspired' but no connection to a value function or policy is given. Clarify how the transition matrix update relates to the reward and whether the normalization preserves a valid probability distribution.","section":"§3.2"},{"comment":"The query budget for each baseline is not stated in the text or table. AQC comparisons are only meaningful if all methods are given the same budget; please state the budget explicitly.","section":"§4.1.1"},{"comment":"Several references are dated 2026, which is consistent with the arXiv stamp but may be worth checking for consistency with the actual publication timeline. No further issue.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The central ideas are interesting, but the lack of code/data and the unresolved train/evaluation overlap for DARWIN-Guard make the headline results currently unverifiable. I would be willing to accept a revised version that adds a disjoint retraining experiment, full methodological details, and variance-aware reporting."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: the core idea — coupling an evolving jailbreak strategy pool with online adversarial guardrail training — is genuinely more than a repackaging of MAJIC. MAJIC works over a fixed strategy pool; DARWIN adds external strategy ingestion and genetic mutation, and closes the loop by training the guardrail on the attacker's current failures. That is a meaningful step toward open-ended red-teaming, and the cross-stage results (later attackers beating earlier guards, later guards holding earlier attacks) are the right kind of evidence to look for.\n\nWhat the paper does well: the setup is thoughtful about the two failure modes of adversarial training. Training on disguised benign prompts to avoid over-refusal, and anchoring on raw prompts via the λ_raw term, are sensible design choices. The reported ASR gains over MAJIC are large (e.g., 30.7% → 76.7% on Claude Sonnet 4.6), and the query-count efficiency is consistent with a strategy pool that reuses what works.\n\nNow the soft spots, and they are real. First, there is no code, no data, and no error bars anywhere. For a paper making near-100% ASR claims, that leaves the headline numbers unverifiable. Second — and this is the bigger issue — the guardrail generalization claim is not currently supported. In §3.3, the training data D is described only as \"a dataset containing harmful and benign queries\"; the paper never states that D is disjoint from the twelve evaluation benchmarks in Table 2, several of which (JailbreakHub, HarmBench, XSTest) are exactly the kind of public harmful-prompt collections that would plausibly be used to build D. The 91.6% average recall could reflect memorization, and the held-out attack set in §4.4 does not address benchmark separation — it only holds out strategy families, not datasets. That is a load-bearing omission for the paper's main defense claim, and it needs a direct answer: specify D, or run a retraining experiment that excludes the evaluation benchmarks. Third, the 100.0% pass rate on every benign benchmark is so clean that the benign set looks non-discriminative; it would help to see harder benign cases. Also unknown: what exactly the \"external sources\" are and whether they could leak evaluation items into the attack strategy pool.\n\nProportionately: the attack-side results are plausible and internally consistent; the defense-side generalization claim is the weak pillar. This is not a paper to ignore — the loop concept is timely and the authors show awareness of the right failure modes — but it is not ready to be taken at face value. For a reader working on red-teaming or guardrail robustness, it is worth a serious look with a critical eye. I would send it to peer review, but with the explicit expectation that the authors disclose the training data, release code or detailed ablations, and address the disjointness question.\n\nWho benefits: researchers building automated red-teaming pipelines and guardrail training. Deserves a serious referee, though the current form needs heavy revision. My recommendation: engage, but treat the numbers as provisional until the data split and code are clarified.","headline":"Genuinely new coupling of evolving jailbreak strategies with online guardrail training, but the evidence is under-specified: no code, no error bars, and the guardrail training set is never shown to be disjoint from the evaluation benchmarks.","tokens_in":14819,"tokens_out":3045,"would_cite":false,"duration_ms":32515,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Treating jailbreak as open-ended evolution lets an adversary crack frontier LLMs near-perfectly while co-training a guardrail that blocks 91.6% of unsafe prompts without hurting benign utility.","keywords":["jailbreak attacks","LLM safety","evolutionary algorithm","adversarial training","guardrails","attack-defense loop","strategy pool","online learning"],"falsifier":"A concrete test would be to run DARWIN-Attack with a strategy pool built exclusively from sources disjoint from the evaluation benchmarks, and to evaluate against a target model family never used in the sandbox filter (e.g., a new closed-source model not in the training loop). If the attack success rate drops dramatically under either condition, the evolutionary advantage would be an artifact of data overlap or sandbox overfitting. Similarly, training a static guardrail on the same raw malicious dataset without the online loop and measuring its recall on the held-out attack set would isolate w","tokens_in":13990,"feed_emoji":"🧬","tokens_out":6615,"duration_ms":65147,"temperature":0.7,"pith_summary":"The paper argues that static jailbreak attacks and fixed guardrail training are fundamentally mismatched to real-world adversaries that continuously expand their attack space. To fix this, it proposes an evolving attack-defense loop: an attacker that discovers, mutates, and composes new jailbreak strategies from an ever-growing pool, and a guardrail that is retrained online on the very attacks the adversary generates. The paper reports that this co-evolution drives attack success rates to near-100% on several frontier models and guardrails (over 90% on GPT-5.5), while the guardrail reaches an average unsafe recall of 91.6% across twelve safety benchmarks and a near-100% pass rate on benign inputs. The central insight is that safety evaluation and protection must be open-ended, not one-time, and that adversarial and defensive capabilities can improve together.","feed_headline":"Evolutionary loop: jailbreak near-100%, guardrail blocks 91.6%","feed_subtitle":"Why care: static red-teaming falls behind; evolving both attacker and guardrail reveals and fixes new jailbreaks.","key_machinery":"The load-bearing machinery is the coupled evolution loop defined by the update equations: the attacker state (A_{t+1} = S_t ∪ ΔS_t, Update(T_t, F_t)) and the guardrail update (θ_{t+1} = argmin_θ L_Guard(θ; Attack(A_{t+1}, G_{θ_t}))). The strategy pool S_t is the evolving attack space; the transition matrix T_t is a Q-learning-inspired controller that reweights strategy sequences based on failure feedback; and the sandbox gate τ_sb filters new strategies by their success against a local aligned LLM. On the defense side, the training objective combines cross-entropy on disguised adversarial samples (to correct misclassifications) with a weighted raw-prompt anchor (to preserve baseline performa","core_discovery":"DARWIN's core discovery is that representing the jailbreak attacker as an evolving state—composed of a growing strategy pool S_t and a Markov transition matrix T_t that learns which strategy sequences work against a given target—transforms red-teaming from a fixed search into an open-ended process. By admitting only strategies that beat a sandbox aligned LLM (ASR_sb above threshold τ_sb), generating new variants through genetic crossover and self-reflection, and composing strategies adaptively after failures, DARWIN-Attack lifts attack success rates on difficult targets like Claude Sonnet 4.6 from ~30% (the best baseline) to 76.7% on HarmBench, and reaches 99.7% on DeepSeek-V4-Pro. Symmetric","pith_inferences":["This framework implies that LLM safety certification should be a continuous, deployment-time process rather than a one-time evaluation, because the attack space is open-ended and new strategies can always emerge.","The sandbox admission filter (τ_sb) anchored on a single aligned LLM could be the weakest transfer point; a natural testable extension is to measure how well strategies admitted by one sandbox transfer to a diverse set of targets, and to make the filter multi-model.","The explicit inclusion of benign disguised queries in training suggests a general design principle for robust classifiers: adversarial training should perturb both positive and negative examples to prevent spurious correlations between superficial patterns and labels.","The Markov-visualized composition of strategies could be adapted to other safety domains, such as detecting model-generated disinformation or adversarial multi-turn dialogue, where an evolving pool of attack patterns would similarly outpace static defenses."],"forward_implications":["If the loop works as claimed, static safety benchmarks substantially underestimate real-world jailbreak risk, and red-teaming should move to evolving adversaries that expand their own strategy space.","Guardrails trained with online adversarial updates can maintain robustness to historical attacks (no catastrophic forgetting) while improving against held-out, never-seen attack families—the paper reports ASR on a held-out set dropping from 54% to 26% as the loop progresses.","The very low average query counts (e.g., 2.2 on DeepSeek-V4-Pro for HarmBench) imply that an evolving strategy pool makes jailbreaking cheap enough for large-scale automated safety auditing.","The near-perfect benign pass rate suggests that adversarial training that includes disguised benign anchors can avoid the over-refusal trap, offering a template for future safety-alignment methods."],"fun_headline_variants":["Evolving jailbreak and guardrail: near-perfect attack, solid defense","Co-evolving attack-defense loop tops jailbreak and guardrail benchmarks","Open-ended evolution: jailbreak and defense both improve","Adaptive attacker and guardrail co-evolve to beat new jailbreaks","Evolving attacker and guardrail: 99.7% jailbreak, 91.6% recall"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that strategies admitted by the local sandbox LLM (using ASR_sb with threshold τ_sb) are representative of the closed-source targets and guardrails, and that the evolving strategy pool does not overlap with the evaluation benchmarks (e.g., JailbreakHub appears both as a strategy source and as an evaluation set); if either assumption fails, the reported attack success rates could reflect sandbox overfitting or benchmark contamination rather than gen","fun_headline_variants_meta":{"raw":{"variants":["Evolving jailbreak and guardrail: near-perfect attack, solid defense","Co-evolving attack-defense loop tops jailbreak and guardrail benchmarks","Open-ended evolution: jailbreak and defense both improve","Adaptive attacker and guardrail co-evolve to beat new jailbreaks","Evolving attacker and guardrail: 99.7% jailbreak, 91.6% recall"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001411,"raw_usage":{"total_tokens":5601,"prompt_tokens":872,"completion_tokens":4729,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":616,"completion_tokens_details":{"reasoning_tokens":4630}},"tokens_in":616,"tokens_out":4729,"duration_ms":32694,"temperature":1.0,"reasoning_tokens":4630,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T11:34:18.288540+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete test would be to run DARWIN-Attack with a strategy pool built exclusively from sources disjoint from the evaluation benchmarks, and to evaluate against a target model family never used in the sandbox filter (e.g., a new closed-source model not in the training loop). If the attack success rate drops dramatically under either condition, the evolutionary advantage would be an artifact of data overlap or sandbox overfitting. Similarly, training a static guardrail on the same raw malicious dataset without the online loop and measuring its recall on the held-out attack set would isolate w","supporting_citations":[],"review_version":1}