{"id":"e50edbef-6b9a-465e-bcb1-c3ef05cf543e","arxiv_id":"2608.01414","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A new white-box defense, DSA, masks a model's strongest refusal neurons and applies dropout during training so safety is redistributed across many compensatory neurons, cutting attack success under heavy pruning from about 250 to below 30 of 313 prompts.","lead":"DSA is a safety-training method that makes open-weight language models keep refusing harmful requests even after an attacker prunes the internal neurons normally responsible for refusal. It works by forcing the model to spread refusal behavior across many neurons, and the paper reports large robustness gains on Qwen, Llama, Gemma, Phi, and multimodal models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline FULL-pruning results are confounded: hyperparameters K, λ, p were selected using the same evaluation metrics, and the FULL attack overlaps the training mask; held-out tuning is needed to support the central claim.","rationale":"The reader's CONDITIONAL verdict is reasonable, and the ABLATE result in Table V is a genuine limitation. However, I see an earlier and more load-bearing issue: the central quantitative claim is evaluated under hyperparameters selected on the same metrics that define the headline. The paper states in Appendix B that 'DSA uses fixed configurations rather than conducting per-method hyperparameter search,' but Table XI and Fig. 5 are exactly a hyperparameter search on Qwen2.5-7B using FULL ASR, GRAD/WANDA ASR, benign answer rate, and ARC, and the 'fixed' configuration is the outcome of that search. This makes the reported ASRs an optimistically selected optimum rather than an unbiased estimate of DSA's performance. The concern is not that the authors are dishonest; it is a standard selection-on-test issue in safety papers, and it is especially consequential because the method's success depends on a fairly narrow hyperparameter window: K=1,000 gives FULL ASR 0 but benign answer rate only 32.5%, while K=15,000 gives FULL ASR 9 and ARC 55.80. A modest change in the selection criterion could shift the reported tradeoff substantially. The paper does receive independent support from the ablation (DSA vs. plain SFT under FULL), the GRAD/WANDA held-out attacks, and the iterative pruning experiments, so I would not reject the work. But the central claim 'DSA substantially improves robustness while preserving utility' should be re-verified with a genuinely held-out hyperparameter selection. That is why my recommended verdict remains CONDITIONAL/UNCHANGED rather than ACCEPT.","tokens_in":21094,"tokens_out":10948,"duration_ms":111597,"concrete_test":"Hold out a tuning split of the evaluation: e.g., use only a random 100 of the 313 StrongREJECT prompts plus one ARC split to select K, λgen, and p on Qwen2.5-7B, then freeze those hyperparameters and evaluate on the remaining 213 prompts. If FULL ASR on the held-out split is materially higher than the reported 9/313 (e.g., >30/213), or if the tuned hyperparameters differ from the default, the headline is selection-dependent. Independently, for each checkpoint compute the overlap between the 3,700 FULL-pruned neurons and the 8,000-neuron training mask, and report ASR when pruning the 3,700 lowest-scoring rather than highest-scoring neurons. If the FULL attack set is >90% contained in the training mask, a disjoint neuron set must be used to validate the 'distributed' claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing gap is that the paper's headline numbers are not an out-of-sample test of DSA. The three main hyperparameters (K=8,000, λgen=0.5, p=0.15) were fixed by inspecting the same evaluation metrics that later appear as results: Table XI and Fig. 5 select K and p on Qwen2.5-7B using FULL ASR, GRAD/WANDA ASR, benign answer rate, and ARC, and these exact values are then used for every reported backbone. Consequently, the reported FULL ASR on Qwen—and by transfer, on LLaMA/Gemma/Phi—is optimistically biased by selection on the test set. Additionally, the FULL attack prunes the top ES/SAS neurons, the same neuron-selection family used to construct the training mask, so the main evaluation largely asks whether the model can survive removal of neurons it was explicitly trained to do without. The paper does provide some out-of-family evidence (GRAD, WANDA, adaptive pruning), but Table V shows ABLATE still achieves 289/313 on Qwen2.5-7B, so the claim that DSA removes the single point of failure is not yet established for all white-box internal attacks. This is fixable with a held-out tuning protocol, but without it the central quantitative claim is not independently verifiable.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DSA (distributed safety alignment), a neuron-level defense that redistributes refusal capability across many FFN neurons instead of concentrating it in a few safety-critical units. DSA computes a direction-aware first-order Taylor score on the pre-down-projection activations, deterministically masks the top-K scored neurons, and applies stochastic dropout to the remaining neurons while fine-tuning FFN parameters on a joint harmful-refusal and benign-utility objective. During inference all masks are removed. The manuscript reports attack success rates under ORI, ES, SAS, and FULL pruning, plus unseen GRAD, WANDA, and ABLATE attacks, across Qwen2.5, LLaMA-3.2, Gemma-7B, Phi-4, and two multimodal models, together with utility benchmarks, over-refusal tests, adaptive-pruning budgets, and iterative-pruning visualizations.","tokens_in":21408,"tokens_out":6592,"duration_ms":62168,"significance":"If the results hold, DSA is a valuable contribution: it moves beyond static safety-neuron defenses, gives a simple mechanism-level justification for why dropout and masking should spread refusal support, and backs the claim with cross-architecture experiments, out-of-family attack evaluation, over-refusal checks, and post-attack utility measurements. The manuscript is honest in Appendix F about the local and FFN-focused nature of the analysis. The central quantitative claim, however, is not yet fully supported because the main hyperparameters are selected using the reported evaluation metrics, because the FULL pruning attack overlaps with the training-time mask construction, and because the ABLATE results on Qwen2.5-7B are a clear failure that is understated in the text.","major_comments":[{"comment":"The main hyperparameters K=8,000, lambda_gen=0.5, and p=0.15 are chosen using the same metrics that later serve as headline results. Table XI selects K because it 'keeps FULL, GRAD, and WANDA ASR at or below 9/313, preserves a 100% benign answer rate, and achieves the highest ARC score', and Fig. 5 selects lambda_gen and p by minimizing FULL ASR on Qwen2.5-7B. The reported FULL ASR values are therefore optimistically biased by selection on the test set, and the transferred results on LLaMA/Gemma/Phi inherit this choice. The authors should adopt a held-out tuning protocol (e.g., a separate validation split or an attack family not used in the headline tables) or report results over the full hyperparameter grid so readers can assess the selection effect.","section":"§4.1, Table XI, Fig. 5"},{"comment":"The FULL pruning attack and the DSA training mask are not independent. DSA deterministically masks the top-K Taylor-scored neurons (K=8,000), while the FULL attack prunes the top 3,700 ES/SAS neurons. Since the Taylor score is built from the same activation/gradient family, the pruned set is likely substantially contained in the training-masked set. The flagship FULL results thus largely test whether the model can survive loss of neurons it was explicitly trained to do without. The GRAD and WANDA results partially address this, but the paper should quantify the overlap between K_core and the FULL attack set, and report a pruning attack based on a held-out or differently constructed neuron ranking, before claiming that the 'single point of failure' has been removed.","section":"§3.3 Eq. (7); Appendix B"},{"comment":"The claimed robustness to general white-box internal attacks is not supported by the ABLATE results. On Qwen2.5-7B, DSA's ABLATE ASR is 289/313, only 10 points below the undefended model and essentially unchanged from SafeNeuron's 295/313; this is not a meaningful defense. The text says DSA 'remains competitive' under ABLATE, which understates the failure. Appendix F narrows the scope to FFN neurons and bounded attacks, but the abstract and conclusion claim robustness against white-box neuron-level attacks without this caveat. The paper should either restrict its claims to neuron-pruning attack families or add a defense/experiment that addresses refusal-direction ablation.","section":"Table V; §4.5 and Appendix F"},{"comment":"All quantitative results are single-run counts with no error bars or multiple seeds. Because DSA training uses stochastic dropout and random data sampling, the differences that carry the ablations (e.g., FULL ASR 9 vs 21 vs 38 in Fig. 4; lambda_gen 9 vs 24 vs 30 in Fig. 5) may be within run-to-run noise. Without standard errors or released checkpoints, the reader cannot tell whether the reported improvements are significant. Please provide at least three seeds for the main comparison and the key ablations, or clearly state any deterministic components and release the exact training/evaluation code and checkpoints.","section":"Tables I–V; §4.4"}],"minor_comments":[{"comment":"References [37] and [38] are the same paper (Sun et al., 'A simple and effective pruning approach for large language models') and should be consolidated to avoid duplicate entries.","section":"References"},{"comment":"The text says the sensitivity experiments contain '11 configurations in total (Tables XI)', but Table XI lists only five K values; the phrasing and reference to 'Tables' should be corrected.","section":"Appendix B"},{"comment":"The ablation figure labels ('−27', '−29', etc.) are unclear. The caption should explain what the numbers above bars represent, and the bar colors/line styles should be defined.","section":"Fig. 4"},{"comment":"There is a typesetting artifact in the abstract warning: 'W ARNING' should read 'WARNING'.","section":"Abstract"},{"comment":"The name of the attack from ref. [42] is inconsistent: the bibliography and some text use 'NeuroStrike' while the main text and Table III headers use 'NeuronStrike'. Please unify the spelling.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the manuscript relies on several references dated 2025–2026 that may be concurrent or not yet widely available, especially SafeNeuron [39], NLSR [43], NeuroStrike [42], TraceRouter [33], and Orthoeraser [34]. Since the main comparisons are against SafeNeuron and NeuroStrike, please verify that these references are publicly accessible and that their use is consistent with the venue's disclosure norms. No code or checkpoints are provided, which is a limitation for a defense paper whose headline numbers depend on stochastic training and hyperparameter choices."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Honest read: DSA is a genuinely new defense idea—train the model to keep refusing after its dominant refusal FFN neurons are removed—and the empirical sweep is broad. But the headline numbers carry a selection-on-evaluation bias, and the 'no single point of failure' claim only covers FFN pruning, not white-box attacks like refusal-direction ablation. Read it as a strong proof-of-concept, not yet an out-of-sample demonstration.\n\nWhat's new: prior neuron defenses preserve safety neurons; DSA deliberately removes the top Taylor-scored neurons and adds dropout so refusal redistributes. That is a real conceptual shift. The adaptive-pruning result (43.7x more neurons to break) is the most convincing piece. The paper also does the right checks: unseen attacks (GRAD, WANDA), over-refusal, post-attack utility, iterative pruning, and a local concentration bound that connects the training objective to pruning robustness.\n\nWhere it's soft. First, the default K, lambda, p were chosen on Qwen2.5-7B using the same metrics that later appear as results. Table XI even shows K=3000 gives 0 FULL ASR but worse WANDA and benign rate; picking K=8000 for the 'best balance' means the reported 9/313 is not an out-of-sample number. Held-out tuning or a separate validation set would fix it. Second, the FULL attack prunes top-3700 ES/SAS neurons, which overlap with the training-time mask (K=8000), so the main test largely asks whether the model can do without the neurons it was trained to do without. That's the intended mechanism, but it limits what the headline claim says about generalization. Third, ABLATE still succeeds on Qwen2.5-7B (289/313), and the authors admit in Appendix F that DSA only targets bounded FFN attacks. So 'no single neuron of failure' is accurate only within that family. Fourth, single runs, no error bars, no code or checkpoints—hard to verify.\n\nDoes the central argument hold? In a qualified way. The adaptive pruning and GRAD/WANDA results show something real beyond memorizing the training mask. But the evaluation, as written, is not clean enough to take the strong claims at face value.\n\nWho should read this: anyone working on white-box defenses or interpretable safety. I'd send it to a serious referee; it deserves time. I'd tell the authors to release code/checkpoints, run repeated seeds, and do held-out hyperparameter selection before acceptance.","headline":"A genuinely new defense idea with broad evidence, but the headline pruning numbers are selected on the same metrics and the 'no single neuron' claim only covers FFN pruning.","tokens_in":21898,"tokens_out":3314,"would_cite":true,"duration_ms":30680,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A training scheme that masks a model's dominant safety neurons and adds stochastic dropout can make refusal persist after large-scale neuron pruning, without lowering general task performance.","keywords":["distributed safety alignment","white-box attacks","safety neurons","neuron pruning","refusal behavior","feed-forward networks","LLM safety","multimodal safety"],"falsifier":"Take a DSA-realigned model and attack it with a rank-one refusal-direction intervention instead of pruning FFN neurons: delete or negate the residual-stream direction most predictive of refusal and measure ASR on the same 313 harmful prompts. Table V in the paper already shows 289/313 for Qwen2.5-7B under ABLATE, so this experiment directly tests whether the claim of distributed neuron-level safety extends beyond the FFN-pruning family.","tokens_in":20987,"feed_emoji":"🛡️","tokens_out":13996,"duration_ms":120261,"temperature":0.7,"pith_summary":"Aligned language models refuse harmful requests through a small set of internal 'safety neurons'; a white-box attacker who locates and prunes those neurons can disable the refusal. The paper proposes distributed safety alignment (DSA), which during realignment deliberately masks the top refusal-supporting neurons in the language-side feed-forward networks and applies stochastic dropout to the rest, forcing the model to re-encode refusal across many compensatory neurons instead of one vulnerable hub. The paper reports that after the strongest pruning attack it tests—removing roughly 3,700 top-scored neurons—the attack success rate falls from 176–279/313 to at most 30/313 on Qwen2.5 models and to at most 37/313 on LLaMA-3.2 models, while ARC, GSM8K, and TruthfulQA scores stay at or above the original model's levels. The same recipe transfers to Gemma-7B, Phi-4, and two vision-language models, including attacks that hide instructions inside images. If these results hold, safety alignment can be made structurally redundant rather than dependent on protectable neuron lists.","feed_headline":"Prune 3,700 safety neurons and an aligned model still refuses","feed_subtitle":"Distributed safety alignment spreads refusal across many neurons, so pruning attacks lose their single point of failure.","key_machinery":"The load-bearing mechanism is a two-part intervention on feed-forward activations. First, the direction-aware Taylor score (Eq. 4) ranks neurons by whether suppressing them would increase the response-only refusal loss; it keeps only positive suppression effects. Second, adversarial redundant safety alignment combines a fixed core mask with token–neuron dropout: the fixed mask removes the currently dominant refusal neurons, and the stochastic mask prevents safety from re-concentrating on any small substitute set. The supporting analysis (Appendix D) models the refusal margin as a local linear sum over neuron contributions $c_{x,j}$; dropout adds a loss penalty proportional to $\\sum_j $c^{2}$_{x,","core_discovery":"The paper's central claim is that refusal behavior can be made redundant at the neuron level. DSA localizes safety-relevant computation to the input of each language-side down-projection layer, treating every feature coordinate as a neuron, and scores each neuron with a direction-aware first-order Taylor statistic $$$S_i^{{(l)}}$=\\frac{1}{N}\\sum_{n=1}^{N}\\frac{1}{T_n}\\sum_{t=1}^{T_n}\\max\\left(0,\\,-$a^{{(l)}}$_{n,t,i}\\,\\frac{\\partial $L^{{(n)}}$_{\\mathrm{ref}}}{\\partial $a^{{(l)}}$_{n,t,i}}\\right),$$ which is large only when suppressing the neuron is predicted to increase the refusal loss. The top-$K$ neurons across all layers form the core refusal set. During realignment, DSA fixes a mask that zeroes these","pith_inferences":["Scope inference: the demonstrated redundancy is tied to the training-time intervention surface—language-side FFN down-projection inputs. The paper's own ABLATE result on Qwen2.5-7B (289/313) and its Appendix F caveat that the analysis is local and FFN-focused imply that rank-one refusal-direction attacks or attention/cross-layer attacks are outside the claimed coverage.","Testable extension: the same mask-plus-dropout recipe could be applied to attention heads or residual-stream coordinates. If the concentration-penalty argument in Appendix D generalizes, refusal should become distributed over those components too; if not, the mechanism is specific to FFN activations.","The iterative-pruning evidence, with distinct compensatory routes recruited across rounds, suggests a finite redundancy capacity. Repeating adaptive pruning beyond the reported 2.75× budget and measuring how ASR rises per round would map how many independent refusal routes DSA can sustain.","Because DSA trains only FFN parameters and leaves the rest of the model frozen, a natural next step is combining it with parameter-efficient continual safety updates; the paper does not test that combination."],"forward_implications":["Removing the top ~3,700 safety-scored neurons no longer collapses refusal: on Qwen2.5 the FULL-pruning attack success rate drops from 248–279/313 to 9–30/313, and on LLaMA-3.2 from 176–221/313 to 7–37/313.","An adaptive attacker has to work much harder: on Qwen2.5-7B the budget needed to degrade refusal to the same threshold grows from 183 neurons to 8,004 neurons, a 43.7× increase, and two successive re-localized FULL-pruning rounds still leave ASR at 38/313.","The defense is not architecture-specific: DSA keeps FULL-pruning ASR at 11/313 on Gemma-7B and 3/313 on Phi-4, and extends to vision-language models where harmful instructions are embedded in images.","Under held-out GRAD and WANDA pruning attacks, DSA keeps ASR at 9/313 and 4/313 on Qwen2.5-7B, indicating the redundancy is not overfit to the ES/SAS selection criteria.","The robustness is bought offline: after realignment the model runs with no masks, hooks, extra parameters, or online neuron search, and general utility is preserved at or above the original backbone."],"supporting_citations":[{"why":"Supplies the neuron-level safety alignment baseline and the experimental protocol DSA compares against.","marker":"[39]"},{"why":"Defines the neuron-level white-box attack family (ES/SAS/FULL) and the VL-Question multimodal setting used to evaluate pruning robustness.","marker":"[42]"},{"why":"Provides the ABLATE refusal-direction attack and the evidence that refusal is mediated by a single direction, the vulnerability DSA is designed to remove.","marker":"[2]"},{"why":"SN-Tune, the safety-specific-neuron tuning baseline, represents the 'strengthen a fixed set of neurons' approach DSA contrasts with.","marker":"[46]"},{"why":"NaturalReasoning supplies the benign utility data used in the joint generality-preserving loss.","marker":"[45]"},{"why":"LLM-LAT supplies harmful prompts used for refusal-neuron localization and realignment.","marker":"[31]"},{"why":"StrongREJECT is the held-out text safety benchmark on which attack success rate is measured.","marker":"[35]"},{"why":"XSTest is the over-refusal benchmark used to show DSA does not become indiscriminately refusing.","marker":"[30]"}],"fun_headline_variants":["Distributed safety alignment makes refusal neuron-proof","Refusal survives even when safety neurons are pruned","No single point of failure: safety neurons are redundant","Model stays safe after neuron-level attacks via DSA","Spread refusal across neurons to beat white-box attacks"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The whole method rests on the premise that refusal behavior can be re-encoded inside the language-side feed-forward subnetwork by masking and dropping those neurons; if the true safety mechanism lives in attention, residual-stream directions, or cross-layer circuits, the training perturbation never touches it.","fun_headline_variants_meta":{"raw":{"variants":["Distributed safety alignment makes refusal neuron-proof","Refusal survives even when safety neurons are pruned","No single point of failure: safety neurons are redundant","Model stays safe after neuron-level attacks via DSA","Spread refusal across neurons to beat white-box attacks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000324,"raw_usage":{"total_tokens":1669,"prompt_tokens":770,"completion_tokens":899,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":514,"completion_tokens_details":{"reasoning_tokens":826}},"tokens_in":514,"tokens_out":899,"duration_ms":8811,"temperature":1.0,"reasoning_tokens":826,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T00:11:51.646432+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a DSA-realigned model and attack it with a rank-one refusal-direction intervention instead of pruning FFN neurons: delete or negate the residual-stream direction most predictive of refusal and measure ASR on the same 313 harmful prompts. Table V in the paper already shows 289/313 for Qwen2.5-7B under ABLATE, so this experiment directly tests whether the claim of distributed neuron-level safety extends beyond the FFN-pruning family.","supporting_citations":[{"cited_title":"NeuroStrike: Neuron-level attacks on aligned LLMs","cited_arxiv_id":null,"evidence_quote":"Defines the neuron-level white-box attack family (ES/SAS/FULL) and the VL-Question multimodal setting used to evaluate pruning robustness."},{"cited_title":"Refusal in language models is mediated by a single direction","cited_arxiv_id":null,"evidence_quote":"Provides the ABLATE refusal-direction attack and the evidence that refusal is mediated by a single direction, the vulnerability DSA is designed to remove."},{"cited_title":"Understanding and enhancing safety mechanisms of LLMs via safety-specific neuron","cited_arxiv_id":null,"evidence_quote":"SN-Tune, the safety-specific-neuron tuning baseline, represents the 'strengthen a fixed set of neurons' approach DSA contrasts with."},{"cited_title":"Weston, and Xian Li","cited_arxiv_id":null,"evidence_quote":"NaturalReasoning supplies the benign utility data used in the joint generality-preserving loss."},{"cited_title":"XSTest: A test suite for identifying exaggerated safety behaviours in large language models","cited_arxiv_id":null,"evidence_quote":"XSTest is the over-refusal benchmark used to show DSA does not become indiscriminately refusing."}],"review_version":1}