Dynamic Adversarial Fine-Tuning Reorganizes Refusal Geometry

Haihua Shen; Junbin Yang; Shan Li; Wenhao Lan; Yijun Yang

arxiv: 2604.27019 · v2 · pith:3EUUJX67new · submitted 2026-04-29 · 💻 cs.LG · cs.CL· cs.CR

Dynamic Adversarial Fine-Tuning Reorganizes Refusal Geometry

Wenhao Lan , Shan Li , Junbin Yang , Haihua Shen , Yijun Yang This is my paper

Pith reviewed 2026-05-21 00:34 UTC · model grok-4.3

classification 💻 cs.LG cs.CLcs.CR

keywords refusal geometryadversarial fine-tuninglanguage model safetyjailbreak attacksinternal representationsrobustness utility tradeoffcausal interventionsover-refusal

0 comments

The pith

Dynamic adversarial fine-tuning relocates refusal carriers from late to early layers while trading robustness for utility.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper studies how Robust Refusal Dynamic Defense, a repeated adversarial fine-tuning process, changes where refusal behavior lives inside a 7B language model. Early training steps drive attack success on fixed harmful queries to zero, yet this also produces the highest rates of refusing harmless queries and a total loss of performance on normal tasks. Later steps partially restore normal-task performance but allow some attack success to return. The authors track this shift by measuring which layers carry the refusal signal and find that the strongest such carrier moves from late layers to early layers around step 250. A reader cares because the results show that safety training does not simply strengthen an output rule but rearranges the model's internal organization, creating an explicit frontier between attack resistance and everyday usefulness.

Core claim

R2D2 preserves a late-layer admissible carrier through step 100 and relocates the best admissible carrier to an early layer by step 250. Supervised fine-tuning alone relocates earlier yet remains less robust. Effective rank stays near 1.24, principal-angle drift is larger under SFT despite weaker robustness, and causal interventions indicate that late-stage R2D2 behavior is governed by a low-dimensional carrier that is still coupled to utility.

What carries the argument

the admissible carrier of refusal behavior, identified by aligning fixed attack suites with a five-anchor refusal-geometry suite and causal interventions that test which layers control refusal

If this is right

Fixed-source HarmBench attack success reaches zero at early checkpoints but coincides with maximal XSTest over-refusal and complete failure on benign-utility tasks.
Later checkpoints recover partial benign utility while adaptive GCG attack success rises to 0.415 at step 250 and 0.613 at step 500.
Step 50 remains closed under both adaptive GCG and AutoDAN, confirming the early robustness peak.
SFT produces larger principal-angle drift yet lower robustness than R2D2.
Late-stage R2D2 behavior is governed by a low-dimensional yet utility-coupled carrier.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Safety methods could be designed to stabilize the carrier at a chosen layer rather than allowing it to drift.
The low effective rank observed here may be a general signature of refusal mechanisms across other alignment techniques.
Testing whether the same layer relocation appears in larger models would clarify how architecture scale interacts with this reorganization.
If the carrier can be moved without the utility cost, targeted layer interventions might improve the robustness-utility frontier.

Load-bearing premise

The five-anchor refusal-geometry suite and the sparse adaptive stress test correctly identify the causal carriers of refusal behavior rather than merely correlating with surface-level refusal rates.

What would settle it

An experiment that ablates the early-layer carrier identified at step 250 and shows that refusal rates remain unchanged while utility is preserved would falsify the claim that this carrier controls the observed behavior.

Figures

Figures reproduced from arXiv: 2604.27019 by Haihua Shen, Junbin Yang, Shan Li, Wenhao Lan, Yijun Yang.

**Figure 1.** Figure 1: Trajectory-level behavioral overview. Panel A shows dense fixed-source HarmBench ASR across training view at source ↗

**Figure 2.** Figure 2: Baseline benign-utility trajectories across anchors. Panel A plots strict and lenient utility. Panel B plots view at source ↗

**Figure 3.** Figure 3: Geometry reorganization across anchor checkpoints. Panel A plots the best admissible carrier location in view at source ↗

**Figure 4.** Figure 4: Causal tradeoff between robustness, over-refusal, and utility. Panel A compares baseline, single-direction, view at source ↗

**Figure 5.** Figure 5: Appendix diagnostic on drift versus robustness. Panel A shows the top three principal angles relative to the view at source ↗

**Figure 6.** Figure 6: Appendix comparison of intervention-side utility. Panel A shows SFT intervention utility at steps 250 and view at source ↗

read the original abstract

Safety-aligned language models must refuse harmful requests without collapsing into broad over-refusal, yet it remains unclear how dynamic adversarial fine-tuning changes the internal carriers of refusal. We study one 7B backbone under supervised fine-tuning (SFT) and under Robust Refusal Dynamic Defense (R2D2), a HarmBench-style adversarial fine-tuning procedure that repeatedly refreshes harmful training cases with current jailbreak attacks. Our protocol aligns fixed-source HarmBench, StrongREJECT, and XSTest with a five-anchor refusal-geometry suite, causal interventions, and a sparse adaptive stress test. R2D2 drives fixed-source HarmBench attack success to zero at early checkpoints, but that regime coincides with maximal XSTest refusal and complete failure on a benign-utility audit. Later checkpoints partially recover benign utility while partially reopening attack success. Sparse adaptive attacks sharpen the same frontier: step~50 remains closed under both adaptive GCG and AutoDAN, whereas adaptive GCG ASR rises to 0.415 at step~250 and 0.613 at step~500. Geometrically, R2D2 preserves a late-layer admissible carrier through step~100 and relocates the best admissible carrier to an early layer by step~250; SFT relocates earlier while remaining less robust. Effective rank remains near 1.24, and SFT exhibits larger principal-angle drift despite worse robustness. Causal interventions show that late-stage R2D2 behavior is controlled by a low-dimensional but utility-coupled carrier. These results support a geometry-reorganization account along a robustness--utility frontier.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

R2D2 shifts the main refusal carrier earlier in the layers over repeated adversarial refreshes on a 7B model, but the geometry measurements risk circularity with the evaluation data.

read the letter

The main thing to know is that this paper tracks how repeated HarmBench-style adversarial fine-tuning moves the effective refusal carrier from late layers to early ones by step 250 on a fixed 7B backbone, while showing the expected robustness-utility trade-off in attack success rates and benign performance. Early checkpoints shut down fixed attacks but over-refuse and fail utility checks; later ones recover some utility at the cost of reopening attacks somewhat. The geometry tracking and causal interventions are the concrete part that stands out from prior static alignment work.

Referee Report

3 major / 2 minor

Summary. The manuscript examines how Robust Refusal Dynamic Defense (R2D2), a HarmBench-style dynamic adversarial fine-tuning procedure, alters refusal behavior in a 7B language model compared to standard SFT. It reports that R2D2 drives fixed-source HarmBench attack success to zero at early checkpoints (coinciding with peak XSTest over-refusal and complete benign-utility failure), with later checkpoints partially recovering utility while reopening attack success (e.g., adaptive GCG ASR rising to 0.415 at step 250). Geometrically, R2D2 preserves a late-layer admissible carrier through step ~100 before relocating the best admissible carrier to an early layer by step ~250; effective rank stays near 1.24, principal-angle drift is smaller than in SFT, and causal interventions indicate control by a low-dimensional utility-coupled carrier. These observations support a geometry-reorganization account along a robustness-utility frontier.

Significance. If the geometric reorganization and causal interventions are shown to be non-circular, the work would supply concrete mechanistic evidence that dynamic adversarial training reorganizes internal refusal carriers rather than simply scaling refusal rates. The protocol that aligns fixed-source benchmarks with a five-anchor geometry suite, sparse adaptive stress tests, and causal interventions is a methodological strength that could be extended to other alignment settings.

major comments (3)

[Methods (five-anchor suite and admissible carrier)] Methods section on the five-anchor refusal-geometry suite: the anchors and admissible-carrier criterion are computed directly from activations on the same refusal and utility datasets used to compute XSTest/HarmBench scores; no independent validation set, parameter-free derivation, or null-effect test on non-anchor directions is described, so the reported layer relocation (late-to-early by step 250) and utility-coupled interpretation risk circularity with the surface metrics they are meant to explain.
[Results (principal-angle drift and effective rank)] Results on principal-angle drift and effective rank: the claim that R2D2 exhibits smaller drift than SFT while maintaining effective rank near 1.24 is presented without error bars, statistical tests, or multiple random seeds; this weakens the differential-robustness interpretation because the quantitative geometry claims are load-bearing for the reorganization account.
[Abstract and causal interventions] Abstract and causal-intervention paragraph: the statement that 'late-stage R2D2 behavior is controlled by a low-dimensional but utility-coupled carrier' rests on interventions whose selection and scope are not fully specified; without explicit confirmation that interventions on non-selected directions produce null effects, the carrier cannot be distinguished from a post-hoc correlate of the observed ASR/utility frontier.

minor comments (2)

[Figures] Figure captions and legends: several panels lack explicit indication of which checkpoints correspond to which curves or colors, making it difficult to map the geometric measurements to the training steps discussed in the text.
[Notation and definitions] Notation: the term 'admissible carrier' is introduced without a concise mathematical definition or reference to an earlier equation; a short formal definition would improve readability.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for their constructive comments on our work. We address each of the major comments point by point below and outline the revisions we will make to the manuscript.

read point-by-point responses

Referee: Methods section on the five-anchor refusal-geometry suite: the anchors and admissible-carrier criterion are computed directly from activations on the same refusal and utility datasets used to compute XSTest/HarmBench scores; no independent validation set, parameter-free derivation, or null-effect test on non-anchor directions is described, so the reported layer relocation (late-to-early by step 250) and utility-coupled interpretation risk circularity with the surface metrics they are meant to explain.

Authors: We recognize the validity of this concern regarding potential circularity. The five-anchor suite uses activations from the refusal and utility datasets to identify carriers, which are then related to the behavioral metrics. To mitigate this, we will revise the Methods section to incorporate an independent validation set for validating the admissible carrier and include a null-effect test on non-anchor directions. This addition will help establish that the geometric reorganization is not circular with the surface-level observations. revision: yes
Referee: Results on principal-angle drift and effective rank: the claim that R2D2 exhibits smaller drift than SFT while maintaining effective rank near 1.24 is presented without error bars, statistical tests, or multiple random seeds; this weakens the differential-robustness interpretation because the quantitative geometry claims are load-bearing for the reorganization account.

Authors: The referee correctly identifies a limitation in the presentation of the geometric results. Due to the substantial computational resources required for multiple independent runs of the dynamic adversarial fine-tuning, we conducted the experiments with a single seed. In the revision, we will add error bars where possible from internal variations and include a discussion of this limitation, along with statistical considerations if additional runs can be performed. revision: partial
Referee: Abstract and causal-intervention paragraph: the statement that 'late-stage R2D2 behavior is controlled by a low-dimensional but utility-coupled carrier' rests on interventions whose selection and scope are not fully specified; without explicit confirmation that interventions on non-selected directions produce null effects, the carrier cannot be distinguished from a post-hoc correlate of the observed ASR/utility frontier.

Authors: We agree that more details on the causal interventions are needed to support the claim. We will update the relevant sections to fully specify the selection and scope of the interventions. Additionally, we will report the results of interventions on non-selected directions to demonstrate null effects, thereby distinguishing the identified carrier from a mere post-hoc correlate. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical measurements on external benchmarks

full rationale

The paper trains models with R2D2 (adversarial fine-tuning on HarmBench-style attacks) and then applies a five-anchor geometry suite plus causal interventions to measure layer-wise carriers on the same standard benchmarks (HarmBench, XSTest, StrongREJECT). These are independent external datasets and evaluation protocols, not self-defined or fitted to produce the reported layer shifts and effective-rank values by construction. No self-citation chain, ansatz smuggling, or renaming of known results appears in the provided abstract or protocol description. The geometry results are post-training observations, not inputs that force the central claim.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The central claim rests on the assumption that the chosen geometry metrics faithfully track refusal behavior and that the training procedure does not introduce artifacts that mimic reorganization. No free parameters are explicitly named in the abstract, but the step numbers at which carriers relocate function as implicit fitted milestones.

axioms (1)

domain assumption The five-anchor refusal-geometry suite isolates the causal carriers of refusal.
Invoked when the paper equates measured carrier locations with control over refusal behavior.

invented entities (1)

admissible carrier no independent evidence
purpose: The low-dimensional subspace or direction that controls refusal decisions.
New term introduced to describe the relocated refusal signal; no independent falsifiable prediction is given for its existence outside the measured geometry.

pith-pipeline@v0.9.0 · 5828 in / 1430 out tokens · 69431 ms · 2026-05-21T00:34:54.325968+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

Geometrically, R2D2 preserves a late-layer admissible carrier through step~100 and relocates the best admissible carrier to an early layer by step~250; effective rank remains near 1.24
IndisputableMonolith/Foundation/RealityFromDistinction.lean reality_from_one_distinction unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

The best admissible R2D2 carrier is late-layer through step 100: (pos=−4,layer=24) ... By steps 250 and 500, the best admissible carrier relocates to (pos=−3,layer=0)

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

2 extracted references · 2 canonical work pages

[1]

InInternational Conference on Learning Representations

Robust LLM safeguarding via refusal feature adversarial training. InInternational Conference on Learning Representations. 19 Block Canonical job(s) Canonical hard- ware Canonical GPU- hours Additional recovered spend Reviewer note SFT training 44779 1×A800 2.176 1.464 Stable canonical run. R2D2 training 31079 4×H100 186.893 96.551 Slurm FAILED only after ...

work page 2026
[2]

SFT interventions remain substantially healthier than the corresponding late-stage R2D2 interventions

Panel B overlays matched R2D2 rows for the same metrics. SFT interventions remain substantially healthier than the corresponding late-stage R2D2 interventions. 20

work page

[1] [1]

InInternational Conference on Learning Representations

Robust LLM safeguarding via refusal feature adversarial training. InInternational Conference on Learning Representations. 19 Block Canonical job(s) Canonical hard- ware Canonical GPU- hours Additional recovered spend Reviewer note SFT training 44779 1×A800 2.176 1.464 Stable canonical run. R2D2 training 31079 4×H100 186.893 96.551 Slurm FAILED only after ...

work page 2026

[2] [2]

SFT interventions remain substantially healthier than the corresponding late-stage R2D2 interventions

Panel B overlays matched R2D2 rows for the same metrics. SFT interventions remain substantially healthier than the corresponding late-stage R2D2 interventions. 20

work page