Distribution-Specific Curvature Control with Finite-Sample Guarantees for Open-Weight Safety

Ali Dadsetan; Domenic Rosati; Frank Rudzicz; Hassan Chowdhry; Hassan Sajjad; Hong Huang; Subhabrata Majumdar; Xijie Zeng

REVIEW 3 major objections 4 minor 31 references

Distribution-Specific Curvature Control with Finite-Sample Guarantees for Open-Weight Safety

T0 review · 3 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Harmful fine-tuning of open-weight models can be blocked by inflating loss curvature only along an estimated harmful activation subspace, leaving specified benign tasks trainable, with finite-sample and rate guarantees.

desk verdict Distribution-specific curvature control with honest conditional theory and unusually thorough empirics; the rate certificate rests on a clearly-flagged but unproven stability assumption for neural losses. read the letter →

arxiv 2607.22929 v1 pith:G5K5PQK5 submitted 2026-07-24 cs.LG

Domenic Rosati , Ali Dadsetan , Hong Huang , Xijie Zeng , Hassan Chowdhry , Subhabrata Majumdar , Hassan Sajjad , Frank Rudzicz This is my paper

classification cs.LG

keywords harmfulfine-tuningopen-weightsafetyspectraldeformationcurvaturecontrolcontrastivesubspacegradientdescentconvergenceratesfinite-sampleguaranteestamperresistance

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Open-weight models can be undone by short fine-tuning runs, but blocking harmful fine-tuning without also blocking benign adaptation has been an open problem. This paper claims that a function-preserving spectral deformation along a contrastive activation subspace creates a selective optimization barrier: it raises the curvature of the harmful-distribution loss in directions the harmful data actually uses, while leaving the represented function unchanged at initialization and adding only a small curvature term to specified benign tasks. The authors derive finite-sample bounds showing the harmful-curvature certificate holds with high probability once harmful and benign second moments are estimated, and a stability–progress dichotomy converts that curvature into a conditional lower bound on the number of constant-step gradient-descent iterations an attacker needs. If the construction works as claimed, fine-tuning on hazardous knowledge or harmful-assistance data stalls while protected benign tasks still train, and the same barrier covers accidental safety degradation and some emergent-misalignment cases within the fixed-architecture first-order threat model.

What carries the argument

The central object is the contrastive second-moment operator S_lambda := lambda M_H - (1-lambda) M_B, whose top-k eigenvectors define the controlled subspace. Around it is wrapped a compensated two-block spectral deformation: the layer weight is replaced by U T_k Sigma Pi^T V^T together with a compensation matrix U Sigma (T_k Sigma Pi^T)^-1 U^T, so the product equals the original weight exactly at initialization. The diagonal inflation T_k = diag(tau I_k, I_{r-k}) multiplies only the controlled coordinates, making the harmful-distribution Hessian norm grow as tau^2 times the harmful subspace energy while the benign bound grows only with benign energy. This machinery yields the finite-sample

What would settle it

Run a constant-step gradient-descent attack on the defended model with learning rates above C_stab/L_- and show harmful loss falls by D before the trajectory leaves the certified region; or exhibit a harmful fine-tuning dataset whose activation second moments lie almost entirely in the low-energy complement of the estimated subspace, so that E_H(Pi_k) - err_k <= 0. Either outcome would invalidate the tau^2 rate bound.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that distribution-specific curvature control can be achieved by choosing the top-k eigenvectors of the contrastive second-moment operator S_lambda = lambda M_H - (1-lambda) M_B as the controlled subspace and applying a compensated spectral deformation along it. The load-bearing statement is Corollary 4.10: with probability at least 1-delta, every constant-step gradient-descent trajectory on the defended objective either destabilizes (steps above a stability threshold fail inside the certified region) or requires at least T_R = Omega(tau^2 (E_H(Pi_k) - err_k) D_perp/(B_perp_g)^2) iterations to reduce the harmful loss by D before leaving the c

Load-bearing premise

The load-bearing premise is that, on the defended network, every constant gradient-descent step larger than the stability threshold fails inside the certified region, and that the Gauss–Newton part of the Hessian dominates its residual on both distributions; both are assumed for general neural losses and proven exactly only in the quadratic model.

Editorial extensions

If this is right

The harmful-curvature floor ||H^{D_harm}|| >= zeta_H tau^2 c_H sigma_-^2/k (E_H - err_k) holds with probability 1-delta once the sample conditions of Theorem 4.8 are met, so the defence is precomputable from finite held-out data.
An attacker restricted to constant-step GD and steps below the stability threshold pays at least Omega(tau^2 (E_H - err_k) D_perp/(B_perp_g)^2) iterations; larger steps are assumed to fail inside the certified region.
Benign leakage is bounded by a term proportional to the benign energy on the controlled subspace plus the same n^-1/2 estimation penalty, which is why benign tasks can keep training at low learning rates instead of collapsing as with global spectral deformation.
Empirically the block holds over all tested learning-rate points and checkpoints, across first-order optimizer variants, and under out-of-distribution harmful fine-tuning, while DART, CommonGen, E2E, SamSum, and held-out WikiSQL retain usable training gains.
The fixed-architecture, full-parameter threat model is essential: LoRA route-around and layer-injection attacks recover harmful behaviour, so the guarantee does not extend to parameter-efficient fine-tuning or architecture changes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The subspace rule is a contrastive-PCA trace maximization, so the same construction could be repurposed as a general tool for steering where gradient descent is allowed to move, for example preserving an old task's loss landscape in continual learning or protecting a model from unwanted unlearning.
The certificate lives entirely in the B_perp_g denominator, which points to a natural attack: try to reduce harmful loss using gradient components orthogonal to the controlled parameter subspace; if such updates stay small while still lowering harmful loss, the tau^2 wall is not as high as the bound claims.
Because the finite-sample penalty scales as n^-1/2 + m^-1/2, a testable deployment prediction is that shrinking the harmful sample count used to estimate Pi below the theorem's threshold should reopen the attack; the paper's ablations already show estimation quality is the dominant lever.
The paper's own measurements reveal a large negative-curvature mode on real models; analyzing that negative-curvature instability separately from the positive-curvature certificate could give a cleaner explanation of why the empirical block appears at smaller learning rates than the quadratic model predicts.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read this one. It's the first real attempt to localize spectral curvature inflation to a distribution-specific subspace, and the empirical work is a cut above what this subfield usually ships.

What's genuinely new: HarmAlign picks a contrastive activation subspace (top eigenvectors of lambda M_H - (1-lambda) M_B), deforms the weight along it, and compensates to keep the function exactly invariant. The finite-sample subspace-energy bound (Theorem 4.8) is a standard Bernstein + Davis-Kahan argument, correctly assembled, and they verify the n^{-1/2} rate empirically. The curvature bound then converts into a per-instance iteration lower bound for constant-step GD. That conditional rate analysis is the paper's main theory contribution, and the quadratic case is proved exactly.

The empirical section is broad: direct fine-tuning, three adaptive attacks, OOD harmful distributions, optimizer variants, checkpoint-max ASR, accidental safety degradation and emergent misalignment. They run the defence on public 8B checkpoints, report the benign-trainability cost honestly, and even test the LoRA and layer-injection attacks that are outside the threat model, showing where the method stops.

The soft spot is real but not hidden. The rate certificate (Corollary 4.10) depends on Assumption K.10: every constant step larger than C_stab/L_- fails. For general neural losses this is not proven; it's exact for quadratics and derived under excitation/cross-coupling conditions in Proposition K.12. The paper says so, clearly, in Remark K.23. But it means the per-instance guarantee is conditional, and the condition is exactly the instability half of the dichotomy. A referee should ask for either a broader proof or a deployed-model falsification attempt at large step sizes.

Two other things to flag. First, the residual-control assumption J.7 (Gauss-Newton dominates the Hessian residual) and the constants c_H, c_B, zeta are assumed; they measure some forward diagnostics but not the full set at the deployed points. Second, code and data are 'to be released upon publication' — for a paper this empirical, that's a serious verification blocker.

None of this is fatal. The paper is honest about its assumptions, the construction is sound, and the empirical separation is convincing. I'd send it to a serious referee with a request to pin down K.10 and release the artifacts.

Would I cite it? Yes, if I work on fine-tuning defenses. Worth a reading group slot, too.

Referee Report

3 major / 4 minor

Summary. The paper proposes HarmAlign, a defense against harmful fine-tuning of open-weight LLMs within a fixed-architecture, finite-budget first-order threat model. HarmAlign estimates a contrastive activation subspace from harmful and benign second moments and applies a function-preserving spectral deformation along that subspace, with a compensation layer preserving the represented function at initialization. The theoretical contribution is a chain: functional invariance (Prop. 4.3), curvature lower/upper bounds in terms of controlled-subspace energy (Prop. 4.6), optimality of the contrastive-PCA subspace (Prop. 4.7), finite-sample subspace-energy estimation via matrix Bernstein and Davis–Kahan (Thm. 4.8, Cor. 4.9), and a conditional per-instance iteration lower bound for constant-step GD under Assumptions K.10 and K.18 (Cor. 4.10). Empirically, within the stated threat model, HarmAlign blocks direct fine-tuning and three adaptive attacks on WMDP-bio relearning and BeaverTails harmful-SFT, retains benign trainability, and transfers to OOD harmful distributions, optimizer variants, checkpoint-max evaluation, accidental degradation, and emergent misalignment. The paper is transparent about the conditional nature of the rate certificate and about attacks outside the threat model (layer injection, LoRA).

Significance. If the theoretical claims are taken at face value, this is a meaningful step beyond global spectral deformation: it is the first construction I know of that localizes curvature inflation to a distribution-specific subspace while retaining an explicit finite-sample estimation penalty, and the adaptive empirical evaluation is unusually thorough for this literature. The proof chain is mostly explicit: the compensated-pair invariance is exact, the contrastive-PCA selection is correct, and the finite-sample bound is a standard Bernstein plus Davis–Kahan argument with a clean projector-based formulation. Credit is also due for the exact quadratic dichotomy (Thm. K.4), the forward-only diagnostics, the direct measurement of the tau^2 curvature law on deployed 8B blocks, and the falsifiable threat-model framing. The obstruction is that the paper's main per-instance rate certificate, Cor. 4.10, rests on Assumption K.10, which is proven for quadratics and derived under special excitation/cross-coupling conditions, but is not proven for general neural losses; the advertised 'optimization barrier' therefore needs to be read with a strong caveat, and the theoretical overclaim should be adjusted

major comments (3)

[Cor. 4.10 / Assumption K.10 / Prop. K.12] The load-bearing instability half of the rate certificate is assumed, not proven, for neural losses. Assumption K.10 posits that every constant step eta > C_stab/L- fails within the certified region (T_R = +infinity). This is exactly the large-step branch of the stability–progress dichotomy. Theorem K.4 proves C_stab = 2 only for a sharp–slow quadratic; Proposition K.12 derives the branch only under explicit excitation and cross-coupling conditions (K.9, condition (i)–(ii)), and Remark K.13 states that an unconditional derivation from the neural objective remains open. The derived branch's constants are validated on local synthetic models (Fig. 14), not on the deployed 8B checkpoints. Consequently Cor. 4.10's displayed lower bound T_R = Omega(tau^2 (E_H(Pi_k)-err_k) D_perp/(B_perp_g)^2) is conditional on an assumption that, for the neural losses where the barrier is claimed, is precisely
[Def. K.5 / Remark K.23] The rate certificate is a bound on the stopped hitting time T_R, not on all harmful success. By Definition K.5, T_R = +infinity if the trajectory exits the certified region R before entering the success set S, and Remark K.23 explicitly says the corollary is silent about trajectories after first exit, including possible re-entry. Thus a trajectory that takes a step larger than C_stab/L-, exits R, and then reduces harmful loss outside R is not a counterexample to the displayed bound, but it is a counterexample to the advertised 'optimization barrier' reading. Assumption K.10 rules out success before exit for large steps, but it does not rule out exit-then-success. The empirical checkpoint-max results mitigate this concern in the tested settings, but the theoretical claim should be stated as a localized, region-restricted iteration bound, not as a barrier against harmful fine-tuning achiev
[Prop. 4.6 / Assumption J.7 / App. L.3] The harmful curvature lower bound (1) is conditional on Assumption J.7 (residual domination) and on the harmful nondegeneracy floor c_H. The paper's evidence for residual domination is exact-tier synthetic (Fig. 11), where sigma_1(R) grows linearly in tau while sigma_1(G) grows quadratically, and the text calls the assumption 'self-enforcing' on that synthetic model. The deployed 8B measurements confirm the tau^2 law for block-Hessian extremes and for the certified-direction Rayleigh quotient, but I did not find a deployed measurement of delta_eff = 1 - sigma_1(R)/sigma_1(G) or of the GGN/residual split at the 8B operating points. Since J.7 is an input to the result that Corollary 4.10 consumes, the statement 'certified harmful-distribution curvature lower bound' at 8B is not fully supported by direct measurement. The paper should either report delta_eff on the deployed checkpoints or ex

minor comments (4)

[Sec. 4, after Def. 4.1] The notation T_k := diag(tau I_k, I_{r-k}) is introduced informally; it would help to define it as a display and to state explicitly that the compensation block is stacked before the deformed block in the deployed layer-injection implementation (Remark J.5 is clear, but main-text readers may miss it).
[Cor. 4.10 / App. K] Corollary 4.10 refers to Assumptions K.10 and K.18 without restating their content; since these assumptions are only defined in the appendix, the main-text statement reads as more unconditional than it is. At minimum, add a sentence in the main text spelling out that K.10 is the stability restriction and K.18 is the controlled-subspace budget.
[App. L.5] The phrase 'deployed-style operating point' in the certificate-assembly paragraph is ambiguous about whether the measured constants (bC_stab, B_g, L_2) come from the local synthetic network or from the actual 8B checkpoints. Clarify which tier each constant is measured on, especially since the assembled certificate is stated to be 'established on the local model only.'
[General] The paper uses 'E_H' and 'E_B' both for expectations and for subspace energies; the notation is consistent after Definition 4.5, but a notation table in the main text would reduce friction for readers who do not consult Table 17.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular reduction: the localization, finite-sample, and conditional rate bounds are derived from the stated construction and assumptions; the K.10 stability restriction is an explicitly disclosed incompleteness, not a circular step.

full rationale

Walking the derivation chain (Proposition 4.3 functional invariance, Proposition 4.6 curvature from subspace energy, Proposition 4.7 contrastive subspace selection, Theorem 4.8/Corollary 4.9 finite-sample estimation, Corollary 4.10 conditional rate control), each link is a mathematical consequence of the reparameterization and stated assumptions. The harmful curvature lower bound is proportional to the controlled-subspace energy E_H(Π_k) by construction, but that is a derived identity, not a fitted prediction: τ, k, λ, and Π are fixed before evaluation, and the empirical τ^2 scaling in Appendix L is a post-hoc validation, not an input to the bound. The rate certificate is transparently conditional on Assumption K.10, which is proven exactly for the quadratic model (Theorem K.4), derived under explicit excitation/cross-coupling conditions (Proposition K.12), and otherwise stated as an assumption: the paper explicitly says 'Outside those conditions the restriction remains an assumption.' This is a completeness/correctness limitation of the strongest advertised claim, not a circular reduction, because Corollary 4.10 states its premises rather than smuggling the conclusion into the definitions. The self-citations to Rosati et al. (2026) define the inherited spectral-deformation class and its factorization limitation; they are not load-bearing for the localization guarantee. No step of the paper's derivation reduces by construction to its own inputs.

Assumptions & free parameters 9 free parameters · 9 assumptions · 0 invented entities

The central claims rest on several assumed constants and structurally load-bearing assumptions. The finite-sample estimation is standard, but the per-instance rate control depends on an assumed stability threshold and an assumed controlled-subspace budget. The method itself reparameterizes existing weights and postulates no new physical or architectural entity.

free parameters (9)

tau (deformation scale) = 3e5 (WMDP), 4e4 (BeaverTails), 2e6 (Qwen emergent misalignment)
Inflates the controlled singular values; selected on development data; directly sets the curvature floor and the benign learning-rate headroom.
k (controlled subspace dimension) = 1 (WMDP/Qwen), 2 (BeaverTails)
Number of contrastive eigenvectors used; chosen by coverage/selectivity diagnostics and spectral structure.
lambda (contrast trade-off) = 0.05 (WMDP), 0.5 (BeaverTails/Qwen)
Weights harmful vs. benign second moments in S_lambda; selected on development data.
layer/module deployment site = L4.o_proj (WMDP); L28.{q,k,v,o}_proj (BeaverTails); L18.q_proj (Qwen)
Which weight matrices are deformed; chosen by forward-only diagnostics, but the block's existence depends on this choice.
c_H (harmful nondegeneracy floor) = unknown (assumed > 0)
Prop 4.6(i) assumes average output curvature on every harmful example is at least c_H; if near zero the certified harmful lower bound is vacuous.
c_B (benign curvature ceiling) = unknown (assumed finite)
Prop 4.6(ii) assumes uniformly bounded benign output curvature; if violated the benign upper bound can fail.
zeta_H, zeta_B (residual-control margins) = unknown (assumed in (0,1])
Assumption J.7 requires the GGN term to dominate the Hessian residual; margin values are not measured on the deployed models.
C_stab (stability threshold) = 2 in exact quadratic; measured 0.98-1.5 on local model
Assumption K.10 defines the maximum stable step size; for neural losses it is an assumed quantity, not a theorem.
D_perp (controlled-subspace budget) = fixed on undefended development attacks
Assumption K.18 caps how much harmful-loss reduction can come from the controlled subspace; needed for the T_R lower bound.

assumptions (9)

standard math Matrix Bernstein and Davis-Kahan subspace perturbation theorems
Used in Theorem 4.8 to convert finite-sample second-moment error into projector error; cited from Tropp (2015) and Yu et al. (2015).
domain assumption Bounded activations ||z||^2 <= B almost surely
Assumption J.14; needed for matrix concentration and for the uniform quadratic-form control in Theorem 4.8.
domain assumption Positive boundary eigengap xi_k = nu_k - nu_{k+1} > 0
Required for Davis-Kahan to convert operator deviation into projector error; near-degenerate subspaces need the projector formulation.
domain assumption Residual domination (Assumption J.7): ||R_H(tau)|| <= (1-zeta_H) lambda_max(G_H(tau)) and ||R_B(s)|| <= (1-zeta_B)||G_B(s)|| for s in [1,tau]
Load-bearing for Prop 4.6 curvature bounds; validated synthetically and argued self-enforcing, not proven on deployed 8B models.
ad hoc to paper Stability restriction (Assumption K.10): every constant step eta > C_stab/L- fails within the certified region
This is the large-step branch of the rate dichotomy. It is proven exactly only for a quadratic model; for neural losses it is an assumption, and it is the main unproved premise of Cor 4.10.
ad hoc to paper Controlled-subspace budget (Assumption K.18): cumulative controlled and cross contributions cannot supply D - D_perp
Needed for the small-step branch of the rate bound; it is measured/fixed from development attacks rather than derived from the learning problem.
domain assumption Excitation (Assumption K.9): |q^T grad L_harm(theta_0)| >= c_g > 0
The controlled direction must be excited by the initial harmful gradient; if the harmful objective starts orthogonal to q, the sharp curvature never binds.
domain assumption Hessian-Lipschitz continuity on the certified region (Lemma K.7 premises)
Used to extend certified initial curvature to a persistent floor L- and ceiling L+ over a radius-r ball; estimated numerically, not proven for transformer losses.
domain assumption Identity-activation gaps for the compensation pair
Function preservation holds only when the compensation is stacked across an identity (linear) gap; across nonlinear activations invariance fails (Appendix L.1).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Distribution-Specific Curvature Control with Finite-Sample Guarantees for Open-Weight Safety." pith.science (2026). https://pith.science/paper/G5K5PQK5

@misc{pith2026260722929,
  author       = {Pith},
  title        = {Pith review of: Distribution-Specific Curvature Control with Finite-Sample Guarantees for Open-Weight Safety},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G5K5PQK5}},
  note         = {Machine review of arXiv:2607.22929}
}

read the original abstract

A short fine-tuning run can undo the safety guards of an open-weight model---retraining a refusal-trained assistant to aid weapons development or produce hate speech. Preventing such harmful fine-tuning while retaining benign adaptability remains difficult: the only prior method with an explicit curvature certificate, spectral deformation, inflates curvature globally and thereby obstructs benign adaptation along with harmful adaptation. We propose HarmAlign, which applies function-preserving spectral deformation along a estimated contrastive activation subspace. We derive finite-sample bounds for the estimated subspace energy and the resulting local harmful-distribution curvature lower bound. A stability--progress dichotomy for constant-step gradient descent turns the certified curvature into conditional convergence-rate control. Empirically, within a fixed-architecture, finite-budget first-order threat model, HarmAlign blocks direct fine-tuning and three data- or objective-adaptive attacks across a hazardous-knowledge relearning setting and a harmful-assistance fine-tuning setting, while the protected benign tasks remain trainable. The block persists across the tested first-order optimizer variants over every attack checkpoint, and under out-of-distribution harmful fine-tuning, and it extends to important cases in our threat model: accidental safety degradation and emergent misalignment.

Figures

Figures reproduced from arXiv: 2607.22929 by the authors.

**Figure 1.** HarmAlign creates a selective optimization barrier. Top: HarmAlign applies a reparameterization along an estimated harmful subspace; at initialization the represented function is unchanged up to numerical error. Bottom: harmful fine-tuning diverges (rising loss, coherentASR 0.00) while benign fine-tuning converges (E2E 0.76). to be slow, and its conditioning is distribution-agnostic, impeding benign alongside harm… view at source ↗

**Figure 2.** Phase diagram of HarmAlign: each regime is active depending on the defender’s data availability and the trade-off λ [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗

**Figure 3.** Empirical estimation-rate study at the de [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗

Figures from the paper (8 more)

**Figure 4.** Figure 4: Concentrated and diffuse harmful activation struc [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]

**Figure 6.** Figure 6: Empirically observed selective optimization barrier (Gemma-3-1B). (a) Steps to recovery vs. deformation τ (three BeaverTails attacks: Direct, Mixed, Sidestep; attack lr = 1/τ ). Solid: harmful recovery (T to coherent-ASR ≥ 0.4) rises with τ then is blocked (×; best-ch…

**Figure 7.** Figure 7: Numerical validation of Lemma K.7. (a) Curvature along the controlled direction q vs. distance from θ0: it stays above the L0/2 floor throughout the certified region ∥θ − θ0∥ ≤ r = L0/(2L2). (b) The HVP finite-difference estimate of the local Hessian-Lipschitz slope m…

**Figure 8.** Figure 8: Function preservation requires an identity gap. Invariance error vs. τ : identity (Q→K, V →O) gaps are exact to machine precision, while any genuine nonlinear gap— even 1-homogeneous leaky-ReLU—incurs O(1) error that grows with τ . estimation (Theorem 4.8), the assump…

**Figure 10.** Figure 10: Bound-chain overhang. Per-link looseness of the curvature guarantee; the value-level coordinate-energy step dominates the conservatism. ratio 0.010; Fig. 9b). The coordinate-energy error—the functional the defence actually depends on—decays faster (slope −0.85), bein…

**Figure 11.** Figure 11: Residual domination is self-enforcing. δeff = 1 − σ1(R)/σ1(G) along exact synthetic attack trajectories: never negative, rising with τ and as the attack loss falls. L.3 Residual Domination Is Self-Enforcing The curvature lower bound of Proposition 4.6 assumes the Gaus…

**Figure 13.** Figure 13 [PITH_FULL_IMAGE:figures/full_fig_p036_13.png]

**Figure 14.** Figure 14: Exact-tier validation of the derived stability branch (Proposition K.12). (a) With the sharp mode on the excited direction (derived branch), no success is achieved before region exit at any η > (2 + β)/L−; the negative control—sharp mode placed off the excited directi…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

31 extracted references · 3 linked inside Pith

[1]

J.; Bagaria, V

Abid, A.; Zhang, M. J.; Bagaria, V. K.; and Zou, J. 2018. Exploring patterns enriched in a dataset with contrastive principal component analysis. Nature Communications, 9: 2134

2018
[2]

C.; Foster, D

Arjevani, Y.; Carmon, Y.; Duchi, J. C.; Foster, D. J.; Srebro, N.; and Woodworth, B. 2023. Lower bounds for non-convex stochastic optimization. Mathematical Programming, 199(1): 165--214

2023
[3]

Bengio, Y.; Clare, S.; Prunkl, C.; Andriushchenko, M.; Bucknall, B.; Murray, M.; Bommasani, R.; Casper, S.; Davidson, T.; Douglas, R.; et al. 2026. International ai safety report 2026. arXiv preprint arXiv:2602.21012

arXiv 2026
[4]

Betley, J.; Tan, D.; Warncke, N.; Sztyber-Betley, A.; Bao, X.; Soto, M.; Labenz, N.; and Evans, O. 2025. Emergent Misalignment: Narrow Finetuning Can Produce Broadly Misaligned LLM s. In International Conference on Machine Learning (ICML)

2025
[5]

M.; Kaur, S.; Li, Y.; Kolter, J

Cohen, J. M.; Kaur, S.; Li, Y.; Kolter, J. Z.; and Talwalkar, A. 2021. Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability. In International Conference on Learning Representations

2021
[6]

Cui, J.; Chiang, W.-L.; Stoica, I.; and Hsieh, C.-J. 2025. OR-Bench : An Over-Refusal Benchmark for Large Language Models. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, 11515--11542. PMLR

2025
[7]

Damian, A.; Nichani, E.; and Lee, J. D. 2023. Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stability. In International Conference on Learning Representations

2023
[8]

Das, R.; Agarwal, N.; Sanghavi, S.; and Dhillon, I. S. 2024. Towards Quantifying the Preconditioning Effect of A dam. arXiv preprint arXiv:2402.07114

arXiv 2024

Show all 31 references

[9]

E.; and Straus, E

Forsythe, G. E.; and Straus, E. G. 1955. On best conditioned matrices. Proceedings of the American Mathematical Society, 6(3): 340--345

1955
[10]

S.; et al

Gehrmann, S.; Adewumi, T.; Aggarwal, K.; Ammanamanchi, P. S.; et al. 2021. The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics. In Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM). Association for Computationa...

2021
[11]

Gliwa, B.; Mochol, I.; Biesek, M.; and Wawer, A. 2019. SAMSum Corpus: A Human-Annotated Dialogue Dataset for Abstractive Summarization. In Proceedings of the 2nd Workshop on New Frontiers in Summarization, 70--79. Association for Computational Linguistics

2019
[12]

A.; and Johnson, C

Horn, R. A.; and Johnson, C. R. 2012. Matrix analysis. Cambridge university press

2012
[13]

Huang, T.; Hu, S.; Ilhan, F.; Tekin, S.; and Liu, L. 2024. Harmful fine-tuning attacks and defenses for large language models: A survey. ACM Computing Surveys

2024
[14]

Ji, J.; Liu, M.; Dai, J.; Pan, X.; Zhang, C.; Bian, C.; Sun, R.; Wang, Y.; and Yang, Y. 2023. BeaverTails : Towards Improved Safety Alignment of LLM via a Human-Preference Dataset. In Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track

2023
[15]

P.; and Ba, J

Kingma, D. P.; and Ba, J. 2015. Adam: A Method for Stochastic Optimization. In International Conference on Learning Representations

2015
[16]

Kokiopoulou, E.; Chen, J.; and Saad, Y. 2011. Trace optimization and eigenproblems in dimension reduction methods. Numerical Linear Algebra with Applications, 18(3): 565--602

2011
[17]

Li, N.; Pan, A.; Gopal, A.; et al. 2024. The WMDP Benchmark: Measuring and Reducing Malicious Use with Unlearning. In International Conference on Machine Learning

2024
[18]

O'Brien, K.; Casper, S.; Anthony, Q.; Korbak, T.; Kirk, R.; Davies, X.; Mishra, I.; Irving, G.; Gal, Y.; and Biderman, S. 2025. Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs. arXiv preprint arXiv:2508.06601

2025
[19]

Qi, X.; Wei, B.; Carlini, N.; Huang, Y.; Xie, T.; He, L.; Jagielski, M.; Nasr, M.; Mittal, P.; and Henderson, P. 2025. On Evaluating the Durability of Safeguards for Open-Weight LLM s. In International Conference on Learning Representations (ICLR)

2025
[20]

Qi, X.; Zeng, Y.; Xie, T.; Chen, P.-Y.; Jia, R.; Mittal, P.; and Henderson, P. 2024. Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! In International Conference on Learning Representations (ICLR)

2024
[21]

J.; Kale, S.; and Kumar, S

Reddi, S. J.; Kale, S.; and Kumar, S. 2018. On the Convergence of A dam and Beyond. In International Conference on Learning Representations

2018
[22]

Rosati, D.; Wehner, J.; Williams, K.; Bartoszcze, .; Atanasov, D.; Gonzales, R.; Majumdar, S.; Maple, C.; Sajjad, H.; and Rudzicz, F. 2024. Representation Noising Effectively Prevents Harmful Fine-Tuning on LLMs. In Advances in Neural Information Processing Systems (NeurIPS)

2024
[23]

Rosati, D.; Zeng, X.; Huang, H.; Dionicio, S.; Majumdar, S.; Rudzicz, F.; and Sajjad, H. 2026. Limits of Convergence-Rate Control for Open-Weight Safety. arXiv preprint arXiv:2602.18868

2026
[24]

R.; Vidgen, B.; Attanasio, G.; Bianchi, F.; and Hovy, D

R \"o ttger, P.; Kirk, H. R.; Vidgen, B.; Attanasio, G.; Bianchi, F.; and Hovy, D. 2024. XSTest : A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Com...

2024
[25]

Tamirisa, R.; Bharathi, B.; Phan, L.; Zhou, A.; Gatti, A.; Suresh, T.; Lin, M.; Wang, J.; Wang, R.; Arel, R.; Zou, A.; Song, D.; Li, B.; Hendrycks, D.; and Mazeika, M. 2025. Tamper-Resistant Safeguards for Open-Weight LLMs. In International Conference on Learning Representations

2025
[26]

Tropp, J. A. 2015. An introduction to matrix concentration inequalities. Foundations and trends in machine learning , 8(1-2): 1--230

2015
[27]

Turner, E.; Soligo, A.; Taylor, M.; Rajamanoharan, S.; and Nanda, N. 2025. Model Organisms for Emergent Misalignment. arXiv preprint arXiv:2506.11613

2025 arXiv
[28]

van der Sluis, A. 1969. Condition numbers and equilibration of matrices. Numerische Mathematik, 14(1): 14--23

1969
[29]

Yu, Y.; Wang, T.; and Samworth, R. J. 2015. A useful variant of the Davis--Kahan theorem for statisticians. Biometrika, 102(2): 315--323

2015
[30]

Zhang, Y.; Chen, C.; Ding, T.; Li, Z.; Sun, R.; and Luo, Z.-Q. 2024. Why Transformers Need A dam: A H essian Perspective. In Advances in Neural Information Processing Systems

2024
[31]

Zloczower, I.; Lenga, E.; Gressel, G.; and Mirsky, Y. 2026. One Step to the Side: Why Defenses Against Malicious Finetuning Fail Under Adaptive Adversaries. arXiv preprint arXiv:2605.14605

2026 arXiv

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.

[1] [1]

J.; Bagaria, V

Abid, A.; Zhang, M. J.; Bagaria, V. K.; and Zou, J. 2018. Exploring patterns enriched in a dataset with contrastive principal component analysis. Nature Communications, 9: 2134

2018

[2] [2]

C.; Foster, D

Arjevani, Y.; Carmon, Y.; Duchi, J. C.; Foster, D. J.; Srebro, N.; and Woodworth, B. 2023. Lower bounds for non-convex stochastic optimization. Mathematical Programming, 199(1): 165--214

2023

[3] [3]

Bengio, Y.; Clare, S.; Prunkl, C.; Andriushchenko, M.; Bucknall, B.; Murray, M.; Bommasani, R.; Casper, S.; Davidson, T.; Douglas, R.; et al. 2026. International ai safety report 2026. arXiv preprint arXiv:2602.21012

arXiv 2026

[4] [4]

Betley, J.; Tan, D.; Warncke, N.; Sztyber-Betley, A.; Bao, X.; Soto, M.; Labenz, N.; and Evans, O. 2025. Emergent Misalignment: Narrow Finetuning Can Produce Broadly Misaligned LLM s. In International Conference on Machine Learning (ICML)

2025

[5] [5]

M.; Kaur, S.; Li, Y.; Kolter, J

Cohen, J. M.; Kaur, S.; Li, Y.; Kolter, J. Z.; and Talwalkar, A. 2021. Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability. In International Conference on Learning Representations

2021

[6] [6]

Cui, J.; Chiang, W.-L.; Stoica, I.; and Hsieh, C.-J. 2025. OR-Bench : An Over-Refusal Benchmark for Large Language Models. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, 11515--11542. PMLR

2025

[7] [7]

Damian, A.; Nichani, E.; and Lee, J. D. 2023. Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stability. In International Conference on Learning Representations

2023

[8] [8]

Das, R.; Agarwal, N.; Sanghavi, S.; and Dhillon, I. S. 2024. Towards Quantifying the Preconditioning Effect of A dam. arXiv preprint arXiv:2402.07114

arXiv 2024