Model Immunization from a Condition Number Perspective

Amber Yijia Zheng; Brian Bullins; Cedar Site Bai; Raymond A. Yeh

REVIEW 3 major objections 4 minor 17 references

Model Immunization from a Condition Number Perspective

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Model immunization reduces to controlling the condition number of the downstream Hessian.

desk verdict The R_ill regularizer and the condition-number framing are worth a look, but the paper's central analytical claim (Prop. 3.2) is false, so the theory as stated does not hold. read the letter →

arxiv 2505.23760 v1 pith:HL76ZVQH submitted 2025-05-29 cs.LG

Amber Yijia Zheng , Cedar Site Bai , Brian Bullins , Raymond A. Yeh This is my paper

classification cs.LG MSC 65F3515A1868T07

keywords modelimmunizationconditionnumberHessianmatrixlinearprobingfine-tuningregularizationAIsafetytransferlearning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Model immunization is the idea of releasing pre-trained models that are deliberately hard to fine-tune on harmful tasks without being hard to adapt to benign ones. This paper tries to turn that idea into a quantitative framework: it defines an immunized model by the condition numbers of the Hessians of two downstream linear-probing problems, one on the harmful task and one on the benign task, and shows that fine-tuning difficulty tracks these numbers. For linear models it proves a sharp description of the Hessian's singular values in terms of the alignment between the feature extractor's singular vectors and the harmful data's covariance eigenvectors, which explains both when immunization is possible and when it is not. The paper then supplies an algorithm with two regularizers, one that provably raises the harmful-task condition number and one that provably lowers the benign-task one, and shows empirically that it immunizes linear models and, despite a theoretical gap, deep networks on ImageNet-scale tasks. A sympathetic reader would care because this is the first framework that makes immunization measurable, optimizable, and comparable across methods, rather than an empirical byproduct of a specific generative model.

What carries the argument

The load-bearing object is the Hessian of linear probing, $H(\theta)=\theta^\top K\theta$ with $K=X^\top X$, and its condition number $\kappa(H)=\sigma_{\max}/\sigma_{\min}$, which controls the gradient-descent convergence rate $1-\sigma_{\min}/\sigma_{\max}$. The carrying identity is Proposition 3.2, $\sigma_i=\sum_j (\sigma_{\theta,i}(u_{\theta,i}^\top q_j)\sqrt{\gamma_j})^2$, which ties every Hessian singular value to the alignment $u_{\theta,i}^\top q_j$ between the feature extractor's singular vectors and the data covariance's eigenvectors. The constructive machinery is a matched pair of differentiable regularizers: the existing $R_{\mathrm{well}}(S)=\tfrac12\|S\|_2^2-\tfrac{1}{2p}\|S\|_F^2$ for shrinking $\kappa$, and the new $R_{\mathrm{ill}}(S)=1/\big(\tfrac{1}{2k}\|S\|_F^2-\tfrac12(\sigma_{\min}^S)^2\big)$ for inflating it, which upper-bounds $1/\log\kappa(S)$ and, when the extremal singular value is unique, has a closed-form gradient and a monotonicity guarantee under gradient descent. Theorem 4.2 computes the gradients with respect to $\theta$ for $H(\theta)=\theta^\top K\theta$, and Theorem 4.3 shows the monotone decrease and increase survive updates to $\theta$ itself with covariance-preconditioned steps $K^{-1}\nabla_\theta R$. A dummy layer implementation makes each preconditioned update a single backward pass, so the algorithm runs in standard autodiff frameworks.

What would settle it

Compute the SVDs of $K_P$ and $K_H$ for any real dataset pair: if the singular vectors are essentially aligned, the angle analysis predicts no feature extractor can achieve $\mathrm{RIR}\gg 1$, so a method that does would refute the theory. For the monotonicity claim, run a single regularizer update $\theta'\leftarrow\theta-\eta K^{-1}\nabla_\theta R$ within the stated step-size bounds and evaluate $\kappa(H_H(\theta'))$ and $\kappa(H_P(\theta'))$; if $\kappa(H_H)$ decreases or $\kappa(H_P)$ increases on any such step, Theorem 4.3 fails.

Watch

Extended reading notes

Core claim

Under the paper's setting, a linear feature extractor $f_\theta(x)=x^\top\theta$ is frozen while a linear probe is fit on an $\ell^2$ loss. The harmful fine-tuning Hessian is $H_H(\theta)=\theta^\top K_H\theta$, where $K_H$ is the harmful data covariance. Proposition 3.2 gives its singular values as $\sigma_i=\sum_j (\sigma_{\theta,i}\,(u_{\theta,i}^\top q_j)\,\sqrt{\gamma_j})^2$, so each Hessian singular value is a projection of the extractor's singular vectors onto the covariance's eigenvectors, weighted by both spectra. Because the same $\theta$ also determines the benign Hessian $H_P(\theta)=\theta^\top K_P\theta$, the achievable immunization is decided by the relative angles between the singular vectors of $K_P$ and $K_H$: if the two covariances are perfectly aligned, no extractor can make the harmful task ill-conditioned and the benign task well-conditioned at the same time. On this basis the paper defines an immunized model as one for which $\kappa(H_H(\theta_I))\gg\kappa(K_H)$, $\kappa(H_P(\theta_I))\le\kappa(K_P)$, and the pre-training loss is preserved, and proposes to find such a $\theta_I$ by minimizing the pre-training loss plus $R_{\mathrm{well}}(H_P)$ and a new regularizer $R_{\mathrm{ill}}(H_H)$ that monotonically decreases and monotonically increases the respective condition numbers. The paper reports relative immunization ratios far above one on House-prices regression and all 90 MNIST binary-task pairs, and ratios of roughly 3.5 to 41 for ResNet18 and ViT immunized against Stanford Cars and Country211 while pre-training accuracy is retained.

Load-bearing premise

The guarantees are proven only for linear probing with plain gradient descent over a known harmful dataset on a frozen linear feature extractor; the joint update of both regularizers with the supervised loss is not covered by the monotonicity theorems, and the linear analysis leaves deep networks without a theory.

Editorial extensions

If this is right

Immunization becomes a single measurable number: the relative immunization ratio $\mathrm{RIR} = \frac{\kappa(H_H(\theta_I))/\kappa(H_H(I))}{\kappa(H_P(\theta_I))/\kappa(H_P(I))}$, which the paper uses to compare immunization methods directly.
If the harmful and benign data covariances have their singular vectors perfectly aligned, the framework says no feature extractor can satisfy both condition-number inequalities, so immunization must come from the data split rather than from the model.
The empirical comparisons show the approach succeeds on all 90 MNIST digit-pair tasks while the baselines fail on most pairs, and on deep networks it measurably slows linear-probing convergence on the harmful task while preserving ImageNet accuracy.
Because $\kappa$ controls convergence speed, the same regularizer pair that immunizes a model also provides the first differentiable tool for deliberately making an optimization problem ill-conditioned, which the paper notes has not been done before.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An immediate audit follows from the angle condition: computing the SVD of $K_P$ and $K_H$ before training gives an upper bound on how large an RIR any linear extractor could achieve, so practitioners could pre-check whether immunization is worth attempting; the paper diagnoses this dependence but does not package it as a pre-check.
The theory covers plain gradient descent with an $\ell^2$ loss, yet the MNIST linear experiments already used the Adam optimizer; whether adaptive optimizers erode the condition-number slowdown is a testable question the paper leaves open.
Because the RIR is computed from the released extractor and a public harmful dataset, a downstream auditor could verify an immunization claim directly from model weights without trusting the trainer's procedure.
The same condition-number inflation could be repurposed for other deterrence goals, such as slowing distillation or model stealing, which lies outside the paper's scope.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper introduces a condition-number perspective on model immunization and a new regularizer, R_ill, designed to make a matrix ill-conditioned. The regularizer itself is a reasonable extension of the R_well idea from Nenov et al. (2024), and the monotonicity guarantees in Theorem 4.1 look plausible. The experiments on linear models and on ImageNet-pretrained ResNet/ViT are fairly extensive, and the authors are honest about the limitations of their threat model (linear probing, frozen features, known harmful data). That honesty is a plus.

The problem is that the paper's key analytical result, Proposition 3.2, is simply false. The proof constructs an orthonormal matrix O by normalizing the rows of M = Σθ Uθ^T Q Γ, but the rows of M are not orthogonal in general. Concretely, take theta as a 45-degree rotation and K = diag(4,1); then M M^T has off-diagonal entries and eigenvalues 4 and 1, while Eq. (10) predicts both singular values as 2.5. So the claimed angle-based interpretation of when immunization is possible does not follow from the math. This is the central "key understanding" advertised in the abstract, and it is unsupported.

The regularizer theorems (4.1–4.3) appear to be separate from Proposition 3.2, and I did not find a clear error in those arguments. The empirical evaluation, however, has its own circularity: the RIR metric is the ratio of condition numbers that the training objective directly regularizes, and hyperparameters are selected to optimize that metric. The fine-tuning accuracy curves in Fig. 3 provide some independent evidence, and those do show slower convergence on the harmful task.

Overall, this is not a desk-reject situation. The R_ill regularizer is a useful contribution, and the empirical study is reasonably careful. But as submitted, the theory does not support the paper's central claim. A serious referee should see it, and the right outcome would be a reject-and-resubmit or a major-revision decision, with the expectation that the authors either fix or remove Proposition 3.2, evaluate with held-out hyperparameters, and report independent success metrics rather than only RIR. If those changes are made, the empirical contribution could be salvageable.

Referee Report

3 major / 4 minor

Summary. The paper formalizes model immunization for linear probing as a condition-number problem: a pre-trained feature extractor is immunized if linear probing on a harmful dataset is made harder (larger Hessian condition number) while probing on a pre-training dataset is not made harder, and pre-training utility is preserved. It proposes two regularizers, R_well (minimizing condition number) and a new R_ill (maximizing condition number), gives closed-form gradients with respect to the feature extractor, proves monotonic increase/decrease guarantees, and evaluates the resulting Algorithm 1 on linear regression, MNIST classification, and ImageNet-pretrained ResNet18/ViT deep networks. The central analytical claim is Proposition 3.2, which asserts a closed-form, angle-based formula for the singular values of theta^T K theta.

Significance. If correct, the framework would be a valuable first principled account of model immunization, with a concrete definition, an evaluation metric, and regularizers whose monotonicity is proved. The paper is clearly written and ships code, closed-form gradient derivations, and detailed experimental settings, which are strengths. However, Proposition 3.2, the load-bearing result that motivates the angle-based interpretation in the abstract and Section 1, is false in general. In addition, a stated property of the new regularizer R_ill (Theorem 4.1(1)) is incorrect as written. The empirical headline metric RIR is essentially the ratio of the condition numbers that the training objective directly regularizes, so the quantitative results are partly circular, though the convergence curves in Figures 1 and 3 provide some independent evidence. Because the central theoretical contribution is not established, the paper cannot be accepted in its current form.

major comments (3)

[Section 3.1, Eq. (10), Appendix B.1]
[Section 4.1, Appendix B.2.1]
[Section 5, Eq. (15), Tables 1 and 2]

minor comments (4)

[Definition 3.1, Eq. (5)]
[References]
[Eq. (15)]
[Appendix B.4, proof of Theorem 4.3(2)]

Circularity Check

2 steps flagged · score 6.0 of 10

The headline RIR metric restates the condition-number objective that the algorithm directly optimizes, and hyperparameters are selected using RIR itself, so the linear-model validation is substantially circular.

self definitional [Sec. 3 Definition 3.1; Sec. 4 Eq. (11); Sec. 5 Eq. (15)]
"a model is said to be immunized if: (a) It is more difficult to apply linear probing on the harmful task DH ... κ(∇2wL(DH,w,θI))≫κ(∇2wL(DH,w,I)); (b) It is not more difficult ... κ(∇2ωL(DP,ω,θI))≤κ(∇2ωL(DP,ω,I)). ... we introduce the relative immunization ratio (RIR) ... RIR≜(κ(HH(θI))/κ(HH(I)))÷(κ(HP(θI))/κ(HP(I))) ... min_{ω,θ} Rill(HH(θ)) + Rwell(HP(θ)) + L(DP,ω,θ)."

Definition 3.1 defines immunization through the condition numbers of the harmful and pretraining Hessians. Equation (11) trains exactly those Hessians through Rill(HH) and Rwell(HP), and Theorem 4.3 guarantees that the Algorithm 1 updates monotonically increase κ(HH) and decrease κ(HP). Equation (15) then reports the ratio of precisely the same two quantities. A high RIR after training is therefore a certificate that the optimizer did what the loss asked; it is not an independent test that the model is hard to fine-tune. The convergence curves in Figures 1 and 3 provide some independent evidence, but the primary quantitative metric is the optimized objective.
fitted input called prediction [Appendix C.2, Immunization training details; reported in Tables 1-3]
"We choose λP and λH by balancing the gradient norm of Rwell and Rill. Specifically, we obtain the scale of λP and λH first and search over multiples of {1,2,3,5}. For linear models, we search over the set of {0.0005,0.001,0.005,0.01} and report the best result. ... The number of epochs is based on early stopping using RIR and the test accuracy."

The reported success metric RIR is also the model-selection criterion: the regularizer weights and the stopping epoch are chosen by optimizing RIR. Consequently, the headline RIR values in Tables 1-3 are fitted outcomes selected on the evaluation metric, not out-of-sample predictions of immunization quality. This amplifies the self-definitional circularity: the method is both trained and selected to maximize the quantity that is later presented as evidence of effectiveness.

full rationale

The paper's self-citations are not load-bearing: Zheng & Yeh (2024) and Zheng & Yeh (2025) are cited as prior empirical work and baselines, not as the authority for the condition-number claims, and the monotonicity theorems in Section 4 are proved in the appendix rather than imported from the cited papers. Proposition 3.2 may be mathematically incorrect, but an invalid proof is a correctness issue, not a circularity, so it does not enter the score. The circularity is in the validation loop: Definition 3.1 defines an immunized model as one with large κ(HH) and small κ(HP); Equation (11) and Algorithm 1 directly manipulate those two condition numbers; Equation (15) measures success as their ratio. The known convergence bound in Equation (2) then turns the trained condition numbers into the convergence slowdowns shown in Figures 1 and 3, so those curves are consequences of the optimized objective rather than independent confirmation of a novel prediction. The deep-net experiments are more independent because the theory does not apply to them and the fine-tuning accuracy curves demonstrate actual slowdown, but the headline tables still rest on the circular RIR metric. This is a partial, not total, circularity, hence a score of 6 rather than 8 or 10.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical or conceptual entities. The free parameters are the regularizer weights and optimizer settings, tuned on the target metric. The domain assumptions about the threat model restrict the applicability of the guarantees.

free parameters (3)

lambda_P, lambda_H regularization weights = HousePrice 100/1e7, MNIST 1/5e7, ResNet18 5e-5/2e6 or 1e-4/2e6, ViT 3e-6/3e8 or 1e-6/1e8 (Table 4)
Chosen by balancing gradient norms and searching over multiples {1,2,3,5} and then selecting the best result on the RIR metric, which is the evaluation target.
learning rate eta = 0.005 (HousePrice), 0.001 (MNIST), 1e-5 (ImageNet)
Standard optimizer hyperparameter, but the choice is reported without sensitivity analysis.
number of training epochs = 100 (HousePrice), 30 (MNIST), 2-3 (ImageNet)
Early stopping is based on RIR and test accuracy, so the stopping criterion depends on the evaluation metric.

assumptions (5)

standard math SVD of theta and eigendecomposition of data covariance are used to express H(theta) as V M M^T V^T (Proposition 3.2).
The decomposition is standard, but the subsequent claim that M M^T is diagonal is false in general.
domain assumption The adversary uses linear probing with a frozen feature extractor and an L2 loss (Section 3).
The entire framework and algorithm are built around this threat model; if the adversary fine-tunes the full network, the guarantees do not apply.
domain assumption The defender has access to the harmful dataset X_H during immunization (Algorithm 1, input).
Algorithm 1 uses X_H to compute K_H, so immunization is limited to known harmful tasks.
domain assumption The pre-trained feature extractor is a linear map theta (theory) and deep-nets are treated as an empirical extension (Sections 3, 5.2).
The paper states this is a practical limitation and that deep-net results are empirical.
ad hoc to paper The maximum and minimum singular values of the Hessian are unique when applying the regularizers (Theorems 4.1, 4.3).
Differentiability and monotonicity guarantees rely on this genericity condition, which is not verified in experiments.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Model Immunization from a Condition Number Perspective." pith.science (2026). https://pith.science/paper/HL76ZVQH

@misc{pith2026250523760,
  author       = {Pith},
  title        = {Pith review of: Model Immunization from a Condition Number Perspective},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HL76ZVQH}},
  note         = {Machine review of arXiv:2505.23760}
}

read the original abstract

Model immunization aims to pre-train models that are difficult to fine-tune on harmful tasks while retaining their utility on other non-harmful tasks. Though prior work has shown empirical evidence for immunizing text-to-image models, the key understanding of when immunization is possible and a precise definition of an immunized model remain unclear. In this work, we propose a framework, based on the condition number of a Hessian matrix, to analyze model immunization for linear models. Building on this framework, we design an algorithm with regularization terms to control the resulting condition numbers after pre-training. Empirical results on linear models and non-linear deep-nets demonstrate the effectiveness of the proposed algorithm on model immunization. The code is available at https://github.com/amberyzheng/model-immunization-cond-num.

Figures

Figures reproduced from arXiv: 2505.23760 by the authors.

**Figure 1.** Norm ratio Eq. (16) vs. Epochs. We visualize the convergence of linear probing of different immunized models using gradient descent with an exact line search. Here, Identity corresponds to not using a feature extractor, i.e., θI = I. Observe that Ours made the convergence faster on DP while slower in DH when compared to the other baselines; consistent with the results in Tab. 1. via gradient descent instead of using… view at source ↗

**Figure 2.** Visualization of log(RIR) of binary classification tasks created from MNIST. Each element in the figure corresponds to the log(RIR) of a model immunized against DH from the pre-training task of DP. We color the block blue if RIR ≫ 1, and red otherwise. Our method succeeds in immunizing the model across all digit pairs, while the baselines failed in most pairs. be observed that Rill Only fails for all digit pairs, IM… view at source ↗

**Figure 3.** Test accuracy vs. Fine-tuning Epochs on DH. We visualize the test accuracy of linear probing on ImageNet of different immunized models using gradient descent. Here DH is the Stanford Cars dataset. 1969), and has recently found applications in deep learning (Saratchandran et al., 2024). Most related to this work, Balazs et al. (2024) propose to regularize the condition number of weight matrices by directly adding th… view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Dummy layer with selective inverse feature covariance matrix in backward function. 25 [PITH_FULL_IMAGE:figures/full_fig_p025_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

17 extracted references · 7 canonical work pages

[7]

Fast and near-optimal diagonal preconditioning.arXiv preprint arXiv:2008.01722,

Jambulapati, A., Li, J., Musco, C., Sidford, A., and Tian, K. Fast and near-optimal diagonal preconditioning.arXiv preprint arXiv:2008.01722,

arXiv 2008
[8]

To- wards safe self-distillation of internet-scale text-to-image diffusion models.arXiv preprint arXiv:2307.05977,

Kim, S., Jung, S., Kim, B., Choi, M., Shin, J., and Lee, J. To- wards safe self-distillation of internet-scale text-to-image diffusion models.arXiv preprint arXiv:2307.05977,

arXiv
[9]

Kingma, D. P. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980,

arXiv
[11]

Savani, Y ., Trockman, A., Feng, Z., Schwarzschild, A., Robey, A., Finzi, M., and Kolter, J. Z. Antidistillation sampling.arXiv preprint arXiv:2504.13146,

arXiv
[14]

(4) [Monotonic Decrease] If σmax S is unique, update S with ∇SRwell(S) such that S′ =S−η 1∇SRwell(S) for 0< η1 < κ(S)−1 (1− 1 p )κ(S)+ 1 p , thenκ(S ′)< κ(S)

(3) [Differentiability] If σmax S =σ 1 > σi for any i >1, i.e., σmax S is unique, then Rwell(S) is differentiable and its gradient is given by∇ SRwell(S) =σ 1u1v⊤ 1 − 1 p S. (4) [Monotonic Decrease] If σmax S is unique, update S with ∇SRwell(S) such that S′ =S−η 1∇SRwell(S) for 0< η1 < κ(S)−1 (1− 1 p )κ(S)+ 1 p , thenκ(S ′)< κ(S). B. Proof of Propositions...

work page 2024
[16]

∂S 1 2 σmin S 2 =∂ S(R′ ill,1 ◦σ)(S) = n UDiag(µ)V ⊤ µ∈∂ σR′ ill,1(σ) o Given that R′ ill,1 is differentiable and apparently also absolutely symmetric with respect to σ, by Lemma B.2, we know 1 2 (σmin S )2 is also differentiable and ∇ 1 2 σmin S 2 =UDiag(∇ σR′ ill,1(σ))V ⊤ =UDiag(σ min S ep)V ⊤ =σ min S ukv⊤ k . 16 Model Immunization from a Condition Num...

work page 2008
[224]

The feature embedding sizes for ResNet18 and ViT are 512 and 768, respectively

We also create the dataset with the built-in function create dataset from Wightman (2019). The feature embedding sizes for ResNet18 and ViT are 512 and 768, respectively. To facilitate balanced training when dataset sizes differ, we implement aCombinedLoader, which pairs batches from two data loaders. The longer dataset dictates training duration, while t...

work page 2019
[1952]

F., and Liu, L

Huang, T., Hu, S., Ilhan, F., Tekin, S. F., and Liu, L. Harm- ful fine-tuning attacks and defenses for large language models: A survey.arXiv preprint arXiv:2409.18169,

arXiv

Show all 17 references

[1984]

Generative AI misuse: A taxonomy of tactics and insights from real-world data.arXiv preprint arXiv:2406.13843,

Marchal, N., Xu, R., Elasmar, R., Gabriel, I., Goldberg, B., and Isaac, W. Generative AI misuse: A taxonomy of tactics and insights from real-world data.arXiv preprint arXiv:2406.13843,

arXiv
[1995]

In this case, for the singular value decompositionS=UDiag(σ)V ⊤, ∇(f◦σ) (S) =UDiag(∇f(σ))V ⊤

without Convexity).If a function f:R p →R is absolutely symmetric, that is, ∀x∈R p and any y as a permutation of x, f(x) =f(y) , then f◦σ is differentiable at matrix S∈R p1×p2 if and only if fis differentiable atσ=σ(S). In this case, for the singular value decompositionS=UDiag...

1995
[2017]

Y ., Okolo, C

Bengio, Y ., Mindermann, S., Privitera, D., Besiroglu, T., Bommasani, R., Casper, S., Choi, Y ., Fox, P., Garfinkel, B., Goldfarb, D., Heidari, H., Ho, A., Kapoor, S., Khalatbari, L., Longpre, S., Manning, S., Mavroudis, V ., Mazeika, M., Michael, J., Newman, J., Ng, K. Y ., O...

2025
[2020]

A, we provide the complete statements of the properties ofR well(S)for minimizing the condition number

11 Model Immunization from a Condition Number Perspective Appendix The appendix is organized as follows: • In Sec. A, we provide the complete statements of the properties ofR well(S)for minimizing the condition number. • In Sec. B, we provide the complete proof for the Theorem...

2024
[2021]

Certified data removal from machine learning models

Guo, C., Goldstein, T., Hannun, A., and Van Der Maaten, L. Certified data removal from machine learning models. arXiv preprint arXiv:1911.03030,

1911 arXiv
[2022]

Unlearn- ing isn’t deletion: Investigating reversibility of machine unlearning in llms.arXiv preprint arXiv:2505.16831,

Xu, X., Yue, X., Liu, Y ., Ye, Q., Hu, H., and Du, M. Unlearn- ing isn’t deletion: Investigating reversibility of machine unlearning in llms.arXiv preprint arXiv:2505.16831,

arXiv
[2023]

The malicious use of artificial intelligence: Forecasting, prevention, and mitigation.arXiv preprint arXiv:1802.07228,

Brundage, M., Avin, S., Clark, J., Toner, H., Eckersley, P., Garfinkel, B., Dafoe, A., Scharre, P., Zeitzoff, T., Filar, B., et al. The malicious use of artificial intelligence: Forecasting, prevention, and mitigation.arXiv preprint arXiv:1802.07228,

arXiv
[2024]

Estimating or propagating gradients through stochastic neurons for con- ditional computation.arXiv preprint arXiv:1308.3432,

Bengio, Y ., L´eonard, N., and Courville, A. Estimating or propagating gradients through stochastic neurons for con- ditional computation.arXiv preprint arXiv:1308.3432,

arXiv
[2025]

Bertran, M

URL https: //www.gov.uk/government/publications/ international-ai-safety-report-2025. Bertran, M. A., Tang, S., Kearns, M., Morgenstern, J. H., Roth, A., and Wu, S. Reconstruction attacks on ma- chine unlearning: Simple models are vulnerable. InProc. NeurIPS,

2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [7]

Fast and near-optimal diagonal preconditioning.arXiv preprint arXiv:2008.01722,

Jambulapati, A., Li, J., Musco, C., Sidford, A., and Tian, K. Fast and near-optimal diagonal preconditioning.arXiv preprint arXiv:2008.01722,

arXiv 2008

[2] [8]

To- wards safe self-distillation of internet-scale text-to-image diffusion models.arXiv preprint arXiv:2307.05977,

Kim, S., Jung, S., Kim, B., Choi, M., Shin, J., and Lee, J. To- wards safe self-distillation of internet-scale text-to-image diffusion models.arXiv preprint arXiv:2307.05977,

arXiv

[3] [9]

Kingma, D. P. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980,

arXiv

[4] [11]

Savani, Y ., Trockman, A., Feng, Z., Schwarzschild, A., Robey, A., Finzi, M., and Kolter, J. Z. Antidistillation sampling.arXiv preprint arXiv:2504.13146,

arXiv

[5] [14]

(4) [Monotonic Decrease] If σmax S is unique, update S with ∇SRwell(S) such that S′ =S−η 1∇SRwell(S) for 0< η1 < κ(S)−1 (1− 1 p )κ(S)+ 1 p , thenκ(S ′)< κ(S)

(3) [Differentiability] If σmax S =σ 1 > σi for any i >1, i.e., σmax S is unique, then Rwell(S) is differentiable and its gradient is given by∇ SRwell(S) =σ 1u1v⊤ 1 − 1 p S. (4) [Monotonic Decrease] If σmax S is unique, update S with ∇SRwell(S) such that S′ =S−η 1∇SRwell(S) for 0< η1 < κ(S)−1 (1− 1 p )κ(S)+ 1 p , thenκ(S ′)< κ(S). B. Proof of Propositions...

work page 2024

[6] [16]

∂S 1 2 σmin S 2 =∂ S(R′ ill,1 ◦σ)(S) = n UDiag(µ)V ⊤ µ∈∂ σR′ ill,1(σ) o Given that R′ ill,1 is differentiable and apparently also absolutely symmetric with respect to σ, by Lemma B.2, we know 1 2 (σmin S )2 is also differentiable and ∇ 1 2 σmin S 2 =UDiag(∇ σR′ ill,1(σ))V ⊤ =UDiag(σ min S ep)V ⊤ =σ min S ukv⊤ k . 16 Model Immunization from a Condition Num...

work page 2008

[7] [224]

The feature embedding sizes for ResNet18 and ViT are 512 and 768, respectively

We also create the dataset with the built-in function create dataset from Wightman (2019). The feature embedding sizes for ResNet18 and ViT are 512 and 768, respectively. To facilitate balanced training when dataset sizes differ, we implement aCombinedLoader, which pairs batches from two data loaders. The longer dataset dictates training duration, while t...

work page 2019

[8] [1952]

F., and Liu, L

Huang, T., Hu, S., Ilhan, F., Tekin, S. F., and Liu, L. Harm- ful fine-tuning attacks and defenses for large language models: A survey.arXiv preprint arXiv:2409.18169,

arXiv