Pith. sign in

REVIEW 5 minor 24 references

In the class-unlearning setting studied, the identity of saliency-selected weights does not explain representation-level forgetting: saliency masks, random masks of equal sparsity, and unconstrained random-label updates leave the forgotten

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · deepseek-v4-flash

2026-08-01 07:40 UTC pith:Z56444EO

load-bearing objection A careful matched-compute ablation showing SalUn's mask identity does not drive representation-level forgetting; the central claim holds, with a real but non-fatal near-ceiling measurement caveat.

arxiv 2607.21353 v1 pith:Z56444EO submitted 2026-07-23 cs.LG

Gradient Concentration, Not Weight Saliency, Explains Representation-Level Class Unlearning

classification cs.LG
keywords machine unlearningclass unlearningweight saliencyrepresentation-level forgettinggradient concentrationlinear probingprototype recoveryequivalence testing
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that, in a standard class-unlearning setup (CIFAR-10/100, ResNet-18, random-label objective), the specific choice of weights to update does not matter for whether the forgotten class remains recoverable from the model's internal representations. Saliency-based masks, random masks of equal sparsity, and unconstrained updates all produce statistically equivalent recoverability, while the forgotten class stays highly recoverable relative to exact retraining. The explanation offered is that forget gradients are already concentrated in the last layers (about 92% of gradient energy on CIFAR-10) before any mask is applied, so every masking strategy acts in the same late representational subspace. Saliency masks also show almost no class specificity. If this is right, better weight-selection rules cannot deliver representation-level unlearning; objectives need to act on latent representations directly.

Core claim

In the class-unlearning setting studied, representation-level forgetting is governed by gradient concentration and representation geometry, not by which weights a saliency mask selects. Saliency masks, random masks of equal sparsity, and unconstrained random-label updates are statistically equivalent (two one-sided tests, ±5 percentage points) on prototype recovery and linear-probe recall; on CIFAR-10 the forgotten class stays 93.2–93.9% recoverable versus 41.5% after exact retraining. Why: forget-gradient energy is already concentrated in late layers (≈92% on CIFAR-10) before masking, so every mask inherits the same subspace; saliency masks are nearly class-agnostic, and a diagonal-Fisher m

What carries the argument

The central object is the gradient-saliency mask: the top 50% of weights by absolute forget-gradient magnitude, applied as a multiplicative gate on a random-label unlearning objective. The paper contrasts it with a random mask of equal sparsity, an unmasked update, a diagonal-Fisher mask, and a causal early-layer mask. The load-bearing measurements are squared-gradient energy by network region, a class-specificity index built from Jaccard overlap between masks, and equivalence testing via the two one-sided tests procedure at a pre-specified margin of ±5 percentage points.

Load-bearing premise

The load-bearing premise is that representation-level recoverability is adequately captured by linear-probe recall and few-shot prototype recovery on frozen features; if the forgotten class leaves traces these probes miss, the conclusion that mask identity does not matter could be a measurement artifact.

What would settle it

Replicate the matched-compute ablation and attack the frozen features with a nonlinear probe or a nearest-subspace classifier; if saliency-masked models show materially lower forgotten-class recall than random-masked models, the equivalence claim fails. Alternatively, repeat on a vision transformer where forget-gradient energy splits across layers; if saliency masks then beat random masks, the mechanism is architecture-dependent.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Saliency-based weight selection cannot be credited for representation-level forgetting in this setting; equivalent outcomes follow from random or unconstrained updates.
  • Output-level forgetting (0% forget accuracy) coexists with representation-level recoverability, so deletion claims need representation-level auditing.
  • The diagonal-Fisher and saliency masks nearly coincide, suggesting the class-agnosticism is a property of gradient-derived selection rules generally, not of one algorithm.
  • A mask that avoids the late subspace still drives forget accuracy to zero, while late-subspace updates only move geometry-sensitive metrics toward retrain-like values without erasing the class.
  • For design, the paper says effective representation-level unlearning will require objectives that act directly on latent representations, not more selective masks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Testable extension: run the same matched-compute protocol on vision transformers or ImageNet-scale models; if forget-gradient energy is not late-concentrated there, the equivalence between saliency and random masks may break, which would delimit the mechanism.
  • If late-layer gradient concentration is generic in converged deep classifiers, then any update derived from a logit-level loss will leave class structure intact; the conclusion would then extend well beyond SalUn to a wide family of parameter-update unlearning methods.
  • Because the saliency masks carry almost no forget-specific signal, a practical corollary is that privacy claims based on which weights a method updates should be audited with feature-space attacks rather than trusted from the mask's construction.
  • A stronger check of the paper's measurement dependence would use nonlinear probes or nearest-subspace attacks on the same frozen features; if those reveal more erasure under saliency than under random masks, the equivalence claim would need qualifying.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

0 major / 5 minor

Summary. The paper reports a matched-compute ablation of SalUn-style gradient-saliency masking for single-class unlearning on CIFAR-10/CIFAR-100 with ResNet-18. Keeping the random-label objective, optimization schedule, sparsity, and compute fixed, the authors compare three conditions that differ only in the weight mask: SalUn's gradient-saliency mask, a random mask of equal sparsity, and an unconstrained update. They find that all three are statistically equivalent on representation-level recoverability metrics (prototype recovery, linear-probe recall), while Gold retraining is substantially lower on CIFAR-10. They attribute this to pre-masking concentration of forget-gradient energy in late layers (≈92% of squared gradient energy on CIFAR-10), to the low class specificity of saliency masks (specificity index 0.09–0.11), and to the fact that saliency masks overlap an independent random mask at chance level. A causal placement intervention freezing layer 4 and the head shows that output-level forgetting does not require the late subspace, while prototype geometry is affected only by late-subspace updates; linear separability of the forgotten class survives under all placements. The paper concludes that, in this setting, mask identity does not explain representation-level forgetting and that representation-level objectives matter more than weight-selection strategy.

Significance. If accepted, this is a valuable controlled mechanistic study: it isolates the saliency-masking component from the unlearning objective, which most prior work treats as inseparable. The paper is unusually careful for an empirical ablation: matched compute, a pre-specified TOST margin calibrated to seed noise, full per-class/per-seed tables in the appendix, archived code and data, honest treatment of the underpowered CIFAR-100 prototype test, and a causal placement intervention that goes beyond correlational evidence. The main claim is negative but constructive: gradient concentration and representation geometry, not weight saliency, explain representation-level recoverability in this setting. The result strengthens the case for representation-level unlearning objectives and provides a concrete caution for interpreting saliency-mask-based privacy guarantees. The reproducibility artifacts and the completeness of the appendix are clear strengths.

minor comments (5)
  1. [§3.7, §A.4, Table 2] The text states that paired t-tests are reported alongside TOST, but Table 2 reports only differences and TOST p-values. Please either add the paired t-test statistics (or confidence intervals) to the table, or revise the wording to avoid promising unreported results. This matters because the CIFAR-100 prototype comparisons are described as 'n.s.' without showing the test output.
  2. [§4.1, Table 1] On CIFAR-10, prototype recovery for the base model is ≈94.3% and the three unlearning conditions sit at 93.2–93.9%, so the equivalence region is at the ceiling of the metric. The Gold row (41.5%) and the CIFAR-100 early-mask result (SalUn 25.5% vs. early 59.0%) show that the metric has dynamic range, and the chance-level mask-overlap analysis in §4.7 and Table C.5(b) independently supports the main conclusion. Still, the paper would be strengthened by an explicit sensitivity check in the near-ceiling regime, e.g., a small partial-retention interpolation between the base model and Gold, or by reporting linear-probe recall for the base model on both datasets. Without this, a reader may worry that the TOST equivalence partly reflects metric compression rather than true equivalence of retained information.
  3. [§3.7, §A.4] The phrase 'pre-registered margin' is used, but no external registration or protocol document is cited. If the margin was only pre-specified internally, replace 'pre-registered' with 'pre-specified' to avoid implying formal registration.
  4. [§4.8, Table 4] The forced early-mask variant is explicitly marked as supporting evidence because it sacrifices 3–6pp retain accuracy on CIFAR-100 and 2/30 runs fail the forget criterion. Table 4 does not show retain accuracy; adding a utility column (or stating the utility gate in the table) would make the caveat immediately interpretable.
  5. [Appendix C, Tables C.2/C.4] The per-seed tables are very useful, but the captions are terse. For readers who use only the printed appendix, it would help to spell out the column abbreviations (Fisher, Early, E-3×, Late-%, SpecificityJ) directly in the Table C.4 caption rather than relying on the Table C.2 note.

Circularity Check

0 steps flagged

No significant circularity: the matched-compute ablation measures the mechanism directly and makes no fitted-as-prediction claim.

full rationale

The paper's central claim is an empirical null result from a controlled ablation, not a derivation that reduces to its own inputs. The three main conditions are constructed independently in §3.4: the saliency mask is defined by Eq. 4 from the forget-loss gradient, the random mask is a uniform binary mask of equal cardinality, and RL-full is the identity mask, with the objective, schedule, and compute budget held fixed. The outcome metrics (prototype recovery, linear-probe recall, CKA, silhouette) are evaluated on held-out features and are not used to fit or select any parameter of the unlearning update. The proposed mechanism, gradient concentration, is measured directly on raw forget-loss gradients before masking (Figure 5), and the specificity index is a descriptive transformation of measured Jaccard overlaps (Table 3), not a fitted quantity used to explain the recovery outcome. The causal placement intervention in §4.8 actively tests the mechanism and even reports a falsifying branch (the late subspace is not necessary for output-level forgetting), showing that the mechanism claim is not insulated from disconfirmation. The only self-citations are the paper's own code/data repository (Habbati et al. 2026), which is not load-bearing for the scientific claim. The TOST margin is pre-specified and calibrated to the inter-seed noise floor; whether that margin or the chosen metrics are sufficiently sensitive is a validity/correctness concern, not a circularity between inputs and outputs. No step in the paper exhibits the required reduction of a claimed result to its definition, a fitted parameter renamed as a prediction, or a load-bearing self-citation chain.

Axiom & Free-Parameter Ledger

1 free parameters · 4 axioms · 0 invented entities

No invented entities or fitted theory parameters. The only analysis-configuration number the central claim depends on is the equivalence margin δ, which is hand-set rather than fitted. The paper's mechanism (gradient concentration, mask overlap) is measured directly from the network, not assumed.

free parameters (1)
  • Equivalence margin δ = ±5 percentage points
    Hand-set analysis threshold that defines the paper's central equivalence claim. Chosen a priori to match the inter-seed SD of prototype recovery (≈3–4pp on CIFAR-10). The TOST equivalence conclusions in Tables 1–2 and Figures 1 and 3 are only valid at this margin; a different margin could alter the conclusions.
axioms (4)
  • domain assumption Representation-level recoverability is measured by linear-probe recall and few-shot prototype recovery on frozen features; these metrics operationalize 'information retained about the forgotten class.'
    Invoked throughout §3.6 and used for all central claims. If these probes miss class information not expressible as linear separability or centroid proximity, the equivalence conclusion could be a measurement artifact.
  • domain assumption The random-label objective of Eq. (2) is a representative unlearning objective for studying saliency-based class unlearning.
    The ablation fixes this objective across conditions (§3.2); the paper's conclusions are bounded to this objective family (stated in §5.3).
  • domain assumption ResNet-18 on CIFAR-10/CIFAR-100 is a representative setting for studying representation-level class unlearning.
    The authors adopt the benchmark used by related auditing work (§3.5) and explicitly list architectures and datasets not covered in §5.3.
  • domain assumption SalUn's released hyperparameters (sparsity 0.5, learning rate 0.013, 10 repair epochs) define a valid operating point for the saliency mechanism.
    The paper takes these values unchanged from the released implementation (Appendix A.2) and sweeps sparsity only partly (three classes for outcomes; full census for raw mask overlap).

pith-pipeline@v1.3.0-alltime-deepseek · 28312 in / 15827 out tokens · 155107 ms · 2026-08-01T07:40:42.334548+00:00 · methodology

0 comments
read the original abstract

Machine unlearning aims to remove the influence of specific training data while preserving model utility. Many state-of-the-art approaches pursue this goal by restricting the forgetting update to a subset of parameters selected through gradient-based saliency. Although such methods are widely adopted, the actual contribution of saliency-based weight selection to representation-level forgetting remains unclear. In this work, we perform the first controlled ablation of the saliency masking mechanism used by SalUn. Using a matched-compute experimental design on CIFAR-10 and CIFAR-100 with ResNet-18, we compare saliency-based masking against random masks of equal sparsity and unconstrained updates, while keeping the unlearning objective, optimization schedule, and computational budget fixed. Across multiple representation-level evaluations, including linear probing, prototype recovery, and layer-wise CKA, the three configurations exhibit statistically equivalent representation-level recoverability. We find that forget gradients are strongly concentrated in the final network layers (approximately 92% of the squared gradient energy on CIFAR-10) before any mask is applied, causing all masking strategies to operate within the same representational subspace. Furthermore, saliency masks show limited class specificity (specificity index 0.09-0.11), selecting highly overlapping parameter subsets across different forget classes. Our findings suggest that, in the studied setting, representation-level forgetting is primarily governed by gradient concentration and representation geometry rather than by the specific identity of saliency-selected weights. More broadly, the results support a growing body of evidence indicating that effective representation-level unlearning requires objectives that act directly on latent representations rather than on increasingly sophisticated weight-selection strategies.

Figures

Figures reproduced from arXiv: 2607.21353 by Alessio Merlo, Billel Habbati, Luca Verderame, Meriem Guerar.

Figure 1
Figure 1. Figure 1: The saliency mask does not change what survives. Mean forget-class prototype recovery (k=5) across ten classes. On CIFAR-10 all three conditions are equiv￾alent (TOST, [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The matched-compute ablation. Three conditions branch from one base model and differ only in the weight mask; learning rate, epochs, and retain term are identical. Any difference in outcome is therefore attributable to mask selection alone. 3.5. Datasets, model, and reference conditions We evaluate single-class unlearning on CIFAR-10 and CIFAR-100 (Krizhevsky, 2009) using ResNet-18 (He et al., 2016), the b… view at source ↗
Figure 3
Figure 3. Figure 3: Saliency and random masks yield similar class-level recovery. Prototype recovery under the SalUn saliency mask and a random mask of equal sparsity. Each point corresponds to one forget class; the shaded region denotes the ±5 percentage-point equivalence band. On CIFAR-10, nearly all classes fall within the equivalence region. On CIFAR-100, per-class differences are larger but do not show a consistent direc… view at source ↗
Figure 4
Figure 4. Figure 4: All matched conditions reshape the same late representation. Layer￾wise linear CKA between each unlearned model and the original model, averaged over ten classes. Early and intermediate layers remain close to the original representation, while the final residual block changes substantially. The SalUn, random-mask, and RL-full profiles are closely aligned, indicating that the three updates affect similar re… view at source ↗
Figure 5
Figure 5. Figure 5: Forget gradients are concentrated in late network regions before mask￾ing. Share of the forget-loss gradient’s squared ℓ2-norm by network region, measured before any unlearning step. Layer 3, layer 4, and the classifier head account for most of the gradient energy on both datasets. As a result, different masks are applied to gradients that already concentrate their update direction in the same late subspac… view at source ↗
Figure 6
Figure 6. Figure 6: Feature-space visualization of class survival. t-SNE of penultimate fea￾tures for CIFAR-10 class 3 (cat) against all retained classes. Under SalUn, random mask, and RL-full, the forget-class samples remain compact and separated from most retained samples. Under Gold, the cluster is less coherent and partially dispersed among neighbor￾ing retained classes. vs. retain-set mask vs. other-class mask vs. random… view at source ↗
Figure 7
Figure 7. Figure 7: Saliency masks show limited class specificity. Jaccard overlap between the forget-class SalUn mask and three comparator masks over class-seed combinations. Other-class and retain-set masks exhibit very similar overlap with the forget-class mask, suggesting that the selected parameters are not strongly specific to the forgotten class. 26 [PITH_FULL_IMAGE:figures/full_fig_p027_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 1 canonical work pages

  1. [2]

    the mask does nothing

    with SGD (momentum0.9, learning rate0.013, weight decay 5×10 −4) for ten repair epochs. SalUn applies the saliency maskmS (top-50% weights by forget-gradient magnitude);Random maskappliesm R of equal ℓ0;RL-fullsetsm=1. To keep the mask faithful, masked-out coordinates are held at their base valueandtheir momentum buffer is zeroed each step, preventing mom...

  2. [4]

    ArXiv:2604.08271

    An illusion of unlearning? assessing machine unlearning through internal representations, in: Inter- national Conference on Artificial Intelligence and Statistics (AISTATS). ArXiv:2604.08271. Ginart, A., Guan, M.Y., Valiant, G., Zou, J.,

  3. [6]

    arXiv preprint arXiv:2402.14015

    Corrective machine unlearning. arXiv preprint arXiv:2402.14015 . Golatkar, A., Achille, A., Soatto, S.,

  4. [7]

    arXiv preprint arXiv:2405.19211

    Gone but not for- gotten: Improved benchmarks for machine unlearning. arXiv preprint arXiv:2405.19211 . Guo, C., Goldstein, T., Hannun, A., van der Maaten, L.,

  5. [9]

    gradient concentration, not weight saliency, explains representation-level class unlearning

    Salun-unlearning- studies: code for “gradient concentration, not weight saliency, explains representation-level class unlearning”. URL:https://doi.org/10.5281/ zenodo.21507241, doi:10.5281/zenodo.21507241. [Computer software]. Zenodo. Version 1.0.0. 35 Han, S., Pool, J., Tran, J., Dally, W.J.,

  6. [10]

    ArXiv:2604.15166

    Class unlearning via depth- aware removal of forget-specific directions, in: CVPR Workshop on Ma- chine Unlearning for Vision (MUV). ArXiv:2604.15166. Hayes, J., Shumailov, I., Triantafillou, E., Khalifa, A., Papernot, N.,

  7. [11]

    arXiv preprint arXiv:2403.01218

    Inexact unlearning needs more careful evaluations to avoid a false sense of privacy. arXiv preprint arXiv:2403.01218 . He, K., Zhang, X., Ren, S., Sun, J.,

  8. [12]

    ArXiv:2507.07754

    OPC: One-point-contraction un- learningtowarddeepfeatureforgetting, in: ICCVWorkshoponUnlearning and Model Editing (U&ME). ArXiv:2507.07754. Kim, Y., Cha, S., Kim, D.,

  9. [13]

    arXiv preprint arXiv:2503.06991

    Are we truly forgetting? a critical re- examination of machine unlearning evaluation protocols. arXiv preprint arXiv:2503.06991 . 36 Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., Hassabis, D., Clopath, C., Kumaran, D., Hadsell, R.,

  10. [14]

    arXiv preprint arXiv:2511.19339 ArXiv:2511.19339

    POUR: A provably optimal method for unlearning representations via neural collapse. arXiv preprint arXiv:2511.19339 ArXiv:2511.19339. Lee, J., Kim, Y., Kim, D.,

  11. [15]

    arXiv preprint arXiv:2602.05375

    Erase at the core: Representation unlearning for machine unlearning. arXiv preprint arXiv:2602.05375 . Li, Q., Geng, J., Woisetschläger, H., Chen, Z., Cai, F., Wang, Y., Nakov, P., Jacobsen, H.A., Karray, F.,

  12. [16]

    arXiv preprint arXiv:2503.01854

    A survey of machine unlearning in large language models: Methods, challenges and future directions. arXiv preprint arXiv:2503.01854 . Mallya, A., Lazebnik, S.,

  13. [17]

    arXiv preprint arXiv:2209.02299

    A survey of machine unlearning. arXiv preprint arXiv:2209.02299 . Papyan, V., Han, X.Y., Donoho, D.L.,

  14. [18]

    arXiv preprint arXiv:2406.17216

    Machine unlearning fails to remove data poisoning attacks. arXiv preprint arXiv:2406.17216 . Raghu, M., Gilmer, J., Yosinski, J., Sohl-Dickstein, J.,

  15. [19]

    arXiv preprint arXiv:2601.22028

    From logits to latents: Contrastive representa- tion shaping for LLM unlearning. arXiv preprint arXiv:2601.22028 . 38 Tarun, A.K., Chundawat, V.S., Mandal, M., Kankanhalli, M.,

  16. [20]

    arXiv preprint arXiv:2606.25001

    Erased, but not gone: Output forgetting is not true forgetting. arXiv preprint arXiv:2606.25001 . 39 This appendix has one purpose: to let every quantitative claim in §4.1– §4.7 be checked against the exact configuration and data that produced it, rather than taken on trust. It follows the paper’s own logic rather than a generic template. Appendix A gives...

  17. [22]

    Optimizer SGD, momentum0.9, wd5×10 −4 Learning rate0.013(SalUn default) Repair epochs10 Mask sparsity0.5; swept{0.1,0.3,0.5,0.7} Forget set one class (5000/500imgs, C10 / C100) Evaluation Prototype recoveryk=5, mean over5attack seeds Linear probe logistic reg. on avgpool features Per-layer CKA linear CKA (Kornblith et al., 2019),n=200 Membership inference...

  18. [23]

    Ten-class means at the primary seed: Base0.66, Gold−0.09, SalUn0.62, RL-full0.68, Random0.67, Fisher0.61, Early0.56, Early3×0.53

    Class Seed Base Gold SalUn-RL RL-full Random Mask Fisher Early Early3× airplane 42 0.644−0.1140.541 0.679 0.689 0.651 0.458 0.479 1337 0.644−0.1140.684 0.711 0.738 0.628 0.481 0.535 999 0.644−0.1140.602 0.630 0.610 0.670 0.514 0.445 automobile 42 0.749−0.0780.722 0.782 0.709 0.648 0.756 0.772 1337 0.749−0.0780.734 0.758 0.718 0.740 0.628 0.661 999 0.749−0...

  19. [2010]

    He haspublishedmorethan120scientificpapersininternational conferences and journals

    He is currently a Professor in Computer Engineering and the Director of the Centre for Defense Higher Studies (CASD), Rome, Italy. He haspublishedmorethan120scientificpapersininternational conferences and journals. His research interests include mo- bile security, the Internet of Things, and cyber-physical sys- tems security. Luca Verderamereceived the M....

  20. [2019]

    ArXiv:1907.05012

    Making AI forget you: Data deletion in machine learning, in: Advances in Neural Information Processing Systems (NeurIPS). ArXiv:1907.05012. Goel, S., Prabhu, A., Torr, P., Kumaraguru, P., Sanyal, A.,

  21. [2022]

    1897–1914

    Membership inference attacks from first principles, in: IEEE Symposium on Security and Privacy (S&P), pp. 1897–1914. Chen, M., Gao, W., Liu, G., Peng, K., Wang, C.,

  22. [2024]

    ArXiv:2310.12508

    SalUn: Em- powering machine unlearning via gradient-based weight saliency in both image classification and generation, in: International Conference on Learn- ing Representations (ICLR). ArXiv:2310.12508. Foster, J., Schoepf, S., Brintrup, A.,

  23. [2025]

    arXiv preprint arXiv:2506.01318

    Unlearning’sblindspots: Over-unlearning and prototypical relearning attack. arXiv preprint arXiv:2506.01318 . Habbati, B., Merlo, A., Verderame, L., Guerar, M.,

  24. [2026]

    arXiv preprint arXiv:2601.21564

    Representation unlearning: Forgetting through information compression. arXiv preprint arXiv:2601.21564 . Baumhauer, T., Schöttle, P., Zeppelzauer, M.,