REVIEW 4 major objections 5 minor 40 references
SCOPE: Entanglement Frontier Escape for Source-Free Class Unlearning
T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Source-free class unlearning has a provable entanglement frontier: every fixed projection pays at least the retain-readout energy along the forget-discriminant subspace, and input-conditional gating escapes it.
desk verdict A clean geometric lower bound for fixed-projection unlearning plus a gated escape whose practical bite depends on an unproven separation property of the frozen head. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is the principal-angle decomposition of the entanglement between $U_F$ and $S_R$, where $U_F$ spans the class-mean differences of the forget classes and $S_R$ spans the retained rows of the frozen linear head. Theorem 1 prices each principal direction by the retained readout energy $\rho_i=\|W_R u_i\|_2^2$ and shows the total $\sum_i \cos^2\theta_i\rho_i$ is the exact floor. The escape is carried by the gate $g_\tau(z)=\sigma(\tau(\max_{c\in F}w_c^\top z-\max_{c\in R}w_c^\top z))$, a head-derived function that decides per input how much of the erased subspace to subtract.
What would settle it
On a model satisfying the Gaussian shared-covariance assumptions, search a broad family of fixed linear projections that drive forget-class linear-probe accuracy to chance; if any projection's retained-logit perturbation falls strictly below $\|P_{U_F}W_R^T\|_F^2$, Theorem 1 is false. On the empirical side, a trained network whose head scores place a large false-positive mass on retain inputs would let the gated map's retain cost exceed the frontier floor, showing the escape is not universal.
Extended reading notes
Core claim
The paper's central claim is that the retain cost of deleting classes from a representation is governed by the principal-angle geometry between the forget-discriminant subspace $U_F$ and the retain-readout subspace $S_R$. Under Assumption 1 (Gaussian class-conditional features with shared covariance) and Assumption 2 (isotropic retain second moment), Theorem 1 states the minimum retain cost among all global deleting linear maps is exactly $\|P_{U_F}W_R^T\|_F^2 = \sum_i \cos^2\theta_i\,\rho_i$, attained by the minimal eraser that removes $U_F$ and can be padded only with directions orthogonal to $S_R$. Consequently every fixed-projection unlearner, including the leading source-free erasers, pays at least this floor whenever the two subspaces are not orthogonal. Proposition 1 then shows the floor is not fundamental to deletion itself: a map $z\mapsto z-g(z)P_Qz$ with an input-dependent gate can erase at zero retain cost when the gate fires only on forget inputs, and its retain cost is bounded by the gate's false-positive mass. SCOPE realizes this with a single logistic gate read from the frozen head's forget-minus-retain margin, builds the erased basis in closed form from forget features and the head's rows, and is evaluated with linear, converged, nonlinear, and sparse-dictionary re-extraction audits.
Load-bearing premise
The load-bearing premise is that class-conditional features are Gaussian with a shared covariance, which makes erasing the forget-discriminant subspace exactly equivalent to deleting the classes, and that the frozen head's weight scores separate forget from retain inputs well enough for the gate to fire chiefly on forget inputs.
Editorial extensions
If this is right
- Any projection-based unlearner that uses a single global map, such as erasing the forget subspace or nulling a class-weight direction, inherits the retain-cost floor of Theorem 1, a floor set only by the geometry of the original model.
- A class can be erased at zero retention cost if and only if the forget-discriminant subspace is orthogonal to the retain-readout subspace; otherwise the cheapest fixed eraser is the minimal forget-discriminant projector.
- Input-dependent gating converts retain cost into a function of the gate's false-positive mass, so the quality of the frozen head as a forget-versus-retain classifier becomes the controlling factor in the trade-off.
- SCOPE is closed form, needs no retain data or gradient training, and in the reported benchmarks costs orders of magnitude less than retraining while keeping forget classes unrecoverable under linear and stronger probes.
- At $K=1$ the within-forget frontier degenerates because $U_F=\{0\}$, so single-class forgetting must be judged by full-class re-extraction rather than by the within-forget criterion.
Reading between the lines
- This suggests a practical diagnostic: before running any projection eraser, compute $\|P_{U_F}W_R^T\|_F^2$ on the original model; it forecasts the retention damage that even the best fixed eraser must incur, which is cheaper than running the unlearning method itself.
- The gate mechanism may carry over to other head-based readouts, such as concept erasure in language or multimodal models, whenever the final layer can be split into forget and retain categories; SCOPE itself does not claim this.
- If real features deviate from the Gaussian model, the theorem shifts from an exact guarantee to a calibration tool; the paper reports close agreement under distribution shift, but transfer to arbitrary architectures is not proven.
- The large gap between the minimal eraser and random matched-rank subspaces suggests that new erasers should be evaluated against the geometric floor rather than only against retraining, because retraining itself leaves forget classes linearly recoverable in several reported settings.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies source-free class unlearning at the representation level, where deletion is judged by whether the forgotten classes remain linearly decodable from the edited features. It proves an "entanglement frontier" (Theorem 1): under Gaussian shared-covariance features and an isotropic retain second moment, every input-independent linear deletion map pays retain cost at least ||P_UF W_R^T||_F^2, and the minimum is attained by erasing the forget-discriminant subspace U_F alone. The authors then observe that an input-conditional gate can escape this frontier, and Proposition 1 gives retain-cost, deletion, and leakage bounds in terms of the gate's false-positive mass and missed-forget mass. They instantiate this idea in SCOPE, a closed-form method whose gate is read from the frozen head's weight scores and whose erased basis combines the empirical forget-discriminant subspace with retain-cost-ranked residual directions. The paper reports extensive experiments on CIFAR-100, Tiny-ImageNet, VGGFace2-100/200, and VCTK with ResNet-18 and Swin-T, claiming that the frontier predicts measured retain cost and that SCOPE leads the source-free erasers under multiple re-extraction audits.
Significance. If the theoretical statements are fully correct, Theorem 1 is a clean, parameter-free lower bound that unifies a broad class of fixed-projection unlearning methods, and Proposition 1 provides a useful template for why input-conditional erasure can be fundamentally less destructive than global projection. The empirical contribution is substantial: five benchmarks, two backbones, two modalities, multiple forget-set sizes, and a re-extraction ladder from linear probes to nonlinear and sparse-dictionary audits, with ablations isolating the gate and the basis. The paper also makes a genuine prediction, namely that retain cost of deleting projections is governed by the cos^2-weighted retain-readout energy, and the reported R^2 values support that prediction in the tested regimes. The main value is conditional, however, because the formal escape result depends on the gate's false-positive and miss rates being small, a property that is not established for trained neural features under the paper's distributional assumptions.
major comments (4)
- [Sections 3 and 4; Appendix] The proofs of Theorem 1, Corollary 1, Proposition 1, the gate convergence statements, the exact notation, the general-Sigma_R derivation, and all experimental configuration details are deferred to an appendix, but the submitted full text contains no appendix. As a result, the central claims cannot be fully verified from the manuscript. A revision must include the appendix or inline proofs.
- [Section 4, Eq. (6); Proposition 1] Proposition 1 gives a formal deletion certificate for the hard gate h, namely BAF(psi o M_h) <= 1/K + epsilon, but for the soft gate it gives only the leakage bound Leak(M_g) <= sqrt(beta kappa_F). The deployed SCOPE uses a finite-tau soft gate, and the leakage bound does not by itself imply that a linear probe cannot recover the forget classes. The text states that the deletion certificate holds under U_F subset Q and that KR audits finite-tau behavior empirically, but the formal guarantee for the actually deployed map is not established. The authors should either prove a finite-tau deletion bound or explicitly restrict the formal deletion claim to the tau-to-infinity limit and treat finite-tau deletion as empirical.
- [Proposition 1 and Section 5.3] The escape from the frontier is quantitative only through the false-positive mass alpha and the missed-forget mass epsilon (and their soft counterparts). Proposition 1 does not prove that trained neural features make the frozen head's weight scores separate forget from retain; it only states what happens conditionally on alpha and epsilon. Assumption 1, the Gaussian shared-covariance model, does not by itself imply small alpha and epsilon. The empirical Section 5.3 shows small soft retain mass on the tested benchmarks, but the paper's abstract and introduction claim a general escape. The authors should either supply conditions under which alpha and epsilon decay, or consistently frame the escape as an empirical phenomenon observed in the tested regimes.
- [Definition 2 and Section 5.2] The theorem's deletion criterion, Definition 2, is a within-forget criterion: a linear probe must fail to separate the forget classes from one another. The paper's primary empirical deletion standard, however, is full-class KR re-extraction on the unlearned features. Since any transform that passes full-class re-extraction also annihilates U_F, Theorem 1 remains a valid lower bound for full-class deletion, but the equality statement "erasing U_F alone attains the floor" applies only to the within-forget criterion. The paper should make this distinction explicit when claiming that the frontier limits the whole class of source-free erasers, since the deployed audit is stronger than the formalized deletion notion.
minor comments (5)
- [Abstract and Section 1] The text contains numerous line-break artifacts, such as "forget-discriminantsubspace", "andith", and "inputand"; these should be fixed in the final version.
- [Figure 1 caption] The caption writes cos2 for cos^2; please use consistent mathematical notation.
- [Section 3, after Eq. (1)] The sentence "Section 5.5 measures this deployed cost against explicit whitening" appears to refer to content that is not in the listed Section 5.5 ablation; either the cross-reference is wrong or the relevant discussion is only in the missing appendix.
- [Section 4, Eq. (4)] The definition of omega(v) takes values in [0,+inf], but the subsequent ranking by descending omega is not fully defined when the same value occurs infinitely often; the text mentions a deterministic tie-break, but the details should be explicit in the main text rather than only in the appendix.
- [Section 5.4] The phrase "the deployed panel saturates" is informal; please specify which metric saturates and why, or use a more precise description.
Circularity Check
No significant circularity: the entanglement bound is a parameter-free projection inequality and the gated escape is a conditional bound, not a fitted or self-citational result.
full rationale
The paper's central derivation is self-contained. Theorem 1 is a parameter-free optimization over deleting projections: under Assumptions 1 and 2, the deletion condition is UF⊆Q and the retain cost is ||P_Q W_R^T||_F^2, so the minimum over deleting projections is exactly ||P_UF W_R^T||_F^2 by orthogonal projection. No parameter is fitted from the measured retain cost. Proposition 1 states upper bounds RC(M_g) ≤ √α κ_R and BAF ≤ 1/K + ε̄ in terms of the gate's false-positive and miss masses; these are Cauchy-Schwarz and union-type inequalities with constants defined from the data, not calibrated to the measured outcomes. The claimed escape is conditional on the head separating forget from retain, which is asserted as a property to be validated empirically, not derived from the theorem; that is a robustness or correctness limitation, not a circular reduction. The empirical R² for 'frontier predicts retain cost' checks whether the identity RC(P)=||P_Q W_R^T||_F^2 holds in deployed feature coordinates, thereby testing Assumption 2, rather than using the measured cost to construct the theorem. No load-bearing self-citation, imported uniqueness theorem, or ansatz smuggled in via citation appears. The practical gap between the formal Gaussian model and real neural features is a validity concern, not circularity. Score 0.
Assumptions & free parameters
free parameters (3)
- q_sched(K)
- r_pool
- tau
assumptions (3)
- domain assumption Assumption 1: class-conditional Gaussian features with shared covariance Sigma.
- domain assumption Assumption 2: retrain second moment is isotropic (Sigma_R = I).
- domain assumption Population containment U_F subset Q and gate separation between forget and retain.
Cite this review
Pith. "Pith review of SCOPE: Entanglement Frontier Escape for Source-Free Class Unlearning." pith.science (2026). https://pith.science/paper/TKKL5QBH
@misc{pith2026260802058,
author = {Pith},
title = {Pith review of: SCOPE: Entanglement Frontier Escape for Source-Free Class Unlearning},
year = {2026},
howpublished = {\url{https://pith.science/paper/TKKL5QBH}},
note = {Machine review of arXiv:2608.02058}
}
read the original abstract
Source-free class unlearning erases whole classes using only the forget data, judged at the representation level, where features can leak a class the head no longer predicts. Existing feature-space erasers answer with one fixed projection, yet forget and retain classes share a representation, so deleting one disturbs the other where they overlap. We prove this tension is a frontier. Every fixed projection that deletes pays a retain cost of at least the retain-readout energy along the forget-discriminant subspace, and erasing that subspace alone attains the floor. The leading source-free erasers all instantiate the form it binds, so the frontier limits the whole class. Conditioning the erasure on the input escapes it. Spectral Conditional Projective Erasure (SCOPE) does so with a single gate, suppressing the forget subspace chiefly on inputs its frozen head's weight scores read as a forget class. It is closed form, needs no retain data or gradient training, and costs orders of magnitude less than retraining. Across five object, face, and speaker benchmarks spanning two modalities and both convolutional and transformer backbones, the frontier predicts the measured retain cost. SCOPE leads the source-free erasers on every benchmark and forget-set size, and at the hardest setting it tops every unlearner, trained methods included.
Figures
Reference graph
Works this paper leans on
-
[1]
Le, Anjie and Peng, Can and Liu, Yuyuan and Noble, J. Alison , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =. 2026 , pages =
work page 2026
-
[2]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =
Lee, Tae-Young and Park, Sundong and Jeon, Minwoo and Hwang, Hyoseok and Park, Gyeong-Moon , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =. 2025 , pages =
work page 2025
-
[3]
, author=
Visualizing data using t-SNE. , author=. Journal of machine learning research , volume=
-
[4]
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , month =
Schroff, Florian and Kalenichenko, Dmitry and Philbin, James , title =. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , month =
-
[5]
2015 IEEE symposium on security and privacy , pages=
Towards making systems forget with machine unlearning , author=. 2015 IEEE symposium on security and privacy , pages=. 2015 , organization=
2015
-
[6]
2021 IEEE symposium on security and privacy (SP) , pages=
Machine unlearning , author=. 2021 IEEE symposium on security and privacy (SP) , pages=. 2021 , organization=
work page 2021
-
[7]
International conference on artificial intelligence and statistics , pages=
Approximate data deletion from machine learning models , author=. International conference on artificial intelligence and statistics , pages=. 2021 , organization=
2021
-
[8]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =
Zhou, Yu and Zheng, Dian and Mo, Qijie and Lu, Renjie and Lin, Kun-Yu and Zheng, Wei-Shi , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =. 2025 , pages =
work page 2025
Show all 40 references
-
[9]
Proceedings of the AAAI conference on artificial intelligence , volume=
Amnesiac machine learning , author=. Proceedings of the AAAI conference on artificial intelligence , volume=
-
[10]
Engineering Applications of Artificial Intelligence , volume=
Are we truly forgetting? a critical re-examination of machine unlearning evaluation protocols , author=. Engineering Applications of Artificial Intelligence , volume=. 2026 , publisher=
2026
-
[11]
Algorithmic Learning Theory , pages=
Descent-to-delete: Gradient-based methods for machine unlearning , author=. Algorithmic Learning Theory , pages=. 2021 , organization=
2021
-
[12]
Proceedings of the 37th International Conference on Machine Learning , pages=
Certified data removal from machine learning models , author=. Proceedings of the 37th International Conference on Machine Learning , pages=
-
[13]
ACM Transactions on Intelligent Systems and Technology , volume=
A survey of machine unlearning , author=. ACM Transactions on Intelligent Systems and Technology , volume=. 2025 , publisher=
2025
-
[14]
International Conference on Machine Learning , pages=
When to Forget? Complexity Trade-offs in Machine Unlearning , author=. International Conference on Machine Learning , pages=. 2025 , organization=
2025
-
[15]
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
Eternal sunshine of the spotless net: Selective forgetting in deep networks , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
-
[16]
International Conference on Learning Representations , volume=
Salun: Empowering machine unlearning via gradient-based weight saliency in both image classification and generation , author=. International Conference on Learning Representations , volume=
-
[17]
Proceedings of the AAAI Conference on Artificial Intelligence , volume=
Can bad teaching induce forgetting? unlearning in deep networks using an incompetent teacher , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=
-
[18]
Advances in neural information processing systems , volume=
Towards unbounded machine unlearning , author=. Advances in neural information processing systems , volume=
-
[19]
Proceedings of the AAAI conference on artificial intelligence , volume=
Fast machine unlearning without retraining through selective synaptic dampening , author=. Proceedings of the AAAI conference on artificial intelligence , volume=
-
[20]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
Boundary unlearning: Rapid forgetting of deep networks via shifting the decision boundary , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
-
[21]
The Second Tiny Papers Track at ICLR , year=
Loss-Free Machine Unlearning , author=. The Second Tiny Papers Track at ICLR , year=
-
[22]
arXiv preprint arXiv:2507.07754 , year=
OPC: One-Point-Contraction Unlearning Toward Deep Feature Forgetting , author=. arXiv preprint arXiv:2507.07754 , year=
-
[23]
Proceedings of the ACM Web Conference 2026 , pages=
Suppression or Deletion: A Restoration-Based Representation-Level Analysis of Machine Unlearning , author=. Proceedings of the ACM Web Conference 2026 , pages=
2026
-
[24]
Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , month =
Xiao, Yaxin and Ye, Qingqing and Hu, Li and Zheng, Huadi and Hu, Haibo and Liang, Zi and Li, Haoyang and Jiao, Yijie , title =. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , month =. 2025 , pages =
2025
-
[25]
Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection
Ravfogel, Shauli and Elazar, Yanai and Gonen, Hila and Twiton, Michael and Goldberg, Yoav. Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020
2020
-
[26]
Transactions on Machine Learning Research , issn=
Deep Unlearning: Fast and Efficient Gradient-free Class Forgetting , author=. Transactions on Machine Learning Research , issn=. 2024 , url=
2024
-
[27]
Adversarial Concept Erasure in Kernel Space
Ravfogel, Shauli and Vargas, Francisco and Goldberg, Yoav and Cotterell, Ryan. Adversarial Concept Erasure in Kernel Space. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022. doi:10.18653/v1/2022.emnlp-main.405
2022 doi
-
[28]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
Closed-form concept erasure via double projections , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
-
[29]
Zhijie Deng and Chris Yuhao Liu and Zirui Pang and Xinlei He and Lei Feng and Qi Xuan and Zhaowei Zhu and Jiaheng Wei , booktitle=
-
[30]
International conference on learning representations , volume=
Programming refusal with conditional activation steering , author=. International conference on learning representations , volume=
-
[31]
Chenlu Ding and Jiancan Wu and Leheng Sheng and Fan Zhang and Yancheng Yuan and Xiang Wang and Xiangnan He , year=
-
[32]
Kang Eun Jeon and Yunsung Kang and Do Yeong Kang and Tae-Young Lee and Gyeong-Moon Park and Jong Hwan Ko , year=. Mo
-
[33]
2009 , publisher=
Learning multiple layers of features from tiny images , author=. 2009 , publisher=
2009
-
[34]
CS 231N , volume=
Tiny imagenet visual recognition challenge , author=. CS 231N , volume=
-
[35]
and Zisserman, Andrew , booktitle =
Cao, Qiong and Shen, Li and Xie, Weidi and Parkhi, Omkar M. and Zisserman, Andrew , booktitle =. 2018 , volume =. doi:10.1109/FG.2018.00020 , url =
2018
-
[36]
2019 , url=
CSTR VCTK Corpus: English Multi-speaker Corpus for CSTR Voice Cloning Toolkit (version 0.92) , author=. 2019 , url=
2019
-
[37]
Proceedings of the IEEE conference on computer vision and pattern recognition , pages=
Deep residual learning for image recognition , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=
-
[38]
2021 IEEE/CVF International Conference on Computer Vision (ICCV) , year=
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows , author=. 2021 IEEE/CVF International Conference on Computer Vision (ICCV) , year=
2021
-
[39]
30th USENIX security symposium (USENIX security 21) , pages=
Systematic evaluation of privacy risks of machine learning models , author=. 30th USENIX security symposium (USENIX security 21) , pages=
-
[40]
Mathematical programming , volume=
On the limited memory BFGS method for large scale optimization , author=. Mathematical programming , volume=. 1989 , publisher=
1989
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.