{"id":"6c5aa59c-0e0d-413b-b6bc-b2809344e3f4","arxiv_id":"2608.02058","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A tight lower bound for fixed-projection class unlearning is proven, and an input-conditional gate (SCOPE) escapes the bound while achieving state-of-the-art source-free deletion.","lead":"This paper proves that any unlearning method that applies one fixed projection to all inputs must pay a measurable retention cost, and shows that a simple per-input gate can avoid that cost. The authors introduce SCOPE, a closed-form, training-free unlearning method that beats existing source-free erasers on five benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SCOPE's escape from the frontier is only as strong as the frozen head's forget/retain separation, a property that is assumed in Proposition 1 but never established for real features.","rationale":"The reader's weakest assumption identifies the same load-bearing point: Assumption 1's Gaussian model and the unproven gate separation are the conditions under which the formal results hold. My stress-test sharpens this into a concrete, testable condition: Proposition 1's escape bounds depend explicitly on the gate's false-positive and miss masses, and the paper provides no theoretical or reproducible evidence that these masses are small for general trained models. This is a genuine gap between theory and the empirical claim of escaping the frontier, but it is the kind of gap that can be closed with additional experiments or a characterization of the gate's error rates, so it does not change the reader's conditional verdict. The theorem itself appears algebraically sound under its assumptions; the concern is about the transfer of the escape result to practice, not about an internal inconsistency in the proof.","tokens_in":14906,"tokens_out":19185,"duration_ms":199331,"concrete_test":"Construct forget/retain splits with deliberately high class confusion on an existing benchmark, e.g., CIFAR-100 classes drawn from the same superclass or VGGFace2 identities with high pairwise similarity. Run SCOPE with the published configuration and measure the deployed gate's retain false-positive mass alpha, forget miss rate bar-epsilon, KR retain accuracy, and forget re-extraction accuracy. If alpha or bar-epsilon rises substantially relative to the paper's splits, and the HM_KR^t gain over the global projection shrinks or deletion fails, the escape claim is not robust without the separation property.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central practical claim is that conditioning on the input escapes Theorem 1. Proposition 1 makes the escape quantitative: retain cost is at most sqrt(alpha) kappa_R and within-forget deletion is at most 1/K + bar-epsilon, where alpha is the gate's false-positive mass on retain inputs and bar-epsilon is its miss rate on forget inputs. These bounds only approach the zero-cost corner when the frozen head's weight scores separate forget from retain. No formal argument proves that trained neural features satisfy this separation; Assumption 1 (Gaussian shared-covariance features) does not imply it. The empirical benchmarks may simply be regimes where the head happens to separate well. If forget and retain classes are confusable, alpha and bar-epsilon grow, the retain-cost bound is no longer negligible, and the deletion bound degrades. Additionally, Proposition 1 states a hard-gate deletion bound but only a leakage bound for the soft gate; the deployed finite-tau gate is soft, so the formal deletion certificate applies strictly only in the tau-to-infinity limit. Thus the escape is conditional on an unproven property of the particular trained model, not a guaranteed consequence of the geometry theorem.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies source-free class unlearning at the representation level, where deletion is judged by whether the forgotten classes remain linearly decodable from the edited features. It proves an \"entanglement frontier\" (Theorem 1): under Gaussian shared-covariance features and an isotropic retain second moment, every input-independent linear deletion map pays retain cost at least ||P_UF W_R^T||_F^2, and the minimum is attained by erasing the forget-discriminant subspace U_F alone. The authors then observe that an input-conditional gate can escape this frontier, and Proposition 1 gives retain-cost, deletion, and leakage bounds in terms of the gate's false-positive mass and missed-forget mass. They instantiate this idea in SCOPE, a closed-form method whose gate is read from the frozen head's weight scores and whose erased basis combines the empirical forget-discriminant subspace with retain-cost-ranked residual directions. The paper reports extensive experiments on CIFAR-100, Tiny-ImageNet, VGGFace2-100/200, and VCTK with ResNet-18 and Swin-T, claiming that the frontier predicts measured retain cost and that SCOPE leads the source-free erasers under multiple re-extraction audits.","tokens_in":15141,"tokens_out":6349,"duration_ms":65396,"significance":"If the theoretical statements are fully correct, Theorem 1 is a clean, parameter-free lower bound that unifies a broad class of fixed-projection unlearning methods, and Proposition 1 provides a useful template for why input-conditional erasure can be fundamentally less destructive than global projection. The empirical contribution is substantial: five benchmarks, two backbones, two modalities, multiple forget-set sizes, and a re-extraction ladder from linear probes to nonlinear and sparse-dictionary audits, with ablations isolating the gate and the basis. The paper also makes a genuine prediction, namely that retain cost of deleting projections is governed by the cos^2-weighted retain-readout energy, and the reported R^2 values support that prediction in the tested regimes. The main value is conditional, however, because the formal escape result depends on the gate's false-positive and miss rates being small, a property that is not established for trained neural features under the paper's distributional assumptions.","major_comments":[{"comment":"The proofs of Theorem 1, Corollary 1, Proposition 1, the gate convergence statements, the exact notation, the general-Sigma_R derivation, and all experimental configuration details are deferred to an appendix, but the submitted full text contains no appendix. As a result, the central claims cannot be fully verified from the manuscript. A revision must include the appendix or inline proofs.","section":"Sections 3 and 4; Appendix"},{"comment":"Proposition 1 gives a formal deletion certificate for the hard gate h, namely BAF(psi o M_h) <= 1/K + epsilon, but for the soft gate it gives only the leakage bound Leak(M_g) <= sqrt(beta kappa_F). The deployed SCOPE uses a finite-tau soft gate, and the leakage bound does not by itself imply that a linear probe cannot recover the forget classes. The text states that the deletion certificate holds under U_F subset Q and that KR audits finite-tau behavior empirically, but the formal guarantee for the actually deployed map is not established. The authors should either prove a finite-tau deletion bound or explicitly restrict the formal deletion claim to the tau-to-infinity limit and treat finite-tau deletion as empirical.","section":"Section 4, Eq. (6); Proposition 1"},{"comment":"The escape from the frontier is quantitative only through the false-positive mass alpha and the missed-forget mass epsilon (and their soft counterparts). Proposition 1 does not prove that trained neural features make the frozen head's weight scores separate forget from retain; it only states what happens conditionally on alpha and epsilon. Assumption 1, the Gaussian shared-covariance model, does not by itself imply small alpha and epsilon. The empirical Section 5.3 shows small soft retain mass on the tested benchmarks, but the paper's abstract and introduction claim a general escape. The authors should either supply conditions under which alpha and epsilon decay, or consistently frame the escape as an empirical phenomenon observed in the tested regimes.","section":"Proposition 1 and Section 5.3"},{"comment":"The theorem's deletion criterion, Definition 2, is a within-forget criterion: a linear probe must fail to separate the forget classes from one another. The paper's primary empirical deletion standard, however, is full-class KR re-extraction on the unlearned features. Since any transform that passes full-class re-extraction also annihilates U_F, Theorem 1 remains a valid lower bound for full-class deletion, but the equality statement \"erasing U_F alone attains the floor\" applies only to the within-forget criterion. The paper should make this distinction explicit when claiming that the frontier limits the whole class of source-free erasers, since the deployed audit is stronger than the formalized deletion notion.","section":"Definition 2 and Section 5.2"}],"minor_comments":[{"comment":"The text contains numerous line-break artifacts, such as \"forget-discriminantsubspace\", \"andith\", and \"inputand\"; these should be fixed in the final version.","section":"Abstract and Section 1"},{"comment":"The caption writes cos2 for cos^2; please use consistent mathematical notation.","section":"Figure 1 caption"},{"comment":"The sentence \"Section 5.5 measures this deployed cost against explicit whitening\" appears to refer to content that is not in the listed Section 5.5 ablation; either the cross-reference is wrong or the relevant discussion is only in the missing appendix.","section":"Section 3, after Eq. (1)"},{"comment":"The definition of omega(v) takes values in [0,+inf], but the subsequent ranking by descending omega is not fully defined when the same value occurs infinitely often; the text mentions a deterministic tie-break, but the details should be explicit in the main text rather than only in the appendix.","section":"Section 4, Eq. (4)"},{"comment":"The phrase \"the deployed panel saturates\" is informal; please specify which metric saturates and why, or use a more precise description.","section":"Section 5.4"}],"recommendation":"major_revision","confidential_remarks":"The paper's main geometric result appears sound and the empirical study is unusually thorough, but the submission is incomplete without the appendix, and the formal guarantees for the deployed soft-gate method are weaker than the text sometimes suggests. I would ask the authors to provide the full appendix, tighten the soft-gate claims, and explicitly qualify the escape result as conditional on gate separation. With those changes, the paper could be a solid contribution to representation-level unlearning."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here is my take on SCOPE. The genuinely new thing is Theorem 1: under a Gaussian shared-covariance model, any fixed linear eraser that deletes the forget classes must pay retain cost at least the retain-readout energy in the forget-discriminant subspace, and erasing exactly that subspace attains the floor. It is a simple projection inequality once written down, but it is the right formalization, and the paper shows the leading source-free erasers (ESC, POUR) are instances of the bound. The empirical work is extensive: five benchmarks, two backbones, and the frontier predicts measured retain cost at R^2 ≈ 0.99. The ablation separating the gate from the erased basis is well designed, and the method itself is closed form and cheap.\n\nThe soft spot is the one the stress-test note flags. Proposition 1 gives the escape for a hard gate; the deployed SCOPE uses a soft logistic gate. For the soft gate the paper has a retain-cost bound and a leakage bound, but the deletion certificate is stated only for the hard gate, the τ → ∞ limit. In practice the soft retain mass is low, so the escape works on these benchmarks, but Assumption 1 does not imply that the frozen head's weight scores separate forget from retain. If forget and retain are confusable, the false-positive mass α grows and the escape degrades. That makes the practical claim conditional on a property of the particular trained model, not a consequence of the geometry theorem.\n\nTwo smaller issues. No code or data is released, and the proofs are in an appendix, so full verification has to wait. And the concurrent conditional method MoRE (Jeon et al. 2026) is cited but not compared; the authors should either add it or explain why not.\n\nI do not think these sink the paper. The theorem is correct under its assumptions, and the frontier claim is well supported. I would send it to peer review, but I would ask for code, for an analysis of when head separation holds or fails, and for a MoRE comparison. Someone in machine unlearning will get real value from the geometric framing even if the 'escape' is more conditional than the abstract suggests.","headline":"A clean geometric lower bound for fixed-projection unlearning plus a gated escape whose practical bite depends on an unproven separation property of the frozen head.","tokens_in":15650,"tokens_out":2906,"would_cite":true,"duration_ms":25784,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Source-free class unlearning has a provable entanglement frontier: every fixed projection pays at least the retain-readout energy along the forget-discriminant subspace, and input-conditional gating escapes it.","keywords":["machine unlearning","source-free class unlearning","feature-space erasure","knowledge retention","forget-discriminant subspace","entanglement frontier","input-conditional gating","SCOPE"],"falsifier":"On a model satisfying the Gaussian shared-covariance assumptions, search a broad family of fixed linear projections that drive forget-class linear-probe accuracy to chance; if any projection's retained-logit perturbation falls strictly below $\\|P_{U_F}W_R^T\\|_F^2$, Theorem 1 is false. On the empirical side, a trained network whose head scores place a large false-positive mass on retain inputs would let the gated map's retain cost exceed the frontier floor, showing the escape is not universal.","tokens_in":14747,"feed_emoji":"🧠","tokens_out":14268,"duration_ms":116285,"temperature":0.7,"pith_summary":"Source-free class unlearning asks whether a model can forget a whole class from its feature representation using only the forget data, and this paper proves the task meets a hard geometric limit. When the directions that separate the forget classes overlap with the directions the retained classifier reads from, any fixed linear projection that deletes the classes must damage the retained logits by at least the retain-readout energy inside the forget-discriminant subspace, $\\|P_{U_F}W_R^T\\|_F^2$, and the cheapest fixed deletion is to erase exactly that subspace. The limit binds the whole family of existing projection-based erasers, because they all apply one fixed map to every input. The paper then shows that an input-conditional erasure, a gate derived from the frozen head's weight scores, can escape the frontier, and instantiates this escape as SCOPE, a closed-form, retain-free unlearner that across five benchmarks predicts the measured retain degradation from geometry alone and outperforms the fixed-projection baselines.","feed_headline":"Machine unlearning hits a retain-cost floor; input gating escapes it","feed_subtitle":"Fixed projections must disturb retention where forget and retain features overlap; a head-derived gate skips that cost.","key_machinery":"The machinery is the principal-angle decomposition of the entanglement between $U_F$ and $S_R$, where $U_F$ spans the class-mean differences of the forget classes and $S_R$ spans the retained rows of the frozen linear head. Theorem 1 prices each principal direction by the retained readout energy $\\rho_i=\\|W_R u_i\\|_2^2$ and shows the total $\\sum_i \\cos^2\\theta_i\\rho_i$ is the exact floor. The escape is carried by the gate $g_\\tau(z)=\\sigma(\\tau(\\max_{c\\in F}w_c^\\top z-\\max_{c\\in R}w_c^\\top z))$, a head-derived function that decides per input how much of the erased subspace to subtract.","core_discovery":"The paper's central claim is that the retain cost of deleting classes from a representation is governed by the principal-angle geometry between the forget-discriminant subspace $U_F$ and the retain-readout subspace $S_R$. Under Assumption 1 (Gaussian class-conditional features with shared covariance) and Assumption 2 (isotropic retain second moment), Theorem 1 states the minimum retain cost among all global deleting linear maps is exactly $\\|P_{U_F}W_R^T\\|_F^2 = \\sum_i \\cos^2\\theta_i\\,\\rho_i$, attained by the minimal eraser that removes $U_F$ and can be padded only with directions orthogonal to $S_R$. Consequently every fixed-projection unlearner, including the leading source-free erasers, pays at least this floor whenever the two subspaces are not orthogonal. Proposition 1 then shows the floor is not fundamental to deletion itself: a map $z\\mapsto z-g(z)P_Qz$ with an input-dependent gate can erase at zero retain cost when the gate fires only on forget inputs, and its retain cost is bounded by the gate's false-positive mass. SCOPE realizes this with a single logistic gate read from the frozen head's forget-minus-retain margin, builds the erased basis in closed form from forget features and the head's rows, and is evaluated with linear, converged, nonlinear, and sparse-dictionary re-extraction audits.","pith_inferences":["This suggests a practical diagnostic: before running any projection eraser, compute $\\|P_{U_F}W_R^T\\|_F^2$ on the original model; it forecasts the retention damage that even the best fixed eraser must incur, which is cheaper than running the unlearning method itself.","The gate mechanism may carry over to other head-based readouts, such as concept erasure in language or multimodal models, whenever the final layer can be split into forget and retain categories; SCOPE itself does not claim this.","If real features deviate from the Gaussian model, the theorem shifts from an exact guarantee to a calibration tool; the paper reports close agreement under distribution shift, but transfer to arbitrary architectures is not proven.","The large gap between the minimal eraser and random matched-rank subspaces suggests that new erasers should be evaluated against the geometric floor rather than only against retraining, because retraining itself leaves forget classes linearly recoverable in several reported settings."],"forward_implications":["Any projection-based unlearner that uses a single global map, such as erasing the forget subspace or nulling a class-weight direction, inherits the retain-cost floor of Theorem 1, a floor set only by the geometry of the original model.","A class can be erased at zero retention cost if and only if the forget-discriminant subspace is orthogonal to the retain-readout subspace; otherwise the cheapest fixed eraser is the minimal forget-discriminant projector.","Input-dependent gating converts retain cost into a function of the gate's false-positive mass, so the quality of the frozen head as a forget-versus-retain classifier becomes the controlling factor in the trade-off.","SCOPE is closed form, needs no retain data or gradient training, and in the reported benchmarks costs orders of magnitude less than retraining while keeping forget classes unrecoverable under linear and stronger probes.","At $K=1$ the within-forget frontier degenerates because $U_F=\\{0\\}$, so single-class forgetting must be judged by full-class re-extraction rather than by the within-forget criterion."],"supporting_citations":[{"why":"Introduces machine unlearning and the retraining-from-scratch reference against which deletion is measured.","marker":"Bourtoule et al. 2021"},{"why":"Defines ESC, the leading source-free fixed-projection eraser and the knowledge-retention re-extraction audit; it is the main baseline the frontier binds and SCOPE is compared against.","marker":"Lee et al. 2025b"},{"why":"Presents POUR/POUR-P, a projection-based eraser derived from neural collapse; another instance of the fixed-projection form Theorem 1 limits.","marker":"Le et al. 2026"},{"why":"Supplies the nullspace-projection operation that fixed-projection erasers build on, placing their form inside Theorem 1's scope.","marker":"Ravfogel et al. 2020"},{"why":"DELETE, a distillation-based unlearner outside the fixed-projection class, provides a non-projection comparison baseline in the benchmarks.","marker":"Zhou et al. 2025"},{"why":"NG and FT, gradient and finetuning unlearning baselines; they define the selective-forgetting setting SCOPE is benchmarked in.","marker":"Golatkar, Achille, and Soatto 2020"},{"why":"CIFAR-100, one of the five evaluation benchmarks on which the frontier prediction and SCOPE comparisons rest.","marker":"Krizhevsky, Hinton et al. 2009"}],"fun_headline_variants":["Fixed-projection unlearning pays a proven floor; input gate escapes it","Source-free class erasure: frontier theorem, then a gate breaks free","SCOPE gates the forget subspace to beat the retain-cost floor","The unlearning frontier: any fixed map loses, but gating wins"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that class-conditional features are Gaussian with a shared covariance, which makes erasing the forget-discriminant subspace exactly equivalent to deleting the classes, and that the frozen head's weight scores separate forget from retain inputs well enough for the gate to fire chiefly on forget inputs.","fun_headline_variants_meta":{"raw":{"variants":["Fixed-projection unlearning pays a proven floor; input gate escapes it","Source-free class erasure: frontier theorem, then a gate breaks free","SCOPE gates the forget subspace to beat the retain-cost floor","The unlearning frontier: any fixed map loses, but gating wins"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000315,"raw_usage":{"total_tokens":1845,"prompt_tokens":1066,"completion_tokens":779,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":682,"completion_tokens_details":{"reasoning_tokens":701}},"tokens_in":682,"tokens_out":779,"duration_ms":7677,"temperature":1.0,"reasoning_tokens":701,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:02:55.180542+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a model satisfying the Gaussian shared-covariance assumptions, search a broad family of fixed linear projections that drive forget-class linear-probe accuracy to chance; if any projection's retained-logit perturbation falls strictly below $\\|P_{U_F}W_R^T\\|_F^2$, Theorem 1 is false. On the empirical side, a trained network whose head scores place a large false-positive mass on retain inputs would let the gated map's retain cost exceed the frontier floor, showing the escape is not universal.","supporting_citations":[{"cited_title":"Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection","cited_arxiv_id":null,"evidence_quote":"Supplies the nullspace-projection operation that fixed-projection erasers build on, placing their form inside Theorem 1's scope."},{"cited_title":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =","cited_arxiv_id":null,"evidence_quote":"DELETE, a distillation-based unlearner outside the fixed-projection class, provides a non-projection comparison baseline in the benchmarks."}],"review_version":1}