{"id":"e739cf38-952d-4efe-ac56-999c833f0b0a","arxiv_id":"2505.07026","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"MAXRR trains the feature extractor on core samples only, so under a generalized unlearning definition, deleting non-core samples is exact and deleting core samples needs only a cheap SVM retrain.","lead":"This paper proposes a training scheme that makes many data deletion requests in machine learning very cheap: the feature extractor is trained only on a small set of important samples, and the final classifier is a support vector machine. It introduces a broader definition of unlearning so that, under that definition, deleting any unimportant sample is exact and free.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Core-set selection uses the full dataset, so a forgotten sample can alter which samples are in D_k; the proof that w_FE is independent of D\\D_k fails, and exact unlearning is not established even for non-core requests.","rationale":"The reader located the risk in Definition 1's weakened comparison class. I agree that the definition is weak, but the sharper problem is internal to the exactness proof: the core set D_k is chosen using the full training data, so the FE is not independent of samples outside D_k. This makes the exact-unlearning claim fail even for the headline scenario D_f⊆D\\(S∪D_k). The generalized definition does not repair the gap unless A' is allowed to depend on D_f through the hardcoded D_k, which contradicts 'independent of D_f'. The proposed test directly checks whether D_k changes when D_f is removed before the ranking step. A failure would require either selecting the core set from data disjoint from any possible deletion request, proving that the ranking is unaffected by D_f, or downgrading the exact claim to approximate unlearning. The empirical study is otherwise coherent, but the central theoretical guarantee needs reworking before the exact-unlearning claim can stand. I therefore keep the reader's CONDITIONAL verdict.","tokens_in":11472,"tokens_out":10807,"duration_ms":118335,"concrete_test":"Re-run the full MAXRR pipeline on D\\D_f for one fixed D_f outside the original S∪D_k: recompute the support-vector frequencies, select D_k', train the FE on D_k', train the SVM on D\\D_f, and compare the resulting model with MAXRR's unlearned model (original w_FE plus SVM retrained on D\\D_f). If any sample's membership in D_k changes, or if the two models differ in test-set predictions or decision-function distance, the exact-unlearning claim fails because D_f influenced core selection. Repeat across several seeds and several choices of D_f to confirm robustness.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The load-bearing claim is in Section IV-A/B: for D_f⊆D\\(S∪D_k), the model is 'per design exactly unlearned' because 'the feature extractor w_FE is entirely independent of D\\D_k'. This step fails. MAXRR selects D_k by ranking support-vector frequencies over multiple training runs on the entire dataset D (Section IV). Thus the identity of D_k is a function of D, including any later-requested D_f. If D_f had been absent from the ranking stage, a different D_k' could have been selected and a different FE trained, so U(A(D),D,D_f) is not generally equal to A(D\\D_f) under the original algorithm. The proof's A'(D_p) trains the FE on the original D_k; this A' is not independent of D_f because D_k was inferred from a dataset containing D_f. The exactness guarantee is therefore conditional on a particular D_k that the unlearning request itself helped determine, and it is not an exact-unlearning guarantee over the full training procedure. The issue is not merely that Definition 1 is weak; it breaks the stated proof even for non-core, non-support deletions. Appendix B's observation that the MIA cannot confirm exact unlearning is consistent with this gap.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MAXRR, an unlearning-aware training procedure for SVM-headed classifiers. The model is split into a feature extractor (FE) and a linear SVM; the FE is trained on a core subset D_k of the most influential samples, determined by support-vector frequency over multiple runs on the full dataset, while the SVM is trained on all data. The authors introduce a generalized exact-unlearning definition (Definition 1) that allows comparison to any algorithm A' trained on any subset D_p of the remaining data, and they claim that for any request D_f disjoint from S ∪ D_k, the model is \"per design exactly unlearned.\" When D_f intersects D_k, they fall back to approximate unlearning and verify it with a confidence-based membership inference attack. The paper reports Fashion MNIST experiments showing competitive accuracy and similar MIA verification results between MAXRR and full retraining.","tokens_in":11739,"tokens_out":10470,"duration_ms":111182,"significance":"If the exact-unlearning claim were correct, MAXRR would be a genuinely useful recipe: most deletion requests become free or cheap for SVM-headed classifiers, and the paper would contribute a clean decomposition of the training procedure into a reusable feature extractor and a retrainable last layer. The writing is clear, the algorithm is simple to implement, and the authors honestly acknowledge the limitations of MIA-based verification, including in Appendix B the fact that their MIA cannot detect even exact unlearning of non-core samples. However, the central claim rests on a weakened definition and on an independence argument that does not hold as written; the substantive exact-unlearning guarantee is therefore not established, and the approximate case is supported only by a verification method the paper itself shows to be unreliable. The contribution is potentially salvageable, but the current manuscript substantially overstates what is proven.","major_comments":[{"comment":"The proof that any D_f ⊆ D\\(S∪D_k) is exactly unlearned asserts that \"the feature extractor w_FE is entirely independent of D\\D_k.\" This is not correct under the procedure described in Section IV: the core set D_k is selected by ranking support-vector frequencies computed over the full dataset D, so the identity of D_k and hence the trained w_FE are functions of D, including any later-requested D_f. The comparison algorithm A'(D_p) constructed in the next paragraph reuses the same D_k; because D_k was inferred from a dataset that contains D_f, that A' is not independent of D_f under the usual meaning of Definition 1. Removing D_f before the ranking stage could select a different core set and a different feature extractor, so U(A(D),D,D_f) is not generally equal to A(D\\D_f) even when D_f contains no core samples and no support vectors. The authors need either to prove that the core-selection procedure is independent of every possible deletion request, e.g. by choosing D_k without using the deletable data, or to explicitly restrict the claim to a fixed precomputed D_k and explain what privacy guarantee that provides.","section":"Section IV-B, exact unlearning paragraph"},{"comment":"The abstract states that MAXRR supports \"exact unlearning\" and that its properties \"closely match\" full retraining, without the crucial caveat that its exactness is relative to Definition 1, a substantial weakening of the standard notion. Definition 1 allows the unlearned model to be compared to any algorithm A' on any subset D_p of the remaining data, and Section IV-B then chooses A' and D_p so that U equals A'(D_p) by construction. This makes the exactness guarantee definitional rather than a property validated against an independent baseline, and it raises the question of what privacy property Definition 1 actually provides. The paper should either adopt the standard comparison A(D\\D_f) (in which the core-selection dependence is fatal), or clearly motivate Definition 1 as a meaningful operationalization and advertise the guarantee as such in the abstract and introduction.","section":"Definition 1 and the abstract"},{"comment":"The evidence for the approximate-unlearning claim is not load-bearing enough. For the case D_f ∩ D_k ≠ ∅, the paper verifies unlearning by showing that a confidence-based MIA yields similar classifications for MAXRR and for a fully retrained model (Figures 3 and 4). However, Appendix B shows that the same MIA cannot reliably detect whether non-core samples were unlearned even when the unlearning is exact. A verification method that fails to separate an exactly unlearned model from a non-unlearned one cannot support the claim that an approximate method \"closely matches\" full retraining. The authors should either provide a stronger verification argument (a different attack, a distributional distance, or a formal approximate-unlearning bound) or explicitly weaken the claim to an empirical utility comparison rather than an unlearning guarantee.","section":"Section IV-B(b) and Appendix B"}],"minor_comments":[{"comment":"The phrase \"theright to be forgotten\" in the abstract and introduction is missing a space; this is a simple typographical error.","section":"Section I, first paragraph"},{"comment":"The decision function uses \"sgin(x)\" where the text presumably means \"sign(x)\"; please correct this typo.","section":"Section II, decision function"},{"comment":"The proof argues that after removing D_f, an improved solution on D\\D_f would also be feasible for D and contradict optimality; this is correct for the objective value, but the argument should explicitly handle the case of multiple optimal dual solutions, since the decision boundary depends on b and the chosen support-vector set.","section":"Section III, Proposition 1 proof"},{"comment":"The text says \"Assume access to a ranking\" but does not specify whether the ranking is part of the training algorithm and whether it may access samples that will later be forgotten; clarifying this would make the exactness claim easier to evaluate.","section":"Section IV-A"},{"comment":"The axis label \"Percentage of runs both algorithms output same verification result\" is confusing because agreement between MAXRR and perfect unlearning is not the same as successful unlearning; please rephrase and clarify in the caption and text.","section":"Figures 3 and 4"}],"recommendation":"major_revision","confidential_remarks":"The paper is clearly written and the experimental setup is transparent, but the central exact-unlearning guarantee is not established because core-set selection uses the full dataset including the data to be forgotten, and the generalized definition makes the guarantee partly definitional. I would ask the authors to either supply a formal independence argument or substantially revise the claims, possibly repositioning the work as an empirical efficiency/utility method with approximate unlearning. The MIA-based verification is known by the authors themselves to be weak, so the approximate claim needs additional support before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: MAXRR is a genuinely new way to structure unlearning-aware training — train the feature extractor on a small core set, put an SVM head on the full dataset — and the SVM-only part is correct. But the exact-unlearning guarantee, the paper's main claim, does not survive contact with how the core set is chosen.\n\nWhat's new and good. The combination of model splitting and core-sample selection is not in the earlier unlearning literature. Proposition 1 is just the standard property that a linear SVM is determined by its support vectors, but using that property inside a split architecture is a reasonable design move. The Fashion MNIST experiments are a credible proof of concept: training the FE on 20k of 60k samples and the SVM on all of them keeps accuracy near 0.88, and unlearning non-core points by retraining only the SVM is cheap. The authors also deserve credit for flagging in Appendix B that their MIA cannot detect even exact unlearning on non-core samples; that is the kind of honesty that makes the rest of the paper easier to trust.\n\nWhere it falls down. The exact-unlearning proof in Section IV-A rests on Definition 1, which replaces the standard benchmark A(D\\D_f) with any algorithm A' on any subset D_p disjoint from D_f. That is a serious weakening. Under that definition, the unlearning procedure can be declared exact just because it matches an algorithm chosen after the fact. The abstract says \"exact unlearning\" without this caveat. I think the weaker notion can be defended, but only with an argument that it still means the deleted samples have no influence; the paper does not make that argument.\n\nThe deeper problem is the independence claim. MAXRR selects the core set D_k by ranking support-vector frequencies over multiple training runs on the entire dataset D. So D_k is a function of D, including every sample that will later appear in D_f. The proof's A' trains the FE on this fixed D_k and then retrains the SVM on D_p. The paper says A' is independent of D_f, but it is not, in the counterfactual sense that matters: if D_f had never been in the training data, a different D_k could have been chosen and a different FE trained. The unlearned model is therefore still shaped by the forgotten points, and the exact-unlearning guarantee is not established even for non-core, non-support requests. This is not a minor gap; it is the load-bearing part of the paper.\n\nThe approximate-unlearning evidence is also weaker than it looks, because the MIA used for verification is shown in Appendix B to fail on exact unlearning; similarity to retraining under such an attack does not demonstrate removal. Missing standard baselines and unstated hyperparameters (e.g., SVM regularization C) add noise.\n\nBottom line: the design principle — train the hard-to-update part on a deliberately small core set — is worth taking seriously, and the paper should go to peer review. But the revision needs to either fix the independence problem or reframe MAXRR as an approximate-unlearning method with a weaker, honestly labeled guarantee.","headline":"A genuinely new unlearning-aware training scheme with a useful core-set design principle, but the exact-unlearning guarantee fails as proved because the core set is selected from the full data and is therefore not independent of the forgotten points.","tokens_in":12240,"tokens_out":7745,"would_cite":true,"duration_ms":82785,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MAXRR makes most unlearning requests exact by construction, with no retraining needed for samples outside the core set and the SVM support vectors.","keywords":["machine unlearning","right to be forgotten","support vector machine","core sample selection","model splitting","membership inference attack","exact unlearning","generalized unlearning definition"],"falsifier":"A decisive calculation: take a non-core, non-support-vector sample, apply MAXRR's exact-unlearning protocol for it, and compute the divergence between the resulting distribution and the distribution of the original algorithm retrained on $D \\setminus D_f$; a nonzero divergence shows the exactness certificate is an artifact of Definition 1 rather than of matching full retraining. A second check is whether $D_k$ remains unchanged when that sample is excluded from the ranking procedure—if not, the proof's $A'$ is not independent of $D_f$.","tokens_in":11243,"feed_emoji":"🧠","tokens_out":12083,"duration_ms":115589,"temperature":0.7,"pith_summary":"MAXRR makes unlearning a training-time design decision rather than a post-hoc repair. A classifier is split into a feature extractor and a linear support-vector machine head; the feature extractor is trained only on the most influential 'core' samples, while the SVM is trained on everything. Because an SVM's decision function is determined entirely by its support vectors, forgetting any sample that is neither a core sample nor a support vector requires no model change at all, and forgetting a support vector requires retraining only the cheap SVM head. Under the paper's generalized definition of exact unlearning—equivalence to any algorithm trained on any subset of the remaining data—these cases are exactly unlearned, and in experiments the remaining approximate cases behave like full retraining under membership-inference auditing.","feed_headline":"Model split makes most unlearning requests free","feed_subtitle":"Train only the feature extractor on core samples; the SVM head absorbs most deletions without retraining.","key_machinery":"The load-bearing object is the dual soft-margin SVM, whose decision function is a weighted combination of training samples that have nonzero dual variables (the support vectors). Proposition 1 establishes that removing any non-support sample from the training set leaves the dual optimum feasible and optimal, so the model literally does not change. Model splitting turns this property into an architecture: the network's last layer is replaced by a linear SVM trained on the feature embeddings. Core sample selection ranks all training samples by how many times they become support vectors across multiple runs, defines $D_k$ as the top $k$, and trains the feature extractor only on $D_k$, making it structurally independent of all non-core samples. Definition 1 generalizes exact unlearning so that equality to any algorithm $A'$ trained on any remaining subset $D_p$ is sufficient; that generalization is what converts 'no retraining needed' into a formal exact-unlearning certificate.","core_discovery":"The paper's central claim is that a large fraction of unlearning requests can be made exact by construction. With the MAXRR composition $w_{\\mathrm{FE}} \\circ w_{\\mathrm{SVM}}$, the feature extractor $w_{\\mathrm{FE}}$ is trained only on a core set $D_k$ of samples that are most frequently selected as support vectors across several initial runs, and the linear SVM $w_{\\mathrm{SVM}}$ is trained on the full dataset's embeddings. Proposition 1 shows that deleting a non-support-vector sample leaves the SVM's optimum unchanged, and the FE is independent of every sample outside $D_k$ by design. Hence for any request $D_f \\subseteq D \\setminus (S \\cup D_k)$ the stored checkpoint is already exactly unlearned in the sense of Definition 1, and for $D_f \\subset S \\setminus D_k$ only the SVM layer needs retraining, which is still exact. The reported Fashion-MNIST experiments put the exact-unlearning coverage at $|S \\cup D_k| = 20.5 \\cdot 10^3$ out of $60 \\cdot 10^3$ training samples on average.","pith_inferences":["Implicit in the exactness proof is that the core set $D_k$ is chosen without using the samples that will later be forgotten; the paper's ranking procedure runs on the full dataset $D$, so unless $D_k$ is fixed externally or before any request, the $A'$ in Definition 1 may not actually be independent of $D_f$.","If a regulator adopts Definition 1, the right to be forgotten becomes an architectural property: compliance is achieved by how the model is built, not by post-hoc deletion. If not, MAXRR's contribution reduces to an efficient approximate-unlearning scheme with the same weak-verification caveats as other approaches.","The exact-unlearning coverage is governed by the overlap $S \\cup D_k$; on datasets where support-vector membership is unstable across runs, the core set must grow to keep accuracy, shrinking the fraction of deletion requests that can be served for free. This is a testable prediction for other datasets.","Appendix B already shows confidence-based MIAs cannot verify genuinely exact unlearning of easy samples, so the approximate-mode verification results in Figs. 3-4 should be read as evidence of behavioral similarity, not as proof that residual information is gone."],"forward_implications":["Deleting any sample outside $S \\cup D_k$ is free: the stored checkpoint is already exactly unlearned under Definition 1 and no retraining is triggered.","When the forgotten sample lies in $S \\setminus D_k$, the whole unlearning cost is one SVM retraining on the remaining embeddings; the feature extractor, usually the expensive part, is untouched.","The unlearned model can be more accurate than a fully retrained model, since the core samples that drive feature quality stay in the feature extractor (0.8798 vs 0.8625 average accuracy in the reported core-unlearning experiment).","Because the guarantee is structural, it transfers to any classifier whose final layer is a linear SVM, not just to the specific architecture tested.","MAXRR is compatible with stricter unlearning methods as a preprocessing step, so a provider can upgrade to stronger guarantees when needed."],"supporting_citations":[{"why":"Supplies the approximate-unlearning baseline and the KL-divergence slack that MAXRR's approximate mode is measured against.","marker":"[11]"},{"why":"States the observation that forgetting unimportant data may not significantly impact the model, which motivates core sample selection.","marker":"[16]"},{"why":"Provides the entropy/confidence-based membership inference attack used as the verification procedure in Algorithm 2.","marker":"[24]"},{"why":"Gives the probabilistic framework for exact and approximate unlearning that Definition 1 generalizes.","marker":"[36]"},{"why":"Supplies the LeNet-5 architecture used as the feature extractor in all Fashion-MNIST experiments.","marker":"[37]"}],"fun_headline_variants":["Most unlearning requests become exact by design","Split model: deletion costs nothing for many samples","Only SVM head retrains: exact unlearning for most","Most deletion requests need zero retraining","Core sample selection makes unlearning exact often"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the generalized definition of exact unlearning—comparing the unlearned model with any algorithm trained on any subset of the remaining data, rather than with the original algorithm retrained on the full remaining set—is an acceptable operationalization, and that the core set used to train the feature extractor is fixed independently of the samples later requested for deletion; if either gives way, the exactness claim no longer holds.","fun_headline_variants_meta":{"raw":{"variants":["Most unlearning requests become exact by design","Split model: deletion costs nothing for many samples","Only SVM head retrains: exact unlearning for most","Most deletion requests need zero retraining","Core sample selection makes unlearning exact often"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000593,"raw_usage":{"total_tokens":2757,"prompt_tokens":902,"completion_tokens":1855,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":518,"completion_tokens_details":{"reasoning_tokens":1786}},"tokens_in":518,"tokens_out":1855,"duration_ms":13232,"temperature":1.0,"reasoning_tokens":1786,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:27:32.276196+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive calculation: take a non-core, non-support-vector sample, apply MAXRR's exact-unlearning protocol for it, and compute the divergence between the resulting distribution and the distribution of the original algorithm retrained on $D \\setminus D_f$; a nonzero divergence shows the exactness certificate is an artifact of Definition 1 rather than of matching full retraining. A second check is whether $D_k$ remains unchanged when that sample is excluded from the ranking procedure—if not, the proof's $A'$ is not independent of $D_f$.","supporting_citations":[{"cited_title":"Eternal sunshine of the spotless net: Selective forgetting in deep networks,","cited_arxiv_id":null,"evidence_quote":"Supplies the approximate-unlearning baseline and the KL-divergence slack that MAXRR's approximate mode is measured against."},{"cited_title":"Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models,","cited_arxiv_id":null,"evidence_quote":"Provides the entropy/confidence-based membership inference attack used as the verification procedure in Algorithm 2."},{"cited_title":"Unrolling sgd: Understanding factors influencing machine unlearning,","cited_arxiv_id":null,"evidence_quote":"Gives the probabilistic framework for exact and approximate unlearning that Definition 1 generalizes."},{"cited_title":"Gradient-based learning applied to document recognition,","cited_arxiv_id":null,"evidence_quote":"Supplies the LeNet-5 architecture used as the feature extractor in all Fashion-MNIST experiments."}],"review_version":1}