{"id":"90eab3e0-6ee0-4be1-acd1-f64d12b7b386","arxiv_id":"2411.11315","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":1.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"The paper provides a structured review and classification of machine unlearning methods, linking them to data lineage management.","lead":"This paper reviews machine unlearning techniques that let AI models forget specific training data, as required by privacy laws like the GDPR. It organizes methods into exact and approximate unlearning and connects them to data lineage tracking.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The review's central taxonomy is not yet reliable: Table 1 and Fig. 6 classify SISA as federated learning, and §4.5.2 gives a contradictory complexity for influence-based unlearning, so the map needs correction before the review's claims can be accepted.","rationale":"The reader's verdict was CONDITIONAL, based on taxonomy completeness. I agree with that verdict but for a slightly different reason. The most load-bearing condition for the central claim is not only that the taxonomy covers the field, but that each included entry is described accurately; a map with wrong labels cannot support the paper's conclusions. The SISA/federated mischaracterization and the contradictory O(d^2)/linear-in-d statement are present on the face of the text and are directly connected to the classification the review offers. They are fixable, so rejection is not warranted; the conditional verdict stands until the authors correct Table 1, Fig. 6, and Sections 4.5.2/4.5.4, and ideally audit the remaining rows. No ad hominem is intended; these are internal accuracy failures of a review, not evidence of bad faith.","tokens_in":14482,"tokens_out":7730,"duration_ms":76688,"concrete_test":"Run a citation audit of Table 1 against the primary sources. For each of the 23 rows, record the exact/approximate assignment, the architectural characterization, and any complexity claim, and compare with the cited paper's abstract and method section. Start with the two contested entries: (1) Bourtoule et al. [3] — does SISA involve multiple data owners or federated aggregation anywhere? (2) Izzo et al. [36] — is the deletion cost O(d^2), O(nd^2), or linear in d? If either row contradicts the review's text, the taxonomy entry must be corrected; if several rows require correction, the review's central map is unreliable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central contribution is a structured taxonomy of exact versus approximate unlearning. That contribution is load-bearing on the fidelity of each taxonomy entry. As written, Section 4.5.2 states that the influence-method unlearning algorithm of Izzo et al. has 'computational cost ... linearly related to the feature dimension d, i.e., O(d^2).' The two claims contradict each other, and the surrounding discussion does not resolve which complexity the authors intended. Similarly, Table 1 and Figure 6 present Bourtoule et al.'s SISA as federated learning, with the figure caption explicitly saying SISA is 'presented in the form of federated learning.' Yet Section 4.4.2 and the cited paper describe SISA as sharded, isolated, sliced, and aggregated training; it does not involve multiple data owners or federated communication. A reader using the review's map would carry away a false architectural picture of a representative exact-unlearning method. Section 4.5.4's evaluation-metric definition is also ambiguous: AccErr is defined as SAPE(Acc*_test, Accu_test), but the interpretation refers to an 'initial model (in which the noise value σ = 0)' that is not defined in the surrounding text. These are internal inconsistencies in the paper's own classification, not merely differences from the external literature.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper is a survey of machine unlearning, motivated by the GDPR right to be forgotten. It introduces the distinction between exact and approximate unlearning, reviews representative methods in each category (e.g., SQ-learning based unlearning, SISA, influence-based methods, scrubbing, and amnesiac unlearning), and discusses data lineage management as a supporting mechanism for privacy-preserving ML. The central contribution is a taxonomy and a summary table classifying 23 works, together with a discussion of open challenges.","tokens_in":14748,"tokens_out":3549,"duration_ms":31380,"significance":"The survey is potentially useful as an entry point for researchers and practitioners: it compiles a broad set of recent references, anchors the discussion in a clear exact-versus-approximate frame, and explicitly connects unlearning to data lineage tools such as MLMD. The value of the paper, however, rests on the fidelity of its classifications and complexity statements, since it provides no new experiments or proofs. The paper's strengths are its organizational clarity and the breadth of coverage within the chosen taxonomy.","major_comments":[{"comment":"The text describing Izzo et al. states that the computational cost 'is linearly related to the feature dimension d, i.e., O(d^2), and is independent of the number of training data n.' O(d^2) is quadratic in d, not linear, so the two claims are contradictory. Since the efficiency gain of approximate unlearning is central to the review's argument, this complexity statement must be corrected and reconciled with the cited source.","section":"Section 4.5.2"},{"comment":"Table 1 classifies Bourtoule et al.'s SISA as 'Federated learning (SISA)', and Figure 6's caption says the SISA approach is 'presented in the form of federated learning.' Section 4.4.2 correctly describes SISA as sharded, isolated, sliced, and aggregated training on a single dataset without federated communication among multiple data owners. This internal inconsistency misleads readers about a representative exact-unlearning method; the same mischaracterization recurs in Section 6.3, where SISA is called a 'distributed unlearning model.'","section":"Table 1 and Figure 6"},{"comment":"The Effectiveness metric is defined as AccErr = SAPE(Acc*_test, Accu_test), but the accompanying interpretation says a lower value means the updated model's accuracy is closer to 'the accuracy of the initial model (in which the noise value σ = 0).' Earlier in the same paragraph, Acc*_test is defined as 'the optimal accuracy of the regression model on the same dataset,' and σ is not introduced anywhere in the section. This makes the metric ambiguous; please clarify what Acc*_test represents and define σ.","section":"Section 4.5.4"}],"minor_comments":[{"comment":"There is a typo: 'makings it more accessible for attackers' should be 'making it more accessible for attackers.'","section":"Section 1.1"},{"comment":"Reference [19] is cited for the idea of dividing data into blocks and training sub-models, but the cited work (Fredrikson et al.) is about model inversion attacks and does not appear to support this claim; please re-check the citation.","section":"Section 4.2"},{"comment":"There is a typo: 'delated data' should be 'deleted data.'","section":"Section 4.5.4"},{"comment":"The entry for He et al. 2021 says 'Intermeidate models'; this should be 'Intermediate models.'","section":"Table 1"},{"comment":"The notation for θ\\k is garbled: 'θ\\k = θ((θ, ..., k, 1, ... n−k)⊤)' does not clearly define the weight vector; please rewrite the expression in standard form.","section":"Section 4.5.2"},{"comment":"Reference [5] (Schelter) lacks a publication venue and year; please complete the citation.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"This is a review paper with no new technical results, so its suitability depends on the journal's appetite for surveys. The taxonomy errors identified in the major comments are local and fixable, but they are load-bearing because the paper's value is precisely as a reliable map of the unlearning literature. I see no issues with self-citation or circularity; the paper cites external work throughout. Once the SISA classification, the complexity statement, and the metric definition are corrected, the survey would be a serviceable contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a readable survey of machine unlearning, and at a high level the exact-versus-approximate split is sensible. I would point a newcomer to it for the GDPR context, the brief security background, and the data lineage discussion, which is a genuinely useful angle most surveys skip. The table of 23 selected works is a handy quick reference even if coverage is selective.\n\nThe soft spots are real and they are in the survey's core content. The stress-test note is right: the SISA classification is wrong. Table 1 and Figure 6 present Bourtoule et al.'s SISA as federated learning, and the figure caption says it is \"presented in the form of federated learning.\" But SISA is sharded, isolated, sliced, and aggregated training on a single dataset; it does not involve multiple data owners or federated communication. That is a factual error about a representative exact-unlearning method, and a reader using this review as a map will carry away a wrong architectural picture. It is also inconsistent with Section 4.4.2, which describes SISA correctly. The O(d^2) statement in Section 4.5.2 is internally contradictory: the text says the cost is \"linearly related to the feature dimension d, i.e., O(d^2).\" Those are two different claims. The evaluation-metrics paragraph also refers to an \"initial model (in which the noise value σ = 0)\" that is never defined. These are not quibbles; they are errors in the paper's central contribution, which is the taxonomy itself.\n\nThat said, none of this is fatal. The architecture of the survey is coherent, the cited works are mostly correct, and the errors are localized and fixable. What the paper does well is organize a messy area for people who are not yet in it. What it does less well is present the methods accurately at the level of detail a careful reader needs.\n\nWho is it for? Someone entering machine unlearning who wants a first orientation. An expert will not learn much, and the errors mean even a newcomer should read with caution until they are fixed. I would accept this for peer review, but the reviewers should push for a revision that fixes the SISA misclassification, reconciles the complexity statement, and defines the evaluation metrics cleanly. With those corrections, it would be a decent survey. Without them, it is a source of misconceptions.","headline":"A useful but imperfect survey of machine unlearning that needs correcting before it can be trusted as a map of the field.","tokens_in":15222,"tokens_out":1774,"would_cite":false,"duration_ms":17818,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This review argues that machine unlearning is best organized as exact retraining versus fast approximate updates, with data lineage as the audit trail that makes deletion practical.","keywords":["machine unlearning","right to be forgotten","GDPR","privacy-preserving machine learning","data lineage","exact unlearning","approximate unlearning","certified removal"],"falsifier":"A peer-reviewed unlearning method that fits none of the four categories — exact retraining, Fisher-information retraining with noise, influence/Newton updates, or stored-batch rollback — would show the survey's organizing taxonomy is incomplete; zero-shot unlearning, which erases data without accessing the original training set, is a plausible candidate to test.","tokens_in":14308,"feed_emoji":"🧠","tokens_out":8534,"duration_ms":75456,"temperature":0.7,"pith_summary":"This review sets out to organize the field of machine unlearning, the practice of making a trained model behave as if specific training data had never been used. It argues that GDPR's Article 17 (the right to be forgotten) and security threats such as data poisoning make unlearning a practical necessity, while full retraining from scratch is too costly. The paper's central contribution is a classification of unlearning into exact methods, which retrain on the remaining data, and approximate methods, which adjust the model more cheaply. It further splits approximate unlearning into three families and connects the whole area to data lineage management as a tool for tracking data flows and supporting deletion. A reader comes away with a structured map of the main approaches and their trade-offs between speed, accuracy, and certifiability.","feed_headline":"Machine unlearning: exact retraining or fast approximation","feed_subtitle":"Review maps how models can erase user data under GDPR and why tracking data lineage makes unlearning auditable.","key_machinery":"The central object is the exact-versus-approximate taxonomy of unlearning, with approximate methods subdivided into three groups (Fisher-information-based retraining with noise, influence-function or Newton updates, and stored-batch parameter rollback). The taxonomy does the argument's work by giving every reviewed method a slot and by making the trade-off visible: exact methods are strongest in fidelity to true retraining and weakest in cost, while approximate methods invert that. A second load-bearing object is the data-lineage record, defined here as a running log of where each data point came from and how it was transformed during training; the paper argues this record is what makes unlearning auditable and helps defend against data-poisoning attacks. The SAPE-based metrics (effectiveness, certifiability, efficiency) supply a common yardstick for comparing updated models with fully retrained models.","core_discovery":"On the paper's own terms, machine unlearning is a security and privacy mechanism that lets a service provider respond to a data-holder's deletion request by producing a model that is as close as possible to one never trained on the deleted data. Exact unlearning achieves this by retraining on the dataset minus the removed points, while approximate unlearning updates the model or dataset directly and is the main focus of current research. Following the classification it adopts, the review groups approximate methods into three families: retraining the remaining data while injecting noise calibrated by the Fisher information matrix, taking a Newton step or influence-function update to remove the deleted data's effect, and storing training batches and their parameter updates so that a deletion can roll back the corresponding updates. It then proposes that effectiveness, certifiability, and efficiency, measured through Symmetric Absolute Percentage Error, form a common evaluation scheme, and argues that data lineage management should track every change so that unlearning and poisoning defense can operate on a complete record.","pith_inferences":["The taxonomy covers methods that operate on the trained model or training batches; unlearning approaches that never touch the original training data, such as zero-shot unlearning, would form a natural fourth category that the review leaves out.","The evaluation scheme takes full retraining as the gold standard, which makes exact unlearning the implicit target; a privacy-focused benchmark that measures residual membership-inference risk rather than closeness to retraining could rank the same methods very differently.","A practical extension the paper only hints at: data-lineage metadata could automatically supply the batch identifiers that stored-batch unlearning methods need, making the rollback step auditable and nearly free to execute.","Because approximate methods only make outputs close to a retrained model, they may still leak information through other channels; testing updated models against membership-inference attacks after each family of updates is a direct way to check whether forgetting is real."],"forward_implications":["For a small deletion request, methods that isolate training data into shards and retrain only the affected shard can answer quickly without retraining the whole model, while very large deletions may be cheaper by full retraining.","Approximate methods trade a formal closeness guarantee against the fully retrained model for lower time and computation cost; the three families differ mainly in what they retain from training to enable the update.","Using the SAPE-based effectiveness, certifiability, and efficiency metrics gives researchers a shared way to compare any unlearning algorithm against full retraining.","Data lineage, a record of data provenance and transformation during training, makes unlearning auditable and also helps detect data poisoning, so privacy protection and security defense can share the same tracking infrastructure.","Unlearning does not automatically restore privacy: the review reports that membership-inference attacks can still recover information from unlearned models, and retraining-from-scratch is more vulnerable to them than distributed or block-based approaches."],"supporting_citations":[{"why":"Formalizes unlearning as a game between service provider and users and supplies the block-based training strategy that makes exact unlearning cheaper.","marker":"[3]"},{"why":"Provides the three-group classification of approximate unlearning and the SAPE evaluation metrics that structure the review.","marker":"[20]"},{"why":"Introduces the first explicit machine unlearning method in summation form, the historical anchor of the survey.","marker":"[24]"},{"why":"Defines certified removal and the Newton-step update, the prototype for one approximate-unlearning family.","marker":"[21]"},{"why":"Contributes the influence-function deletion update and two evaluation tests (L2 distance, feature injection) used for approximate methods.","marker":"[36]"},{"why":"Proposes amnesiac deletion by rolling back stored batch parameter updates, exemplifying the third approximate family.","marker":"[6]"},{"why":"Shows unlearning methods can be attacked with membership inference, grounding the paper's discussion of privacy risks of unlearning.","marker":"[7]"},{"why":"Establishes data provenance as a defense against data poisoning, the basis for the data-lineage discussion.","marker":"[1]"}],"fun_headline_variants":["Machine unlearning: retrain or approximate the delete?","Right to be forgotten meets AI: unlearning review","Exact vs approximate: machine unlearning techniques","How models forget: a machine unlearning survey"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument depends on the assumption that the exact/approximate split with three approximate families covers the important work in the field; if a substantial line of unlearning research falls outside those four bins, the review's map and its conclusions about open challenges are incomplete.","fun_headline_variants_meta":{"raw":{"variants":["Machine unlearning: retrain or approximate the delete?","Right to be forgotten meets AI: unlearning review","Exact vs approximate: machine unlearning techniques","How models forget: a machine unlearning survey"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000333,"raw_usage":{"total_tokens":1849,"prompt_tokens":943,"completion_tokens":906,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":559,"completion_tokens_details":{"reasoning_tokens":844}},"tokens_in":559,"tokens_out":906,"duration_ms":9129,"temperature":1.0,"reasoning_tokens":844,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T18:39:04.777384+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A peer-reviewed unlearning method that fits none of the four categories — exact retraining, Fisher-information retraining with noise, influence/Newton updates, or stored-batch rollback — would show the survey's organizing taxonomy is incomplete; zero-shot unlearning, which erases data without accessing the original training set, is a plausible candidate to test.","supporting_citations":[{"cited_title":"In: 2015 IEEE Symposium on Security and Privacy, pp","cited_arxiv_id":null,"evidence_quote":"Introduces the first explicit machine unlearning method in summation form, the historical anchor of the survey."},{"cited_title":"In: Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pp","cited_arxiv_id":null,"evidence_quote":"Shows unlearning methods can be attacked with membership inference, grounding the paper's discussion of privacy risks of unlearning."},{"cited_title":"In: Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp","cited_arxiv_id":null,"evidence_quote":"Establishes data provenance as a defense against data poisoning, the basis for the data-lineage discussion."}],"review_version":1}