{"id":"8c10d6c5-0592-4292-8403-76035a0cfab5","arxiv_id":"2412.13790","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"ISPF combines a new generator loss that discourages synthesis of forgetting-class samples with a logit-redistribution step that lets every synthetic sample contribute to distillation, improving retained accuracy over GKT.","lead":"The paper proposes a new method, ISPF, to make data-free machine unlearning more efficient: removing a class from a trained image classifier without access to the original training data. It improves on prior work by both suppressing generation of to-be-forgotten samples and reusing all synthetic samples with modified teacher labels.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"PostFilter's logit redistribution removes direct forgetting-class supervision but not pixel-level forgetting features; ISPF's own ResNet18 AIN values indicate residual latent knowledge, so the unlearning guarantee is not fully established.","rationale":"The reader's weakest assumption identifies exactly the same load-bearing concern: PostFilter operates on teacher logits, not on the pixels of synthetic images, so images that the teacher assigns to the forgetting class may still carry forgetting-class visual features into the student. The paper's own ResNet18 AIN values (0.45 and 0.28 versus 1.0 for Retrain) are the strongest internal evidence that these features survive, because they show that the unlearned model can relearn the forgetting class much faster than a model trained without that class. This does not make the paper internally inconsistent, nor does it contradict the efficiency or retaining-accuracy results; it means the unlearning guarantee side of the central claim is weaker than the headline suggests. I agree with the reader's conditional verdict: the paper is a credible engineering contribution with code, ablations, and consistent improvements over GKT, but the unlearning guarantee needs stronger evidence. My concern does not move the verdict; it reinforces the conditionality already identified.","tokens_in":20981,"tokens_out":4674,"duration_ms":48298,"concrete_test":"Use the released code to train ISPF, GKT, Retrain, and Original on SVHN-AllCNN and CIFAR-10-ResNet18. Freeze the feature extractor (all layers up to the final linear layer) of each model, fit a linear logistic-regression probe to classify the forgetting class versus all other classes on a balanced subset of the real training data, and evaluate on the held-out forgetting test set. If ISPF's probe accuracy/AUC is close to Retrain's, PostFilter's logit surgery removes usable f features; if it is substantially higher (e.g., more than 10 points above Retrain), pixel-level f information leaks through the redistributed retaining logits and the AIN residuals are explained. Report probe results for each per-class unlearning setting in Table 9.","verdict_should_be":"UNCHANGED","load_bearing_attack":"ISPF's central claim (Abstract/Conclusion) is that it makes data-free unlearning both more efficient and more accurate on retaining classes while still achieving unlearning. The load-bearing step is PostFilter (Eq. 8-9): it sets all forgetting-class logits to min(t), spreads the deleted mass Delta uniformly over retaining classes, and trains the student with DKL(softmax(hat_t) || S(x)) (Eq. 10). This removes the direct logit signal for class f, but the input pixels of a synthetic image that the teacher classifies as f are still present, and ISPF deliberately uses all synthetic samples, including those. The student can therefore learn f-discriminative features in its penultimate layers while the output head assigns near-zero probability to f. Af=0 and MIA I=100% only test the output; they do not test feature-level leakage. The paper's own ResNet18 AIN results (0.45 for SVHN, 0.28 for CIFAR-10 in Table 1, versus 1.0 for Retrain) show that ISPF's unlearned models relearn f much faster than a retrained model, which is direct evidence that residual forgetting information survives. The claim that ISPF 'effectively tackles' data-free unlearning therefore rests on an assumption—that logit redistribution removes f information even for f-class images—that the paper does not validate.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ISPF, a data-free unlearning method built on data-free knowledge distillation. It argues that the existing GKT method is inefficient because its PreFilter over-filters synthetic samples, and proposes two components: Inhibited Synthesis (IS), which adds a generator loss term to suppress synthesis of forgetting-class samples, and PostFilter (PF), which keeps all synthetic samples but redistributes the teacher logits by setting forgetting-class logits to the minimum value and spreading the removed mass over retaining classes. The student is trained against the redistributed softmax target. Experiments on SVHN, CIFAR-10, and CIFAR-100 with AllCNN and ResNet18 report Af=0.0 in all settings and substantial gains in Ar over GKT (e.g., SVHN-AllCNN Ar from 55.23 to 92.68, CIFAR-10-AllCNN from 57.62 to 86.02), plus comparisons of MIA, AIN, wall-clock efficiency, and ablations of the two components.","tokens_in":21313,"tokens_out":2733,"duration_ms":27774,"significance":"If the central claim is correct, ISPF would be a clear improvement over the existing data-free unlearning method GKT, with a simple and reproducible recipe. The paper's strengths include extensive experiments (multiple datasets, architectures, per-class and multi-class forgetting, three seeds), public code, and a concrete diagnosis of GKT's inefficiency that is partially supported by the synthetic-sample statistics in Figures 2 and 3. However, the significance is limited by the fact that the load-bearing claim about the unlearning guarantee is not established: the PostFilter operation removes explicit forgetting-class logit signals but does not remove pixel-level forgetting features, and the paper's own AIN results on ResNet18 (0.45 on SVHN, 0.28 on CIFAR-10, versus 1.0 for Retrain) indicate residual forgetting information in the unlearned model. The theoretical analysis in Section 3.1 is informal and relies on an unproven assumption about the student's outputs. These issues do not invalidate the empirical efficiency gains, but they do require that the central claim be narrowed or the guarantee be verified with additional evidence.","major_comments":[{"comment":"The analysis of why DFKD leads to over-filtering rests on Eq. (6), which asserts that for every synthetic sample, the student's forgetting-class output satisfies 0 < S_f(x) < epsilon. This is assumed rather than derived, and it cannot hold early in training when the student is randomly initialized; at that stage the adversarial generator can push S_f high. The subsequent conclusion that maximizing Eq. (5) forces T_f up is therefore not a rigorous consequence. The paper should either provide a derivation with explicit conditions, or clearly label this as a heuristic motivation rather than a theoretical analysis. This matters because the claim of 'strictly analyzing' the inefficiency, stated in the Introduction and Conclusion, is load-bearing for the IS design.","section":"Section 3.1, Eq. (6)"},{"comment":"PostFilter removes direct logit supervision for the forgetting class, but the input pixels of a synthetic image that the teacher classifies as f still contain f-discriminative features, and ISPF deliberately trains on all such images. The student can therefore encode forgetting-class features in intermediate representations while producing near-zero output for class f. The reported Af=0.0 and MIA I=100% only test the output layer; they do not test feature-level leakage. The ResNet18 AIN values in Table 1 (0.45 for SVHN, 0.28 for CIFAR-10, versus 1.0 for Retrain) show that the ISPF-unlearned models relearn the forgetting class substantially faster than a retrained model, which is direct evidence that residual forgetting information survives. To support the unlearning guarantee, the paper should add an internal-representation or feature-level evaluation (e.g., linear probing on penultimate features, or comparison of neuron activations against the Retrain model), or state a precise condition under which the redistributed target is independent of forgetting-class information.","section":"Section 3.3, Eqs. (8)-(10), Table 1"},{"comment":"The IS loss is designed so that minimizing the added term reduces T_f given small S_f, but the behavior of the loss is not analyzed when S_f is not uniformly small, which is the regime early in training and for classes the generator has not yet mastered. The claim that IS 'reduces the synthesis of forgetting-class information' is partly a consequence of the objective construction, which is fine as a design, but the paper should not present the observed reduction in forgetting-class sample counts as an empirical discovery without also reporting the dynamics of T_f and S_f over training for the IS variant.","section":"Section 3.2, Eq. (7)"},{"comment":"The comparison includes only GKT as a data-free unlearning baseline (plus BlockF). This is acceptable if GKT is the only prior fully data-free method, but the paper should state this explicitly and note that the claimed advantage may not transfer to other DFKD foundations or to approximate-unlearning methods that use limited real data. Appendix H addresses one DFKD variant, which is helpful, but the main text should acknowledge this scope limitation.","section":"Section 4.2, Baselines"}],"minor_comments":[{"comment":"There is a typo, 'Unlearinng' in the Preliminaries heading; the phrase 'the the' also appears in the introduction of the filter concept. These should be corrected.","section":"Abstract and Section 1"},{"comment":"The footnote on the first page cites an extended version at arxiv.org/abs/0000.00000, which is a placeholder. This should be either filled in or removed before publication.","section":"Introduction, Extended version link"},{"comment":"The notation for the redistributed logits is not fully specified: after Eq. (9), the student is trained with DKL(softmax(hat_t) || S(x)), but it is not stated whether hat_t is also temperature-scaled or used directly as a soft target. Clarifying this would improve reproducibility.","section":"Section 3.3, Eq. (9)"},{"comment":"The fourth column of Figure 2 reports average batch entropy, but the text says 'information entropy of the retaining classes' while Eq. (11) sums over k not in Yf. This is consistent only if the entropy is computed over the teacher's retaining-class distribution; please state this explicitly in the caption and text.","section":"Section 4.3, Figure 2"},{"comment":"Table 9 shows large per-class variance for several methods, especially GKT. The main-text claim that ISPF is 'consistent across classes' would be better supported by reporting the per-class ranges or a statistical comparison, not just the averaged means in Table 1.","section":"Appendix D and Table 9"}],"recommendation":"major_revision","confidential_remarks":"The paper is an incremental but potentially useful contribution to data-free unlearning. My main concern is that the unlearning guarantee, which is the paper's central selling point, is not substantiated beyond output-level metrics. The ResNet18 AIN values in Table 1 are, in my reading, evidence of residual forgetting information; this is a fixable issue (add feature-level evaluation or soften the claim), so I am not recommending rejection. I would also urge the editor to check whether the authors have properly cited the prior work on data-free unlearning beyond GKT, as the related-work appendix appears thin for a subfield that has grown in the last two years."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about this paper. The core diagnosis is right: GKT's filtering strategy wastes a large fraction of synthetic data, and the paper demonstrates this clearly with synthesis counts and filtered-sample counts across training. The proposed fix—Inhibited Synthesis plus PostFilter—is a real engineering improvement, not a tweak. On SVHN-AllCNN, retained accuracy goes from 55.23 to 92.68 while maintaining Af at 0 and MIA I at 100%. The code is public, the ablations are honest, and the additional comparisons against ZSKT and a simpler logit-zeroing implementation strengthen the empirical case.\n\nThe main soft spot is the unlearning guarantee. The AIN values on ResNet18 tell a worrying story: ISPF gets 0.45 on SVHN and 0.28 on CIFAR-10, versus 1.0 for Retrain. That means the unlearned model relearns the forgetting class much faster than a retrained model, which is direct evidence that PostFilter removes the output-level signal but not all feature-level information. The paper acknowledges this but does not validate the stronger claim that logit redistribution fully eliminates forgetting information. I would want to see a feature-level leakage test, such as linear probing on the penultimate layer, before accepting the unlearning guarantee as fully established. The informal theory in Section 3.1 is another weak point: Eq. (6) is assumed rather than derived, so the analysis is more suggestive than rigorous.\n\nThe baseline set is thin—GKT is the only true data-free unlearning competitor—and the extended-version link is a placeholder. These are minor but should be fixed.\n\nWho is this for: researchers working on data-free unlearning or data-free knowledge distillation. It deserves a serious referee because the diagnosis is plausible, the techniques are novel, and the results are reproducible. My recommendation is to engage with it. A conditional accept feels right: the retained-accuracy improvement is substantial and worth publishing, but the unlearning guarantee needs stronger evidence and the theory should be tightened.","headline":"A credible engineering contribution that correctly diagnoses GKT's over-filtering and fixes it with two novel techniques, but the unlearning guarantee is only partially validated on ResNet18 and the theory is informal.","tokens_in":21812,"tokens_out":2250,"would_cite":true,"duration_ms":20358,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Two fixes lift data-free unlearning's retained accuracy.","keywords":["machine unlearning","data-free unlearning","knowledge distillation","synthetic data","logit redistribution","class forgetting","retaining accuracy","adversarial generator"],"falsifier":"Run GKT with a filter threshold relaxed to the point where almost no synthetic samples are discarded and compare retaining accuracy; if Ar does not rise toward the ISPF level, over-filtering is not the main cause of GKT's inefficiency. Separately, fine-tune an ISPF model on a small forgetting-class set and count steps to match the original model's forgetting accuracy: a relearning time much shorter than a retrained model's would show that the PostFilter's logit rewrite leaves latent forgetting information.","tokens_in":20801,"feed_emoji":"🔒","tokens_out":8467,"duration_ms":69642,"temperature":0.7,"pith_summary":"This paper argues that GKT, the existing data-free unlearning method, is inefficient because it throws away too many synthetic samples before distillation. The adversarial generator used in data-free knowledge distillation naturally drifts toward producing forgetting-class images as the student learns to suppress that class, and GKT's threshold filter discards those images along with many retaining-class images. The paper proposes ISPF, which combines an Inhibited Synthesis loss that stops the generator from over-producing forgetting-class images and a PostFilter that rewrites the teacher's logits so every synthetic image can be used in distillation. In experiments, ISPF keeps forgetting accuracy at 0.0 while raising retaining accuracy substantially—for example from 55.23 to 92.68 on SVHN with AllCNN, and from 57.62 to 86.02 on CIFAR-10 with AllCNN. If correct, the result shows that enriching retaining-related information in the distillation signal is the key to efficient data-free unlearning.","feed_headline":"Two fixes lift data-free unlearning's retained accuracy","feed_subtitle":"Rewriting teacher logits instead of filtering synthetic samples keeps forgetting at zero while retaining far more.","key_machinery":"The load-bearing mechanism is the teacher–student logit pipeline in adversarial data-free distillation, modified at both ends. Inhibited Synthesis (IS) changes the generator objective: instead of the standard adversarial loss, the generator minimizes Eq. 7, which keeps the positive retaining-class terms but flips the forgetting-class terms so they actively push the teacher's forgetting-class logits down, suppressing synthesis of forgetting-class images. PostFilter (PF) changes the teacher's supervision: for every synthetic sample, Eq. 8 sums the logit mass by which each forgetting-class output exceeds the batch minimum, and Eq. 9 distributes that total evenly among the retaining classes while setting forgetting-class logits to the minimum; the student then distills from the redistributed softmax target via Eq. 10. Together they ensure that all synthetic samples enter the distillation and that the distillation signal carries retaining-class information without an explicit forgetting-class component.","core_discovery":"The paper's central claim is that GKT's inefficiency stems from over-filtering: the generator increasingly synthesizes forgetting-class samples, and the PreFilter then discards them, which also removes retaining-class information from the distillation pool. The mechanism is analyzed from the adversarial DFKD objective: since the student's outputs on the forgetting class are near zero, maximizing the teacher–student divergence forces the teacher's forgetting-class logits up, driving the generator toward the forgetting distribution. To fix this, the paper introduces Inhibited Synthesis, which flips the sign of the forgetting-class terms in the generator loss (Eq. 7), and PostFilter, which redistributes the teacher's logits by moving the mass above the minimum from the forgetting classes to all retaining classes (Eqs. 8–9) before computing the student's KL loss (Eq. 10). With these two changes, ISPF reports Af = 0.0 across every setting it tests while improving retaining accuracy over GKT on all three datasets, including a multi-class CIFAR-100 setting where Ar rises from 49.86 to 62.58. The authors conclude that enriching retaining-related information in the distillation process significantly improves the student's acquisition of retaining-class knowledge.","pith_inferences":["Beyond the paper's experiments, the over-filtering diagnosis should apply to any data-free unlearning built on adversarial distillation, so the sign-flip generator loss could be grafted onto other DFKD generators beyond DFQ and ZSKT.","PostFilter's uniform redistribution is one choice among many; spreading the deleted logit mass proportionally to the retaining-class logits, or with a temperature, might better preserve the teacher's ranking and yield different Ar/AIN trade-offs that the paper does not explore.","The paper's own ResNet18 AIN values (0.28 on CIFAR-10, 0.45 on SVHN) are below the retrained model's 1.0, which leaves open whether a latent forgetting signal survives the logit rewrite; a relearning-speed probe or representation-level test after full fine-tuning would settle that."],"forward_implications":["Retaining accuracy after data-free unlearning can approach retrained-model levels while forgetting accuracy stays at 0.0; e.g., on SVHN-AllCNN Ar goes from 55.23 (GKT) to 92.68, and on CIFAR-10-AllCNN from 57.62 to 86.02.","Wall-clock time to reach a given Ar is reduced, because no synthetic samples are discarded and each distillation batch carries more retaining-class information, as shown by the higher batch entropy in Figure 2.","The two components are separable and complementary: Inhibited Synthesis alone suppresses forgetting-class synthesis, PostFilter alone raises usable retaining information, and their combination gives the best Ar in the ablation.","The approach transfers to multi-class unlearning: on CIFAR-100 with ten distributed forgetting classes, ISPF raises Ar to 62.58 versus 49.86 for GKT.","The method is not tied to one base distillation framework; replacing DFQ with ZSKT still yields large Ar gains over GKT (Appendix H)."],"supporting_citations":[{"why":"Introduces GKT, the PreFilter baseline this paper identifies as inefficient, and defines the AIN metric used for unlearning guarantees.","marker":"Chundawat et al. 2023"},{"why":"Supplies the adversarial-inversion DFKD objective that GKT builds on and that the paper's Eq. 4 analysis starts from; also used directly in Appendix H.","marker":"Micaelli and Storkey 2019"},{"why":"Provides DFQ, the representative DFKD method used as the fundamental distillation framework in the main experiments.","marker":"Choi et al. 2020"},{"why":"Provides the SVHN dataset used for the main accuracy and efficiency evaluations.","marker":"Netzer et al. 2011"},{"why":"Provides the CIFAR-10 and CIFAR-100 datasets used for single- and multi-class unlearning.","marker":"Krizhevsky, Hinton et al. 2009"},{"why":"Provides the AllCNN architecture used for the primary experiments.","marker":"Springenberg et al. 2015"},{"why":"Provides ResNet18, used to assess whether the method transfers to a different backbone.","marker":"He et al. 2016"}],"fun_headline_variants":["Flipping logit signs fixes data-free unlearning's over-filtering","Two simple fixes make data-free unlearning retain far more","Data-free unlearning: redistribute logits instead of discarding","New method cuts info loss in data-free unlearning","Logit redistribution keeps data-free unlearning at zero forgetting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that rewriting the teacher's logits—setting forgetting-class outputs to the minimum and spreading that mass evenly over retaining classes—removes all forgetting information from every synthetic image, even an image synthesized from the forgetting class, although the paper's own ResNet18 Anamnesis Index values below 1.0 suggest the removal may not be complete.","fun_headline_variants_meta":{"raw":{"variants":["Flipping logit signs fixes data-free unlearning's over-filtering","Two simple fixes make data-free unlearning retain far more","Data-free unlearning: redistribute logits instead of discarding","New method cuts info loss in data-free unlearning","Logit redistribution keeps data-free unlearning at zero forgetting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000263,"raw_usage":{"total_tokens":1573,"prompt_tokens":891,"completion_tokens":682,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":507,"completion_tokens_details":{"reasoning_tokens":611}},"tokens_in":507,"tokens_out":682,"duration_ms":7198,"temperature":1.0,"reasoning_tokens":611,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T12:49:40.342671+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run GKT with a filter threshold relaxed to the point where almost no synthetic samples are discarded and compare retaining accuracy; if Ar does not rise toward the ISPF level, over-filtering is not the main cause of GKT's inefficiency. Separately, fine-tune an ISPF model on a small forgetting-class set and count steps to match the original model's forgetting accuracy: a relearning time much shorter than a retrained model's would show that the PostFilter's logit rewrite leaves latent forgetting information.","supporting_citations":[{"cited_title":"S.; Tarun, A","cited_arxiv_id":null,"evidence_quote":"Introduces GKT, the PreFilter baseline this paper identifies as inefficient, and defines the AIN metric used for unlearning guarantees."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the adversarial-inversion DFKD objective that GKT builds on and that the paper's Eq. 4 analysis starts from; also used directly in Appendix H."},{"cited_title":"P.; El - Khamy, M.; and Lee, J","cited_arxiv_id":null,"evidence_quote":"Provides DFQ, the representative DFKD method used as the fundamental distillation framework in the main experiments."},{"cited_title":"Y.; et al","cited_arxiv_id":null,"evidence_quote":"Provides the SVHN dataset used for the main accuracy and efficiency evaluations."},{"cited_title":"T.; Dosovitskiy, A.; Brox, T.; and Riedmiller, M","cited_arxiv_id":null,"evidence_quote":"Provides the AllCNN architecture used for the primary experiments."}],"review_version":1}