Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

FUNU: Boosting Machine Unlearning Efficiency by Filtering Unnecessary Unlearning

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that many deletion requests are unnecessary and can be filtered by a similarity-to-neighbors test, shrinking unlearning cost without a distinguishable model.

desk verdict Clever filtering idea with honest experiments, but the formal privacy guarantee does not survive contact with the proof. read the letter →

arxiv 2501.16614 v1 pith:2DTE25T4 submitted 2025-01-28 cs.LG

Zitong Li , Qingqing Ye , Haibo Hu This is my paper · ORCID
classification cs.LG
keywords machineunlearningunnecessarydataremovalrequestfilteringprototypediscoverymodelmemorizationsimilaritythresholdSISAKLdivergencebound
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that many data removal requests can be skipped: if a data point has enough similar neighbors in the remaining dataset, retraining without it yields a model essentially indistinguishable from the model before unlearning. FUNU is the proposed filter: it selects a subset $D^+_u$ of the requested samples that can be left in, and unlearns only the rest, shrinking the workload of unlearning methods whose cost grows with request count. A theory bound (Theorem 3.1) says the KL divergence between the unlearned and fully retrained model on the filtered samples is at most $n[(\lambda_1+\lambda_2)(\sqrt{2-2\theta})+\delta]$, with $n=|D^+_u|$. Experiments on MNIST, CIFAR-10, and CIFAR-100 show the filter keeps model behavior close to retraining while reducing time in a SISA case study.

What carries the argument

The load-bearing machinery is a full pairwise distance matrix $M_d$ computed once in the feature space of the original model $M_o$, taken just before the final fully connected layer. A similarity condition $\mathrm{Sim}_{\theta,\alpha}(x,D)$ returns true when at least $\alpha$ samples in $D$ have a cosine score with $x$ above $\theta$. Parameters are set from a one-epoch reference model $M_{ref}$: $\theta_c$ is the average within-class score among samples $M_{ref}$ predicts correctly, $\theta$ is the average over classes, and $\alpha$ is the average count of pairs above $\theta$ within those correct sets. Filtering removes from $D_u$ any $x$ with $\mathrm{Sim}_{\theta,\alpha}(x,D_r^{c(x)})$ true, using only same-class remaining samples. The privacy proof maps each filtered $x$ to an injected similar neighbor $\mathrm{nei}(x)$ in $D_r$ and decomposes the KL divergence into a Lipschitz-bounded feature gap and a $\delta$-bounded output gap.

What would settle it

Retrain a model without the filtered samples, extract its pre-final-layer features on $D^+_u$, and compare them with the original model's features; if the average cosine distance is not near zero, the identity premise behind Theorem 3.1 is violated, and the bound need not hold. A direct check would also compute $\mathrm{KL}_{D^+_u}(p_u \| p_r)$ and compare it with the theorem's $\epsilon$.

Watch

Extended reading notes

Core claim

The central claim is that a large share of data removal requests are 'unnecessary' in a precise sense: removing them would not produce a model distinguishable from the retrained model, as measured by a readout distance. FUNU operationalizes this by classifying a requested sample as unnecessary when, in the original model's pre-classifier feature space, it has more than $\alpha$ same-class neighbors in the remaining dataset whose pairwise cosine score (the paper's sample distance, with larger values meaning more similar) exceeds $\theta$. The parameters $\theta$ and $\alpha$ are chosen automatically from a reference model trained for one epoch, avoiding manual thresholds. Theorem 3.1 then bounds the KL divergence between the output distributions of the post-unlearning model and the fully retrained model on the filtered samples by $n[(\lambda_1+\lambda_2)(\sqrt{2-2\theta})+\delta]$, where $n$ is the size of the filtered set, $\lambda_1$ and $\lambda_2$ are Lipschitz constants of the log-output maps, $\theta$ is the similarity threshold, and $\delta$ bounds the log-output gap on the remaining data. The paper reports that this reduces the proportion of requests still needing unlearning to about 0.44 in random removal, and that applying FUNU before SISA cuts unlearning time by about 24% while keeping accuracy within 2% of retraining on the remaining and test sets.

Load-bearing premise

The argument collapses if retraining or unlearning shifts the intermediate feature representations, because the proof assumes the original, retrained, and unlearned models share identical pre-final-layer features and measures all distances in the original model's feature space.

Editorial extensions

If this is right

  • For unlearning methods whose cost scales with the number of requests, such as SISA, filtering with FUNU reduces the number of slices that must be retrained; the paper measures a 31% drop in influenced slices and a 24% drop in time.
  • FUNU's request-reduction ratio adapts to the scenario: in class removal it keeps more requests for unlearning than in random removal, because same-class neighbors of a requested sample may themselves be deleted.
  • Because it reduces the number of requests, FUNU can extend the effective deletion capacity of model-shifting unlearning methods that cap the number of removals under a privacy guarantee.
  • The model produced after filtering is experimentally closer to the retrained model than a Certified Removal baseline, measured by output accuracy and MIA accuracy/F1 on the requested samples.
  • The automatic threshold selection removes the manual tuning burden that makes prototype-discovery baselines sensitive to parameters in this setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorially, the same 'similar neighbor in the remaining set' test could be reused as a cheap pre-screen for influence-based data valuation, flagging points whose deletion would change an influence estimate below noise.
  • Editorially, because the bound lives in $M_o$'s feature space, a practical robustness check would run the similarity condition in $M_r$'s feature space as well; if the selected $D^+_u$ changes substantially, the guarantee is fragile.
  • Editorially, the unnecessary-unlearning definition is readout-specific; a point may be skippable under KL on outputs yet still alter membership-inference or attribute-inference risk, so deployment should choose readouts matching the privacy threat.
  • Editorially, the one-epoch reference model is a memorization heuristic; a testable extension is to replace it with a small held-out validation set to set $\theta$ and $\alpha$ without any training on $D_o$.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces FUNU, a method for filtering data removal requests in machine unlearning so that samples whose deletion would not noticeably change the model can be skipped, thereby reducing unlearning time for methods whose cost scales with the number of requests. The method uses features from the original model to build a distance matrix, trains a one-epoch reference model to set similarity thresholds without manual tuning, and selects a subset D+_u of removal requests that have sufficiently many similar neighbors in the remaining dataset. The authors claim that unlearning only the remaining requests D-_u yields a model within a KL-divergence bound of the fully retrained model, and they provide a proof of this bound in Theorem 3.1. Experiments on MNIST, CIFAR-10, and CIFAR-100, including a case study with SISA, show moderate efficiency gains and model similarity.

Significance. The problem addressed is practically relevant: reducing the number of unlearning requests can benefit request-dependent unlearning methods such as SISA and extend the deletion capacity of parameter-scrubbing methods. The paper contains useful ideas, notably the explicit notion of 'unnecessary unlearning', the use of a reference model to avoid manual threshold selection, and the adaptation to both random and class removal scenarios. However, the central theoretical contribution, the privacy guarantee in Theorem 3.1, is not established: the proof in Appendix A contains invalid inequalities and conflates probability spaces. The experimental evaluation does not directly validate the claimed bound. If the theoretical claim were repaired, the paper could make a meaningful contribution; as it stands, the main load-bearing result is unsupported.

major comments (4)
  1. [Appendix A, Eqs. (10)-(13) and (20)-(21)] The proof of Theorem 3.1 uses the step M_u(x) * (log M_r(nei(x)) - log M_r(x)) <= log M_r(nei(x)) - log M_r(x), justified by M_u(x) <= 1. This inequality is valid only when the log-ratio is nonnegative; for negative log-ratios the inequality reverses. Since the summands can be negative, the subsequent bound n * lambda_1 * sqrt(2 - 2*theta) does not follow. The identical invalid step is used again in bounding the delta term at Eqs. (20)-(21). Thus the theorem is not proven even under the stated assumptions.
  2. [Appendix A, Eq. (5) and surrounding text] The quantity KL_{D_+^u}(p_u || p_r) is treated as E_{D_+^u}[p_u log(p_u/p_r)], where p_u and p_r are scalar model outputs for individual samples. Standard KL divergence is defined over output distributions over classes for a fixed input, or as an average over a distribution of inputs. The proof and theorem statement conflate these, making the claimed epsilon-unnecessary-unlearning bound ill-defined. A precise definition of the divergence used is required before the bound can be evaluated.
  3. [Section 3.4] The assumption that the features produced by M_o are identical to those produced by M_r and M_u (because M_o's training set covers the other two) is not justified. Retraining or unlearning changes model parameters and typically shifts intermediate representations; the proof's Lipschitz and cosine-distance arguments are carried out in M_o's feature space, so the bound does not apply to the actual models. This is a load-bearing premise for Theorem 3.1 and needs at least empirical validation or a relaxation that accounts for feature shift.
  4. [Section 4] The experiments do not check the inequality in Theorem 3.1. They report accuracy and MIA-based similarity between M_u and M_r, which are not quantitative estimates of the KL divergence bound. Moreover, the paper does not report values or estimates of lambda_1, lambda_2, delta, or the effective theta, so the tightness or non-vacuousness of the bound is never assessed. Consequently, even if the proof were correct, the experimental section would provide only indirect support for the central claim.
minor comments (5)
  1. [Section 3.1] The definition of sample_dist is confusing: the text says 'the larger the cosine distance, the more similar the two samples are,' but the standard convention is that cosine distance = 1 - cosine similarity, so a larger value means less similar. Please clarify the definition or rename the quantity.
  2. [Section 3.2, Eqs. (1)-(4)] The definitions of theta_c and alpha_c involve distances among all pairs in D_ref^c; it is not stated whether pairs (i,i) are included, and alpha_c counts pairs, which are O(|D|^2) quantities, while later alpha is used as a threshold on the number of neighbor samples. The relationship between these two notions should be spelled out.
  3. [Section 4.2.2] The sentence 'the absolute online timing of FUNU is around four seconds, which is still acceptable' is vague; it should specify the dataset or configuration for which this timing was measured.
  4. [Appendix A, Theorem A.1 header] The appendix labels the theorem as 'Theorem 1.1' in the heading while it is A.1 in the main text; the numbering is inconsistent.
  5. [Throughout] There are several typos and grammatical artifacts, e.g., 'we start with an original model' should be 'We start', and 'the belong to the same class' in Section 3.3. A careful proofreading pass is recommended.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: FUNU's filtering thresholds come from a one-epoch reference model, not from the target KL divergence; the theorem's assumptions are stated conditions rather than fitted predictions.

full rationale

The derivation chain is not circular. FUNU's similarity thresholds θ and α are estimated from a one-epoch reference model by averaging pairwise distances and neighbor counts among correctly predicted samples (Section 3.2), not by fitting to the KL divergence or to the final model distance, so the selection of D+u is an independent heuristic. The claimed guarantee in Theorem 3.1 is a conditional statement: given Lipschitz constants, a cosine threshold θ, and a δ bound on log-output difference on Dr, it bounds KL on D+u. The δ premise does partially presuppose that Mu and Mr are close on Dr, and the 'identical features' assumption in Section 3.4 is strong, but these are explicit modeling assumptions rather than quantities fitted to the target quantity; the proof attempts an independent Lipschitz bridging from Dr-neighbors to D+u. The self-citations to the authors' prior work (e.g., references [2], [31], [47]) appear in related-work context and are not load-bearing for Theorem 3.1. Separately, the Appendix A proof contains algebraic errors—M_u(x) ≤ 1 does not imply each KL term is bounded by the unweighted log-ratio when that ratio is negative, and KL over data samples is conflated with KL over class distributions—but those are proof-validity concerns, not circularity. Overall, the central filtering method is not circular.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on four domain assumptions, three data-derived or hand-chosen parameters, and a theorem whose proof is not valid. The most consequential assumptions are feature equality across differently trained models and the unmeasured delta bound. No new physical or algorithmic entities are postulated beyond the filtering concept itself.

free parameters (3)
  • theta (similarity threshold) = Data-derived, avg_c avg pairwise cosine among correctly predicted reference samples, Eq. (1)-(2)
    Controls how similar a neighbor must be for a sample to be filtered. It is not manually tuned, but there is no analysis connecting this statistic to the epsilon-unnecessary guarantee.
  • alpha (neighbor count threshold) = Data-derived, avg_c count of pairs above theta, Eq. (3)-(4)
    Controls how many similar neighbors are required. Also data-derived, but no theoretical link to epsilon or to the KL bound is established.
  • reference model training epochs = 1
    The reference model is trained for exactly one epoch by manual design. The choice is motivated by efficiency and memorization literature, but sensitivity to this choice is not studied.
assumptions (5)
  • domain assumption Features produced by M_o, M_r, and M_u are identical before the final FC layer because M_o's training set contains the others' training sets.
    Invoked in Section 3.4 to justify bounding logits of M_r and M_u in M_o's feature space. No evidence is given, and retraining or unlearning typically changes representations.
  • domain assumption The logarithms of outputs of M_u and M_r are uniformly close on D_r, i.e., ||log M_r(x) - log M_u(x)|| <= delta, with delta small.
    Section 3.4. delta is never estimated and is essential to the KL bound in Theorem 3.1.
  • ad hoc to paper A sample correctly predicted by a one-epoch reference model has sufficient similar neighbors in the dataset.
    Section 3.2. This inference from model memorization literature is the basis for setting theta and alpha, but it is not tested directly.
  • domain assumption Model output distributions on different samples are independent, so the readout KL can be summed over D+_u.
    Section 3.4, following references [22,38,43]. Used implicitly in the expectation and summation steps of Appendix A.
  • domain assumption The logarithms of final FC layer outputs are lambda_1- and lambda_2-Lipschitz in the feature space.
    Assumed in Theorem 3.1. No values are provided, and the proof uses this assumption to convert feature distance into output distance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FUNU: Boosting Machine Unlearning Efficiency by Filtering Unnecessary Unlearning." pith.science (2026). https://pith.science/paper/2DTE25T4

@misc{pith2026250116614,
  author       = {Pith},
  title        = {Pith review of: FUNU: Boosting Machine Unlearning Efficiency by Filtering Unnecessary Unlearning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2DTE25T4}},
  note         = {Machine review of arXiv:2501.16614}
}
read the original abstract

Machine unlearning is an emerging field that selectively removes specific data samples from a trained model. This capability is crucial for addressing privacy concerns, complying with data protection regulations, and correcting errors or biases introduced by certain data. Unlike traditional machine learning, where models are typically static once trained, machine unlearning facilitates dynamic updates that enable the model to ``forget'' information without requiring complete retraining from scratch. There are various machine unlearning methods, some of which are more time-efficient when data removal requests are fewer. To decrease the execution time of such machine unlearning methods, we aim to reduce the size of data removal requests based on the fundamental assumption that the removal of certain data would not result in a distinguishable retrained model. We first propose the concept of unnecessary unlearning, which indicates that the model would not alter noticeably after removing some data points. Subsequently, we review existing solutions that can be used to solve our problem. We highlight their limitations in adaptability to different unlearning scenarios and their reliance on manually selected parameters. We consequently put forward FUNU, a method to identify data points that lead to unnecessary unlearning. FUNU circumvents the limitations of existing solutions. The idea is to discover data points within the removal requests that have similar neighbors in the remaining dataset. We utilize a reference model to set parameters for finding neighbors, inspired from the area of model memorization. We provide a theoretical analysis of the privacy guarantee offered by FUNU and conduct extensive experiments to validate its efficacy.

Figures

Figures reproduced from arXiv: 2501.16614 by the authors.

Figure 1
Figure 1. Example of unnecessary unlearning In this work, we aim to detect those samples in data removal requests that would result in unnecessary unlearning. Existing works [10, 17, 42] on data prototype discovery, as reviewed in Sec￾tion 2.2.1, can be applied to our problem. However, these approaches arXiv:2501.16614v1 [cs.LG] 28 Jan 2025 [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Existing solutions and our method in class removal [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The procedure of FUNU 3.1 Calculate distance matrix The distance matrix, denoted as 𝑀𝑑 , is a matrix whose dimension equals the size of the original dataset, i.e., 𝑀𝑑 ∈ R 𝑁 ×𝑁 , where 𝑁 = |𝐷𝑜 |. Each cell of 𝑀𝑑 represents a distance between two samples, such that 𝑀𝑑 𝑖,𝑗 = 𝑠𝑎𝑚𝑝𝑙𝑒_𝑑𝑖𝑠𝑡(𝑥𝑖 , 𝑥𝑗), where 𝑥𝑖 , 𝑥𝑗 ∈ 𝐷𝑜 . We now introduce the function above 𝑠𝑎𝑚𝑝𝑙𝑒_𝑑𝑖𝑠𝑡 in detail. We first use the original model 𝑀𝑜 to genera… view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: 𝑃 − of different methods In [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 4
Figure 4. Figure 4: Timing of FUNU and existing solutions 4.2.2 Time cost. We calculate the average execution time of these methods under varying numbers of data removal requests in the random removal scenario and present the results in [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 6
Figure 6. Figure 6: Timing of applying FUNU to SISA our method could lead to a model similar to the retrained model, consequently preserving model privacy. 5 Related work Related research fields to this work are machine unlearning and data prototype discovery. Machine unlearning. Machine …

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. When unlearning is free: leveraging low influence points to reduce computational costs

    cs.LG 2025-12 conditional novelty 5.0 of 10

    Low-influence training points can be dropped from forget/retain sets before unlearning, cutting runtime up to ~50% with little measured loss in accuracy or MIA-based privacy.

Reference graph

Works this paper leans on

52 extracted references · 37 canonical work pages · cited by 1 Pith paper

  1. [1]

    Chirag Agarwal, Daniel D’souza, and Sara Hooker. 2022. Estimating example difficulty using variance of gradients. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 10368–10378

    work page 2022

  2. [2]

    Li Bai, Haibo Hu, Qingqing Ye, Haoyang Li, Leixia Wang, and Jianliang Xu

    work page

  3. [3]

    Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hen- grui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. 2021. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP) . IEEE, 141–159

    2021

  4. [4]

    PRESTON BUKATY. 2019. The California Consumer Privacy Act (CCPA): An implementation guide. IT Governance Publishing. http://www.jstor.org/stable/j. ctvjghvnn

    work page 2019

  5. [5]

    Ricardo JGB Campello, Davoud Moulavi, Arthur Zimek, and Jörg Sander. 2015. Hierarchical density estimates for data clustering, visualization, and outlier de- tection. ACM Transactions on Knowledge Discovery from Data (TKDD) 10, 1 (2015), 1–51

    work page 2015

  6. [6]

    Ricardo J. G. B. Campello, Davoud Moulavi, Arthur Zimek, and Jörg Sander. 2015. Hierarchical Density Estimates for Data Clustering, Visualization, and Outlier Detection. ACM Transactions on Knowledge Discovery from Data (TKDD) 10 (2015), 1 – 51. https://api.semanticscholar.org/CorpusID:2887636

    work page 2015

  7. [7]

    Yinzhi Cao and Junfeng Yang. 2015. Towards making systems forget with machine unlearning. In 2015 IEEE symposium on security and privacy . IEEE, 463–480

    work page 2015

  8. [8]

    Yinzhi Cao, Alexander Fangxiao Yu, Andrew Aday, Eric Stahl, Jon Merwine, and Junfeng Yang. 2018. Efficient repair of polluted machine learning systems via causal unlearning. In Proceedings of the 2018 on Asia conference on computer and communications security. 735–747

    work page 2018

Show all 52 references
  1. [9]

    Nicholas Carlini, Ulfar Erlingsson, and Nicolas Papernot. 2019. Distribution density, tails, and outliers in machine learning: Metrics and applications. arXiv preprint arXiv:1910.13427 (2019)

    2019 arXiv

  2. [10]

    Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. [n. d.]. Quantifying Memorization Across Neural Language Models. In The Eleventh International Conference on Learning Represen- tations

  3. [11]

    Terzis, Florian Tramèr, and Chiyuan Zhang

    Nicholas Carlini, Matthew Jagielski, Nicolas Papernot, A. Terzis, Florian Tramèr, and Chiyuan Zhang. 2022. The Privacy Onion Effect: Memorization is Rela- tive. ArXiv abs/2206.10469 (2022). https://api.semanticscholar.org/CorpusID: 249890361

    2022 arXiv

  4. [12]

    Min Chen, Zhikun Zhang, Tianhao Wang, Michael Backes, Mathias Humbert, and Yang Zhang. 2022. Graph unlearning. In Proceedings of the 2022 ACM SIGSAC conference on computer and communications security . 499–513

    2022

  5. [13]

    European Parliament and Council of the European Union. [n. d.]. Regulation (EU) 2016/679 of the European Parliament and of the Council . https://data.europa.eu/ eli/reg/2016/679/oj

    2016

  6. [14]

    Vitaly Feldman. 2020. Does learning require memorization? a short tale about a long tail. In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing. 954–959

    2020

  7. [15]

    Vitaly Feldman and Chiyuan Zhang. 2020. What neural networks memorize and why: Discovering the long tail via influence estimation. Advances in Neural Information Processing Systems 33 (2020), 2881–2891

    2020

  8. [16]

    Felps, Amelia D

    Daniel L. Felps, Amelia D. Schwickerath, Joyce D. Williams, Trung N. Vuong, Alan Briggs, M. Hunt, Evan Sakmar, David D. Saranchak, and Tyler Shumaker. 2020. Class Clown: Data Redaction in Machine Unlearning at Enterprise Scale. ArXiv abs/2012.04699 (2020). https://api.semantic...

    2020 arXiv

  9. [17]

    Isha Garg, Deepak Ravikumar, and Kaushik Roy. [n. d.]. Memorization Through the Lens of Curvature of Loss Function Around Samples. In Forty-first Interna- tional Conference on Machine Learning

  10. [18]

    Aditya Golatkar, Alessandro Achille, Avinash Ravichandran, Marzia Polito, and Stefano Soatto. 2021. Mixed-privacy forgetting in deep networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 792–801

    2021

  11. [19]

    Aditya Golatkar, Alessandro Achille, and Stefano Soatto. 2020. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 9304–9312

    2020

  12. [20]

    Aditya Golatkar, Alessandro Achille, and Stefano Soatto. 2020. Forgetting outside the box: Scrubbing deep networks of information accessible from input-output observations. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Pa...

    2020

  13. [21]

    Laura Graves, Vineel Nagisetty, and Vijay Ganesh. 2020. Amnesiac Ma- chine Learning. In AAAI Conference on Artificial Intelligence . https://api. semanticscholar.org/CorpusID:224817947

    2020

  14. [22]

    Chuan Guo, Tom Goldstein, Awni Hannun, and Laurens Van Der Maaten. 2020. Certified data removal from machine learning models. In Proceedings of the 37th International Conference on Machine Learning . 3832–3842

    2020

  15. [23]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition . 770–778

    2016

  16. [24]

    Yingzhe He, Guozhu Meng, Kai Chen, Jinwen He, and Xingbo Hu. 2021. Deep- obliviate: a powerful charm for erasing data residual memory in deep neural networks. arXiv preprint arXiv:2105.06209 (2021)

    2021 arXiv

  17. [25]

    Tyler B Johnson and Carlos Guestrin. 2018. Training Deep Models Faster with Robust, Approximate Importance Sampling. In Advances in Neu- ral Information Processing Systems , S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.), Vol. 31. Curr...

    2018

  18. [26]

    Angelos Katharopoulos and François Fleuret. 2018. Not All Samples Are Created Equal: Deep Learning with Importance Sampling. In International Conference on Machine Learning. https://api.semanticscholar.org/CorpusID:3663876

    2018

  19. [27]

    Been Kim, Rajiv Khanna, and Oluwasanmi O Koyejo. 2016. Examples are not enough, learn to criticize! criticism for interpretability. Advances in neural information processing systems 29 (2016)

    2016

  20. [28]

    Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning multiple layers of features from tiny images. (2009)

    2009

  21. [29]

    Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient- based learning applied to document recognition. Proc. IEEE 86, 11 (1998), 2278– 2324

    1998

  22. [30]

    Oscar Li, Hao Liu, Chaofan Chen, and Cynthia Rudin. 2018. Deep learning for case-based reasoning through prototypes: A neural network that explains its predictions. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 32

    2018

  23. [31]

    Ronghua Li, Haibo Hu, and Qingqing Ye. 2024. RFTrack: Stealthy Location Inference and Tracking Attack on Wi-Fi Devices.IEEE Transactions on Information Forensics and Security 19 (2024), 5925–5939. https://doi.org/10.1109/TIFS.2024. 3404810

    2024 doi

  24. [32]

    Yugeng Liu, Rui Wen, Xinlei He, Ahmed Salem, Zhikun Zhang, Michael Backes, Emiliano De Cristofaro, Mario Fritz, and Yang Zhang. 2022. ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models. In USENIX Security Symposium (USENIX Security) . USEN...

    2022

  25. [33]

    Pratyush Maini, Saurabh Garg, Zachary Lipton, and J Zico Kolter. 2022. Charac- terizing datapoints via second-split forgetting. Advances in Neural Information Processing Systems 35 (2022), 30044–30057

    2022

  26. [34]

    Ronak Mehta, Sourav Pal, Vikas Singh, and Sathya N. Ravi. 2022. Deep Un- learning via Randomized Conditionally Independent Hessians. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 10412–10421. https://doi.org/10.1109/CVPR52688.2022.01017

    2022

  27. [35]

    Geoff Pleiss, Tianyi Zhang, Ethan Elenberg, and Kilian Q Weinberger. 2020. Identifying Mislabeled Data using the Area Under the Margin Ranking. In Advances in Neural Information Processing Systems , H. Larochelle, M. Ran- zato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. ...

    2020

  28. [36]

    Deepak Ravikumar, Efstathia Soufleri, Abolfazl Hashemi, and Kaushik Roy. 2024. Unveiling Privacy, Memorization, and Input Curvature Links. In Forty-first In- ternational Conference on Machine Learning . https://openreview.net/forum?id= 4dxR7awO5n

    2024

  29. [37]

    Sebastian Schelter, Stefan Grafberger, and Ted Dunning. 2021. Hedgecut: Main- taining randomised trees for low-latency machine unlearning. In Proceedings of the 2021 International Conference on Management of Data . 1545–1557

    2021

  30. [38]

    Ayush Sekhari, Jayadev Acharya, Gautam Kamath, and Ananda Theertha Suresh

  31. [39]

    Pierre Stock and Moustapha Cisse. 2018. Convnets and imagenet beyond accuracy: Understanding mistakes and uncovering biases. In Proceedings of the European conference on computer vision (ECCV) . 498–512

    2018

  32. [40]

    Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, 11 (2008)

    2008

  33. [41]

    Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y Zhao. 2019. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In 2019 IEEE symposium on security and privacy (SP) . IEEE, 707–723

    2019

  34. [42]

    Jiaheng Wei, Yanjun Zhang, Leo Yu Zhang, Ming Ding, Chao Chen, Kok-Leong Ong, Jun Zhang, and Yang Xiang. 2024. Memorization in deep learning: A survey. arXiv preprint arXiv:2406.03880 (2024)

    2024 arXiv

  35. [43]

    Jiancan Wu, Yi Yang, Yuchun Qian, Yongduo Sui, Xiang Wang, and Xiangnan He. 2023. Gif: A general graph unlearning strategy via influence function. In Proceedings of the ACM Web Conference 2023 . 651–661

    2023

  36. [44]

    Yinjun Wu, Edgar Dobriban, and Susan Davidson. 2020. Deltagrad: Rapid retrain- ing of machine learning models. In International Conference on Machine Learning . PMLR, 10355–10366

    2020

  37. [45]

    Huan Xu and Shie Mannor. 2010. Robustness and generalization. Machine Learn- ing 86 (2010), 391 – 423. https://api.semanticscholar.org/CorpusID:254739858

    2010

  38. [46]

    Heng Xu, Tianqing Zhu, Lefeng Zhang, Wanlei Zhou, and Philip S. Yu. 2023. Machine Unlearning: A Survey. ACM Comput. Surv. 56, 1, Article 9 (aug 2023), WWW ’25, April 28-May 2, 2025, Sydney, NSW, Australia Zitong Li, Qingqing Ye, and Haibo Hu 36 pages. https://doi.org/10.1145/3603620

    2023 doi

  39. [47]

    Hongyang Yan, Anli Yan, Li Hu, Jiaming Liang, and Haibo Hu. 2024. MTL-Leak: Privacy Risk Assessment in Multi-Task Learning. IEEE Trans. Dependable Secur. Comput. 21, 1 (Jan. 2024), 204–215. https://doi.org/10.1109/TDSC.2023.3247869

    2024

  40. [48]

    Chih-Kuan Yeh, Joon Kim, Ian En-Hsu Yen, and Pradeep K Ravikumar

  41. [49]

    curvature

    Binchi Zhang, Zihan Chen, Cong Shen, and Jundong Li. 2024. Verification of Machine Unlearning is Fragile. In Proceedings of the 41st International Confer- ence on Machine Learning (Proceedings of Machine Learning Research, Vol. 235) , Ruslan Salakhutdinov, Zico Kolter, Katheri...

    2024

  42. [2018]

    In Advances in Neural Information Processing Systems , S

    Representer Point Selection for Explaining Deep Neural Networks. In Advances in Neural Information Processing Systems , S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.), Vol. 31. Cur- ran Associates, Inc. https://proceedings.neurips.cc/p...

    2018

  43. [2021]

    Advances in Neural Information Processing Systems 34 (2021), 18075–18086

    Remember what you want to forget: Algorithms for machine unlearning. Advances in Neural Information Processing Systems 34 (2021), 18075–18086

    2021

  44. [2024]

    ACM Comput

    Membership Inference Attacks and Defenses in Federated Learning: A Survey. ACM Comput. Surv. 57, 4, Article 89 (Dec. 2024), 35 pages. https: //doi.org/10.1145/3704633

    2024 doi

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.