Pith. sign in

REVIEW 4 major objections 4 minor 60 references

Machine Unlearning for Streaming Forgetting

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that sequential deletion requests can be handled as a distribution-shift estimation problem, and that the resulting SAFE algorithm achieves an O(√T + V_T) unlearning regret bound without convexity or access to the…

desk verdict The SAFE algorithm is a creative take on streaming unlearning, but the advertised O(sqrt(T)+V_T) regret bound is false as stated; the fixed-variance Gaussian perturbation alone forces linear regret even for convex quadratics. read the letter →

arxiv 2507.15280 v1 pith:I4GBO2KL submitted 2025-07-21 cs.LG

Shaofei Shen , Chenhao Zhang , Yawen Zhao , Alina Bialkowski , Weitong Chen , Miao Xu This is my paper · ORCID
classification cs.LG
keywords machineunlearningstreamingforgettingdynamicregretdistributionshiftdatadeletionapproximateonlinelearningGaussiandensityratio
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper addresses machine unlearning when deletion requests arrive one at a time over a long sequence, rather than as one batch. It proposes SAFE, which treats each round of forgetting as a distribution-shift problem: the unknown retrained model is approximated by reweighting the original model's predictions with label ratios and class-conditional Gaussian density ratios estimated on randomly projected data. All updates are single gradient steps from the original model, so the original training set is never accessed again after initialization. The paper's central claim is that this procedure has unlearning regret $O(\sqrt{T} + V_T)$ over $T$ requests, where $V_T$ measures how much the ideal retrained model drifts, and that this holds without assuming the loss is convex. If correct, this would make continuous data deletion practical for deep models at roughly constant per-request cost.

What carries the argument

The central object is the streaming risk estimator $R_t(w)$ from Eq. (10)/(13): the loss on remaining data plus a KL-divergence penalty pulling the model's predictions toward the estimated unlearned target. Its optimization is carried by the posterior-shift approximation $q_t^{(y)}(x) = \frac{n_t(y)}{n_0(y)} \frac{|D_0|}{|D_t|} \frac{\mathcal{N}(z\mid \mu_t^{(y)}, \Sigma_t^{(y)})}{\mathcal{N}(z\mid \mu_0^{(y)}, \Sigma_0^{(y)})}$, which uses Bayes' theorem to express the class-conditional distribution after deletion as a ratio of Gaussian densities in a random low-dimensional projection. This ratio turns the unobservable retrained model into a computable reweighting of the original model's outputs, enabling single-step gradient updates from $w_0$ and recursive mean/covariance updates (Eqs. 22 and 23) that never revisit $D_0$. The Gaussian perturbation $b_t$ is the mechanism that yields the $(\epsilon,\delta)$-approximate unlearning guarantee via the same argument as the Gaussian mechanism in differential privacy.

What would settle it

Run SAFE on a non-convex network over $T$ rounds with the stated learning rate and measure the average per-round risk gap by retraining a reference model each round; if the gap does not remain $O(\sqrt{T})$ for a deletion sequence with $V_T=0$, or if the total regret grows superlinearly, then the unstated convexity or smoothness step is doing the work.

Watch

Extended reading notes

Core claim

On the paper's own terms, SAFE is the first streaming unlearning method with a dynamic regret guarantee that does not require convexity or access to the original training data. The key move is replacing the per-step ideal model $w^*_t$ (the model retrained on $D_t$) by an explicit reweighting of the original model's predictive distribution: $f(x; w^*_t) \approx q_t^{(y)}(x) f(x; w_0)$, with $q_t^{(y)}(x)$ equal to the label proportion ratio times the ratio of two Gaussian densities fitted to randomly projected class-conditional features. The risk estimator $R_t(w)$ combines a retention term on remaining data and a forgetting term measuring KL divergence to this reweighted target; both can be updated recursively from the deletion sets alone. The main theorem states that under bounded gradients and bounded parameters, with learning rate $\gamma = \sqrt{W}/(K\sqrt{T})$ and Gaussian perturbation of variance $\phi_t$, the expected per-step gap $\mathbb{E}[R_t(w_t) - R_t(w^*_t)]$ is $O(\sqrt{T})$ and the accumulated regret over $T$ rounds is $O(\sqrt{T} + V_T)$, with $V_T = \sum_t \|w^*_t - w^*_{t-1}\|_2$. The paper also claims each step satisfies $(\epsilon,\delta)$-approximate unlearning.

Load-bearing premise

The load-bearing premise is that the per-step risk gap can be bounded by an inner product of the gradient with the parameter error—an inequality the proof uses without stating and that only holds for convex or smooth losses—together with the assumption that the original model already equals the true class-conditional distribution.

Editorial extensions

If this is right

  • Each deletion request costs computation proportional to $|F_t|$, not to the size of the original dataset, after one initial gradient computation.
  • The original training data does not need to be stored or accessed during unlearning; only the deletion sets and per-class Gaussian statistics are kept.
  • Across $T$ streaming requests, the accumulated quality gap versus retraining grows like $O(\sqrt{T} + V_T)$, so long unlearning sequences remain under control when the ideal model drifts slowly.
  • Every round is $(\epsilon,\delta)$-approximately unlearned, giving a formal privacy-like guarantee per deletion step.
  • On the evaluated image, text, and tabular benchmarks, the method matches or beats batch and streaming baselines on remaining, forgetting, and test accuracy, and is the fastest on the image datasets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the regret bound survives scrutiny, the same distribution-shift estimator could be adapted to incremental learning, where data arrives rather than being deleted, by reversing the sign of the recursive updates.
  • An implicit testable consequence is that SAFE's regret degrades gracefully only when the retrained models change slowly, so deletion orders that target rare classes should be harder than random-subset deletions because they move the class-conditional Gaussians more.
  • The Gaussianity of the random projection is checked empirically with a multivariate normality test; a stress test on a dataset whose latent projections are visibly non-Gaussian would show whether the approximation error bound in the first theorem is what actually drives performance.
  • The paper does not compare against storing a small coreset of remaining data; a practical extension would test whether fitting the reweighting term with a coreset instead of parametric Gaussians keeps the same regret with fewer distributional assumptions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper formulates machine unlearning with sequentially arriving deletion requests as a streaming problem, models the post-deletion retrained model as a distribution shift from the original model, and proposes SAFE. SAFE stores no original training set: it keeps a precomputed gradient at the initial parameters, recursively updates deletion-set gradients and per-class Gaussian statistics in a low-dimensional projection, and at each round makes a single normalized gradient step from the initial parameters with added Gaussian noise. The authors claim a surrogate risk approximation (Theorem 1), a per-step O(√T) error and cumulative O(√T + V_T) unlearning regret under only bounded gradients and without convexity (Theorems 2 and 3), and an (ε,δ)-approximate unlearning guarantee (Theorem 4). Experiments on MNIST, Fashion, CIFAR10, TinyImagenet, SST-2, and CoverType compare SAFE with Retrain, Unroll, CaMU, Lcode, and Desc on accuracy, forgetting accuracy, test accuracy, MIA, and runtime.

Significance. The problem is timely and the empirical evaluation is unusually broad for an unlearning paper: it includes ablations, varying numbers of rounds and deletion sizes, class-targeted streaming unlearning, tabular and text data, and privacy-attack metrics, and the authors provide code. If the theory were correct, the paper would be the first to give a dynamic-regret guarantee for data-free streaming unlearning, which would be an important advance. However, the theoretical core is not sound: the advertised cumulative bound is contradicted by a simple convex quadratic example, and the non-convex claim rests on an unjustified descent-type inequality. The paper's demonstrated value is therefore empirical, not theoretical, and cannot support the central claims as written.

major comments (4)
  1. [Section 3.3, Theorem 2; Algorithm 1, step 9; Appendix A.3] The claimed cumulative bound O(√T + V_T) is false for the algorithm as stated because the Gaussian perturbation has constant variance. Take the one-dimensional quadratic risk R_t(w) = (w − w*_t)^2 with w_0 = 1 and w*_t = 0 for all t, so V_T = 0. The normalized gradient of R_t at w_0 is +1, hence w_t = 1 − γ − b_t and E[R_t(w_t) − R_t(w*_t)] = (1 − γ)^2 + φ_t, with φ_t = W√(2 ln(1.25/δ))/ε independent of t. Summing over T gives a Θ(T) term, not O(√T). The proof in Appendix A.3 hides this by absorbing the term B_t, which contains E||b_t − b_{t−1}||^2 = 2d φ_t (or d times this, depending on dimension convention), into constants W^2 and dropping it; this step is not valid. This counterexample is in the simplest convex quadratic setting, so the error is not a subtle non-convexity issue.
  2. [Appendix A.3, proof of Theorem 2; main text Section 3.3] The proof bounds the per-step function error by E|R_t(w_t) − R_t(w*_t)| ≤ U E|(∇_{t+1})^T(w_t − w*_t)|. This inequality is a convexity-based descent inequality (or requires an unstated smoothness/descent lemma); it is not a consequence of the stated assumption ∥∇R_t(w)∥ ≤ U. Consequently, the advertised claim that the O(√T + V_T) bound holds 'without the strong restriction of convex loss function' is not proven. Since the experiments are on non-convex neural networks, this is a load-bearing gap, not a cosmetic one.
  3. [Section 3.3, Theorem 1; Appendix A.3] Theorem 1 assumes p0(y|x) = f(x; w0), an idealized condition that is not satisfied by trained neural networks, and the proof silently uses additional conditions: the total deleted size must satisfy sqrt(|D_t|) ≫ sum_i |F_i| and sum_i |F_i| < |D_t|. These conditions do not appear in the theorem statement. Furthermore, the derivation of the bound for |R~_t(w) − R_t(w)| jumps from a Berry-Esseen error O(|D_t|^{−1/2}) for the estimated conditional density to the displayed bound C∑|F_i|/|D_t|^{3/2} without justifying the intermediate inequality involving log ratios. The theorem is therefore not established as stated.
  4. [Theorem 4 and Definition 1] Theorem 4 asserts that each step satisfies (ε,δ)-approximate unlearning, but its proof is only a citation to the Gaussian mechanism for differential privacy. The definition requires closeness between the output distribution of the unlearning algorithm and the distribution of the retrained model A(D_t), not a differential-privacy guarantee of the gradient update. No coupling, sensitivity analysis, or explicit construction of the two distributions is provided, so the (ε,δ) claim is unsupported. Independently, the fixed-variance noise needed for that claim is precisely what forces the linear regret in Major Comment 1, so the two parts of the theorem cannot both hold as stated.
minor comments (4)
  1. [Table 3] The notation table describes d_KL as 'cosine similarity loss,' but in the main text it is the Kullback-Leibler divergence; the table entry should be corrected.
  2. [Section 4.2] The discussion of Figure 1 refers to the baseline 'Descent,' while the method is called 'Desc' throughout the rest of the paper; the naming should be made consistent.
  3. [Appendix A.3] The heading 'Theoritical Proof' is misspelled, and Theorems 2, 3, and 4 restate the three items of the main-text Theorem 2 as separate theorems; unifying the presentation would improve readability.
  4. [Section 3.3] Theorem 2(i) states a per-step bound of O(√T), which grows with the horizon T; as written this bound is vacuous for any bounded risk and should be clarified as a worst-case-horizon statement rather than a rate in t.

Circularity Check

0 steps flagged · score 1.0 of 10

No load-bearing circularity: the regret bound is a self-contained analysis of an explicit update rule, not an input renamed as output; the self-citations are background/baseline only.

full rationale

The central advertised result, E[sum_t (R_t(w_t)-R_t(w*_t))] <= O(sqrt(T)+V_T), is derived from an explicit algorithm (Eq. 27: w_t = w0 - gamma g_t/||g_t|| - b_t) by a telescoping argument over ||w_t - w*_t||^2, with V_T defined as the total variation of the per-step optimal models and appearing naturally from the dynamic-regret terms. V_T is not defined in terms of the algorithm's output, and no parameter of the bound is fitted to the data whose regret is measured. The only self-citations (e.g., [43] for the risk decomposition and MIA protocol, [44] in related work) are background design choices and baselines; they do not carry the theoretical claim. Theorem 1's assumption p0(y|x)=f(x;w0) is an explicit idealization, not a restatement of the conclusion. The proof does contain correctness gaps that are not circularity: the per-step reduction of R_t(w_t)-R_t(w*_t) to an inner product of a gradient with (w_t-w*_t) is only justified under convexity/smoothness that is not stated, and the Gaussian perturbation term B_t is dropped in the summed bound in Theorem 3. These issues are mathematical-error concerns rather than equivalence-to-inputs circularity, so they do not raise the circularity score.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new physical or ontological entities are introduced; the reweighting factor q_t(x) is a derived quantity from the Gaussian density ratio. The central assumptions are the perfect initial model, Gaussian projected features, negligible marginal feature shift, and an unstated descent condition for non-convex losses.

free parameters (4)
  • lambda (forgetting regularization weight) = 1000/120000 (Sec 4.1); 2000/6000/100000/1000000 (Appendix B.2)
    Selected per dataset; inconsistency between main text and appendix.
  • K (learning rate factor in gamma=sqrt(W)/(K sqrt(T))) = 2.5, 8, 2.5, 4 (Sec 4.1); Appendix B.5 says K=2.5 for TinyImagenet instead of 4
    Tuned per dataset on the unlearning performance; K affects the step size.
  • random projection dimension m = not reported
    The dimension of the random projection matrix V is not specified; the Gaussianity claim depends on it.
  • perturbation scale (W, epsilon, delta) = not specified in experiments
    The Gaussian perturbation variance phi_t is defined with W, epsilon, delta, but the values used in experiments are not reported.
assumptions (5)
  • domain assumption The initial model is perfectly calibrated: p0(y|x) = f(x;w0)
    Used in Theorem 1 to bound the surrogate risk error; Section 3.3.
  • domain assumption Random-projected, standardized class-conditional features z follow a multivariate Gaussian distribution
    Used to compute density ratios pt(x|y)/p0(x|y) in Eq. 24; validated with Mardia test but not theoretically guaranteed; Section 3.2.2.
  • domain assumption Marginal feature shifts are negligible: pt(x)/p0(x) is approximately constant and absorbed into normalization
    Eq. 18 in Section 3.2.2.
  • ad hoc to paper The loss landscape satisfies a descent-type inequality that allows bounding R_t(wt)-R_t(w*) by a gradient inner product (or losses are convex)
    Invoked implicitly in Theorem 2 proof; not stated among the assumptions; Section 3.3, Appendix A.3.
  • ad hoc to paper The cumulative forgetting set is small relative to the remaining data: sum_{i=1}^t |F_i| < |D_t| and sqrt(|D_t|) >> sum|F_i|
    Stated inside the proof of Theorem 1 without being listed as an assumption; Appendix A.3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Machine Unlearning for Streaming Forgetting." pith.science (2026). https://pith.science/paper/I4GBO2KL

@misc{pith2026250715280,
  author       = {Pith},
  title        = {Pith review of: Machine Unlearning for Streaming Forgetting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I4GBO2KL}},
  note         = {Machine review of arXiv:2507.15280}
}
abstract

Machine unlearning aims to remove knowledge of the specific training data in a well-trained model. Currently, machine unlearning methods typically handle all forgetting data in a single batch, removing the corresponding knowledge all at once upon request. However, in practical scenarios, requests for data removal often arise in a streaming manner rather than in a single batch, leading to reduced efficiency and effectiveness in existing methods. Such challenges of streaming forgetting have not been the focus of much research. In this paper, to address the challenges of performance maintenance, efficiency, and data access brought about by streaming unlearning requests, we introduce a streaming unlearning paradigm, formalizing the unlearning as a distribution shift problem. We then estimate the altered distribution and propose a novel streaming unlearning algorithm to achieve efficient streaming forgetting without requiring access to the original training data. Theoretical analyses confirm an $O(\sqrt{T} + V_T)$ error bound on the streaming unlearning regret, where $V_T$ represents the cumulative total variation in the optimal solution over $T$ learning rounds. This theoretical guarantee is achieved under mild conditions without the strong restriction of convex loss function. Experiments across various models and datasets validate the performance of our proposed method.

Figures

Figures reproduced from arXiv: 2507.15280 by the authors.

Figure 1
Figure 1. Comparisons of time costs for different unlearning algorithms (seconds) in streaming random subset unlearning. SAFE achieves the lowest time costs on MNIST, CIFAR10, and TinyImagenet, with at least 30% decrease compared with the method of the second-lowest cost. on the CIFAR10 dataset, the advantage of SAFE is even more pro￾nounced. SAFE required only 2.55 seconds per request, whereas the second-fastest algorithm, D… view at source ↗
Figure 2
Figure 2. Model performance against unlearning rounds on MNIST. (a) Remaining data accuracy (b) Forgetting data accuracy (c) Test data accuracy [PITH_FULL_IMAGE:figures/full_fig_p016_2.png] view at source ↗
Figure 3
Figure 3. Model performance against unlearning rounds on Fashion. (a) Remaining data accuracy (b) Forgetting data accuracy (c) Test data accuracy [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Model performance against unlearning rounds on CIFAR10. (a) Remaining data accuracy (b) Forgetting data accuracy (c) Test data accuracy [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]
Figure 5
Figure 5. Figure 5: Model performance against unlearning rounds on TinyImagenet [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 53 canonical work pages

  1. [1]

    D. Baby, S. Garg, T. Yen, S. Balakrishnan, Z. C. Lipton, and Y . Wang. Online label shift: Optimal dynamic regret meets practical algorithms. Arxiv, 2305.19570, 2023

    arXiv 2023

  2. [2]

    Y . Bai, Y . Zhang, P. Zhao, M. Sugiyama, and Z. Zhou. Adapting to online label shift with provable guarantees. In NeurIPS, 2022

    work page 2022

  3. [3]

    Besbes, Y

    O. Besbes, Y . Gur, and A. Zeevi. Non-stationary stochastic optimization. Oper. Res., 63:1227–1244, 2015

    work page 2015

  4. [4]

    Bourtoule, V

    L. Bourtoule, V . Chandrasekaran, C. A. Choquette-Choo, H. Jia, A. Travers, B. Zhang, D. Lie, and N. Papernot. Machine unlearning. In SP, 2021

    work page 2021

  5. [5]

    M. Chen, Z. Zhang, T. Wang, M. Backes, M. Humbert, and Y . Zhang. When machine unlearning jeopardizes privacy. In CCS, 2021

    work page 2021

  6. [6]

    M. Chen, W. Gao, G. Liu, K. Peng, and C. Wang. Boundary unlearning: Rapid forgetting of deep networks via shifting the decision boundary. In CVPR, 2023

    work page 2023

  7. [7]

    Chien, H

    E. Chien, H. Wang, Z. Chen, and P. Li. Langevin unlearning: A new perspective of noisy gradient descent for machine unlearning. Arxiv, 2401.10371, 2024

    work page arXiv 2024

  8. [8]

    Chourasia and N

    R. Chourasia and N. Shah. Forget unlearning: Towards true data-deletion in machine learning. In ICML, 2023

    work page 2023

Show all 60 references
  1. [9]

    V . S. Chundawat, A. K. Tarun, M. Mandal, and M. S. Kankanhalli. Can bad teaching induce forgetting? unlearning in deep networks using an incompetent teacher. In AAAI, 2023

    2023

  2. [10]

    V . S. Chundawat, A. K. Tarun, M. Mandal, and M. S. Kankanhalli. Zero-shot machine unlearning. IEEE Trans. Inf. Forensics Secur., 18: 2345–2354, 2023

    2023

  3. [11]

    Dasgupta and D

    S. Dasgupta and D. J. Hsu. On-line estimation with the multivariate gaussian distribution. In N. H. Bshouty and C. Gentile, editors, COLT. Springer, 2007

    2007

  4. [12]

    de la Torre

    L. de la Torre. A guide to the california consumer privacy act of 2018. Available at SSRN 3275571, 2018

    2018

  5. [13]

    J. C. Duchi, S. Shalev-Shwartz, Y . Singer, and A. Tewari. Composite objective mirror descent. In COLT, 2010

    2010

  6. [14]

    Dwork and A

    C. Dwork and A. Roth. The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sci., 9(3-4):211–407, 2014

    2014

  7. [15]

    C. Fan, J. Liu, Y . Zhang, E. Wong, D. Wei, and S. Liu. Salun: Empow- ering machine unlearning via gradient-based weight saliency in both image classification and generation. In ICLR, 2024

    2024

  8. [16]

    T. Finch. Incremental calculation of weighted mean and variance. Uni- versity of Cambridge, 2009

    2009

  9. [17]

    J. H. Friedman and J. W. Tukey. A projection pursuit algorithm for exploratory data analysis. IEEE Transactions on computers , 100(9): 881–890, 1974

    1974

  10. [18]

    X. Gao, X. Li, and S. Zhang. Online learning with non-convex losses and non-stationary regret. In AISTATS, 2018

    2018

  11. [19]

    U. Ghai, Z. Lu, and E. Hazan. Non-convex online learning via algorith- mic equivalence. In NeurIPS, 2022

    2022

  12. [20]

    Graves, V

    L. Graves, V . Nagisetty, and V . Ganesh. Amnesiac machine learning. In EAAI, 2021

    2021

  13. [21]

    C. Guo, T. Goldstein, A. Y . Hannun, and L. van der Maaten. Certified data removal from machine learning models. In ICML, 2020

    2020

  14. [22]

    Gupta, C

    V . Gupta, C. Jung, S. Neel, A. Roth, S. Sharifi-Malvajerdi, and C. Waites. Adaptive machine unlearning. In NeurIPS, 2021

    2021

  15. [23]

    Hazan, A

    E. Hazan, A. Kalai, S. Kale, and A. Agarwal. Logarithmic regret algo- rithms for online convex optimization. In COLT, 2006

    2006

  16. [24]

    Hazan, A

    E. Hazan, A. Agarwal, and S. Kale. Logarithmic regret algorithms for online convex optimization. Mach. Learn., 69:169–192, 2007

    2007

  17. [25]

    K. He, X. Zhang, S. Ren, and J. Sun. Identity mappings in deep residual networks. In ECCV, 2016

    2016

  18. [26]

    S. C. H. Hoi, D. Sahoo, J. Lu, and P. Zhao. Online learning: A compre- hensive survey. Neurocomputing, 459:249–289, 2021

    2021

  19. [27]

    J. Jia, J. Liu, P. Ram, Y . Yao, G. Liu, Y . Liu, P. Sharma, and S. Liu. Model sparsity can simplify machine unlearning. In NeurIPS, 2023

    2023

  20. [28]

    D. P. Kingma and M. Welling. Auto-encoding variational bayes. In ICLR, 2014

    2014

  21. [29]

    Krizhevsky, G

    A. Krizhevsky, G. Hinton, et al. Learning multiple layers of features from tiny images. https://www.cs.toronto.edu/ kriz/cifar.html, 2009

    2009

  22. [30]

    Kurmanji, P

    M. Kurmanji, P. Triantafillou, and E. Triantafillou. Towards unbounded machine unlearning. Arxiv, 2302.09880, 2023

    2023 arXiv

  23. [31]

    Le and X

    Y . Le and X. Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015

    2015

  24. [32]

    Y . LeCun. The mnist database of handwritten digits. http://yann. lecun. com/exdb/mnist/, 1998

    1998

  25. [33]

    LeCun, Y

    Y . LeCun, Y . Bengio, et al. Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks , 1995

    1995

  26. [34]

    Y . Li, C. Wang, and G. Cheng. Online forgetting process for linear regression models. In AISTATS, 2021

    2021

  27. [35]

    B. Liu, Q. Liu, and P. Stone. Continual learning and private unlearning. In CoLLAs, 2022

    2022

  28. [36]

    K. V . Mardia. Measures of multivariate skewness and kurtosis with applications. Biometrika, 57(3):519–530, 1970

    1970

  29. [37]

    Mehta, S

    R. Mehta, S. Pal, V . Singh, and S. N. Ravi. Deep unlearning via random- ized conditionally independent hessians. In CVPR, pages 10422–10431, 2022

    2022

  30. [38]

    R. M. Neal. Bayesian learning for neural networks , volume 118. Springer Science & Business Media, 2012

    2012

  31. [39]

    S. Neel, A. Roth, and S. Sharifi-Malvajerdi. Descent-to-delete: Gradient- based methods for machine unlearning. In V . Feldman, K. Ligett, and S. Sabato, editors, ALT 2021. PMLR, 2021

    2021

  32. [40]

    T. T. Nguyen, T. T. Huynh, P. L. Nguyen, A. W. Liew, H. Yin, and Q. V . H. Nguyen. A survey of machine unlearning.Arxiv, 2209.02299, 2022

    2022 arXiv

  33. [41]

    Quiñonero-Candela, M

    J. Quiñonero-Candela, M. Sugiyama, A. Schwaighofer, and N. Lawrence. Covariate shift and local learning by distribution matching. Dataset Shift in Machine Learning, pages 131–160, 2008

    2008

  34. [42]

    Sekhari, J

    A. Sekhari, J. Acharya, G. Kamath, and A. T. Suresh. Remember what you want to forget: Algorithms for machine unlearning. In NeurIPS, 2021

    2021

  35. [43]

    S. Shen, C. Zhang, A. Bialkowski, W. Chen, and M. Xu. Camu: Disen- tangling causal effects in deep model unlearning. Arxiv, 2401.17504, 2024

    2024 arXiv

  36. [44]

    S. Shen, C. Zhang, Y . Zhao, A. Bialkowski, W. T. Chen, and M. Xu. Label-agnostic forgetting: A supervision-free unlearning in deep models. In ICLR, 2024

    2024

  37. [45]

    Shokri, M

    R. Shokri, M. Stronati, C. Song, and V . Shmatikov. Membership infer- ence attacks against machine learning models. In SP, 2017

    2017

  38. [46]

    Sugiyama, S

    M. Sugiyama, S. Nakajima, H. Kashima, P. Buenau, and M. Kawanabe. Direct importance estimation with model selection and its application to covariate shift adaptation. Advances in neural information processing systems, 20, 2007

    2007

  39. [47]

    A. K. Tarun, V . S. Chundawat, M. Mandal, and M. S. Kankanhalli. Fast yet effective machine unlearning. TNNLS, 2023

    2023

  40. [48]

    Thudi, G

    A. Thudi, G. Deza, V . Chandrasekaran, and N. Papernot. Unrolling SGD: understanding factors influencing machine unlearning. In EuroS&P, 2022

    2022

  41. [49]

    V oigt and A

    P. V oigt and A. V on dem Bussche. The eu general data protection regula- tion (gdpr). A Practical Guide, 1st Ed., Cham: Springer International Publishing, 2017

    2017

  42. [50]

    R. Wu, C. Guo, Y . Su, and K. Q. Weinberger. Online adaptation to label distribution shift. In NeurIPS, 2021

    2021

  43. [51]

    H. Xiao, K. Rasul, and R. V ollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. Arxiv, 1708.07747, 2017

    2017 arXiv

  44. [52]

    H. Xu, T. Zhu, L. Zhang, W. Zhou, and P. S. Yu. Machine unlearning: A survey. ACM Comput. Surv., 2024

    2024

  45. [53]

    Zhang, W

    C. Zhang, W. Chen, W. E. Zhang, and M. Xu. Countering relearning with perception revising unlearning. In ACML, 2024

    2024

  46. [54]

    Zhang, S

    C. Zhang, S. Shen, Y . Zhao, W. T. Chen, and M. Xu. GENIU: A restricted data access unlearning for imbalanced data. Arxiv, abs/2406.07885, 2024

    2024 arXiv

  47. [55]

    Zhang, S

    C. Zhang, S. Shen, W. Chen, and M. Xu. Toward efficient data-free unlearning. In AAAI, 2025

    2025

  48. [56]

    Zhang, S

    L. Zhang, S. Lu, and Z. Zhou. Adaptive online learning in dynamic environments. In NeurIPS, 2018

    2018

  49. [57]

    H. Zhao, B. Ni, H. Wang, J. Fan, F. Zhu, Y . Wang, Y . Chen, G. Meng, and Z. Zhang. Continual forgetting for pre-trained vision models. In CVPR, 2024

    2024

  50. [58]

    Zinkevich

    M. Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In ICML, 2003

    2003

  51. [59]

    Zinkevich

    M. Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In ICML, 2003. A Appendix A.1 Related Work A.1.1 Machine Unlearning Machine unlearning requires the removal of information about the forgetting data from the original model while preserving ...

    2003

  52. [60]

    However, these approaches still face limitations

    fine-tunes the model with noisy gradients for unlearning, which can be extended to streaming unlearning with limited error increases. However, these approaches still face limitations. Some are restricted to convex loss functions [39], while others still rely on full training d...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.