Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Towards Efficient and Exact Forgetting Services in Pre-Trained-Model-based Continual Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a continual-learning classifier with a frozen pre-trained feature extractor can erase specific training data exactly, using only the forgotten samples, via a recursive closed-form update proven equivalent to…

desk verdict A correct but narrow exact-unlearning identity for frozen-feature ridge classifiers, packaged as a new problem; referee it, but require the authors to fix the appendix algebra and scale back the backbone-unlearning claims. read the letter →

arxiv 2505.12239 v2 pith:E2E5BMM4 submitted 2025-05-18 cs.LG cs.AIcs.CR

Yajiang Huang , Jianheng Tang , Kejia Fan , Huiping Zhuang , Anfeng Liu , Tian Wang , Yunhuai Liu , Mianxiong Dong
show 1 more author
Houbing Herbert Song
This is my paper · ORCID
classification cs.LGcs.AIcs.CR
keywords machineunlearningcontinuallearninganalyticpre-trainedmodelsclosed-formleastsquaresknowledgetrackingmatrixexactforgetting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces continual unlearning (CU), the setting where, after continual learning, forgetting requests arrive sequentially, and claims that a model built from a frozen pre-trained feature extractor plus a ridge-regression classifier can answer each request exactly using only the samples to be forgotten. It identifies gradient-based updates as the root cause of why existing unlearning methods either need to revisit the retained data (violating CL privacy) or trade fidelity for efficiency, and proposes Analytic Continual Unlearning (ACU) as a gradient-free alternative. ACU maintains a compressed knowledge tracking matrix and applies a recursive update that provably reproduces, after every request, the classifier that would have been obtained by retraining on the remaining data. If the claim holds, exact privacy-compliant forgetting becomes a lightweight algebraic operation that degrades neither under frequent requests nor without access to historical data.

What carries the argument

The knowledge tracking matrix $T_i = (\sum_{j\in\hat{D}_i} f_j^\top f_j + \gamma I)^{-1}$, a $d_F\times d_F$ matrix that compresses the entire retained set into a second-order statistic, is the engine of the method; it is updated in closed form through the Woodbury matrix identity (Lemma 2), which converts the removal of the forgetting set's feature gram matrix $\check{F}_i^\top\check{F}_i$ into a recursive update involving only $\check{F}_i$. On top of it, the classifier update in Theorem 1 re-weights the previous model by the forgotten features' covariance and subtracts the forgotten features' target contribution, a two-term 'amplify and erase' decomposition that the paper derives by rewriting the retained-set solution. The enabling structure is the frozen pre-trained backbone, which makes feature extraction gradient-free and keeps the whole pipeline analytic, so that data influence is an explicit algebraic quantity rather than a trace left in SGD trajectories.

What would settle it

Pre-train the feature backbone on the same distribution as the data slated for forgetting, run ACU to unlearn a subset, and then probe the backbone's features with a membership-inference classifier trained to tell forgotten samples from never-seen ones on the basis of feature statistics. If this probe separates the two groups with accuracy clearly above chance, the influence of the forgotten data survives in the deployed feature extractor, which contradicts the service-level claim that ACU erases it; under exact forgetting the probe should perform at chance.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is Theorem 1: if the classifier is the analytic least-squares solution $W_0 = (\sum_j f_j^\top f_j + \gamma I)^{-1} \sum_j f_j^\top y_j$ trained during the continual learning phase, and the knowledge tracking matrix $T_0$ stores the corresponding inverse-covariance term, then for each unlearning request $i$ the recursive update $$W_i = \left(I + T_i \sum_{j\in \check{D}_i} f_j^\top f_j\right) W_{i-1} - T_i \sum_{j\in \check{D}_i} f_j^\top y_j, \qquad T_i = T_{i-1} + T_{i-1}\check{F}_i^\top (I - \check{F}_i T_{i-1} \check{F}_i^\top)^{-1} \check{F}_i T_{i-1}$$ is exactly equivalent to the ridge-regression classifier retrained from scratch on the retained set $\hat{D}_i = D \setminus \bigcup_{k\le i}\check{D}_k$. The update is interpretable: the first term amplifies the knowledge that must stay, the second erases the knowledge that must go. Because both updates use only the forgotten features $\check{F}_i$, the retained data never needs to be revisited, and because the tracking matrix is a compressed second-order statistic, it cannot be unrolled to recover the original samples. Experiments confirm the theorem numerically: ACU attains exactly zero deviation from the re-trained model on parameter distance, retained/forgetting/test accuracy, and membership-inference indicators, while gradient-based baselines degrade sharply as requests accumulate.

Load-bearing premise

The paper's guarantee rests on the assumption that the pre-trained feature extractor is frozen and never modified during learning or unlearning, so erasing the linear classifier's dependence on the forgotten samples removes all of their influence on the deployed model; the paper states this in Section 3.1 and flags in Section 5 that if the backbone itself holds domain-specific or private knowledge, ACU does not erase it.

Editorial extensions

If this is right

  • Each forgetting request costs $O(d_F^3)$ plus terms depending only on the size of the forgetting set, independent of how much data the model has seen, so even a stream of adversarial requests arriving one fragment at a time stays cheap to serve.
  • After every request the model equals the classifier that never saw the forgotten data, which means membership-inference attacks on the classifier should perform at chance and the unlearning can be verified without trusting the service provider.
  • The model remains in its optimal analytic form after each request, so the system can alternate freely between learning and forgetting phases without resets or retraining.
  • Any continual learner built on a frozen pre-trained extractor plus an analytic classifier inherits exact forgetting as a corollary, provided it maintained the tracking matrix recursively during learning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The mechanism is really inverse-covariance maintenance: the same two-term recursion should apply verbatim to any ridge-regression head trained on frozen features, whether linear probes, kernel regressors, or federated models, making sequential exact deletion a generic algebraic service rather than a CL-specific construction.
  • The guarantee covers only the classifier; if the frozen backbone has memorised the forgotten inputs, a membership-inference probe on the feature vectors themselves should still separate them, so a natural extension pairs ACU with low-rank representation corrections and measures residual leakage after both stages.
  • Exactness holds in real arithmetic; in practice the recursion's accumulated floating-point drift in $T_i$ over hundreds of requests is unmeasured, and the deviation from the retrained model would set a safe request-count envelope for deployment.
  • If the theorem stands, request-fragmentation denial-of-service attacks lose their point, as each fragment costs the same small closed-form operation, and the adversarial frontier shifts entirely to the backbone, which is why the authors' suggestion of an adjustable backbone is the consequential open direction.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper introduces Continual Unlearning (CU), a setting in which a model produced by a continual-learning phase must sequentially remove the influence of designated samples, with access only to the forgotten samples and not to the retained set. The proposed method, Analytic Continual Unlearning (ACU), assumes a frozen pre-trained feature extractor and a linear analytic classifier trained by ridge regression. ACU maintains a d_F x d_F Knowledge Tracking Matrix T_i = (sum over retained set of f^T f + gamma I)^{-1}, which is updated by a Woodbury-type recursion using only the forgetting features, together with the recursive model update W_i = (I + T_i sum over forget set of f^T f) W_{i-1} - T_i sum over forget set of f^T y (Eqs. (5)-(6)). Theorem 1 claims that this update exactly reproduces the ridge solution retrained on the retained set. Experiments on CIFAR-10 and CIFAR-100 compare ACU with Finetuning, SCRUB, NegGrad, WoodFisher, RandomLabel, and others on parameter, accuracy, MIA, and runtime gaps, reporting zero deltas for ACU across all metrics.

Significance. The central algebraic identity is correct and useful: for a fixed feature map, ACU gives an exact, closed-form, sequential decremental update that needs neither retained data nor gradient iteration, with O(d_F^3 + |forget set| d_F^2) per-request cost and O(d_F^2) memory. If the stated assumptions hold, this is a clean result for the analytic-CL community and a meaningful step toward efficient unlearning services. The paper is also commendable for separating the recursive update from the oracle re-trained target, which makes the exactness claim falsifiable by direct comparison, and for reporting cumulative efficiency under 25-50 requests. The main caveats are that the exactness statement is scoped to the linear head on a frozen backbone, and that the privacy-preservation claims go beyond what is actually proven.

major comments (3)
  1. [Abstract, Section 3.1, Section 5] The title and abstract claim 'exact forgetting' and 'privacy preservation' without qualification, but Theorem 1 and Eq. (2) concern only the linear analytic classifier on a frozen pre-trained backbone. Section 5 acknowledges that ACU does not unlearn knowledge acquired during pre-training and instead asserts that public pre-trained models come with 'certified security and privacy guarantees'; no citation or argument supports this assertion, and a backbone trained on data overlapping the forget set will retain that information after ACU. Please qualify the claims, e.g., 'exact unlearning of the analytic head on a frozen backbone,' and either remove or support the certified-guarantee sentence.
  2. [Section 3.2] The statement that the Knowledge Tracking Matrix has rank R <= d_F << N and therefore 'cannot be inverted to recover the original dataset, thereby preserving historical data privacy' is an informal privacy claim. For gamma > 0, T_i is nonsingular, so its rank is d_F; the compression is in the dimension of the representation, not in rank. Whether T_i leaks membership or features depends on the feature map, the attacker's auxiliary information, and the privacy definition. As written, the privacy guarantee is not established. Please provide a formal privacy model and proof, or weaken the statement to say that ACU does not store raw samples and does not require access to the retained set.
  3. [Appendix E.1, Table 1] The experimental setting is ambiguous on a load-bearing point: Appendix E.1 states that 'both the pre-trained base model and the original model are trained for 300 epochs using the SGD optimizer,' but Theorem 1 requires the initial W_0 to be exactly the ridge solution in Eq. (3). If W_0 is obtained by SGD, the ACU update is not guaranteed to match the re-trained model, and the zero gaps in Table 1 need an explanation. If W_0 is instead the analytic solution, this should be stated explicitly. In addition, if the 'optimal re-trained model' in Table 1 is computed by the same closed form (8), the zero deltas are a verification of the algebra rather than an independent empirical test; please clarify the construction of both models.
minor comments (5)
  1. [Appendix A, Lemma 3, Eq. (36)] In the inductive step from i to i+1, the sums over forgotten features are written over the i-th forgetting set instead of the (i+1)-th forgetting set; the final equality is correct only after changing the index from D_check_i to D_check_{i+1} throughout that display.
  2. [Appendix A, Theorem 1 proof, Eqs. (43)-(44)] The identity F_check_i^T F_check_i = sum_{j in D_hat_i} f_j^T f_j is incorrect; the sum should be over the forgetting set D_check_i. The same wrong retained-set index appears again in the final line of Eq. (44), where the theorem statement and Eq. (6) use D_check_i. Please correct both occurrences and re-check all subsequent set indices.
  3. [Appendix A, Lemma 2] The word 'reversible' should be 'invertible' in the statement of the Woodbury identity, and the sentence 'So (18) can be rewritten as' appears to refer to the wrong displayed equation; the intended manipulation concerns Eq. (22).
  4. [Appendix B] The stated computational complexities for W_i and T_i appear to be swapped: the term O(d_F^3 + n d_F^2 + n^2 d_F + n^3) corresponds to the Woodbury update of T_i, while the update of W_i has cost O(d_F^3 + n d_F^2 + n d_C d_F + d_C d_F), with n = |D_check_i|. Please correct the assignment.
  5. [Appendix C, Eq. (45)] The recursive formula for the CL phase uses the symbol F_check_i in the update, but the learning set is denoted D_tilde_p and the feature matrix F_tilde_p; please use a consistent subscript so the recursion is over p.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the recursive ACU update is derived from first principles as an algebraic identity with the independently defined ridge-retrained classifier; the only caveat is the frozen-backbone scope of 'exact forgetting.'

full rationale

The paper's central derivation is self-contained. Theorem 1 (Section 3.4) proves that the recursive update in Eq. (9), with the Knowledge Tracking Matrix updated by Eq. (5), equals the closed-form ridge-regression solution in Eq. (8) on the retained set. The target model is independently defined by the optimization objective in Eq. (2), and Lemma 1 derives its closed form; Lemma 3 proves, via the Woodbury identity (Lemma 2), that the recursively updated T_i equals the inverse regularized Gram matrix of the retained set. The proof then algebraically rewrites the retained-set solution into the recursive form of Eq. (9). No parameter is fitted to the retained or forgotten data and then presented as a prediction; the equality is a mathematical identity, not a statistical reduction. The zero experimental gaps in Table 1 follow directly from this identity, so they are checks of the theorem rather than fitted predictions. The paper's own Section 5 limitation is explicit: ACU does not unlearn knowledge in the frozen pre-trained backbone, and Appendix E.1 deliberately uses disjoint base and CL partitions. This is a scope restriction on the privacy claim, not circularity in the derivation. Self-citations to prior analytic-learning work (e.g., [49] for recursive Moore-Penrose updates) are contextual and are not load-bearing here, because the needed identities are re-proved in the appendix. The finding is therefore no significant circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 1 invented entities

The method relies on standard linear algebra (Woodbury identity) and on the assumption that the CL phase produces a linear ridge classifier over frozen features. The only tuning parameters are the ridge coefficient and feature-map details. No new physical entities are introduced.

free parameters (2)
  • regularization γ = not reported
    The ridge penalty in Eq. (2) must be set; it affects model accuracy but not the unlearning identity.
  • feature mapping G(·) hyperparameters = not specified
    Random projection or kernel parameters in Eq. (1) affect features; the paper does not report their values or whether they are tuned.
assumptions (4)
  • standard math Woodbury matrix identity
    Used in Lemma 2 and to derive the recursive updates in Eqs. (5) and (6).
  • domain assumption The CL phase model is a linear ridge classifier on frozen features, trained with MSE
    Eq. (2) defines the objective; if the CL model is non-linear or trained with CE, the closed-form unlearning derivation does not apply.
  • domain assumption The forgetting requests are disjoint subsets of the CL training set
    Section 3 defines ˇD_i with disjointness; overlapping requests would require re-deriving the update.
  • domain assumption The pre-trained backbone is frozen and not unlearned
    Section 3.1 and Section 5 state the backbone is fixed; the paper explicitly leaves backbone unlearning as future work.
invented entities (1)
  • Knowledge Tracking Matrix T_i
    purpose: Stores the inverse regularized second-moment matrix of the retained features, enabling recursive unlearning without the retained data
    It is a mathematical bookkeeping matrix defined in Eq. (4), not an independently observable physical or conceptual entity; its privacy properties are asserted informally.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Efficient and Exact Forgetting Services in Pre-Trained-Model-based Continual Learning." pith.science (2026). https://pith.science/paper/E2E5BMM4

@misc{pith2026250512239,
  author       = {Pith},
  title        = {Pith review of: Towards Efficient and Exact Forgetting Services in Pre-Trained-Model-based Continual Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E2E5BMM4}},
  note         = {Machine review of arXiv:2505.12239}
}
read the original abstract

In Continual Learning (CL), using a Pre-Trained Model (PTM) as the feature extractor has become a popular practice. Accompanied by analytic classifiers, the PTM-based methods have achieved state-of-the-art performance in CL, in pursuit of the non-forgetting goal. Meanwhile, actively forgetting specific knowledge acquired during the CL phase is also essential in most service construction paradigms, for example, Mobile Crowd Sensing (MCS), where mobile edge nodes continuously collect sensory data and demand not only non-forgetting adaptation but also specific knowledge forgetting for privacy preservation. Thus, a unique problem, called Continual Unlearning (CU), arises when the forgetting requests show sequentially in CL. However, existing unlearning methods focus on single-shot joint forgetting and prove highly inadequate when applied to CU, including (1) violating the historical data privacy in CL and (2) vulnerably being overwhelmed or degraded with adversarially frequent requests. To handle the challenges of CU, we propose a gradient-free approach, called Analytic Continual Unlearning (ACU), for efficient and exact forgetting with historical data privacy preservation in PTM-based CL. In response to each unlearning request, our ACU recursively derives the analytical (i.e., closed-form) solutions via least squares in an interpretable manner. By meticulous design, our ACU is compatible with both sample-level and class-level unlearning requests. The theoretical and experimental evaluations validate our ACU's superiority in unlearning effectiveness, model fidelity, and system efficiency.

Figures

Figures reproduced from arXiv: 2505.12239 by the authors.

Figure 1
Figure 1. Following the CL philosophy that online task data is discarded after training [ [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 1
Figure 1. Illustration of model state evolution under the CL and CU phases. The model begins [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The framework of our proposed ACU method. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figures from the paper (3 more)
Figure 3
Figure 3. Figure 3: The dynamics analysis for a total of 25 CU requests. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png]
Figure 4
Figure 4. Figure 4: The dynamics analysis for a total of 50 CU requests. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 3
Figure 3. Figure 3: This observation indicates that these baselines are sensitive to the number of CU requests [PITH_FULL_IMAGE:figures/full_fig_p009_3.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. APFL: Analytic Personalized Federated Learning via Dual-Stream Least Squares

    cs.LG 2025-08 conditional novelty 6.0 of 10

    A closed-form dual-stream least-squares method for personalized federated learning that is provably invariant to how data are distributed among clients.

Reference graph

Works this paper leans on

57 extracted references · 48 canonical work pages · cited by 1 Pith paper

  1. [1]

    A comprehensive survey of continual learning: Theory, method and application

    Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: Theory, method and application. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(8):5362–5383, 2024

    2024

  2. [2]

    Fairness continual learning approach to semantic scene understanding in open-world environments

    Thanh-Dat Truong, Hoang-Quan Nguyen, Bhiksha Raj, and Khoa Luu. Fairness continual learning approach to semantic scene understanding in open-world environments. Advances in Neural Information Processing Systems, 36:65456–65467, 2023

    work page 2023

  3. [3]

    Learning to prompt knowledge transfer for open-world continual learning

    Yujie Li, Xin Yang, Hao Wang, Xiangkun Wang, and Tianrui Li. Learning to prompt knowledge transfer for open-world continual learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 13700–13708, 2024

    work page 2024

  4. [4]

    SLCA: Slow learner with classifier alignment for continual learning on a pre-trained model

    Gengwei Zhang, Liyuan Wang, Guoliang Kang, Ling Chen, and Yunchao Wei. SLCA: Slow learner with classifier alignment for continual learning on a pre-trained model. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 19091–19101, 2023

    work page 2023

  5. [5]

    Learning to prompt for continual learning

    Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 139–149, 2022

    2022

  6. [6]

    Expandable subspace ensemble for pre-trained model-based class-incremental learning

    Da-Wei Zhou, Hai-Long Sun, Han-Jia Ye, and De-Chuan Zhan. Expandable subspace ensemble for pre-trained model-based class-incremental learning. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 23554–23564, 2024

    work page 2024

  7. [7]

    RanPAC: Random projections and pre-trained models for continual learning

    Mark D McDonnell, Dong Gong, Amin Parvaneh, Ehsan Abbasnejad, and Anton Van den Hengel. RanPAC: Random projections and pre-trained models for continual learning. Advances in Neural Information Processing Systems, 36:12022–12053, 2023

    work page 2023

  8. [8]

    Continual learning with pre-trained models: A survey

    Da-Wei Zhou, Hai-Long Sun, Jingyi Ning, Han-Jia Ye, and De-Chuan Zhan. Continual learning with pre-trained models: A survey. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24 , pages 8363–8371. International Joint Conferences on Artificial Intelligence Organization, 8 2024. Survey Track

    work page 2024

Show all 57 references
  1. [9]

    PILOT: a pre-trained model- based continual learning toolbox

    Hai-Long Sun, Da-Wei Zhou, De-Chuan Zhan, and Han-Jia Ye. PILOT: a pre-trained model- based continual learning toolbox. Science China Information Sciences, 68(4):147101, 2025

    2025

  2. [10]

    Machine unlearning: Taxonomy, metrics, applications, challenges, and prospects

    Na Li, Chunyi Zhou, Yansong Gao, Hui Chen, Zhi Zhang, Boyu Kuang, and Anmin Fu. Machine unlearning: Taxonomy, metrics, applications, challenges, and prospects. IEEE Transactions on Neural Networks and Learning Systems, pages 1–21, 2025

    2025

  3. [11]

    Towards un- bounded machine unlearning

    Meghdad Kurmanji, Peter Triantafillou, Jamie Hayes, and Eleni Triantafillou. Towards un- bounded machine unlearning. In Thirty-seventh Conference on Neural Information Processing Systems, 2023

    2023

  4. [12]

    Learning to unlearn for robust machine unlearning

    Mark He Huang, Lin Geng Foo, and Jun Liu. Learning to unlearn for robust machine unlearning. In European Conference on Computer Vision (ECCV), pages 202–219. Springer, 2024

    2024

  5. [13]

    Fecam: Exploiting the heterogeneity of class distributions in exemplar-free continual learning.Advances in Neural Information Processing Systems, 36:6582–6595, 2023

    Dipam Goswami, Yuyang Liu, Bartłomiej Twardowski, and Joost Van De Weijer. Fecam: Exploiting the heterogeneity of class distributions in exemplar-free continual learning.Advances in Neural Information Processing Systems, 36:6582–6595, 2023

    2023

  6. [14]

    Resurrecting old classes with new data for exemplar-free continual learning

    Dipam Goswami, Albin Soutif-Cormerais, Yuyang Liu, Sandesh Kamath, Bart Twardowski, Joost Van De Weijer, et al. Resurrecting old classes with new data for exemplar-free continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pa...

    2024

  7. [15]

    GACL: Exemplar-free generalized analytic continual learning

    Huiping Zhuang, Yizhu Chen, Di Fang, Run He, Kai Tong, Hongxin Wei, Ziqian Zeng, and Cen Chen. GACL: Exemplar-free generalized analytic continual learning. In Advances in Neural Information Processing Systems. Curran Associates, Inc., December 2024

    2024

  8. [16]

    DS-AL: A dual-stream analytic learning for exemplar-free class-incremental learning

    Huiping Zhuang, Run He, Kai Tong, Ziqian Zeng, Cen Chen, and Zhiping Lin. DS-AL: A dual-stream analytic learning for exemplar-free class-incremental learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 17237–17244, 2024

    2024

  9. [17]

    Machine un- learning of features and labels

    Alexander Warnecke, Lukas Pirch, Christian Wressnegger, and Konrad Rieck. Machine un- learning of features and labels. In Proc. of the 30th Network and Distributed System Security (NDSS), 2023. 10

    2023

  10. [18]

    Machine unlearning

    Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In 2021 IEEE symposium on security and privacy (SP), pages 141–159. IEEE, 2021

    2021

  11. [19]

    ARCANE: An efficient architecture for exact machine unlearning

    Haonan Yan, Xiaoguang Li, Ziyao Guo, Hui Li, Fenghua Li, and Xiaodong Lin. ARCANE: An efficient architecture for exact machine unlearning. In Lud De Raedt, editor, Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, pages 4006–4...

    2022

  12. [20]

    Machine unlearning via algorithmic stability

    Enayat Ullah, Tung Mai, Anup Rao, Ryan A Rossi, and Raman Arora. Machine unlearning via algorithmic stability. In Conference on Learning Theory, pages 4126–4142. PMLR, 2021

    2021

  13. [21]

    Model sparsity can simplify machine unlearning

    Jinghan Jia, Jiancheng Liu, Parikshit Ram, Yuguang Yao, Gaowen Liu, Yang Liu, Pranay Sharma, and Sijia Liu. Model sparsity can simplify machine unlearning. In Thirty-seventh Conference on Neural Information Processing Systems, 2023

    2023

  14. [22]

    SalUn: Empowering machine unlearning via gradient-based weight saliency in both image classification and generation

    Chongyu Fan, Jiancheng Liu, Yihua Zhang, Eric Wong, Dennis Wei, and Sijia Liu. SalUn: Empowering machine unlearning via gradient-based weight saliency in both image classification and generation. In The Twelfth International Conference on Learning Representations, 2024

    2024

  15. [23]

    Unrolling SGD: Understanding Factors Influencing Machine Unlearning

    Anvith Thudi, Gabriel Deza, Varun Chandrasekaran, and Nicolas Papernot. Unrolling SGD: Understanding Factors Influencing Machine Unlearning . In 2022 IEEE 7th European Sympo- sium on Security and Privacy (EuroS&P), pages 303–319, Los Alamitos, CA, USA, June 2022. IEEE Computer Society

    2022

  16. [24]

    What makes unlearning hard and what to do about it

    Kairan Zhao, Meghdad Kurmanji, George-Octavian B˘arbulescu, Eleni Triantafillou, and Peter Triantafillou. What makes unlearning hard and what to do about it. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  17. [25]

    ERM-KTP: Knowledge-level machine unlearning via knowledge transfer

    Shen Lin, Xiaoyu Zhang, Chenyang Chen, Xiaofeng Chen, and Willy Susilo. ERM-KTP: Knowledge-level machine unlearning via knowledge transfer. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20147–20155, 2023

    2023

  18. [26]

    ACIL: Analytic class-incremental learning with absolute memorization and privacy protection

    Huiping Zhuang, Zhenyu Weng, Hongxin Wei, Renchunzi Xie, Kar-Ann Toh, and Zhiping Lin. ACIL: Analytic class-incremental learning with absolute memorization and privacy protection. Advances in Neural Information Processing Systems, 35:11602–11614, 2022

    2022

  19. [27]

    GKEAL: Gaussian kernel embedded analytic learning for few-shot class incremental task

    Huiping Zhuang, Zhenyu Weng, Run He, Zhiping Lin, and Ziqian Zeng. GKEAL: Gaussian kernel embedded analytic learning for few-shot class incremental task. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7746–7755, 2023

    2023

  20. [28]

    F-OAL: Forward-only online analytic learning with fast training and low memory footprint in class incremental learning

    Huiping Zhuang, Yuchen Liu, Run He, Kai Tong, Ziqian Zeng, Cen Chen, Yi Wang, and Lap- Pui Chau. F-OAL: Forward-only online analytic learning with fast training and low memory footprint in class incremental learning. Advances in Neural Information Processing Systems, 37:41517–...

    2024

  21. [29]

    MMAL: Multi-modal analytic learning for exemplar-free audio- visual class incremental tasks

    Xianghu Yue, Xueyi Zhang, Yiming Chen, Chengwei Zhang, Mingrui Lao, Huiping Zhuang, Xinyuan Qian, and Haizhou Li. MMAL: Multi-modal analytic learning for exemplar-free audio- visual class incremental tasks. In Proceedings of the 32nd ACM International Conference on Multimedia,...

    2024

  22. [30]

    Order-robust class incremental learning: Graph-driven dynamic similarity grouping

    Guannan Lai, Yujie Li, Xiangkun Wang, Tianrui Li Junbo Zhang, and Xin Yang. Order-robust class incremental learning: Graph-driven dynamic similarity grouping. In 2025 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  23. [31]

    Knowledge memorization and rumination for pre-trained model-based class-incremental learning

    Zijian Gao, Wangwang Jia, Xingxing Zhang, Dulan Zhou, Kele Xu, Feng Dawei, Yong Dou, Xinjun Mao, and Huaimin Wang. Knowledge memorization and rumination for pre-trained model-based class-incremental learning. In 2025 Proceedings of the IEEE/CVF Conference on Computer Vision an...

    2025

  24. [32]

    TSVD: Bridging theory and practice in continual learning with pre-trained models

    Liangzu Peng, Juan Elenter, Joshua Agterberg, Alejandro Ribeiro, and Rene Vidal. TSVD: Bridging theory and practice in continual learning with pre-trained models. In The Thirteenth International Conference on Learning Representations, 2025

    2025

  25. [33]

    Boosting multiple views for pretrained-based continual learning

    Quyen Tran, Tung Lam Tran, Khanh Doan, Toan Tran, Dinh Phung, Khoat Than, and Trung Le. Boosting multiple views for pretrained-based continual learning. InThe Thirteenth International Conference on Learning Representations, 2025. 11

    2025

  26. [34]

    Certified data removal from machine learning models

    Chuan Guo, Tom Goldstein, Awni Hannun, and Laurens Van Der Maaten. Certified data removal from machine learning models. In Proceedings of the 37th International Conference on Machine Learning, ICML’20. JMLR.org, 2020

    2020

  27. [35]

    Varshney, Mohit Bansal, Sanmi Koyejo, and Yang Liu

    Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Yuguang Yao, Chris Yuhao Liu, Xiaojun Xu, Hang Li, Kush R. Varshney, Mohit Bansal, Sanmi Koyejo, and Yang Liu. Rethinking machine unlearning for large language models. Nature Machine Intellig...

    2025

  28. [36]

    Continual forgetting for pre-trained vision models

    Hongbo Zhao, Bolin Ni, Junsong Fan, Yuxi Wang, Yuntao Chen, Gaofeng Meng, and Zhaoxiang Zhang. Continual forgetting for pre-trained vision models. In2024 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 28631–28642, 2024

    2024

  29. [37]

    A survey on federated unlearning: Challenges, methods, and future directions

    Ziyao Liu, Yu Jiang, Jiyuan Shen, Minyi Peng, Kwok-Yan Lam, Xingliang Yuan, and Xiaoning Liu. A survey on federated unlearning: Challenges, methods, and future directions. ACM Computing Surveys, 57(1):1–38, 2024

    2024

  30. [38]

    Graph unlearning

    Min Chen, Zhikun Zhang, Tianhao Wang, Michael Backes, Mathias Humbert, and Yang Zhang. Graph unlearning. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, CCS ’22, page 499–513, New York, NY , USA, 2022. Association for Computing Machinery

    2022

  31. [39]

    Learning to unlearn: Instance-wise unlearning for pre-trained classifiers

    Sungmin Cha, Sungjun Cho, Dasol Hwang, Honglak Lee, Taesup Moon, and Moontae Lee. Learning to unlearn: Instance-wise unlearning for pre-trained classifiers. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 11186–11194, 2024

    2024

  32. [40]

    A unified framework for continual learning and machine unlearning

    Romit Chatterjee, Vikram Chundawat, Ayush Tarun, Ankur Mali, and Murari Mandal. A unified framework for continual learning and machine unlearning. arXiv preprint arXiv:2408.11374, 2024

    2024 arXiv

  33. [41]

    Pseudoinverse learning algorithm for feedforward neural networks

    Ping Guo, Michael R Lyu, and NE Mastorakis. Pseudoinverse learning algorithm for feedforward neural networks. Advances in Neural Networks and Applications, 1(321-326), 2001

    2001

  34. [42]

    A progressive stacking pseudoinverse learning framework via active learning in random subspaces

    Zhenjiao Cai, Sulan Zhang, Ping Guo, Jifu Zhang, and Lihua Hu. A progressive stacking pseudoinverse learning framework via active learning in random subspaces. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 54(5):2822–2832, 2024

    2024

  35. [43]

    Bayesian pseudoinverse learners: From uncertainty to deterministic learning

    Qian Yin, Bingxin Xu, Kaiyan Zhou, and Ping Guo. Bayesian pseudoinverse learners: From uncertainty to deterministic learning. IEEE Transactions on Cybernetics, 52(11):12205–12216, 2022

    2022

  36. [44]

    Universal approximation using radial-basis-function networks

    Jooyoung Park and Irwin W Sandberg. Universal approximation using radial-basis-function networks. Neural computation, 3(2):246–257, 1991

    1991

  37. [45]

    Learning from the kernel and the range space

    Kar-Ann Toh. Learning from the kernel and the range space. In 2018 IEEE/ACIS 17th International Conference on Computer and Information Science (ICIS), pages 1–6. IEEE, 2018

    2018

  38. [46]

    Noniterative deep learning: Incorporating re- stricted boltzmann machine into multilayer random weight neural networks

    Xi-Zhao Wang, Tianlun Zhang, and Ran Wang. Noniterative deep learning: Incorporating re- stricted boltzmann machine into multilayer random weight neural networks. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 49(7):1299–1308, 2017

    2017

  39. [47]

    An analytic formulation of con- volutional neural network learning for pattern recognition

    Huiping Zhuang, Zhiping Lin, Yimin Yang, and Kar-Ann Toh. An analytic formulation of con- volutional neural network learning for pattern recognition. Information Sciences, 686:121317, 2025

    2025

  40. [48]

    DensePILAE: a feature reuse pseudoinverse learning algorithm for deep stacked autoencoder

    Jue Wang, Ping Guo, and Yanjun Li. DensePILAE: a feature reuse pseudoinverse learning algorithm for deep stacked autoencoder. Complex & Intelligent Systems, pages 1–11, 2022

    2022

  41. [49]

    Blockwise recursive moore–penrose inverse for network learning

    Huiping Zhuang, Zhiping Lin, and Kar-Ann Toh. Blockwise recursive moore–penrose inverse for network learning. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 52(5):3237– 3250, 2021

    2021

  42. [50]

    Locality sensitive sparse encoding for learning world models online

    Zichen Liu, Chao Du, Wee Sun Lee, and Min Lin. Locality sensitive sparse encoding for learning world models online. In The Twelfth International Conference on Learning Representations, 2024

    2024

  43. [51]

    AFL: A single-round analytic approach for federated learing with pre-trained models

    Run He, Kai Tong, Di Fang, Han Sun, Ziqian Zeng, Haoran Li, Tianyi Chen, and Huiping Zhuang. AFL: A single-round analytic approach for federated learing with pre-trained models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025. 12

    2025

  44. [52]

    On loss functions for deep neural networks in classification

    Katarzyna Janocha and Wojciech Marian Czarnecki. On loss functions for deep neural networks in classification. arXiv preprint arXiv:1702.05659, 2017

    2017 arXiv

  45. [53]

    Evaluation of neural architectures trained with square loss vs cross-entropy in classification tasks

    Like Hui and Mikhail Belkin. Evaluation of neural architectures trained with square loss vs cross-entropy in classification tasks. In International Conference on Learning Representations, 2021

    2021

  46. [54]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009

    2009

  47. [55]

    WoodFisher: Efficient second-order approximation for neural network compression

    Sidak Pal Singh and Dan Alistarh. WoodFisher: Efficient second-order approximation for neural network compression. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 18098–18109. Curr...

    2020

  48. [56]

    Eternal sunshine of the spotless net: Selective forgetting in deep networks

    Aditya Golatkar, Alessandro Achille, and Stefano Soatto. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9301–9309, 2020

    2020

  49. [57]

    Membership inference attacks against machine learning models

    Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP), pages 3–18, 2017. 13 Table 2: Description of important notations Notations Description D The co...

    2017

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.