Pith. sign in

REVIEW 3 major objections 6 minor 61 references

AFCL: Analytic Federated Continual Learning for Spatio-Temporal Invariance of Non-IID Data

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper proves that a federated continual learning system can be built entirely without gradients, and that its global model is exactly the centralized joint-learning solution regardless of how data are split across clients or tasks.

desk verdict The recursive analytic aggregation is a genuine, useful result, but Theorem 2 as stated overclaims order invariance by ignoring label-column permutation. read the letter →

arxiv 2505.12245 v1 pith:MXPLP3MA submitted 2025-05-18 cs.LG cs.AI

Jianheng Tang , Huiping Zhuang , Jingyu He , Run He , Jingchao Wang , Kejia Fan , Anfeng Liu , Tian Wang
show 5 more authors
Leye Wang Zhanxing Zhu Shanghang Zhang Houbing Herbert Song Yunhuai Liu
This is my paper · ORCID
classification cs.LGcs.AI
keywords analyticlearningfederatedcontinualcatastrophicforgettingspatio-temporalinvariancenon-IIDdataclass-incrementalclosed-formleastsquaresfrozenpre-trainedbackbone
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that catastrophic forgetting in federated continual learning is caused by gradient updates on non-IID data, and that a fully gradient-free design removes the problem rather than patching it. It proposes AFCL, in which clients freeze a pre-trained backbone, fit a linear classifier by closed-form least squares, and upload only Gram-matrix and weight statistics. The server recursively aggregates these statistics into a Global Knowledge Matrix; the paper proves the resulting global model equals the model that centralized joint learning would produce on all data from all clients. The claimed consequence is spatio-temporal invariance: the final model does not depend on how heterogeneous the data are across clients or tasks, nor on client order. A sympathetic reader would care because it turns the hardest FCL failure mode into an algebraic identity, with single-epoch training and single-round aggregation as side benefits.

What carries the argument

The load-bearing object is the Global Knowledge Matrix $G_k$, updated by $G_k = [A_k G_{k-1} + B_k \hat{W}_k,\; B_k \check{W}_k]$ with $A_k$ and $B_k$ defined through cumulative regularized Gram matrices $\tilde{R}_k = \sum_{i=1}^k R_i$, where $R_i = F_i^\top F_i + \gamma I$. The proof uses the Woodbury matrix identity to show by induction that $G_k = (\tilde{R}_k)^{-1} F_{1:k}^\top Y_{1:k}$; then the server's reconstruction $W_k = [\tilde{R}_k - (k-1)\gamma I]^{-1} \tilde{R}_k G_k$ collapses algebraically to the ridge-regression solution over all data seen so far. The known-unknown class splitting, implemented through dynamic one-hot encoding functions, keeps the block label matrix non-overlapping so the same recursion supports class-continual growth.

What would settle it

On a small synthetic dataset, implement the recursive aggregation of equations (8)-(10) and compare the result with direct computation of equation (14); any nonzero difference beyond floating-point tolerance, or any change in the final $W_K$ when the clients are processed in a different order, would refute Theorems 1 and 2. Equivalently, with a deliberately weak frozen backbone, AFCL's accuracy should be predictable from linear-probe accuracy on those frozen features; if AFCL systematically exceeds that ceiling, the claim that the classifier is a pure least-squares fit would be wrong.

Watch

Extended reading notes

Core claim

The paper claims that spatio-temporal catastrophic forgetting in federated continual learning can be eliminated, not mitigated, by removing gradients from training. On the client side, each virtual client forward-passes its data through a frozen pre-trained backbone, encodes labels with known-unknown one-hot splitting, and solves two ridge regressions in closed form, yielding local weights for previously known and newly introduced classes plus a regularized Gram matrix. On the server side, a recursively updated Global Knowledge Matrix absorbs each client's local statistics, and the final global classifier $W_k$ computed from it is exactly $(F_{1:k}^\top F_{1:k} + \gamma I)^{-1} F_{1:k}^\top Y_{1:k}$, the centralized joint-learning solution over all data seen so far. Because the objective is invariant to row permutation of the data, the global model is independent of how samples are split across clients and tasks, and independent of client registration order.

Load-bearing premise

The load-bearing premise is that the frozen pre-trained backbone already extracts features good enough for every task in the stream; the proof of exact equivalence is algebraic, but if the backbone is not well pre-trained for the task distribution, the exact solution is uniformly weak and spatio-temporal invariance becomes vacuous.

Editorial extensions

If this is right

  • Average accuracy should stay constant as spatial heterogeneity intensifies, such as smaller Dirichlet $\alpha$, because the model is the same function of the pooled data; the experiments show AFCL's accuracy staying flat while baselines collapse.
  • Increasing the number of temporal tasks should not degrade the final model, since the pooled-data objective is unchanged; the results show no accuracy drop when the task count rises from 5 to 10.
  • Client registration order cannot affect the final weights; permuting the clients yields the same $W_K$ up to floating-point roundoff.
  • Each client needs only one forward pass and one upload per round, and the server only one aggregation, so communication and computation scale without multi-round synchronization or gradient backpropagation.
  • New classes can appear in any round through the known-unknown split without retraining old clients or storing raw samples, because the recursion consumes only Gram matrices and local closed-form weights.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same recursive aggregation should transfer to any ridge-regression head on frozen features, including regression, multi-label classification, or kernelized features, because the proof uses only the block structure of the label matrix and the Woodbury identity.
  • A testable prediction is that AFCL's accuracy ceiling equals the linear-probe accuracy of the frozen features; pairing the recursion with random projections or kernels could raise that ceiling while preserving the invariance result.
  • The privacy claim deserves closer scrutiny: the uploaded regularized Gram matrix and local weight matrices are sufficient statistics of local data, and in low-sample or low-dimensional regimes they may reveal more than the statement that the uploads 'cannot be used to infer' local data.
  • The asynchronous virtual-client treatment means one server update handles both new data from existing clients and newly joined clients; a natural extension is to compare this exact aggregation with federated averaging when clients can share only a subset of their statistics.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. AFCL is a federated continual learning method that avoids gradient updates entirely. Each client extracts features with a frozen pretrained backbone, solves a ridge-regression classifier for 'known' and 'unknown' classes, and uploads local weights plus regularized Gram matrices. The server recursively updates a global knowledge matrix and then converts it to the global model. The paper proves (Lemma 2, Theorem 1) that the recursive update equals the closed-form centralized ridge solution over all data seen so far, and claims (Theorem 2) that the final model is invariant to data partitioning and client order. Experiments on CIFAR-100, Tiny-ImageNet, and ImageNet-R compare AFCL against seven baselines and show large accuracy and runtime gains, with AFCL's accuracy exactly constant across heterogeneity settings for each dataset.

Significance. Theorem 1 is a genuinely useful result: it gives a simple, single-pass aggregation that exactly reproduces centralized ridge regression over non-IID federated data, and the induction proof in Appendix C is self-contained and correct. The known-unknown class splitting extends analytic learning to class-continual FCL without storing raw data. The experiments support the partition-invariance corollary: the AFCL numbers in Table 1 are identical across all alpha and T settings, exactly as the theory predicts. At the same time, the order-invariance claim is overstated: Appendix D only analyzes row permutations with a fixed label-column ordering, whereas the actual algorithm constructs one-hot columns incrementally by registration order. Thus the advertised 'identical and invariant' statement needs to be weakened to invariance up to a class-column permutation, or the proof must model column permutations. The acknowledged dependence on a well-pretrained backbone (Section 5) is a real scope limitation but not a mathematical flaw.

major comments (3)
  1. [Theorem 2 / Appendix D / Eqs. (2), (38), (45)] The proof of Theorem 2 models spatio-temporal heterogeneity and client reordering as a row permutation P of the pooled matrices F and Y (Eq. 45). However, in the actual algorithm the one-hot columns are created in first-appearance order by Eq. (2), so changing which client or task first introduces a class also permutes the columns of Y_{1:K} and hence the columns of W. As a literal numerical statement, 'the final global model ... is independent of spatio-temporal data heterogeneity and client registration order' is false; what holds is that the predictive function is unchanged up to the induced permutation of output classes. Please restate Theorem 2 accordingly and update the proof, or prove the stronger statement by explicitly tracking the column permutation. This is load-bearing because the abstract and Section 3.5 advertise exact invariance.
  2. [Section 5 / experimental scope] The paper states in Section 5 that AFCL relies on a well-pretrained frozen backbone. Since Theorem 1 only equates AFCL with the centralized ridge solution on the fixed features, a poor backbone makes the invariance property vacuous. The experiments use a single pretrained ResNet-18, so the empirical claim that AFCL is 'consistently superior' is not tested across backbone quality. I recommend adding ablations with weaker or differently pretrained backbones, or at least a quantitative statement of the dependence, to delineate the scope of the claimed advantage.
  3. [Section 4.2 / Table 1] All AFCL rows in Table 1 are exactly constant across settings, which is strong evidence for the theoretical invariance. However, no standard deviations or seed counts are reported for the baselines, some of which are stochastic (e.g., FedCBC's generative replay, FedMGP's prompts). The margins over the second-best baseline would be more convincing with at least three seeds and mean +/- std; please also state whether the AFCL entries are single deterministic runs or exact values. This affects the strength of the empirical 'consistent superiority' claim but not the core derivations.
minor comments (6)
  1. [Section 3.4] The sentence 'the final global model W_K obtained by our AFCL is equivalent to empirical risk minimization in (1) with the full datasets D_{1:k} from all K clients' should read D_{1:K}, since the context is the final model over all K clients.
  2. [Figure 2] The labels 'Wights for the Known' and 'Wights for the Unknown' contain a typo and should read 'Weights for the Known' and 'Weights for the Unknown'.
  3. [Appendix A] The word 'addtion' should be 'addition' in the sentence describing temporal heterogeneity.
  4. [Algorithm 1 / Section 3.1] Algorithm 1 line 2 and Section 3.1 refer to Tanh(..., Theta_T), but no Tanh mapping is defined or used in the method description; remove it or introduce it explicitly.
  5. [Appendix E] The notation 'W in R^{l_e in (d_k-d_{k-1})}' is a typo; it should be 'check W_k in R^{l_e times (d_k-d_{k-1})}'.
  6. [Table 1] The 'Improve' column should be defined in the caption; the text reports an average margin of 93.38%, which appears to be the mean of the relative improvements in the table, but the calculation is not stated.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the central equivalence is proven by induction rather than assumed; the only caveats are non-load-bearing self-citations and a non-circular proof gap in Theorem 2's client-order statement.

full rationale

Theorem 1 is not circular. The recursive update (8)-(10) is a constructed algorithm, and Lemma 2 proves by induction, using the Woodbury identity, that G_k=(F_{1:k}^T F_{1:k}+kγI)^{-1} F_{1:k}^T Y_{1:k}; Theorem 1 then multiplies by [R̃_k-(k-1)γI]^{-1} R̃_k and obtains exactly the centralized closed-form solution (F_{1:k}^T F_{1:k}+γI)^{-1} F_{1:k}^T Y_{1:k}. The proof does not assume the conclusion; it derives it. No parameter is fitted and then renamed as a prediction. The self-citations to ACIL, AFL, and the blockwise recursive Moore-Penrose inverse are background technique; Lemma 2's algebra is reproduced in the appendix and does not rest on those citations, so they are not load-bearing. One non-circular weakness should be noted: the proof of Theorem 2 (Appendix D) models client reordering only as a row permutation P of F and Y (F*=PF, Y*=PY), but the known-unknown splitting in Eq. (2) builds one-hot columns incrementally in first-appearance order, so swapping clients can permute label columns as well. This means the literal numerical identity claim for W_K under arbitrary registration order is under-proved; what follows from Theorem 1 is invariance of the predictive function up to the induced column permutation. That is a proof gap, not a circularity, because it does not use the target claim as an input. Finally, the paper's stated reliance on a well-pretrained frozen backbone is a stated limitation, not a circular inference.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The paper's central theorem is a derivation from the standard closed-form least-squares solution and the Woodbury matrix identity. It introduces one hyperparameter (gamma) and relies on the frozen-backbone assumption and the known-unknown label block structure. There are no invented physical entities, mediators, or conserved quantities.

free parameters (1)
  • gamma = not specified for main results (sweep in Table 2)
    Regularization coefficient in the least-squares objective (Eq. 1, 5, 6). Chosen by hand; default value used in Table 1 is not stated.
assumptions (4)
  • domain assumption The frozen pre-trained backbone yields fixed features that are identical across clients and sufficient for the downstream classification tasks.
    Invoked in Section 3.1 and 3.3; all clients share Backbone(·, ΘB) and never update it. The equivalence proof and the experiments depend on these features.
  • domain assumption The learning objective is regularized MSE over one-hot labels for a linear classifier.
    Eq. (1), Section 3.1. The closed-form solution and MSE-to-accuracy link are assumed; no cross-entropy training is considered.
  • ad hoc to paper The class-continual label matrix has the block structure of Eq. (38): rows from earlier clients have zeros in columns of classes introduced later.
    Required by Lemma 2's induction in Appendix C. It is guaranteed by the known-unknown class splitting protocol in Section 3.2, not by the data itself.
  • standard math The matrices (F_k^T F_k + gamma I) are invertible for all k.
    Used in Lemma 1 and the recursive aggregation. With gamma > 0, F_k^T F_k is positive semi-definite, so the sum is positive definite. For gamma = 0, invertibility requires the Gram matrix to be full rank.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AFCL: Analytic Federated Continual Learning for Spatio-Temporal Invariance of Non-IID Data." pith.science (2026). https://pith.science/paper/MXPLP3MA

@misc{pith2026250512245,
  author       = {Pith},
  title        = {Pith review of: AFCL: Analytic Federated Continual Learning for Spatio-Temporal Invariance of Non-IID Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MXPLP3MA}},
  note         = {Machine review of arXiv:2505.12245}
}
read the original abstract

Federated Continual Learning (FCL) enables distributed clients to collaboratively train a global model from online task streams in dynamic real-world scenarios. However, existing FCL methods face challenges of both spatial data heterogeneity among distributed clients and temporal data heterogeneity across online tasks. Such data heterogeneity significantly degrades the model performance with severe spatial-temporal catastrophic forgetting of local and past knowledge. In this paper, we identify that the root cause of this issue lies in the inherent vulnerability and sensitivity of gradients to non-IID data. To fundamentally address this issue, we propose a gradient-free method, named Analytic Federated Continual Learning (AFCL), by deriving analytical (i.e., closed-form) solutions from frozen extracted features. In local training, our AFCL enables single-epoch learning with only a lightweight forward-propagation process for each client. In global aggregation, the server can recursively and efficiently update the global model with single-round aggregation. Theoretical analyses validate that our AFCL achieves spatio-temporal invariance of non-IID data. This ideal property implies that, regardless of how heterogeneous the data are distributed across local clients and online tasks, the aggregated model of our AFCL remains invariant and identical to that of centralized joint learning. Extensive experiments show the consistent superiority of our AFCL over state-of-the-art baselines across various benchmark datasets and settings.

Figures

Figures reproduced from arXiv: 2505.12245 by the authors.

Figure 1
Figure 1. The spatial-temporal data heterogeneity and catastrophic forgetting in FCL. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The framework of our proposed AFCL. 3.1 Motivation and Overview We provide a detailed analysis of spatio-temporal data heterogeneity in Appendix A. Based on it, we identify that the root cause of spatio-temporal catastrophic forgetting lies in the inherent vulnerability and sensitivity of gradients to non-IID data. Motivated by this insight, our AFCL is designed to fundamentally address spatio-temporal catastrophic … view at source ↗
Figure 3
Figure 3. Average accuracy of AFCL and the baselines among different tasks. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Average knowledge retention of AFCL and the baselines among different tasks. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: The gradient update perspective of catastrophic forgetting with non-IID data in FCL. [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 44 canonical work pages

  1. [1]

    Ten challenging problems in federated foundation models, 2025

    Tao Fan, Hanlin Gu, Xuemei Cao, Chee Seng Chan, Qian Chen, Yiqiang Chen, Yihui Feng, Yang Gu, Jiaxiang Geng, Bing Luo, Shuoling Liu, Win Kent Ong, Chao Ren, Jiaqi Shao, Chuan Sun, Xiaoli Tang, Hong Xi Tae, Yongxin Tong, Shuyue Wei, Fan Wu, Wei Xi, Mingcong Xu, He Yang, Xin Yang, Jiangpeng Yan, Hao Yu, Han Yu, Teng Zhang, Yifei Zhang, Xiaojin Zhang, Zhenzh...

    work page 2025

  2. [2]

    Fedfed: Feature distillation against data heterogeneity in federated learning

    Zhiqin Yang, Yonggang Zhang, Yu Zheng, Xinmei Tian, Hao Peng, Tongliang Liu, and Bo Han. Fedfed: Feature distillation against data heterogeneity in federated learning. In Advances in Neural Information Processing Systems, volume 36, pages 60397–60428. Curran Associates, Inc., 2023

    work page 2023

  3. [3]

    Advances and open challenges in federated foundation models, 2024

    Chao Ren, Han Yu, Hongyi Peng, Xiaoli Tang, Bo Zhao, Liping Yi, Alysa Ziying Tan, Yulan Gao, Anran Li, Xiaoxiao Li, Zengxiang Li, and Qiang Yang. Advances and open challenges in federated foundation models, 2024

    work page 2024

  4. [4]

    Balancing similarity and complementarity for federated learning

    Kunda Yan, Sen Cui, Abudukelimu Wuerkaixi, Jingfeng Zhang, Bo Han, Gang Niu, Masashi Sugiyama, and Changshui Zhang. Balancing similarity and complementarity for federated learning. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org, 2024

    work page 2024

  5. [5]

    FedCal: achieving local and global calibration in federated learning via aggregated parameterized scaler

    Hongyi Peng, Han Yu, Xiaoli Tang, and Xiaoxiao Li. FedCal: achieving local and global calibration in federated learning via aggregated parameterized scaler. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org, 2024

    work page 2024

  6. [6]

    Towards efficient asynchronous federated learning in heterogeneous edge environments

    Yajie Zhou, Xiaoyi Pang, Zhibo Wang, Jiahui Hu, Peng Sun, and Kui Ren. Towards efficient asynchronous federated learning in heterogeneous edge environments. In IEEE INFOCOM 2024 - IEEE Conference on Computer Communications, pages 2448–2457, 2024

    work page 2024

  7. [7]

    Heterogeneity-guided client sampling: Towards fast and efficient non-iid federated learning

    Huancheng Chen and Haris Vikalo. Heterogeneity-guided client sampling: Towards fast and efficient non-iid federated learning. Advances in Neural Information Processing Systems , 37:65525–65561, 2024

    work page 2024

  8. [8]

    FedGMKD: An efficient prototype feder- ated learning framework through knowledge distillation and discrepancy-aware aggregation

    Jianqiao Zhang, Caifeng Shan, and Jungong Han. FedGMKD: An efficient prototype feder- ated learning framework through knowledge distillation and discrepancy-aware aggregation. Advances in Neural Information Processing Systems, 37:118326–118356, 2024

    work page 2024

Show all 61 references
  1. [9]

    Confusion-resistant federated learning via diffusion-based data harmonization on non-iid data

    Canran Xiao et al. Confusion-resistant federated learning via diffusion-based data harmonization on non-iid data. Advances in Neural Information Processing Systems, 37:137495–137520, 2024

    2024

  2. [10]

    Personalized federated continual learning via multi-granularity prompt

    Hao Yu, Xin Yang, Xin Gao, Yan Kang, Hao Wang, Junbo Zhang, and Tianrui Li. Personalized federated continual learning via multi-granularity prompt. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’24, page 4023–4034, New York, NY , ...

    2024

  3. [11]

    Target: Federated class-continual learning via exemplar-free distillation

    Jie Zhang, Chen Chen, Weiming Zhuang, and Lingjuan Lyu. Target: Federated class-continual learning via exemplar-free distillation. In Proceedings of the IEEE/CVF International Confer- ence on Computer Vision, pages 4782–4793, 2023

    2023

  4. [12]

    Federated continual learning with weighted inter-client transfer

    Jaehong Yoon, Wonyong Jeong, Giwoong Lee, Eunho Yang, and Sung Ju Hwang. Federated continual learning with weighted inter-client transfer. In International Conference on Machine Learning, pages 12073–12086. PMLR, 2021

    2021

  5. [13]

    A data-free approach to mitigate catastrophic forgetting in federated class incremental learning for vision tasks

    Sara Babakniya, Zalan Fabian, Chaoyang He, Mahdi Soltanolkotabi, and Salman Avestimehr. A data-free approach to mitigate catastrophic forgetting in federated class incremental learning for vision tasks. Advances in Neural Information Processing Systems, 36, 2024

    2024

  6. [14]

    Cross-FCL: Toward a cross-edge federated continual learning framework in mobile edge computing systems

    Zhouyangzi Zhang, Bin Guo, Wen Sun, Yan Liu, and Zhiwen Yu. Cross-FCL: Toward a cross-edge federated continual learning framework in mobile edge computing systems. IEEE Transactions on Mobile Computing, 23(1):313–326, 2024

    2024

  7. [15]

    Federated continual learning via prompt-based dual knowledge transfer

    Hongming Piao, Yichen Wu, Dapeng Wu, and Ying Wei. Federated continual learning via prompt-based dual knowledge transfer. In Forty-first International Conference on Machine Learning, 2024

    2024

  8. [16]

    Federated continual learn- ing via knowledge fusion: A survey

    Xin Yang, Hao Yu, Xin Gao, Hao Wang, Junbo Zhang, and Tianrui Li. Federated continual learn- ing via knowledge fusion: A survey. IEEE Transactions on Knowledge and Data Engineering, 36(8):3832–3850, 2024. 10

    2024

  9. [17]

    Handling spatial-temporal data heterogeneity for federated continual learning via tail anchor

    Hao Yu, Xin Yang, Le Zhang, Hanlin Gu, Tianrui Li, Lixin Fan, and Qiang Yang. Handling spatial-temporal data heterogeneity for federated continual learning via tail anchor. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  10. [18]

    Overcoming spatial-temporal catastrophic forgetting for federated class-incremental learning

    Hao Yu, Xin Yang, Xin Gao, Yihui Feng, Hao Wang, Yan Kang, and Tianrui Li. Overcoming spatial-temporal catastrophic forgetting for federated class-incremental learning. In Proceedings of the 32nd ACM International Conference on Multimedia, MM ’24, page 5280–5288, New York, NY ...

    2024

  11. [19]

    Online hyperparameter optimization for class-incremental learning

    Yaoyao Liu, Yingying Li, Bernt Schiele, and Qianru Sun. Online hyperparameter optimization for class-incremental learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 8906–8913, 2023

    2023

  12. [20]

    GKEAL: Gaussian kernel embedded analytic learning for few-shot class incremental task

    Huiping Zhuang, Zhenyu Weng, Run He, Zhiping Lin, and Ziqian Zeng. GKEAL: Gaussian kernel embedded analytic learning for few-shot class incremental task. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7746–7755, 2023

    2023

  13. [21]

    Federated class-incremental learning

    Jiahua Dong, Lixu Wang, Zhen Fang, Gan Sun, Shichao Xu, Xiao Wang, and Qi Zhu. Federated class-incremental learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10164–10173, 2022

    2022

  14. [22]

    Continual federated learning based on knowledge distillation

    Yuhang Ma, Zhongle Xie, Jue Wang, Ke Chen, and Lidan Shou. Continual federated learning based on knowledge distillation. In IJCAI, pages 2182–2188, 2022

    2022

  15. [23]

    Text-enhanced data-free approach for federated class-incremental learning

    Minh-Tuan Tran, Trung Le, Xuan-May Le, Mehrtash Harandi, and Dinh Phung. Text-enhanced data-free approach for federated class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23870–23880, 2024

    2024

  16. [24]

    Non-iid data and continual learning processes in federated learning: A long road ahead

    Marcos F Criado, Fernando E Casado, Roberto Iglesias, Carlos V Regueiro, and Senén Barro. Non-iid data and continual learning processes in federated learning: A long road ahead. Infor- mation Fusion, 88:263–280, 2022

    2022

  17. [25]

    Ranpac: Random projections and pre-trained models for continual learning

    Mark D McDonnell, Dong Gong, Amin Parvaneh, Ehsan Abbasnejad, and Anton Van den Hengel. Ranpac: Random projections and pre-trained models for continual learning. Advances in Neural Information Processing Systems, 36:12022–12053, 2023

    2023

  18. [26]

    Where to begin? on the impact of pre-training and initialization in federated learning

    John Nguyen, Jianyu Wang, Kshitiz Malik, Maziar Sanjabi, and Michael Rabbat. Where to begin? on the impact of pre-training and initialization in federated learning. In The Eleventh International Conference on Learning Representations, 2023

    2023

  19. [27]

    FedBERT: When federated learning meets pre-training

    Yuanyishu Tian, Yao Wan, Lingjuan Lyu, Dezhong Yao, Hai Jin, and Lichao Sun. FedBERT: When federated learning meets pre-training. ACM Transactions on Intelligent Systems and Technology, 13(4), August 2022

    2022

  20. [28]

    On the importance and applicability of pre-training for federated learning

    Hong-You Chen, Cheng-Hao Tu, Ziwei Li, Han Wei Shen, and Wei-Lun Chao. On the importance and applicability of pre-training for federated learning. InThe Eleventh International Conference on Learning Representations, 2023

    2023

  21. [29]

    An empirical investigation of the role of pre-training in lifelong learning

    Sanket Vaibhav Mehta, Darshan Patil, Sarath Chandar, and Emma Strubell. An empirical investigation of the role of pre-training in lifelong learning. Journal of Machine Learning Research, 24(214):1–50, 2023

    2023

  22. [30]

    ACIL: Analytic class-incremental learning with absolute memorization and privacy protection

    Huiping Zhuang, Zhenyu Weng, Hongxin Wei, Renchunzi Xie, Kar-Ann Toh, and Zhiping Lin. ACIL: Analytic class-incremental learning with absolute memorization and privacy protection. Advances in Neural Information Processing Systems, 35:11602–11614, 2022

    2022

  23. [31]

    Pseudoinverse learning algorithm for feedforward neural networks

    Ping Guo, Michael R Lyu, and NE Mastorakis. Pseudoinverse learning algorithm for feedforward neural networks. Advances in Neural Networks and Applications, 1(321-326), 2001

    2001

  24. [32]

    A progressive stacking pseudoinverse learning framework via active learning in random subspaces

    Zhenjiao Cai, Sulan Zhang, Ping Guo, Jifu Zhang, and Lihua Hu. A progressive stacking pseudoinverse learning framework via active learning in random subspaces. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 54(5):2822–2832, 2024

    2024

  25. [33]

    Bayesian pseudoinverse learners: From uncertainty to deterministic learning

    Qian Yin, Bingxin Xu, Kaiyan Zhou, and Ping Guo. Bayesian pseudoinverse learners: From uncertainty to deterministic learning. IEEE Transactions on Cybernetics, 52(11):12205–12216, 2022

    2022

  26. [34]

    Universal approximation using radial-basis-function networks

    Jooyoung Park and Irwin W Sandberg. Universal approximation using radial-basis-function networks. Neural computation, 3(2):246–257, 1991. 11

    1991

  27. [35]

    Learning from the kernel and the range space

    Kar-Ann Toh. Learning from the kernel and the range space. In 2018 IEEE/ACIS 17th International Conference on Computer and Information Science (ICIS), pages 1–6. IEEE, 2018

    2018

  28. [36]

    Noniterative deep learning: Incorporating re- stricted boltzmann machine into multilayer random weight neural networks

    Xi-Zhao Wang, Tianlun Zhang, and Ran Wang. Noniterative deep learning: Incorporating re- stricted boltzmann machine into multilayer random weight neural networks. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 49(7):1299–1308, 2017

    2017

  29. [37]

    An analytic formulation of con- volutional neural network learning for pattern recognition

    Huiping Zhuang, Zhiping Lin, Yimin Yang, and Kar-Ann Toh. An analytic formulation of con- volutional neural network learning for pattern recognition. Information Sciences, 686:121317, 2025

    2025

  30. [38]

    Densepilae: a feature reuse pseudoinverse learning algorithm for deep stacked autoencoder

    Jue Wang, Ping Guo, and Yanjun Li. Densepilae: a feature reuse pseudoinverse learning algorithm for deep stacked autoencoder. Complex & Intelligent Systems, pages 1–11, 2022

    2022

  31. [39]

    Blockwise recursive moore–penrose inverse for network learning

    Huiping Zhuang, Zhiping Lin, and Kar-Ann Toh. Blockwise recursive moore–penrose inverse for network learning. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 52(5):3237– 3250, 2021

    2021

  32. [40]

    AFL: A single-round analytic approach for federated learing with pre-trained models

    Run He, Kai Tong, Di Fang, Han Sun, Ziqian Zeng, Haoran Li, Tianyi Chen, and Huiping Zhuang. AFL: A single-round analytic approach for federated learing with pre-trained models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  33. [41]

    Locality sensitive sparse encoding for learning world models online

    Zichen Liu, Chao Du, Wee Sun Lee, and Min Lin. Locality sensitive sparse encoding for learning world models online. In The Twelfth International Conference on Learning Representations, 2024

    2024

  34. [42]

    Order-robust class incremental learning: Graph-driven dynamic similarity grouping

    Guannan Lai, Yujie Li, Xiangkun Wang, Tianrui Li Junbo Zhang, and Xin Yang. Order-robust class incremental learning: Graph-driven dynamic similarity grouping. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  35. [43]

    Knowledge memorization and rumination for pre-trained model-based class-incremental learning

    Zijian Gao, Wangwang Jia, Xingxing Zhang, Dulan Zhou, Kele Xu, Feng Dawei, Yong Dou, Xinjun Mao, and Huaimin Wang. Knowledge memorization and rumination for pre-trained model-based class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat...

    2025

  36. [44]

    TSVD: Bridging theory and practice in continual learning with pre-trained models

    Liangzu Peng, Juan Elenter, Joshua Agterberg, Alejandro Ribeiro, and Rene Vidal. TSVD: Bridging theory and practice in continual learning with pre-trained models. In The Thirteenth International Conference on Learning Representations, 2025

    2025

  37. [45]

    Boosting multiple views for pretrained-based continual learning

    Quyen Tran, Tung Lam Tran, Khanh Doan, Toan Tran, Dinh Phung, Khoat Than, and Trung Le. Boosting multiple views for pretrained-based continual learning. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  38. [46]

    Continual learning with pre-trained models: A survey

    Da-Wei Zhou, Hai-Long Sun, Jingyi Ning, Han-Jia Ye, and De-Chuan Zhan. Continual learning with pre-trained models: A survey. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24 , pages 8363–8371. International Joint Conference...

    2024

  39. [47]

    On loss functions for deep neural networks in classification

    Katarzyna Janocha and Wojciech Marian Czarnecki. On loss functions for deep neural networks in classification. arXiv preprint arXiv:1702.05659, 2017

    2017 arXiv

  40. [48]

    Evaluation of neural architectures trained with square loss vs cross-entropy in classification tasks

    Like Hui and Mikhail Belkin. Evaluation of neural architectures trained with square loss vs cross-entropy in classification tasks. In International Conference on Learning Representations, 2021

    2021

  41. [49]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009

    2009

  42. [50]

    Tiny imagenet visual recognition challenge

    Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015

    2015

  43. [51]

    The many faces of robustness: A critical analysis of out-of-distribution generalization

    Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out-of-distribution generalization. In Proceedings of the IEEE/CVF international c...

    2021

  44. [52]

    Online class incremental learning on stochastic blurry task boundary via mask and visual prompt tuning

    Jun-Yeong Moon, Keon-Hee Park, Jung Uk Kim, and Gyeong-Moon Park. Online class incremental learning on stochastic blurry task boundary via mask and visual prompt tuning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11731– 11741, 2023. 12

    2023

  45. [53]

    Ensemble distillation for robust model fusion in federated learning

    Tao Lin, Lingjing Kong, Sebastian U Stich, and Martin Jaggi. Ensemble distillation for robust model fusion in federated learning. In Advances in Neural Information Processing Systems, volume 33, pages 2351–2363. Curran Associates, Inc., 2020

    2020

  46. [54]

    Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017

    2017

  47. [55]

    Communication-Efficient Learning of Deep Networks from Decentralized Data

    Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-Efficient Learning of Deep Networks from Decentralized Data. In Aarti Singh and Jerry Zhu, editors, Proceedings of the 20th International Conference on Artificial Intelligence a...

    2017

  48. [56]

    Learning without forgetting

    Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(12):2935–2947, 2018

    2018

  49. [57]

    Overcoming catastrophic forgetting by incremental moment matching

    Sang-Woo Lee, Jin-Hwa Kim, Jaehyun Jun, Jung-Woo Ha, and Byoung-Tak Zhang. Overcoming catastrophic forgetting by incremental moment matching. In I. Guyon, U. V on Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors,Advances in Neural Information...

    2017

  50. [58]

    Deep residual learning for im- age recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for im- age recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016

    2016

  51. [59]

    Re-fed+: A better replay strategy for federated incremental learning

    Yichen Li, Haozhao Wang, Yining Qi, Wei Liu, and Ruixuan Li. Re-fed+: A better replay strategy for federated incremental learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1–12, 2025

    2025

  52. [60]

    An aggregation-free federated learning for tackling data heterogeneity

    Yuan Wang, Huazhu Fu, Renuga Kanagavelu, Qingsong Wei, Yong Liu, and Rick Siow Mong Goh. An aggregation-free federated learning for tackling data heterogeneity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 26233–26242, June 2024

    2024

  53. [61]

    Preservation of the global knowledge by not-true distillation in federated learning

    Gihun Lee, Minchan Jeong, Yongjin Shin, Sangmin Bae, and Se-Young Yun. Preservation of the global knowledge by not-true distillation in federated learning. In S. Koyejo, S. Mo- hamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processin...

    2022

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.