Server-Proximal Aggregation for Federated Domain-Incremental Learning under Partial Participation: Task-Uniform Convergence and Backward Transfer

Jian Li; Longtao Xu

REVIEW 2 major objections 7 minor 20 references

Server-Proximal Aggregation for Federated Domain-Incremental Learning under Partial Participation: Task-Uniform Convergence and Backward Transfer

T0 review · 2 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SPECIAL's one server-side proximal anchor guarantees backward transfer and a task-uniform convergence rate of $O(\sqrt{E/(NT)})$ for federated domain-incremental learning under partial participation.

desk verdict The headline rate doesn't follow from the paper's own theorem and the BKT bound rests on an unverified strong assumption, but the algorithm and experiments are worth a look. read the letter →

arxiv 2601.22274 v2 pith:HEGP2NZU submitted 2026-01-29 cs.LG

Longtao Xu , Jian Li This is my paper · ORCID

classification cs.LG

keywords federateddomain-incrementallearningserver-sideproximalanchorbackwardknowledgetransfertask-uniformconvergencepartialparticipationnon-convexoptimizationmemory-freecontinualFedAvg

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces SPECIAL, a federated continual learning algorithm for the setting where tasks are distinct input domains with the same label space and only some clients participate each round. Its central thesis is that a single server-side proximal anchor—a weighted average of the round's aggregated update and the previous global model—is enough to guarantee two properties that had been missing for this setting. First, training on a new task cannot increase the loss on any earlier task beyond a drift-controlled term that shrinks as rounds, local epochs, and per-round participation grow (Theorem 1). Second, the algorithm converges on the cumulative objective across all tasks at rate $O(\sqrt{E/(NT)})$ under partial participation, matching single-task FedAvg in communication efficiency (Theorem 2 and Corollary 1). If these guarantees hold, memory-free federated continual learning can be deployed on drifting domains without replay buffers or task-specific heads.

What carries the argument

The central mechanism is the server-side proximal anchor: after averaging the $N$ participating clients' update vectors, the server solves the quadratic problem $\min_u (\|u-\bar\theta^{t+1}_i\|^2 + \lambda\|u-\theta_{i-1}\|^2)$, whose closed form is $\theta^{t+1}_i = (\bar\theta^{t+1}_i + \lambda\theta_{i-1})/(1+\lambda)$. This one-step blend is the entire difference from FedAvg. It converts the previous global model into a compact memory that stabilizes the task-initial point, giving a uniform within-task drift bound $\|\theta^t_i-\theta^0_i\|^2 \le \gamma_G^2\gamma_L^2E^2B^2/\lambda^2$ that is independent of round and task. That drift control is what lets the proof transfer single-task FedAvg-style convergence to the cumulative objective and keeps the backward-transfer correction shrinking with $t$, $E$, and $N$.

What would settle it

On a synthetic federated domain-incremental problem with two tasks whose gradients are orthogonal, the $\epsilon$-alignment premise of Theorem 1 is violated by construction; if SPECIAL still shows the earlier-task loss decreasing beyond what Eq. (7) allows, or if the loss increase exceeds the right-hand side, then Eq. (7) is not a valid certificate in that regime. Concretely, track the cosine similarity between $\nabla f_{1:K-1}(\theta^0_K)$ and each local stochastic gradient and check the inequality round by round.

Watch

Extended reading notes

Core claim

The paper's discovery is that the placement of a proximal regularizer—on the server rather than on each client—turns a one-line modification of FedAvg into a principled federated domain-incremental learning method. At the end of each round the server replaces the plain aggregate $\bar\theta^{t+1}_i$ with $\theta^{t+1}_i = (\bar\theta^{t+1}_i + \lambda\theta_{i-1})/(1+\lambda)$, a quadratic blend toward the final model of the previous task. With this anchor, Theorem 1 bounds the expected loss on earlier tasks by their starting loss plus a vanishing correction of order $2\epsilon^2\sigma_L^2\|\nabla f_{1:K-1}(\theta^0_K)\|^2 / ((K-1)tEMNLB^2)$ under an $\epsilon$-alignment condition, and Theorem 2 gives a task-uniform non-convex stationarity bound whose best achievable rate is $O(\sqrt{E/(NT)})$. The rate explicitly separates stochastic and client variance ($\sigma_L$, $\sigma_G$) from inter-task drift ($\sigma_T$).

Load-bearing premise

The backward-transfer guarantee in Theorem 1 only holds if, at every local step of every participating client, the new task's gradient points in the same direction as the combined gradient of all earlier tasks at the start of the task; the paper's own experiments show negative backward transfer on all four datasets, which suggests that condition is not satisfied there.

Editorial extensions

If this is right

Under the alignment condition of Theorem 1, training on a new task increases the loss on any earlier task by at most a term that shrinks with more rounds, more local epochs, and more participating clients.
SPECIAL reaches a task-uniform non-convex convergence rate of $O(\sqrt{E/(NT)})$ under partial participation, matching single-task FedAvg when measured per participating update.
The bound's residual separates optimization noise ($\sigma_L$, $\sigma_G$) from inter-task drift ($\sigma_T$), so the choice of proximal weight $\lambda$ and the round/epoch trade-off can be matched to the expected amount of domain shift.
All of this is achieved without replay buffers, synthetic data, or task-specific heads: the server stores only the previous global model, so communication volume and model size stay identical to FedAvg.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the update rule $\theta^{t+1}_i = (1/(1+\lambda))\bar\theta^{t+1}_i + (\lambda/(1+\lambda))\theta_{i-1}$ is an exponential moving average across tasks, so SPECIAL can be viewed as continuously decaying old-task knowledge at a fixed rate; this suggests a testable rule for setting $\lambda$ from the measured cosine alignment between tasks—larger alignment, smaller $\lambda$, and
Editorial inference: Theorem 1's $\epsilon$-alignment condition can be checked at runtime by storing only the vector $\nabla f_{1:K-1}(\theta^0_K)$ and computing cosine similarities against incoming local gradients; because the paper reports negative backward transfer on all four datasets, a direct measurement would either confirm that the condition fails in practice or reveal that the bound is lo
Editorial inference: the momentum-like interpretation suggests a natural extension to federated class-incremental learning, where the anchor would need to separate representation drift from classifier drift; the same proof structure might apply if the inter-task drift constant $\sigma_T$ is replaced by a class-specific drift measure.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

I just read arXiv:2601.22274. The punchline: the central theoretical claims don't survive contact with the paper's own equations. The algorithm—a server-side proximal anchor on FedAvg—is simple and the experiments are reasonable, but the advertised task-uniform convergence rate is an artifact of the corollary's algebra, and the BKT bound rests on an assumption the authors don't verify.

What's genuinely new: the FDIL setting with partial participation is under-theorized, and the paper correctly identifies that existing FCL analyses focus on the last task. The proposed SPECIAL is a clean one-line modification to FedAvg, and the empirical study covers four domain-shift benchmarks with sensible baselines and ablations. If all you take away is the empirical claim that a server-side anchor beats a client-side proximal term, that's a useful data point.

The soft spots are serious. In Corollary 1, substituting the stated gamma_L and gamma_G into the vanishing term of Theorem 2 gives an O(1/sqrt(N E T)) term, not O(sqrt(E/(NT)))—the E-dependence is inverted. Worse, the chosen gamma_G violates Theorem 2's own condition gamma_G <= 1/(K-1) unless sqrt(N E) <= lambda L, which isn't true for the paper's typical hyperparameters. And the residual Psi in Eq. (9) contains a K B^2 term with no factor of gamma_L or gamma_G. That means Theorem 2's bound has a floor that does not go to zero as T grows. So even if the step-size algebra were fixed, the bound does not establish convergence to stationarity. The K B^2 floor is not a minor technical annoyance; it kills the claim that SPECIAL achieves a O(1/sqrt(T)) rate.

The BKT bound (Theorem 1) similarly sits on an epsilon-alignment condition that must hold every round, every local epoch, every client, with respect to the gradient of the prior-task sum evaluated at the task start. That's a strong, per-sample correlation requirement. The paper doesn't test it, and the uniformly negative BWT values in Table 1 suggest it doesn't hold in practice. The theorem may be true as a conditional statement, but its relevance to the experiments is unestablished.

The related work is cited appropriately, and I didn't see evidence of circular reasoning or self-citation padding. The writing is clear.

Who is this for? Practitioners looking for a memory-free heuristic for federated domain shift might get something from the empirical part. Theorists shouldn't take the rates at face value. I'd send it to peer review—the problem is important enough and the empirical method is worth a serious look—but I'd expect the reviewers to require a corrected theory or a careful retraction of the rate claims. As it stands, I would not cite the theoretical results.

Referee Report

2 major / 7 minor

Summary. The paper studies federated domain-incremental learning (FDIL) under partial participation and proposes SPECIAL, a server-side proximal anchor added to FedAvg. Two theoretical contributions are claimed: Theorem 1 gives a backward-knowledge-transfer bound on the increase of prior-task loss, and Theorem 2 together with Corollary 1 claims the first task-uniform non-convex convergence rate O(sqrt(E/(NT))) for FDIL with partial participation. Experiments on Digit-10, VLCS, PACS, and DN4IL compare SPECIAL to memory-based and memory-free baselines in terms of ACC and BWT, with ablations over the proximal weight, communication rounds, local epochs, and heterogeneity level.

Significance. If the advertised rate were valid, SPECIAL would be a useful memory-free baseline with a rigorous task-uniform guarantee, and the BKT theorem would be a meaningful extension of centralized continual-learning theory. The paper is clearly written, the algorithm is simple, and the experiments are reasonably thorough. However, the central convergence claim does not follow from the paper's own analysis: the proposed step sizes produce the wrong dependence on E and T, the step-size constraints are violated, and the residual in Theorem 2 has a non-vanishing floor. The BKT result rests on a strong alignment condition that is unverified and appears inconsistent with the reported negative BWT on all four datasets. These are load-bearing issues for the main theoretical claims.

major comments (2)

[Section 4.3, Corollary 1 and Eq. (9)] The advertised rate O(sqrt(E/(NT))) does not follow from Theorem 2. Substituting the stated gamma_L = lambda/(sqrt(KT)EL) and gamma_G = sqrt(NE)/((K-1)lambda L) into the denominator 2(1+lambda)(1-1/K)E gamma_G gamma_L T of Eq. (9) gives E gamma_G gamma_L T = sqrt(N E T)/(sqrt(K)(K-1)L^2), so the first term in Eq. (9) is O(sqrt(K)/sqrt(N E T)), with E in the denominator, not O(sqrt(E/(NT))). The step-size constraints of Theorem 2 are also violated by these choices: gamma_G <= 1/(K-1) requires sqrt(NE) <= lambda L, and gamma_G gamma_L <= (1+lambda)/(3EL) requires 3 sqrt(N) sqrt(E/T)/((K-1)L sqrt(K)) <= 1+lambda, which fails for small T. More fundamentally, the residual Psi in Eq. (9) contains the additive term (2/(1-1/K)) K B^2, with no factor of gamma_L, gamma_G, or 1/T. Consequently the right-hand side of Eq. (9) cannot go below this positive floor as T grows, so no vanishing rate, in particular no O(sqrt(E/(NT))) rate, can be concluded from Theorem 2. This is not a typo: the same per-round K B^2 residual is present in the summed bound in Appendix E.3, so the claimed Corollary 1 is an algebraic artifact of the present analysis.
[Section 4.2, Theorem 1] The epsilon-alignment condition is imposed for every communication round tau, every local epoch e, and every client m, requiring every new-task stochastic gradient to be positively correlated with the aggregate earlier-task gradient at the task-K start. This is a very strong per-sample assumption. It is not verified in the experimental section, and Table 1 reports negative BWT for SPECIAL on all four datasets, suggesting the condition does not hold in the benchmarks. As a result, Theorem 1 does not apply to the empirical evaluation, and the statement in Section 5 that the empirical BWT trends 'match' the theory is not justified. The paper should either verify the alignment condition, test a proxy for it, or state clearly that the BKT guarantee is conditional on an assumption that is not met in the reported experiments.

minor comments (7)

[Section 3] In the description of Algorithm 1, 'stochastic-gradient decent' should read 'stochastic-gradient descent'.
[Assumption 5] The word 'roud' should be 'round'.
[Section 4.4] 'Bellow' should be 'Below'.
[Figure 3 caption] The caption says results are demonstrated on 'three datasets' but lists four datasets; it should say 'four datasets' or correct the list.
[Table 3] The notation table defines T as 'Number of global epochs', but the paper uses T for communication rounds and E for local epochs; this should be corrected to avoid confusion.
[Section 5] The main results paragraph says SPECIAL attains the highest ACC on 'all three datasets', but Table 1 reports four datasets; it should say 'all four datasets'.
[Eq. (7)] The denominator notation $(K-1)tEMNLB^2$ is ambiguous; parentheses should be added, e.g., $((K-1) t E M N L B^2)$, to clarify that all quantities multiply.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: SPECIAL's bounds are derived from explicit assumptions; the disputed Corollary 1 issue is a correctness concern, not a reuse of the conclusion.

full rationale

SPECIAL's backward-knowledge-transfer bound (Theorem 1, Eq. 7) is obtained from the stated epsilon-alignment condition and explicit step-size constraints via a descent-style argument; the conclusion is not used as an input, and no fitted parameter is relabeled as a prediction. The task-uniform convergence bound (Theorem 2, Eq. 9) is derived from smoothness, bounded gradients, client sampling lemmas, and the proximal update rule; Corollary 1 subsequently chooses specific learning rates. Even if the reviewer's algebraic objection is correct—the substituted rates may not yield O(sqrt(E/(NT))), and the residual Psi in Eq. (9) contains a learning-rate-independent K B^2 term—that is a correctness flaw in the corollary, not circularity: the claimed rate is not identical by construction to an input, nor is it obtained by fitting to the target result. There are no load-bearing self-citations or imported uniqueness theorems; the server-side anchor is defined in Eq. (6) and analyzed directly. The paper's Limitations section candidly states the bounded-gradient, L-smoothness, and task-drift assumptions and the per-dataset tuning of lambda. I therefore find no significant circularity.

Assumptions & free parameters 1 free parameters · 7 assumptions · 0 invented entities

The theoretical claims rest on six standard assumptions plus a strong task-alignment condition; lambda is the only fitted hyperparameter. No new physical entities are introduced.

free parameters (1)

lambda (proximal weight) = 0.25 (Digit-10), 0.40 (VLCS), 0.05 (PACS), 0.15 (DN4IL)
Tuned per dataset by grid search; it is the only new hyperparameter of SPECIAL and controls the stability-plasticity trade-off in experiments, though the theory treats it as a fixed constant.

assumptions (7)

standard math Bounded stochastic gradients: ||nabla f_{i,m}(theta,xi)|| <= B (Assumption 1)
Standard in distributed optimization; used throughout the proofs to bound the norm of client updates and drift terms.
standard math L-smoothness of each local objective (Assumption 2)
Standard for non-convex analysis; implies f_{1:K} is KL-smooth (Lemma 2), which supports the descent lemma in Theorem 2.
standard math Unbiased local gradient estimator (Assumption 3)
Standard for SGD analysis; used to separate stochastic noise from true gradients in the proofs.
domain assumption Bounded stochastic variance sigma_L^2 (Assumption 4)
Gradient noise bound per client; standard in FL analyses and used in the BKT bound and the residual of Theorem 2.
domain assumption Bounded intra-task heterogeneity sigma_G^2 (Assumption 5)
Binds client gradients to the task average; standard in FL analyses, appearing in client-drift terms.
domain assumption Bounded inter-task gradient difference sigma_T^2 (Assumption 6)
Bounds the drift between tasks for every parameter vector; this is the paper's way of formalizing task relatedness. The residual in Theorem 2 grows with sigma_T, so the useful range of the bound requires this constant to be small.
ad hoc to paper epsilon-alignment of new-task gradients with earlier-task gradient (Theorem 1 condition)
Requires that at every round, epoch, and client, the current-task gradient has cosine similarity at least epsilon with the cumulative prior-task gradient at the task start. This condition is strong and not verified in experiments; it effectively assumes positive transfer from the new domain to old domains.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Server-Proximal Aggregation for Federated Domain-Incremental Learning under Partial Participation: Task-Uniform Convergence and Backward Transfer." pith.science (2026). https://pith.science/paper/HEGP2NZU

@misc{pith2026260122274,
  author       = {Pith},
  title        = {Pith review of: Server-Proximal Aggregation for Federated Domain-Incremental Learning under Partial Participation: Task-Uniform Convergence and Backward Transfer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HEGP2NZU}},
  note         = {Machine review of arXiv:2601.22274}
}

read the original abstract

Real-world federated systems seldom operate on static data: input distributions drift while privacy rules forbid raw-data sharing. We study this setting as Federated Domain-Incremental Learning (FDIL), where (i) clients are heterogeneous, (ii) tasks arrive sequentially with shifting domains, yet (iii) the label space remains fixed. Two theoretical pillars remain missing for FDIL under realistic deployment: a guarantee of backward knowledge transfer (BKT) and a convergence rate that holds across the sequence of all tasks with partial participation. We introduce SPECIAL (Server-Proximal Efficient Continual Aggregation for Learning), a simple, memory-free FDIL algorithm that adds a single server-side ``anchor'' to vanilla FedAvg: in each round, the server nudges the uniformly sampled participated clients update toward the previous global model with a lightweight proximal term. This anchor curbs cumulative drift without replay buffers, synthetic data, or task-specific heads, keeping communication and model size unchanged. Our theory shows that SPECIAL (i) preserves earlier tasks: a BKT bound caps any increase in prior-task loss by a drift-controlled term that shrinks with more rounds, local epochs, and participating clients; and (ii) learns efficiently across all tasks: the first communication-efficient non-convex convergence rate for FDIL with partial participation, O((E/NT)^(1/2)), with E local epochs, T communication rounds, and N participated clients per round, matching single-task FedAvg while explicitly separating optimization variance from inter-task drift. Experimental results further demonstrate the effectiveness of SPECIAL.

Figures

Figures reproduced from arXiv: 2601.22274 by the authors.

**Figure 2.** Test accuracy on Task 1, Task 2, and their union for Digit-10 under client-side vs. server-side proximal terms. At the start of task K, we initialize the model with the parameters learned on task K−1, that is, θ 0 K = θK−1. Inspired by regularizationbased continual-learning methods, we add a proximal term to the objective to discourage large departures from θK−1. This lightweight anchor curbs catastrophic forgettin… view at source ↗

**Figure 3.** Relation between ACC and BWT under varying values of [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Visualized performance w.r.t T, E, α. Influence of the proximal weight λ [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 9 canonical work pages

[1]

Elastic weight consolidation (ewc): Nuts and bolts.arXiv preprint arXiv:2105.04093,

Abhishek Aich. Elastic weight consolidation (ewc): Nuts and bolts.arXiv preprint arXiv:2105.04093,

arXiv
[3]

CoDeC: Communication-Efficient Decentralized Continual Learning

Sakshi Choudhary, Sai Aparna Aketi, Gobinda Saha, and Kaushik Roy. Codec: communication- efficient decentralized continual learning.arXiv preprint arXiv:2303.15378,

work page Pith review arXiv
[7]

Reading digits in natural images with unsupervised feature learning

Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervised feature learning. InNIPS workshop on deep learning and unsupervised feature learning, volume 2011, pp

2011
[8]

Better generative replay for continual federated learning

12 Preprint Daiqing Qi, Handong Zhao, and Sheng Li. Better generative replay for continual federated learning. arXiv preprint arXiv:2302.13001,

arXiv
[9]

Adaptive federated optimization.arXiv preprint arXiv:2003.00295,

Sashank Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Kone ˇcn`y, Sanjiv Kumar, and H Brendan McMahan. Adaptive federated optimization.arXiv preprint arXiv:2003.00295,

arXiv 2003
[11]

Rehearsal-free Federated Domain-incremental Learning

Rui Sun, Haoran Duan, Jiahua Dong, Varun Ojha, Tejal Shah, and Rajiv Ranjan. Rehearsal-free federated domain-incremental learning.arXiv preprint arXiv:2405.13900,

work page Pith review arXiv
[14]

Accurate forgetting for heterogeneous federated con- tinual learning.arXiv preprint arXiv:2502.14205,

Abudukelimu Wuerkaixi, Sen Cui, Jingfeng Zhang, Kunda Yan, Bo Han, Gang Niu, Lei Fang, Changshui Zhang, and Masashi Sugiyama. Accurate forgetting for heterogeneous federated con- tinual learning.arXiv preprint arXiv:2502.14205,

arXiv
[19]

15 Preprint Table 3: Notations and Terminologies. Notation Description M Number of all clients N Number of participated clients per round T Number of global epochs E Number of local updates K Number of tasks [n] Set of integers{1,...,n} θi Global model after completing training on taski θt i Global model at roundtin taski θt i,m Local model on clientmat r...

work page 2021

Show all 20 references

[20]

CLIENT-VS. SERVER-SIDEPROXIMALTERMS

Then we have: Et f1:K θt+1 K ≤f 1:K θt K − 1 2 γGγLE 1 +λ 1− 1 K ∇f1:K θt K 2 − γGγLE 1 +λ · 1 2−30Kγ 2 LL2E2− 3γGγL (M−N)L (1 +λ)N(M−1) · 120E3L2Kγ 2 L + 4EK ∇fK θt K 2 + γGγLE 1 +λ L2 (K−1) 2γ2 Gγ2 LE2 λ2 +K+ 3γGγL (M−N)EKL (1 +λ)N(M−1) ! B2 + γGγLE 1 +λ 5γ2 LKEL 2 + 60γGγ3 ...

2010
[2011]

A distillation-based ap- proach integrating continual learning and federated learning for pervasive services.arXiv preprint arXiv:2109.04197,

Anastasiia Usmanova, Franc ¸ois Portet, Philippe Lalanda, and German Vega. A distillation-based ap- proach integrating continual learning and federated learning for pervasive services.arXiv preprint arXiv:2109.04197,

arXiv
[2014]

Achieving linear speedup with partial worker participa- tion in non-iid federated learning.arXiv preprint arXiv:2101.11203,

Haibo Yang, Minghong Fang, and Jia Liu. Achieving linear speedup with partial worker participa- tion in non-iid federated learning.arXiv preprint arXiv:2101.11203,

arXiv
[2017]

Dual cognitive architecture: Incorporating biases and multi-memory systems for lifelong learning.arXiv preprint arXiv:2310.11341,

Shruthi Gowda, Bahram Zonooz, and Elahe Arani. Dual cognitive architecture: Incorporating biases and multi-memory systems for lifelong learning.arXiv preprint arXiv:2310.11341,

arXiv
[2018]

A data-free approach to mitigate catastrophic forgetting in federated class incremental learning for vision tasks

Sara Babakniya, Zalan Fabian, Chaoyang He, Mahdi Soltanolkotabi, and Salman Avestimehr. A data-free approach to mitigate catastrophic forgetting in federated class incremental learning for vision tasks. InThirty-seventh Conference on Neural Information Processing Systems, 2023...

arXiv
[2019]

On the convergence of continual federated learning using incrementally aggregated gradients.arXiv preprint arXiv:2411.07959,

Satish Kumar Keshri, Nazreen Shah, and Ranjitha Prasad. On the convergence of continual federated learning using incrementally aggregated gradients.arXiv preprint arXiv:2411.07959,
[2020]

On the convergence of fedavg on non-iid data.arXiv preprint arXiv:1907.02189, 2019a

Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang. On the convergence of fedavg on non-iid data.arXiv preprint arXiv:1907.02189, 2019a. Xilai Li, Yingbo Zhou, Tianfu Wu, Richard Socher, and Caiming Xiong. Learn to grow: A continual structure learning framewor...

1907 arXiv
[2021]

However, most FCL methods lack theoretical guarantees

and FLwF-2T (Usmanova et al., 2021), achieve knowledge transfer between tasks via parameter partitioning and knowledge distillation which can be regarded as two main directions of regularization-based methods. However, most FCL methods lack theoretical guarantees. Keshri et al...

2024
[2022]

14 Preprint A RELATED WORK A.1 CONTINUALLEARNING Continual learning aims to train a model on a sequence of tasks under restriction to access to previous data with the goal of mitigating catastrophic forgetting (Wang et al., 2024; Chen & Liu, 2018; Schwarz et al., 2018; Li et a...

2022
[2023]

Local sgd converges fast and communicates little.arXiv preprint arXiv:1805.09767,

Sebastian U Stich. Local sgd converges fast and communicates little.arXiv preprint arXiv:1805.09767,

arXiv
[2024]

Unbiased look at dataset bias

Antonio Torralba and Alexei A Efros. Unbiased look at dataset bias. InCVPR 2011, pp. 1521–1528. IEEE,

2011
[2025]

A proximal stochastic gradient method with progressive variance reduc- tion.SIAM Journal on Optimization, 24(4):2057–2075,

Lin Xiao and Tong Zhang. A proximal stochastic gradient method with progressive variance reduc- tion.SIAM Journal on Optimization, 24(4):2057–2075,

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Elastic weight consolidation (ewc): Nuts and bolts.arXiv preprint arXiv:2105.04093,

Abhishek Aich. Elastic weight consolidation (ewc): Nuts and bolts.arXiv preprint arXiv:2105.04093,

arXiv

[2] [3]

CoDeC: Communication-Efficient Decentralized Continual Learning

Sakshi Choudhary, Sai Aparna Aketi, Gobinda Saha, and Kaushik Roy. Codec: communication- efficient decentralized continual learning.arXiv preprint arXiv:2303.15378,

work page Pith review arXiv

[3] [7]

Reading digits in natural images with unsupervised feature learning

Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervised feature learning. InNIPS workshop on deep learning and unsupervised feature learning, volume 2011, pp

2011

[4] [8]

Better generative replay for continual federated learning

12 Preprint Daiqing Qi, Handong Zhao, and Sheng Li. Better generative replay for continual federated learning. arXiv preprint arXiv:2302.13001,

arXiv

[5] [9]

Adaptive federated optimization.arXiv preprint arXiv:2003.00295,

Sashank Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Kone ˇcn`y, Sanjiv Kumar, and H Brendan McMahan. Adaptive federated optimization.arXiv preprint arXiv:2003.00295,

arXiv 2003

[6] [11]

Rehearsal-free Federated Domain-incremental Learning

Rui Sun, Haoran Duan, Jiahua Dong, Varun Ojha, Tejal Shah, and Rajiv Ranjan. Rehearsal-free federated domain-incremental learning.arXiv preprint arXiv:2405.13900,

work page Pith review arXiv

[7] [14]

Accurate forgetting for heterogeneous federated con- tinual learning.arXiv preprint arXiv:2502.14205,

Abudukelimu Wuerkaixi, Sen Cui, Jingfeng Zhang, Kunda Yan, Bo Han, Gang Niu, Lei Fang, Changshui Zhang, and Masashi Sugiyama. Accurate forgetting for heterogeneous federated con- tinual learning.arXiv preprint arXiv:2502.14205,

arXiv

[8] [19]

15 Preprint Table 3: Notations and Terminologies. Notation Description M Number of all clients N Number of participated clients per round T Number of global epochs E Number of local updates K Number of tasks [n] Set of integers{1,...,n} θi Global model after completing training on taski θt i Global model at roundtin taski θt i,m Local model on clientmat r...

work page 2021