Pith. sign in

REVIEW 2 major objections 5 minor 79 references

Separating client data skew from task-order mismatch changes which federated continual learning methods look best.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 07:31 UTC pith:6UFHPKBF

load-bearing objection Solid FCL evaluation infrastructure: shared streams with independent α/ρ controls and B10 reporting that actually change method rankings. the 2 major comments →

arxiv 2607.08784 v1 pith:6UFHPKBF submitted 2026-06-13 cs.LG cs.AIcs.DC

HERO: A Heterogeneity-Aware Benchmark Library for Federated Continual Learning

Thinh T. H. Nguyen , Le-Tuan Nguyen , Minh-Duong Nguyen , Nhi Trinh , Anh Tran Nam Nguyet , Dung D. Le , Kok-Seng Wong This is my paper
classification cs.LG cs.AIcs.DC
keywords federated continual learningclass-incremental learningbenchmark libraryclient data skewtask-order mismatchcatastrophic forgettingbottom-client accuracyDomain-IL
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Federated continual learning papers are hard to compare because they often change datasets, task splits, client partitions, task orders, backbones, memory rules, and metrics at the same time. HERO is a benchmark library that builds shared streams by separating three choices that are usually coupled: the task split, the client data split, and the client task sequence. In its main comparable suite, alpha controls how uneven client data is within each task and rho controls how much clients disagree on task order. On CIFAR-100 and TinyImageNet, methods that lead under easy synchronized conditions are not always the most stable under stronger heterogeneity, and average accuracy can hide weak performance on the bottom clients. A separate graph Domain-IL study on molecular scaffolds shows the same interface can expose domain-shift difficulty without collapsing incompatible settings into one leaderboard.

Core claim

When methods share the same HERO-Core streams, relative behavior changes as evaluation moves from easy synchronized settings to strong client data skew and task-order mismatch. Average final accuracy is not enough: bottom-10% client accuracy and average forgetting expose different failures, and task-order mismatch favors different strategies than synchronized evaluation.

What carries the argument

HERO stream construction that separates task split, Dirichlet client data split controlled by alpha, and client task sequences controlled by rho, so the two heterogeneity sources can be varied independently while every method loads the same released stream files.

Load-bearing premise

The core benchmark assumes every client eventually sees every task and advances at the same speed, so order mismatch only reorders shared shards rather than modeling missing tasks or different client rates.

What would settle it

Rebuild the same method comparison on streams where some clients permanently never receive some tasks or progress at different local rates; if method rankings and bottom-client patterns stay the same as the alpha-rho grid, those two axes already capture the main effect, but if rankings flip the fixed-progression assumption is doing the work.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A single synchronized FCL leaderboard is not enough to claim robustness under heterogeneous client streams.
  • Bottom-client accuracy should be reported with average accuracy and forgetting, because they can disagree.
  • Methods that win under easy synchronized data may lose ground when task-order mismatch rises.
  • Released fixed streams let new methods be scored without regenerating hidden splits or schedules.
  • Domain-shift and image class-incremental studies can share an interface while staying on separate leaderboards.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Generators that allow missing-task clients would test whether current B10 gains survive incomplete local coverage common in real deployments.
  • An alpha-rho stress grid could become as standard for FCL as Dirichlet non-IID splits became for static federated learning.
  • Personalization or prompt-based methods may widen the average-to-bottom gap if weak clients systematically get harder local orders.
  • Layering communication and energy accounting on the same frozen streams would link accuracy robustness to system cost.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper introduces HERO, a heterogeneity-aware benchmark library for federated continual learning (FCL). Its core design separates three often-coupled choices—task split, client data split (controlled by Dirichlet concentration α), and client task sequence (controlled by order-mismatch parameter ρ)—and releases fixed streams so methods share the same construction. HERO-Core evaluates representative FCIL methods on CIFAR-100 and TinyImageNet under easy/mild/joint-hard anchors and a full 3×3 α–ρ grid, reporting final average accuracy (AFA), average forgetting (AF), and bottom-10% client accuracy (B10). A separate OGB-MolPCBA Domain-IL portability study tests scaffold-domain granularity with AP-based metrics. The central empirical claim is that method behavior and preferred strategies change from synchronized easy settings to heterogeneous α–ρ settings, that AFA can hide weak bottom-client performance, and that the same interface can describe non-vision Domain-IL without collapsing incompatible settings into one leaderboard.

Significance. If the results hold under the stated protocol, HERO addresses a genuine comparability gap in FCL: prior work often varies dataset, splits, orders, backbones, memory, and reporting simultaneously. Strengths that should be credited include (i) shared, seed-fixed streams with construction checks that realized skew/mismatch track nominal α/ρ (Table 5; Appendix C), (ii) multi-seed tables and full-grid robustness summaries (Tables 1–3, 7; Figure 2), (iii) explicit multi-metric reporting including B10 rather than a single average, (iv) a tiered Core/Portability/Extended design that refuses to mix incompatible modalities into one score, and (v) released streams, configs, method implementations, and reporting scripts. These are concrete infrastructure contributions for setting-aware FCL evaluation, even though HERO-Core’s external validity is intentionally limited (every client eventually sees every task at fixed speed).

major comments (2)
  1. [§4 Baseline Experiments; Appendix H/F] §4–§5 and Tables 1–3, 7: the load-bearing claim that “method choice changes across settings” and that task-order mismatch favors different strategies than synchronized evaluation is supported by shared streams and multi-seed results, but Appendix H / Table 13 leave open whether relative ranking under joint-hard is robust to equalized resource assumptions. Core methods differ in replay, generators, text side-information (LANDER), and balancing modules; documenting assumptions is good practice, yet a short controlled check (e.g., matched replay budgets or a “no-extra-module” ablation on the same streams) would strengthen the claim that ranking shifts are algorithmic rather than budget-driven.
  2. [Appendix C.3; Table 5] Appendix C.3 (minimum-support repair) and Table 5: at α=0.10 the client-skew check rises as intended, but the paper does not report how often empty client–task shards are repaired or how many samples are moved. Because ρ only reorders already-generated shards, frequent repair at low α could slightly couple “data skew” with residual support equalization and affect B10 interpretations in alpha-hard/joint-hard cells. A brief audit statistic (fraction of repaired shards; samples moved per task) would make the α axis fully inspectable.
minor comments (5)
  1. [§1 Motivation and Benchmark Gap] §1, paragraph after the benchmark-gap discussion: “Thefore, we introduce HERO” is a typo for “Therefore.”
  2. [§5 What HERO Reveals About Heterogeneity] In §5, Table 6 is discussed before Table 5 in the narrative order; renumber or reorder so construction checks (Table 5) appear before effect signatures (Table 6).
  3. [Figure 1] Figure 1 caption and legend use “T agFed” with a space; align spelling with “TagFed” used in tables.
  4. [§4.1 Anchor-Setting Results] Appendix K single-axis tables are useful; a one-sentence pointer in §4.1 that alpha-hard vs rho-hard isolate the two axes (already in Appendix K) would help readers who stop at the main text.
  5. [§3.1 Generated benchmark stream] Eq. (2) defines B(ξ,α,ρ) compactly; a short note that test shards D^te_{k,t} are fixed across methods (not regenerated) would further emphasize non-confounded comparison.

Circularity Check

0 steps flagged

No significant circularity: HERO is an empirical benchmark library whose α/ρ controls, construction checks, and AFA/AF/B10 metrics do not reduce method rankings to inputs by construction.

full rationale

HERO does not present a first-principles derivation or a fitted quantity renamed as a prediction. Stream construction (task split, Dirichlet client split via α, task-order sampling via ρ) is an explicit generator; construction checks (Table 5; Appendix C) only verify that realized skew and order mismatch track the intended knobs, which is validation of the generator rather than a tautological claim about method quality. Method rankings on CIFAR-100/TinyImageNet and the OGB-MolPCBA portability study are obtained by training fixed methods on shared released streams and computing standard post-hoc metrics (AFA, AF, B10, AP). Minimum-support repair and fixed client progression are documented protocol assumptions, not self-definitional reductions of the ranking-shift claim. Self-citations appear only as optional method-zoo entries, not as load-bearing uniqueness theorems. The central empirical claim—that behavior and best methods change from easy synchronized to heterogeneous α-ρ settings, and that AFA can hide weak B10—is therefore independent of its inputs and is not circular.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 3 invented entities

The paper is an empirical systems/benchmark contribution. Its claims rest on standard FL/CL modeling choices, a small set of hand-chosen heterogeneity controls and protocol constants, and invented benchmark constructs (HERO tiers, α/ρ axes, B10) rather than physical postulates. No formal theorem is proved; free parameters are protocol settings that define the evaluation regime.

free parameters (5)
  • α client-data Dirichlet concentration grid
    Chosen evaluation levels (100, 10, 0.1) define easy-to-hard client skew; not derived from data, but central to all heterogeneity claims.
  • ρ task-order mismatch grid
    Chosen levels (0, 0.5, 1.0) and Kendall-distance acceptance intervals define task-sequence heterogeneity; hand-set protocol controls.
  • HERO-Core training budget (K=20, T=10, C=0.25, E=5, R=40)
    Fixed participation and communication settings shape absolute scores and relative method behavior; documented defaults, not learned from a held-out principle.
  • Bottom-10% client threshold
    B10 uses the lowest 10% of clients by final accuracy; the percentile is a design choice that affects robustness conclusions.
  • OGB-MolPCBA scaffold domain counts G∈{5,15,25}
    Domain granularity levels are selected to demonstrate portability difficulty; they define the case-study stress test.
axioms (5)
  • domain assumption Client data heterogeneity within each task can be usefully modeled by class-wise Dirichlet sampling with concentration α.
    Standard FL non-IID construction (§3.2, Eq. 3); underpins the claim that α isolates client data skew.
  • domain assumption Task-order mismatch can be usefully modeled by reordering already-generated client-task shards around a shared reference order controlled by ρ.
    Core design choice (§3.2, Eq. 4 and Appendix C.4); assumes reordering without missing tasks captures the intended failure mode.
  • domain assumption Final average accuracy, average forgetting, and bottom-10% client accuracy are sufficient primary metrics for comparing image-based FCIL methods.
    Metric suite in §3.4 / Appendix D; B10 is especially important to the paper's fairness narrative.
  • ad hoc to paper Incompatible modalities/losses/metrics should not be merged into one leaderboard; portability studies remain separate tiers.
    Tiered Core/Portability/Extended organization (§3, §7); shapes how results are interpreted rather than a mathematical necessity.
  • ad hoc to paper Minimum-support repair that moves a few samples to empty client-task shards preserves intended heterogeneity without confounding task-order experiments.
    Appendix C.3 repair rule; needed for small α but is an engineering assumption about not distorting difficulty.
invented entities (3)
  • HERO benchmark library (Core / Portability / Extended tiers) no independent evidence
    purpose: Provide reusable stream construction, method execution, and reporting for FCL evaluation.
    Central contribution; independent evidence would be public adoption and external re-use of the released streams.
  • HERO-Core α-ρ stream family B(ξ,α,ρ) no independent evidence
    purpose: Separately control client data skew and client task-sequence mismatch under fixed task splits.
    Defines the main comparable benchmark cells used for all ranking claims.
  • Bottom-10% client accuracy (B10) no independent evidence
    purpose: Expose weak-client failures hidden by average accuracy.
    Key metric supporting the claim that averages can hide poor clients; standard-style but paper-specific emphasis.

pith-pipeline@v1.1.0-grok45 · 31220 in / 3570 out tokens · 34494 ms · 2026-07-13T07:31:36.910860+00:00 · methodology

0 comments
read the original abstract

Federated continual learning (FCL) evaluates how distributed clients learn from changing data streams while retaining previously learned knowledge. Existing evaluations are difficult to compare because they often change datasets, task splits, client data splits, task orders, backbones, memory assumptions, and reporting rules simultaneously. We introduce \textbf{HERO}, a heterogeneity-aware benchmark library for FCL. HERO builds benchmark streams by separating three choices that are often coupled, namely the task split, the client data split, and the client task sequence. In HERO-Core, the main comparable benchmark, $\alpha$ controls client data skew and $\rho$ controls task-order mismatch. We evaluate representative FCL methods on CIFAR-100 and TinyImageNet using final average accuracy, average forgetting, and bottom-10\% client accuracy. We also include a graph-based Domain-IL portability case study on OGB-MolPCBA, where scaffold-domain granularity changes the input distribution while the prediction task remains fixed. Our results show that method behavior changes across easy and heterogeneous settings, that average accuracy can hide weak bottom-client performance, that task-order mismatch favors different strategies from synchronized evaluation, and that the same HERO interface can expose domain-shift difficulty beyond image-based FCIL. HERO releases benchmark streams, configurations, method implementations, and reporting scripts to support reproducible and setting-aware FCL evaluation.

Figures

Figures reproduced from arXiv: 2607.08784 by Anh Tran Nam Nguyet, Dung D. Le, Kok-Seng Wong, Le-Tuan Nguyen, Minh-Duong Nguyen, Nhi Trinh, Thinh T. H. Nguyen.

Figure 1
Figure 1. Figure 1: Anchor-setting profiles from easy to mild to joint-hard settings. Each point represents one [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Full-grid robustness summary. Each point represents one method. The horizontal axis [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Graph-based Domain-IL portability on OGB-MolPCBA. Increasing the number of scaf [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Average precision on scaffold-grouped OGB-MolPCBA under increasing domain granu [PITH_FULL_IMAGE:figures/full_fig_p027_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Forgetting on scaffold-grouped OGB-MolPCBA. Forgetting becomes larger and more [PITH_FULL_IMAGE:figures/full_fig_p027_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Local-global gap on scaffold-grouped OGB-MolPCBA. The gap becomes more variable [PITH_FULL_IMAGE:figures/full_fig_p028_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Illustration of homogeneous and heterogeneous federated continual learning streams. In [PITH_FULL_IMAGE:figures/full_fig_p028_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Homogeneous and heterogeneous stream comparisons under different communication [PITH_FULL_IMAGE:figures/full_fig_p029_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Illustration of incremental classifiers and full classifiers. Incremental classifiers add heads [PITH_FULL_IMAGE:figures/full_fig_p029_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Comparison between full classifiers and incremental classifiers under global and local [PITH_FULL_IMAGE:figures/full_fig_p030_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

79 extracted references · 9 linked inside Pith

  1. [1]

    Communication-efficient learning of deep networks from decentralized data

    H Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficient learning of deep networks from decentralized data. InProceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), volume 54, pages 1273–

  2. [2]

    URLhttps://proceedings.mlr.press/v54/mcmahan17a/mcmahan17a.pdf

    PMLR, 2017. URLhttps://proceedings.mlr.press/v54/mcmahan17a/mcmahan17a.pdf. 1, 21

    2017

  3. [3]

    Advances and open problems in federated learning.Foundations and trends in machine learning, 14(1-2):1–210, 2021

    Peter Kairouz and H Brendan McMahan. Advances and open problems in federated learning.Foundations and trends in machine learning, 14(1-2):1–210, 2021. 1

    2021

  4. [4]

    Federated continual learning with weighted inter-client transfer

    Jaehong Yoon, Wonyong Jeong, Giwoong Lee, Eunho Yang, and Sung Ju Hwang. Federated continual learning with weighted inter-client transfer. InInternational conference on machine learning, pages 12073–12086. PMLR, 2021. 1, 3, 15, 21

    2021

  5. [5]

    Continual federated learning based on knowledge distillation

    Yuhang Ma, Zhongle Xie, Jue Wang, Ke Chen, and Lidan Shou. Continual federated learning based on knowledge distillation. InIjcai, pages 2182–2188, 2022. 15

    2022

  6. [6]

    Federated continuous learning with broad network architecture.IEEE Transactions on Cybernetics, 51(8):3874–3888, 2021

    Junqing Le, Xinyu Lei, Nankun Mu, Hengrun Zhang, Kai Zeng, and Xiaofeng Liao. Federated continuous learning with broad network architecture.IEEE Transactions on Cybernetics, 51(8):3874–3888, 2021. 1, 3, 15

    2021

  7. [7]

    Federated learning with non-iid data.arXiv preprint arXiv:1806.00582, 2018

    Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. Federated learning with non-iid data.arXiv preprint arXiv:1806.00582, 2018. 1

    Pith/arXiv arXiv 2018

  8. [8]

    Measuring the effects of non-identical data distri- bution for federated visual classification.arXiv preprint arXiv:1909.06335, 2019

    Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Measuring the effects of non-identical data distri- bution for federated visual classification.arXiv preprint arXiv:1909.06335, 2019

    Pith/arXiv arXiv 1909

  9. [9]

    Federated optimization in hetero- geneous networks

    Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. Federated optimization in hetero- geneous networks. InProceedings of Machine Learning and Systems (MLSys), 2020. URLhttps: //proceedings.mlsys.org/paper/2020. 15, 21

    2020

  10. [10]

    Scaffold: Stochastic controlled averaging for federated learning

    Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for federated learning. InInter- national conference on machine learning, pages 5132–5143. PMLR, 2020

    2020

  11. [11]

    Tackling the objective incon- sistency problem in heterogeneous federated optimization.Advances in neural information processing systems, 33:7611–7623, 2020

    Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, and H Vincent Poor. Tackling the objective incon- sistency problem in heterogeneous federated optimization.Advances in neural information processing systems, 33:7611–7623, 2020. 1, 15

    2020

  12. [12]

    Overcoming catastrophic forgetting in neural networks.Proceedings of the national academy of sciences, 114(13):3521–3526,

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks.Proceedings of the national academy of sciences, 114(13):3521–3526,

  13. [13]

    Learning without forgetting.IEEE transactions on pattern analysis and machine intelligence, 40(12):2935–2947, 2017

    Zhizhong Li and Derek Hoiem. Learning without forgetting.IEEE transactions on pattern analysis and machine intelligence, 40(12):2935–2947, 2017

    2017

  14. [14]

    Gradient episodic memory for continual learning.Advances in neural information processing systems, 30, 2017

    David Lopez-Paz and Marc’Aurelio Ranzato. Gradient episodic memory for continual learning.Advances in neural information processing systems, 30, 2017. 2, 15

    2017

  15. [15]

    icarl: Incre- mental classifier and representation learning

    Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incre- mental classifier and representation learning. InProceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 2001–2010, 2017. 2, 15

    2001

  16. [16]

    A continual learning survey: Defying forgetting in classification tasks

    Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence, 44(7):3366–3385, 2021. 15

    2021

  17. [17]

    Three types of incremental learning

    Gido M Van de Ven, Tinne Tuytelaars, and Andreas S Tolias. Three types of incremental learning. Nature Machine Intelligence, 4(12):1185–1197, 2022. doi: 10.1038/s42256-022-00568-3. URLhttps: //www.nature.com/articles/s42256-022-00568-3. 1, 2, 15

    doi:10.1038/s42256-022-00568-3 2022

  18. [18]

    Federated class-incremental learning

    Jiahua Dong, Lixu Wang, Zhen Fang, Gan Sun, Shichao Xu, Xiao Wang, and Qi Zhu. Federated class-incremental learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 10164–10173, 2022. URLhttps://openaccess.thecvf.com/content/ CVPR2022/papers/Dong_Federated_Class-Incremental_Learning_CVPR_2022_paper.pdf. 1, 3, 15, 21 10

    2022

  19. [19]

    Sara Babakniya, Zalan Fabian, Chaoyang He, Mahdi Soltanolkotabi, and Salman Aves- timehr. A data-free approach to mitigate catastrophic forgetting in federated class incre- mental learning for vision tasks.Advances in Neural Information Processing Systems, 36: 66408–66425, 2023. URLhttps://proceedings.neurips.cc/paper_files/paper/2023/file/ d160ea01902c33...

    2023

  20. [20]

    TARGET: Federated class-continual learning via exemplar-free distillation

    Jie Zhang, Chen Chen, Weiming Zhuang, and Lingjuan Lyu. TARGET: Federated class-continual learning via exemplar-free distillation. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 4782–4793, 2023. URLhttps://openaccess.thecvf.com/ content/ICCV2023/papers/Zhang_TARGET_Federated_Class-Continual_Learning_via_ Exemplar-F...

    2023

  21. [21]

    Text-enhanced data- free approach for federated class-incremental learning

    Minh-Tuan Tran, Trung Le, Xuan-May Le, Mehrtash Harandi, and Dinh Phung. Text-enhanced data- free approach for federated class-incremental learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23870–23880, 2024. 15, 21

    2024

  22. [22]

    Towards effi- cient replay in federated incremental learning

    Yichen Li, Qunwei Li, Haozhao Wang, Ruixuan Li, Wenliang Zhong, and Guannan Zhang. Towards effi- cient replay in federated incremental learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12820–12829, 2024. 3, 15

    2024

  23. [23]

    Traceable federated continual learning

    Qiang Wang, Bingyan Liu, and Yawen Li. Traceable federated continual learning. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12872–12881,

  24. [24]

    3, 15, 21

    URLhttps://openaccess.thecvf.com/content/CVPR2024/papers/Wang_Traceable_ Federated_Continual_Learning_CVPR_2024_paper.pdf. 3, 15, 21

  25. [25]

    No one left behind: Real-world federated class-incremental learning.IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(4):2054–2070, 2023

    Jiahua Dong, Hongliu Li, Yang Cong, Gan Sun, Yulun Zhang, and Luc Van Gool. No one left behind: Real-world federated class-incremental learning.IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(4):2054–2070, 2023. 15, 21

    2054

  26. [26]

    Class-wise balancing data replay for federated class-incremental learning.arXiv preprint arXiv:2507.07712, 2025

    Zhuang Qi, Ying-Peng Tang, Lei Meng, Han Yu, Xiaoxiao Li, and Xiangxu Meng. Class-wise balancing data replay for federated class-incremental learning.arXiv preprint arXiv:2507.07712, 2025. 3, 15, 21

    arXiv 2025

  27. [27]

    Resource- constrained federated continual learning: What does matter?ArXiv, abs/2501.08737, 2025

    Yichen Li, Yuying Wang, Jiahua Dong, Haozhao Wang, Yining Qi, Rui Zhang, and Ruixuan Li. Resource- constrained federated continual learning: What does matter?ArXiv, abs/2501.08737, 2025. 1, 3, 15

    arXiv 2025

  28. [28]

    Open graph benchmark: Datasets for machine learning on graphs.Advances in neural information processing systems, 33:22118–22133, 2020

    Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs.Advances in neural information processing systems, 33:22118–22133, 2020. 2, 5, 16, 26

    2020

  29. [29]

    LEAF: A benchmark for federated settings.arXiv preprint arXiv:1812.01097, 2018

    Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu, Tian Li, Jakub Kone ˇcný, H Brendan McMa- han, Virginia Smith, and Ameet Talwalkar. LEAF: A benchmark for federated settings.arXiv preprint arXiv:1812.01097, 2018. URLhttps://arxiv.org/abs/1812.01097. 2, 15

    Pith/arXiv arXiv 2018

  30. [30]

    Fedscale: Benchmarking model and system performance of federated learning at scale

    Fan Lai, Yinwei Dai, Sanjay Singapuram, Jiachen Liu, Xiangfeng Zhu, Harsha Madhyastha, and Mosharaf Chowdhury. Fedscale: Benchmarking model and system performance of federated learning at scale. InInternational conference on machine learning, pages 11814–11827. PMLR, 2022. 15

    2022

  31. [31]

    Flower: A friendly federated learning research framework.arXiv preprint arXiv:2007.14390, 2020

    Daniel J Beutel, Taner Topal, Akhil Mathur, Xinchi Qiu, Javier Fernandez-Marques, Yan Gao, Lorenzo Sani, Hei Li Kwing, Titouan Parcollet, Pedro PB de Gusmão, and Nicholas D Lane. Flower: A friendly federated learning research framework.arXiv preprint arXiv:2007.14390, 2020. 15

    Pith/arXiv arXiv 2007

  32. [32]

    Flamby: Datasets and bench- marks for cross-silo federated learning in realistic healthcare settings.Advances in Neural Information Processing Systems, 35:5315–5334, 2022

    Jean Ogier du Terrail, Samy-Safwan Ayed, Edwige Cyffers, Felix Grimberg, Chaoyang He, Regis Loeb, Paul Mangold, Tanguy Marchand, Othmane Marfoq, Erum Mushtaq, et al. Flamby: Datasets and bench- marks for cross-silo federated learning in realistic healthcare settings.Advances in Neural Information Processing Systems, 35:5315–5334, 2022. 2, 15

    2022

  33. [33]

    Efficient lifelong learning with a-gem.arXiv preprint arXiv:1812.00420, 2018

    Arslan Chaudhry, Marc’Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem.arXiv preprint arXiv:1812.00420, 2018. 2, 15

    Pith/arXiv arXiv 2018

  34. [34]

    Dark experience for general continual learning: a strong, simple baseline.Advances in neural information processing systems, 33:15920–15930, 2020

    Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline.Advances in neural information processing systems, 33:15920–15930, 2020. 2, 15

    2020

  35. [35]

    Learning a unified classifier incrementally via rebalancing

    Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning a unified classifier incrementally via rebalancing. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 831–839, 2019. 2, 15 11

    2019

  36. [36]

    Large scale incremental learning

    Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 374–382, 2019. 2, 15

    2019

  37. [37]

    Asynchronous federated con- tinual learning

    Donald Shenaj, Marco Toldo, Alberto Rigon, and Pietro Zanuttigh. Asynchronous federated con- tinual learning. InCVPR 2023 Workshop on Federated Learning for Computer Vision (FedVi- sion), 2023. URLhttps://openaccess.thecvf.com/content/CVPR2023W/FedVision/papers/ Shenaj_Asynchronous_Federated_Continual_Learning_CVPRW_2023_paper.pdf. 3, 15

    2023

  38. [38]

    Fedknow: Federated continual learning with signature task knowledge integration at edge

    Yaxin Luopan, Rui Han, Qinglong Zhang, Chi Harold Liu, Guoren Wang, and Lydia Y Chen. Fedknow: Federated continual learning with signature task knowledge integration at edge. In2023 IEEE 39th International Conference on Data Engineering (ICDE), pages 341–354. IEEE, 2023

    2023

  39. [39]

    Accurate forgetting for heterogeneous federated continual learning

    Abudukelimu Wuerkaixi, Sen Cui, Jingfeng Zhang, Kunda Yan, Bo Han, Gang Niu, Lei Fang, Changshui Zhang, and Masashi Sugiyama. Accurate forgetting for heterogeneous federated continual learning. In The Twelfth International Conference on Learning Representations, May 2024. 3, 15, 21

    2024

  40. [40]

    Federated continual learning goes online: Uncertainty-aware mem- ory management for vision tasks and beyond

    Giuseppe Serra and Florian Buettner. Federated continual learning goes online: Uncertainty-aware mem- ory management for vision tasks and beyond. InThe Thirteenth International Conference on Learning Representations, May 2025. 3, 15, 21

    2025

  41. [41]

    Better generative replay for continual federated learning

    Daiqing Qi, Handong Zhao, and Sheng Li. Better generative replay for continual federated learning. In The Eleventh International Conference on Learning Representations, 2023. 3, 15, 21

    2023

  42. [42]

    Overcoming catastrophic forgetting in federated class-incremental learning via federated global twin generator.arXiv preprint arXiv:2407.11078, 2024

    Thinh Nguyen, Khoa D Doan, Binh T Nguyen, Danh Le-Phuoc, and Kok-Seng Wong. Overcoming catastrophic forgetting in federated class-incremental learning via federated global twin generator.arXiv preprint arXiv:2407.11078, 2024. 3, 15

    Pith/arXiv arXiv 2024

  43. [43]

    Re-fed+: A better replay strategy for federated incremental learning.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    Yichen Li, Haozhao Wang, Yining Qi, Wei Liu, and Ruixuan Li. Re-fed+: A better replay strategy for federated incremental learning.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025. 3, 15, 21

    2025

  44. [44]

    Pilora: Prototype guided incremental lora for federated class-incremental learning

    Haiyang Guo, Fei Zhu, Wenzhuo Liu, Xu-Yao Zhang, and Cheng-Lin Liu. Pilora: Prototype guided incremental lora for federated class-incremental learning. InEuropean Conference on Computer Vision, pages 141–159. Springer, 2024. 3, 15, 21

    2024

  45. [45]

    Federated continual learning via prompt-based dual knowledge transfer

    Hongming Piao, Yichen Wu, Dapeng Wu, and Ying Wei. Federated continual learning via prompt-based dual knowledge transfer. InForty-first International Conference on Machine Learning, Jul. 2024

    2024

  46. [46]

    Closed-form merging of parameter-efficient modules for federated continual learning

    Riccardo Salami, Pietro Buzzega, Matteo Mosconi, Jacopo Bonato, Luigi Sabetta, and Simone Calderara. Closed-form merging of parameter-efficient modules for federated continual learning. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  47. [47]

    pfedmxf: Personal- ized federated class-incremental learning with mixture of frequency aggregation

    Yifei Zhang, Hao Zhu, Alysa Ziying Tan, Dianzhi Yu, Longtao Huang, and Han Yu. pfedmxf: Personal- ized federated class-incremental learning with mixture of frequency aggregation. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 30640–30650, 2025. 21

    2025

  48. [48]

    Fedagc: Federated continual learning with asymmetric gradient correction

    Chengchao Zhang, Fanhua Shang, Hongying Liu, Liang Wan, and Wei Feng. Fedagc: Federated continual learning with asymmetric gradient correction. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 3841–3850, 2025. 3, 15, 21

    2025

  49. [49]

    Federated continual graph learning

    Yinlin Zhu, Miao Hu, and Di Wu. Federated continual graph learning. InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V . 2, pages 4203–4213, 2025. 5, 7

    2025

  50. [50]

    Motion: Multi-sculpt evolutionary coarsening for federated continual graph learning

    Guancheng Wan, Fengyuan Ran, Ruikang Zhang, Wenke Huang, Xuankun Rong, Guibin Zhang, Yuxin Wu, Bo Du, and Mang Ye. Motion: Multi-sculpt evolutionary coarsening for federated continual graph learning. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. 5, 7

    2025

  51. [51]

    Federated learning based on dynamic regularization.arXiv preprint arXiv:2111.04263, 2021

    Durmus Alp Emre Acar, Yue Zhao, Ramon Matas Navarro, Matthew Mattina, Paul N Whatmough, and Venkatesh Saligrama. Federated learning based on dynamic regularization.arXiv preprint arXiv:2111.04263, 2021. 15

    Pith/arXiv arXiv 2021

  52. [52]

    Ditto: Fair and robust federated learning through personalization

    Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith. Ditto: Fair and robust federated learning through personalization. InInternational conference on machine learning, pages 6357–6368. PMLR,

  53. [53]

    Re-evaluating continual learning scenarios: A categorization and case for strong baselines.arXiv preprint arXiv:1810.12488, 2018

    Yen-Chang Hsu, Yen-Cheng Liu, Anita Ramasamy, and Zsolt Kira. Re-evaluating continual learning scenarios: A categorization and case for strong baselines.arXiv preprint arXiv:1810.12488, 2018. URL https://arxiv.org/abs/1810.12488. 15 12

    Pith/arXiv arXiv 2018

  54. [54]

    Don’t forget, there is more than forgetting: new metrics for continual learning.arXiv preprint arXiv:1810.13166, 2018

    Natalia Díaz-Rodríguez, Vincenzo Lomonaco, David Filliat, and Davide Maltoni. Don’t forget, there is more than forgetting: new metrics for continual learning.arXiv preprint arXiv:1810.13166, 2018. URL https://arxiv.org/abs/1810.13166. 15

    Pith/arXiv arXiv 2018

  55. [55]

    Online continual learning in image classification: An empirical survey.Neurocomputing, 469:28–51, 2022

    Zheda Mai, Ruiwen Li, Jihwan Jeong, David Quispe, Hyunwoo Kim, and Scott Sanner. Online continual learning in image classification: An empirical survey.Neurocomputing, 469:28–51, 2022. 15

    2022

  56. [56]

    The clear benchmark: Continual learning on real-world imagery

    Zhiqiu Lin, Jia Shi, Deepak Pathak, and Deva Ramanan. The clear benchmark: Continual learning on real-world imagery. InThirty-fifth conference on neural information processing systems datasets and benchmarks track (round 2), 2021. 15

    2021

  57. [57]

    Wilds: A benchmark of in-the-wild distribution shifts

    Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubra- mani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. Wilds: A benchmark of in-the-wild distribution shifts. InInternational conference on machine learning, pages 5637–5664. PMLR, 2021. 15

    2021

  58. [58]

    Learning multiple layers of features from tiny images.(2009),

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images.(2009),

    2009

  59. [59]

    Tiny imagenet visual recognition challenge.CS 231N, 7(7):3, 2015

    Yann Le, Xuan Yang, et al. Tiny imagenet visual recognition challenge.CS 231N, 7(7):3, 2015. 16

    2015

  60. [60]

    Deeper, broader and artier domain generalization

    Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier domain generalization. InProceedings of the IEEE international conference on computer vision, pages 5542– 5550, 2017. 16

    2017

  61. [61]

    Moment matching for multi-source domain adaptation

    Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. InProceedings of the IEEE/CVF international conference on computer vision, pages 1406–1415, 2019. 16

    2019

  62. [62]

    Col- lective classification in network data.AI magazine, 29(3):93–93, 2008

    Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Col- lective classification in network data.AI magazine, 29(3):93–93, 2008. 16

    2008

  63. [63]

    Imagenet: A large-scale hier- archical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hier- archical image database. In2009 IEEE conference on computer vision and pattern recognition, pages 248–255. IEEE, 2009. 16

    2009

  64. [64]

    Divergence measures based on the shannon entropy.IEEE Transactions on Information theory, 37(1):145–151, 2002

    Jianhua Lin. Divergence measures based on the shannon entropy.IEEE Transactions on Information theory, 37(1):145–151, 2002. 17

    2002

  65. [65]

    Fedala: Adaptive local aggregation for personalized federated learning

    Jianqing Zhang, Yang Hua, Hao Wang, Tao Song, Zhengui Xue, Ruhui Ma, and Haibing Guan. Fedala: Adaptive local aggregation for personalized federated learning. InProceedings of the AAAI conference on artificial intelligence, volume 37, pages 11237–11244, 2023. 21

    2023

  66. [66]

    Fedas: Bridging inconsistency in personalized federated learning

    Xiyuan Yang, Wenke Huang, and Mang Ye. Fedas: Bridging inconsistency in personalized federated learning. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11986–11995, 2024. 21

    2024

  67. [67]

    Fedl2p: Federated learning to personalize.Advances in Neural Information Processing Systems, 36:14818–14836, 2023

    Royson Lee, Minyoung Kim, Da Li, Xinchi Qiu, Timothy Hospedales, Ferenc Huszár, and Nicholas Lane. Fedl2p: Federated learning to personalize.Advances in Neural Information Processing Systems, 36:14818–14836, 2023. 21

    2023

  68. [68]

    Eliminating domain bias for federated learning in representation space

    Jianqing Zhang, Yang Hua, Jian Cao, Hao Wang, Tao Song, Zhengui XUE, Ruhui Ma, and Haibing Guan. Eliminating domain bias for federated learning in representation space. InThirty-seventh Confer- ence on Neural Information Processing Systems, 2023. URLhttps://openreview.net/forum?id= nO5i1XdUS0. 21

    2023

  69. [69]

    Feder- ated domain generalization with data-free on-server matching gradient

    Trong Binh Nguyen, Duong Minh Nguyen, Jinsun Park, Viet Quoc Pham, and Won-Joo Hwang. Feder- ated domain generalization with data-free on-server matching gradient. InThe Thirteenth International Conference on Learning Representations, 2025. 21

    2025

  70. [70]

    Fedssi: Rehearsal- free continual federated learning with synergistic synaptic intelligence

    Yichen Li, Yuying Wang, Haozhao Wang, Yining Qi, Tianzhe Xiao, and Ruixuan Li. Fedssi: Rehearsal- free continual federated learning with synergistic synaptic intelligence. InForty-second International Conference on Machine Learning, Jul. 2025. 21

    2025

  71. [71]

    Ezzeldin, and Salman Avestimehr

    Yavuz Faruk Bakman, Duygu Nur Yaldiz, Yahya H. Ezzeldin, and Salman Avestimehr. Federated orthog- onal training: Mitigating global catastrophic forgetting in continual federated learning. InThe Twelfth In- ternational Conference on Learning Representations, 2024. URLhttps://openreview.net/forum? id=nAs4LdaP9Y. 21 13

    2024

  72. [72]

    Spatio-temporal gradient matching for fed- erated continual learning

    Duong Minh Nguyen, Le-Tuan Nguyen, and Quoc-Viet Pham. Spatio-temporal gradient matching for fed- erated continual learning. InICML 2025 Workshop on Collaborative and Federated Agentic Workflows,

    2025

  73. [73]

    Handling spatial- temporal data heterogeneity for federated continual learning via tail anchor

    Hao Yu, Xin Yang, Le Zhang, Hanlin Gu, Tianrui Li, Lixin Fan, and Qiang Yang. Handling spatial- temporal data heterogeneity for federated continual learning via tail anchor. InProceedings of the Com- puter Vision and Pattern Recognition Conference, pages 4874–4883, 2025. 21

    2025

  74. [74]

    Can: Leveraging clients as navigators for gen- erative replay in federated continual learning

    Xuankun Rong, Jianshu Zhang, Kun He, and Mang Ye. Can: Leveraging clients as navigators for gen- erative replay in federated continual learning. InForty-second International Conference on Machine Learning, 2025. 21 14 A Extended Related Work Federated learning benchmarks and libraries.Federated learning has motivated several bench- mark and software effor...

    2025

  75. [75]

    Mechanism for continual learning, transfer, or heterogeneity

  76. [76]

    Adaptation to the HERO protocol

  77. [77]

    Method-specific assumptions, such as replay, generation, personalization, or pretraining

  78. [78]

    Hyperparameters and tuning budget

  79. [79]

    21 Table 14: Sanity checks for HERO-Core

    Code path in the repository. 21 Table 14: Sanity checks for HERO-Core. Values are reported over five benchmark seeds. Check CIFAR-100 TinyImageNet Method-separation AFA gap 19.39±1.6315.79±1.41 Easy-to-joint AFA drop 11.70±3.478.97±1.98 Mean AFA seed std 2.13±0.431.77±0.37 Mean AFA-B10 gap 5.73±2.365.73±2.81 Table 15: Default training settings for HERO-Co...