REVIEW 3 major objections 5 minor 1 cited by
Exemplar-condensed Federated Class-incremental Learning
T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read This paper claims that federated clients can replace stored real images with condensed synthetic exemplars and forget less, reporting 49.17% average accuracy on CIFAR-100 and 27.97% on the last task.
desk verdict A plausible method for federated class-incremental learning, but the headline numbers contradict each other and the empirical case is not verifiable as written. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
ECoral's load-bearing component is the condensed exemplar set $\mathcal{M}_{\text{cond}}$, a fixed-size set of synthetic images produced per client by three losses: gradient matching ($\mathcal{L}_{\text{cond}}$) aligns the gradients induced by condensed and real mini-batches in a local condensation model; relationship matching ($\mathcal{L}_{\text{rel}}$) keeps the feature relationships among condensed images consistent with those among real images; and meta-knowledge compensate matching ($\mathcal{L}_{\text{MKCL}}$) pulls each condensed feature toward unbiased class prototypes and away from others. The prototypes come from FINCH clustering on features generated by a Shared-VAE, a disentangled generative model that each client updates with global parameters, letting clients produce features for local and unseen classes. An adjustable memory allocation re-balances slots between old and new classes at each task. This machinery is supervised by knowledge distillation from the previous task's model, and the whole local update is aggregated with FedAvg.
What would settle it
Run a controlled federated class-incremental experiment in which the global model is frozen during the condensation phase, then compare ECoral's condensed exemplars against an equal-sized set of randomly sampled real exemplars: if the condensed examples do not yield higher average accuracy and lower forgetting when the local gradient-matching model is decoupled from the global model, the central transfer claim is falsified.
Extended reading notes
Core claim
The central claim is that replacing sampled real exemplars with condensed synthetic exemplars in each client's fixed memory improves both average accuracy and retention in federated class-incremental learning. ECoral condenses local data online by matching the gradients that real and condensed images induce in a local condensation model, matching feature relationships between condensed and real images, and pulling condensed features toward unbiased class prototypes while pushing them away from other classes. A Shared-VAE, updated with global parameters, generates disentangled features for local and unseen classes, and FINCH clustering turns those features into prototypes; this is meant to counteract the meta-information heterogeneity that arises when each client condenses from a different non-IID slice. The paper reports consistent gains over seven baselines across CIFAR-100, TinyImageNet, and Caltech-256, with the largest margins under the most skewed non-IID partition, and it positions ECoral as a memory component that can be combined with existing replay methods.
Load-bearing premise
The method assumes that gradients computed on a small local condensation model, which is re-initialized whenever new classes arrive and trained only on real local images, produce exemplars whose training signal still matches the federated global model's update directions after aggregation; the paper does not analyze how re-initialization or non-IID client updates affect that transfer.
Editorial extensions
If this is right
- A fixed per-client memory can rehearse more class knowledge when its slots hold condensed images instead of raw samples, because each synthetic image is optimized to carry task-specific gradient information.
- Under strong non-IID skew (σ=0.2), ECoral's average accuracy is 43.52%, its largest margin over baselines, suggesting condensation helps most when clients see very different class distributions.
- Ablations attribute the gain to all four components: full ECoral adds 12.45 percentage points over replay, with compensation matching alone contributing 5.01 percentage points.
- Because condensed exemplars are synthetic and derived only from the client's local data, the method reduces the privacy exposure of rehearsal memory compared with storing raw images.
- The method is designed as a plug-in memory module, so existing federated class-incremental approaches that use replay can swap in condensed exemplars without redesigning their training loop.
Reading between the lines
- We infer that ECoral's advantage should shrink as the local condensation model's gradient updates diverge from the global model's update path; a direct test would be to freeze the global model during the condensation phase and measure the resulting accuracy gap.
- The privacy argument is stronger than the evidence: condensed images are called indecipherable, but dataset condensation can still encode identifiable cues about source images, so a formal privacy analysis would be needed to support the claim.
- The paper notes its advantage decreases on more complex datasets, so we infer the method will be most attractive for small-image benchmarks and may need stronger distillation losses for high-resolution or fine-grained domains.
- A testable extension is to shuffle class order across clients, since the current client-increment strategy follows a fixed old, in-between, and new grouping, and to check whether the condensation losses remain aligned with the global model.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Exemplar-Condensed federated class-incremental learning (ECoral), a replay-based method that replaces sampled real exemplars with condensed synthetic exemplars generated via gradient matching, feature matching, and a meta-knowledge contrastive loss, supported by a shared VAE for client-wise feature disentanglement. The claimed contribution is that ECoral improves accuracy and reduces forgetting compared to existing FCIL baselines, while also being privacy-friendly because it stores condensed data instead of raw images. The manuscript reports experiments on CIFAR-100, TinyImageNet, and Caltech-256 under non-IID settings, with multiple evaluation metrics and an ablation study.
Significance. The conceptual direction—using dataset condensation rather than sample selection for replay in federated class-incremental learning—is timely and potentially useful, especially for privacy and memory-constrained settings. If the empirical claims were reliable, the method would offer a clear improvement over existing rehearsal baselines and could be integrated into other approaches. However, the current manuscript contains severe internal inconsistencies in the reported experimental numbers, which makes the central empirical claim unverifiable as written. The paper also relies on an unexamined assumption that locally condensed exemplars remain effective after FedAvg aggregation. The idea is worth pursuing, but the evidence presented does not currently support the headline conclusions.
major comments (3)
- [Abstract and Section 6 (CIFAR-100 50-task results)] The reported 50-task CIFAR-100 numbers are mutually contradictory. The abstract states ECoral achieves 32.42% accuracy with 50 tasks, while Section 6 states that in the 50-task setup ECoral achieves an average accuracy of 91.00% and a final-task accuracy of 64.40%, outperforming BiC (90.50%) and iCaRL (90.00%). These are about 59 percentage points apart for the same setting. Furthermore, the rightmost panel of Figure 3 has a y-axis maximum of 80%, so neither 91.00% average nor 64.40% final accuracy can be read from the plotted curves. This is a load-bearing inconsistency because the paper's central claim of long-term superiority is based on this experiment. The authors must correct the abstract, Section 6, and Figure 3 so that all reported values agree, and they should explain the source of the discrepancy.
- [Table 1 (CIFAR-100, 10-task results)] Table 1's sub-tables (A, Aincre, Aa) appear misaligned relative to their headers, which casts doubt on the claimed improvements. For example, in the A sub-table at σ=0.5, ECoral's Aavg is 49.17 and Alast is 27.97, but in the Aincre sub-table, ECoral's Aincre_last at σ=0.5 is 43.52, which exactly equals the Aavg value reported for σ=0.2 (43.52). This suggests that the columns have been shifted, making it impossible to determine the true values for each metric and condition. Since the quantitative claims of improvement (e.g., 5.32% and 10.29% over baselines) are computed from these numbers, the table must be restructured with clearly matching metrics, and the authors should verify that each reported value is placed under the correct header.
- [Section 4.1, Eqs. (3)-(5)] The core mechanism of ECoral is gradient matching performed on a local condensation model that is re-initialized whenever new classes appear and is trained only on real local images. The paper asserts that the resulting condensed exemplars will also be effective for the global model after FedAvg aggregation, but no analysis or control experiment is provided to justify this transfer. Given that the empirical results are currently inconsistent, the authors should at least include a sanity check—for example, a comparison of ECoral's condensed exemplars against random real exemplars under identical memory budgets in a single-client and a multi-client setting—to demonstrate that the condensation step actually improves global accuracy rather than merely matching local gradients. This would address the load-bearing assumption that locally condensed exemplars survive aggregation.
minor comments (5)
- [Abstract] There is a typo in the phrase 'Exemplar-Condensed federated class-increment al learning' where 'incremental' is split across a line break; this should be corrected to a single word.
- [Section 5.2 and Table 2 caption] The number of classes per task for TinyImageNet is inconsistent: Section 5.2 states 10 tasks with 20 classes per task, while Table 2's caption says 'Tiny-ImageNet with 10 tasks (10 classes per task).' Please clarify the correct task/class split.
- [Equation (8)] The cosine similarity formula is not formatted clearly; the denominator appears to be missing the multiplication sign and the temperature parameter τ is not defined in the surrounding text. Please rewrite the equation and define τ explicitly.
- [Figure labels] Several figure labels contain 'uni00A0' artifacts (e.g., 'Task/uni00A0Step' in Figures 3 and 4). These should be replaced with proper spaces or non-breaking space characters.
- [Table 3] The ablation table header (A, G, F, C, K) is not self-explanatory; the component names are only given in the caption. Consider adding a legend or using full component names in the header for readability.
Circularity Check
No circularity: ECoral's claims rest on external benchmark measurements rather than on definitions or self-citations.
full rationale
The paper's derivation chain is a method design followed by empirical evaluation. The condensed exemplars are produced by minimizing Lmem = Lcond + Lrel + βLMKCL (Eq. 10), where Lcond matches gradients between condensed and real mini-batches, Lrel matches feature relationships, and LMKCL contrasts class prototypes. The reported accuracies are then measured on held-out test sets of CIFAR-100, TinyImageNet, and Caltech-256; these measurements are not defined in terms of Lmem and are not the optimization objective, so no result reduces to its own input by construction. The hyperparameter β is chosen by grid search and disclosed ('Based on grid search, we set β = 0.5 for all experiments'), which is standard tuning rather than a fitted parameter being renamed a prediction. The bibliography contains no self-citations by the authors that carry the argument; the cited techniques (dataset condensation, SSD, Shared-VAE, FINCH, iCaRL, GLFC) are external prior work. The central comparative claim is therefore self-contained against external benchmarks, and no circular step can be exhibited. I separately note, as a correctness risk only, that the manuscript contains internally inconsistent reported numbers (abstract: 32.42% for 50-task CIFAR-100; Section 6: 'average accuracy of 91.00%' for the same 50-task setup, and Table 1 sub-table shifts), but inconsistency is not circularity under the rubric.
Assumptions & free parameters
free parameters (5)
- beta =
0.5
- lambda =
3
- tau =
not specified
- EWC constraint factor =
300
- learning rate =
0.003
assumptions (5)
- domain assumption Gradient matching on a locally trained condensation model transfers to the federated global model
- ad hoc to paper Shared-VAE can generate reliable features for unseen classes
- domain assumption FINCH clustering produces class-representative unbiased prototypes
- domain assumption LDA non-IID partitions approximate real-world federated data skew
- standard math Standard ML background: cross-entropy, KL divergence, FedAvg aggregation
Cite this review
Pith. "Pith review of Exemplar-condensed Federated Class-incremental Learning." pith.science (2026). https://pith.science/paper/DD2A5MG7
@misc{pith2026241218926,
author = {Pith},
title = {Pith review of: Exemplar-condensed Federated Class-incremental Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/DD2A5MG7}},
note = {Machine review of arXiv:2412.18926}
}
read the original abstract
We propose Exemplar-Condensed federated class-incremental learning (ECoral) to distil the training characteristics of real images from streaming data into informative rehearsal exemplars. The proposed method eliminates the limitations of exemplar selection in replay-based approaches for mitigating catastrophic forgetting in federated continual learning (FCL). The limitations particularly related to the heterogeneity of information density of each summarized data. Our approach maintains the consistency of training gradients and the relationship to past tasks for the summarized exemplars to represent the streaming data compared to the original images effectively. Additionally, our approach reduces the information-level heterogeneity of the summarized data by inter-client sharing of the disentanglement generative model. Extensive experiments show that our ECoral outperforms several state-of-the-art methods and can be seamlessly integrated with many existing approaches to enhance performance.
Figures
Figures from the paper (3 more)
Forward citations
Cited by 1 Pith paper
-
Unleashing the Power of Continual Learning on Non-Centralized Devices: A Survey
A review of non-centralized continual learning that taxonomizes data-, model-, and device-level methods and benchmarks twelve federated continual learning methods on six datasets.
Reference graph
Works this paper leans on
-
[1]
Gradient based sample selection for online continual learning
[Aljundi et al., 2019] Rahaf Aljundi, Min Lin, Baptiste Gou- jaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. Advances in neural informa- tion processing systems, 32,
work page 2019
-
[5]
Don’t forget, there is more than forgetting: new metrics for continual learning
[D´ıaz-Rodr´ıguez et al., 2018] Natalia D´ıaz-Rodr´ıguez, Vin- cenzo Lomonaco, David Filliat, and Davide Maltoni. Don’t forget, there is more than forgetting: new metrics for continual learning. arXiv preprint arXiv:1810.13166,
arXiv 2018
-
[6]
Fed- erated class-incremental learning
[Dong et al., 2022] Jiahua Dong, Lixu Wang, Zhen Fang, Gan Sun, Shichao Xu, Xiao Wang, and Qi Zhu. Fed- erated class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10164–10173,
work page 2022
-
[7]
Federated learning via synthetic data
[Goetz and Tewari, 2020] Jack Goetz and Ambuj Tewari. Federated learning via synthetic data. arXiv preprint arXiv:2008.04489,
arXiv 2020
-
[13]
Blockchained federated learning for inter- net of things: A comprehensive survey
[Jiang et al., 2024] Yanna Jiang, Baihe Ma, Xu Wang, Guangsheng Yu, Ping Yu, Zhe Wang, Wei Ni, and Ren Ping Liu. Blockchained federated learning for inter- net of things: A comprehensive survey. ACM Computing Surveys, 56(10):1–37,
work page 2024
-
[14]
Overcom- ing catastrophic forgetting in neural networks
[Kirkpatrick et al., 2017] James Kirkpatrick, Razvan Pas- canu, Neil Rabinowitz, Joel Veness, Guillaume Des- jardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcom- ing catastrophic forgetting in neural networks. Proceed- ings of the national academy of sciences , 114(13):3521– 3526,
2017
-
[15]
Learning multiple layers of features from tiny im- ages
[Krizhevsky et al., 2009] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny im- ages
2009
-
[18]
Meta knowledge condensation for federated learning
[Liu et al., 2022] Ping Liu, Xin Yu, and Joey Tianyi Zhou. Meta knowledge condensation for federated learning. arXiv preprint arXiv:2209.14851,
arXiv 2022
Show all 38 references
-
[19]
Gradient episodic memory for continual learning
[Lopez-Paz and Ranzato, 2017] David Lopez-Paz and Marc’Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30,
2017
-
[20]
Continual federated learning based on knowledge distillation
[Ma et al., 2022] Yuhang Ma, Zhongle Xie, Jue Wang, Ke Chen, and Lidan Shou. Continual federated learning based on knowledge distillation. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, volume 3,
2022
-
[21]
Reducing catastrophic forgetting with learning on synthetic data
[Masarczyk and Tautkute, 2020] Wojciech Masarczyk and Ivona Tautkute. Reducing catastrophic forgetting with learning on synthetic data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 252–253,
2020
-
[22]
Communication-efficient learning of deep networks from decentralized data
[McMahan et al., 2017] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Ar- cas. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273–1282. PMLR,
2017
-
[23]
Federated learning for internet of things: A comprehensive survey
[Nguyen et al., 2021] Dinh C Nguyen, Ming Ding, Pub- udu N Pathirana, Aruna Seneviratne, Jun Li, and H Vin- cent Poor. Federated learning for internet of things: A comprehensive survey. IEEE Communications Surveys & Tutorials, 23(3):1622–1658,
2021
-
[25]
Pytorch: An imperative style, high- performance deep learning library
[Paszke et al., 2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. Advances in neural in- formatio...
2019
-
[26]
Better generative replay for continual federated learning
[Qi et al., 2023] Daiqing Qi, Handong Zhao, and Sheng Li. Better generative replay for continual federated learning. arXiv preprint arXiv:2302.13001,
2023
-
[28]
Efficient parameter-free clustering using first neighbor relations
[Sarfraz et al., 2019] Saquib Sarfraz, Vivek Sharma, and Rainer Stiefelhagen. Efficient parameter-free clustering using first neighbor relations. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8934–8943,
2019
-
[29]
Dataset distillation
[Wang et al., 2018] Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba, and Alexei A Efros. Dataset distillation. arXiv preprint arXiv:1811.10959,
2018 arXiv
-
[30]
Cafe: Learning to condense dataset by aligning features
[Wang et al., 2022] Kai Wang, Bo Zhao, Xiangyu Peng, Zheng Zhu, Shuo Yang, Shuo Wang, Guan Huang, Hakan Bilen, Xinchao Wang, and Yang You. Cafe: Learning to condense dataset by aligning features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recogn...
2022
-
[31]
Large scale incremental learning
[Wu et al., 2019] Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 374–382,
2019
-
[32]
Feddm: Iterative distribution matching for communication-efficient feder- ated learning
[Xiong et al., 2023] Yuanhao Xiong, Ruochen Wang, Min- hao Cheng, Felix Yu, and Cho-Jui Hsieh. Feddm: Iterative distribution matching for communication-efficient feder- ated learning. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , page...
2023
-
[33]
Feder- ated continual learning with weighted inter-client transfer
[Yoon et al., 2021] Jaehong Yoon, Wonyong Jeong, Gi- woong Lee, Eunho Yang, and Sung Ju Hwang. Feder- ated continual learning with weighted inter-client transfer. In International Conference on Machine Learning , pages 12073–12086. PMLR,
2021
-
[34]
Target: Federated class- continual learning via exemplar-free distillation
[Zhang et al., 2023] Jie Zhang, Chen Chen, Weiming Zhuang, and Lingjuan Lyu. Target: Federated class- continual learning via exemplar-free distillation. In Pro- ceedings of the IEEE/CVF International Conference on Computer Vision, pages 4782–4793,
2023
-
[35]
Dataset condensation with distribution matching
[Zhao and Bilen, 2023] Bo Zhao and Hakan Bilen. Dataset condensation with distribution matching. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 6514–6523,
2023
-
[36]
Dataset condensation with gradient match- ing
[Zhao et al., 2020] Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. Dataset condensation with gradient match- ing. arXiv preprint arXiv:2006.05929,
2020 arXiv
-
[37]
Federated Class- incremental Learning (FCIL) aims to collaboratively train a global model using streaming data that sequentially intro- duces new classes
A Full Preliminaries Federated Class-incremental Learning. Federated Class- incremental Learning (FCIL) aims to collaboratively train a global model using streaming data that sequentially intro- duces new classes. In this context, a model training process consists of a series ...
2022
-
[38]
Therefore, condensed datasets from non-IID sources introduce a meta- information heterogeneity problem that adversely affects the global model’s performance, mirroring the challenges posed by non-IID data in traditional federated learning. B More Experiments Details B.1 Data D...
2017
-
[2007]
Summarizing stream data for memory- constrained online continual learning
[Gu et al., 2024] Jianyang Gu, Kai Wang, Wei Jiang, and Yang You. Summarizing stream data for memory- constrained online continual learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 12217–12225,
2024
-
[2009]
Tiny imagenet visual recognition challenge
[Le and Yang, 2015] Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3,
2015
-
[2015]
Learning without forgetting
[Li and Hoiem, 2017] Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence , 40(12):2935–2947,
2017
-
[2016]
beta-vae: Learning basic visual concepts with a con- strained variational framework
[Higgins et al., 2017] Irina Higgins, Loic Matthey, Arka Pal, Christopher P Burgess, Xavier Glorot, Matthew M Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with a con- strained variational framework. ICLR (Poster), 3,
2017
-
[2017]
Fedsynth: Gra- dient compression via synthetic data in federated learning
[Hu et al., 2022] Shengyuan Hu, Jack Goetz, Kshitiz Malik, Hongyuan Zhan, Zhe Liu, and Yue Liu. Fedsynth: Gra- dient compression via synthetic data in federated learning. arXiv preprint arXiv:2204.01273,
2022 arXiv
-
[2018]
Dataset distillation by matching training trajectories
[Cazenavette et al., 2022] George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei A Efros, and Jun-Yan Zhu. Dataset distillation by matching training trajectories. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4750–4759,
2022
-
[2019]
Understanding disentan- gling in beta-vae
[Burgess et al., 2018] Christopher P Burgess, Irina Higgins, Arka Pal, Loic Matthey, Nick Watters, Guillaume Des- jardins, and Alexander Lerchner. Understanding disentan- gling in beta-vae. arXiv preprint arXiv:1804.03599,
2018 arXiv
-
[2020]
Caltech-256 object category dataset
[Griffin et al., 2007] Gregory Griffin, Alex Holub, Pietro Perona, et al. Caltech-256 object category dataset. Tech- nical report, Technical Report 7694, California Institute of Technology Pasadena,
2007
-
[2021]
Federated learning for smart healthcare: A survey
[Nguyen et al., 2022] Dinh C Nguyen, Quoc-Viet Pham, Pubudu N Pathirana, Ming Ding, Aruna Seneviratne, Zi- huai Lin, Octavia Dobre, and Won-Joo Hwang. Federated learning for smart healthcare: A survey. ACM Computing Surveys (Csur), 55(3):1–37,
2022
-
[2022]
Rie- mannian walk for incremental learning: Understanding forgetting and intransigence
[Chaudhry et al., 2018] Arslan Chaudhry, Puneet K Doka- nia, Thalaiyasingam Ajanthan, and Philip HS Torr. Rie- mannian walk for incremental learning: Understanding forgetting and intransigence. In Proceedings of the Euro- pean conference on computer vision (ECCV) , pages 532– 547,
2018
-
[2023]
icarl: Incremental classifier and representation learning
[Rebuffi et al., 2017] Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Pro- ceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 2001–2010,
2017
-
[2024]
Deep residual learning for image recog- nition
[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778,
2016
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.