REVIEW 2 major objections 5 minor 64 references
Where synthetic samples go in federated learning matters more than how many are generated; an entropy-adaptive budget recovers most accuracy at 94% less cost.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-11 01:37 UTC pith:24E54YLF
load-bearing objection Clean allocation paper: entropy-adaptive fill-to-threshold recovers most of Full-Balance accuracy at ~6% of the samples, and matched-budget gaps show WHERE really matters. the 2 major comments →
WHERE to Generate Matters: Budget-Aware Synthetic Augmentation for Label Skewed Federated Learning
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Synthetic augmentation for label-skewed federated learning is primarily a budget-allocation problem: how much each client generates and where the samples go matter more than generating until every class is balanced. FedEAS computes a per-class generation budget from each client’s normalized entropy and mean class count; skewed clients receive larger budgets, samples fill only classes below that threshold, and the total generation budget is the sum of these local decisions rather than a fixed global quota. On CIFAR-10 and CIFAR-100 under Dirichlet label skew this recovers most of the accuracy gain of full class balancing while cutting generation by 94.1%, and under matched budgets it outperfo
What carries the argument
The entropy-adaptive budget rule bk = floor(sqrt(mean class count) × (1 − normalized entropy) × β). It turns each client’s local label distribution into a per-class generation count that automatically gives more budget to skewed clients, routes samples only to scarce classes, and lets a single scalar β set the overall operating point.
Load-bearing premise
The main experiments assume a high-quality class-conditional generator trained on the entire centralized training set, which is often unavailable or privacy-prohibited in real federated deployments.
What would settle it
Replace the full-training-set generator with one trained only on private client data (or under differential privacy) and re-run the matched-budget CIFAR experiments; if FedEAS no longer recovers most of the full-balance accuracy gain at roughly 6% of the generation volume, or if its gap over uniform allocation collapses, the allocation claim does not survive realistic generators.
If this is right
- Full class balancing is unnecessary; most of its accuracy gain is available at a small fraction of the generation cost.
- At any fixed generation budget, placing samples by local label entropy substantially outperforms uniform or missing-class-only placement.
- Accuracy rises monotonically with the budget parameter, so the operating point can be chosen from available generation resources alone.
- In the balanced (IID) limit the policy reduces to ordinary FedAvg and can be left on by default.
- Data-level entropy budgets can be combined with optimization-level corrections to exceed either method alone.
Where Pith is reading between the lines
- The same fill-to-threshold idea may transfer to non-image federated settings whenever a class-conditional generator exists.
- Because the rule reads only local label counts, it can sit on top of any future privacy-preserving generator without redesign.
- The observed diminishing-returns curve of imbalance reduction suggests an automatic cost-aware stopping rule that would remove the need to hand-pick β.
- Matched-budget protocols should become standard when comparing synthetic-augmentation policies, because total sample count alone does not determine accuracy.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper addresses label skew in federated learning by treating synthetic augmentation as a budget-allocation problem rather than full class balancing. FedEAS assigns each client a per-class generation budget bk = floor(sqrt(n_bar_k) * (1 - H_tilde_k) * beta) from local label counts, then fills only classes below that threshold. The total generation budget therefore emerges from the per-client rule. On CIFAR-10/100 under Dirichlet partitions the method recovers most of Full-Balance accuracy while using 94.1% fewer samples (13 437 vs 226 783 on CIFAR-10) and, under a matched-budget protocol that holds generator and total samples fixed, outperforms Uniform allocation by up to 18.82%. Supporting evidence includes multi-setting accuracy tables, a beta sweep, factor ablations, a closed-form diminishing-returns argument, and an empirical gradient-dissimilarity measurement.
Significance. If the matched-budget results hold, the work cleanly isolates allocation (WHERE) as a first-order design choice for synthetic augmentation under label skew and supplies a simple, label-count-only rule that practitioners can tune with a single scalar. The 3.7x wall-clock reduction and the transparent selection criteria for beta are concrete engineering contributions. Strengths include the matched-budget protocol that holds generator and total samples fixed, the factor ablation that isolates entropy and sub-linear size scaling, the supplementary closed-form diminishing-returns argument, and the empirical Gamma-hat measurement. The generator-realism caveat is acknowledged by the authors and partially stress-tested with SD-turbo, so the internal claim that allocation drives accuracy remains intact.
major comments (2)
- Section 4.1 and Limitations: the primary generator is a class-conditional DDPM trained on each dataset's full training set. While this isolates allocation from generator design and the SD-turbo transfer partially decouples the gap, absolute accuracy drops and privacy-preserving generator training remains open. A short additional experiment or clearer quantitative bound on how much of the reported gain survives under a federated or differentially-private generator would strengthen the deployment claim without altering the allocation result.
- Section 2.2, Eqs. (5)-(6): the link from imbalance score I to client-drift Gamma rests on three idealizing assumptions (orthogonal per-class gradients, shared gradient norm, shared per-class losses). The authors correctly note that these do not hold tightly and supply an empirical Gamma-hat measurement in the supplement; however, that measurement shows both Uniform and FedEAS nearly eliminate directional disagreement, so the 7% accuracy gap is attributed to coverage rather than alignment. A brief discussion of whether I remains a reliable surrogate once directional disagreement is already low would tighten the motivation.
minor comments (5)
- Figure 1(a) caption and main text: the hatched/solid bar distinction is clear once read carefully, but a short legend inside the figure would help readers who scan only the graphic.
- Algorithm 1 line 16: aggregation weights use original N_k; a one-sentence reminder that synthetic samples do not alter client weights would prevent a common misreading.
- Section 4.3: the two selection conditions for beta (remove >=50% of I(0), synthetic fraction <25%) are transparent but appear only in the text; restating them next to Figure 5(b) would improve reproducibility.
- Supplementary Table 2 reports peak accuracy; a single sentence in the main text noting that last-20-round means and peaks tell the same ranking story would reassure readers who prefer peak metrics.
- Typos: 'Accuracyalsorisesmonotonically' (page 2) and occasional missing spaces after periods in the arXiv source should be cleaned.
Circularity Check
No significant circularity: FedEAS is an explicit heuristic budget rule validated by matched-budget empirical comparisons, not a derivation that reduces to its inputs.
full rationale
The paper proposes a hand-designed allocation policy bk = floor(sqrt(n_bar_k) * (1 - H_tilde_k) * beta) motivated by three observations on an imbalance score I(B) that is computed solely from label counts (Eqs. 4-9, Section 2.2-3.1). beta is selected by two transparent, pre-training criteria on the same counts (remove >=50% of I(0) while synthetic fraction <25%; Section 4.3 and Fig. 5b). Accuracy claims are then measured on held-out global test sets under a matched-budget protocol that isolates allocation (Uniform/Missing-only/FedEAS share the identical generator and total B_FE). Nothing in the equations forces the reported test accuracies, the 94.1% budget reduction, or the gaps versus Uniform; those are empirical outcomes against independent baselines. The I-to-Gamma link is presented with explicit idealizing assumptions and is not used as a fitted predictor of accuracy. There are no self-definitional loops, no parameters fitted to the target metric and then re-reported as predictions, no load-bearing uniqueness theorems, and no self-citation chains that close the argument. The work is self-contained against external benchmarks.
Axiom & Free-Parameter Ledger
free parameters (3)
- β (budget scalar) =
12
- size-factor exponent (√n̄) =
0.5
- imbalance-reduction and synthetic-fraction thresholds =
0.5 / 0.25
axioms (4)
- domain assumption Label skew is the dominant source of client drift that can be mitigated by class-count rebalancing.
- domain assumption A fixed pre-trained class-conditional generator produces samples whose utility is determined primarily by allocation, not by per-sample fidelity differences across policies.
- ad hoc to paper First-order change in imbalance score I correctly ranks the value of placing a sample in a scarce versus abundant class.
- ad hoc to paper Idealizing assumptions (orthogonal per-class gradients, shared gradient norm, shared per-class losses) link I to the FedAvg heterogeneity term Γ.
invented entities (1)
-
FedEAS entropy-adaptive per-class budget rule
no independent evidence
read the original abstract
Label skew in federated learning (FL) causes client drift and degrades global accuracy. Synthetic data augmentation can reduce this imbalance; however, full class balancing requires substantial computation cost. We propose FedEAS, a policy that assigns each client an entropy-adaptive per-class generation budget computed from its local label distribution. The budget jointly decides \emph{how much} each client generates and \emph{WHERE} the samples go. Accordingly, the total generation budget follows from the per-client budgets rather than being fixed in advance. FedEAS recovers most of the accuracy gain of full class balancing while reducing the generation budget by 94.1\%. At the same total generation budget, it outperforms Uniform allocation by up to 18.82\% across CIFAR-10 and CIFAR-100.
Figures
Reference graph
Works this paper leans on
-
[1]
Artificial intelligence and statistics , pages=
Communication-efficient learning of deep networks from decentralized data , author=. Artificial intelligence and statistics , pages=. 2017 , organization=
work page 2017
-
[2]
Federated Learning with Non-IID Data
Federated learning with non-iid data , author=. arXiv preprint arXiv:1806.00582 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[3]
Proceedings of Machine learning and systems , volume=
Federated optimization in heterogeneous networks , author=. Proceedings of Machine learning and systems , volume=
-
[4]
International conference on machine learning , pages=
Scaffold: Stochastic controlled averaging for federated learning , author=. International conference on machine learning , pages=. 2020 , organization=
work page 2020
-
[5]
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
Model-contrastive federated learning , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
-
[6]
International conference on machine learning , pages=
Data-free knowledge distillation for heterogeneous federated learning , author=. International conference on machine learning , pages=. 2021 , organization=
work page 2021
-
[7]
Stable Diffusion-based Data Augmentation for Federated Learning with Non-IID Data
Stable diffusion-based data augmentation for federated learning with non-iid data , author=. arXiv preprint arXiv:2405.07925 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[8]
Advances in neural information processing systems , volume=
Denoising diffusion probabilistic models , author=. Advances in neural information processing systems , volume=
-
[9]
Is synthetic data from generative models ready for image recognition? , author=. The 11th International Conference on Learning Representations (ICLR 2023)(01/05/2023-05/05/2023, Kigali, Rwanda) , year=
work page 2023
-
[10]
Transactions on Machine Learning Research , year=
Synthetic Data from Diffusion Models Improves ImageNet Classification , author=. Transactions on Machine Learning Research , year=
-
[11]
Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining , pages=
Fedrs: Federated learning with restricted softmax for label distribution non-iid data , author=. Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining , pages=
-
[12]
International Conference on Machine Learning , pages=
Federated learning with label distribution skew via logits calibration , author=. International Conference on Machine Learning , pages=. 2022 , organization=
work page 2022
-
[13]
International Conference on Learning Representations , year=
On the Convergence of FedAvg on Non-IID Data , author=. International Conference on Learning Representations , year=
-
[14]
International Conference on Learning Representations , year=
FedMix: Approximation of Mixup under Mean Augmented Federated Learning , author=. International Conference on Learning Representations , year=
-
[15]
Proceedings of the Computer Vision and Pattern Recognition Conference , pages=
A simple data augmentation for feature distribution skewed federated learning , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=
-
[16]
International Conference on Learning Representations , volume=
Effective data augmentation with diffusion models , author=. International Conference on Learning Representations , volume=
-
[17]
Advances in Neural Information Processing Systems , volume=
No fear of heterogeneity: Classifier calibration for federated learning with non-iid data , author=. Advances in Neural Information Processing Systems , volume=
-
[18]
Proceedings of the IEEE/CVF international conference on computer vision , pages=
No fear of classifier biases: Neural collapse inspired federated learning with synthetic and fixed classifier , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=
-
[19]
Proceedings of the AAAI Conference on Artificial Intelligence , volume=
Exploiting label skews in federated learning with model concatenation , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=
-
[20]
Proceedings of the AAAI Conference on Artificial Intelligence , volume=
Exploring vacant classes in label-skewed federated learning , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=
-
[21]
Tianfei Zhou and Ender Konukoglu , booktitle=. Fed. 2023 , url=
work page 2023
-
[22]
Advances in Neural Information Processing Systems , volume=
Fedavp: Augment local data via shared policy in federated learning , author=. Advances in Neural Information Processing Systems , volume=
-
[23]
Differentially Private Diffusion Models Generate Useful Synthetic Images
Differentially private diffusion models generate useful synthetic images , author=. arXiv preprint arXiv:2302.13861 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[24]
Foundations and trends in machine learning , volume=
Advances and open problems in federated learning , author=. Foundations and trends in machine learning , volume=. 2021 , publisher=
work page 2021
-
[25]
Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification
Measuring the effects of non-identical data distribution for federated visual classification , author=. arXiv preprint arXiv:1909.06335 , year=
work page internal anchor Pith review Pith/arXiv arXiv 1909
-
[26]
Transactions on Machine Learning Research , issn=
On the Unreasonable Effectiveness of Federated Averaging with Heterogeneous Data , author=. Transactions on Machine Learning Research , issn=. 2024 , url=
work page 2024
-
[27]
2022 IEEE 38th international conference on data engineering (ICDE) , pages=
Federated learning on non-iid data silos: An experimental study , author=. 2022 IEEE 38th international conference on data engineering (ICDE) , pages=. 2022 , organization=
work page 2022
-
[28]
International Conference on Learning Representations , year=
Federated Learning Based on Dynamic Regularization , author=. International Conference on Learning Representations , year=
-
[29]
Advances in neural information processing systems , volume=
Tackling the objective inconsistency problem in heterogeneous federated optimization , author=. Advances in neural information processing systems , volume=
-
[30]
International Conference on Learning Representations , year=
mixup: Beyond Empirical Risk Minimization , author=. International Conference on Learning Representations , year=
-
[31]
Proceedings of the IEEE/CVF international conference on computer vision , pages=
Cutmix: Regularization strategy to train strong classifiers with localizable features , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=
-
[32]
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
High-resolution image synthesis with latent diffusion models , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
-
[33]
International Conference on Learning Representations , year=
Denoising Diffusion Implicit Models , author=. International Conference on Learning Representations , year=
-
[34]
Advances in neural information processing systems , volume=
Diffusion models beat gans on image synthesis , author=. Advances in neural information processing systems , volume=
-
[35]
NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications , year=
Classifier-Free Diffusion Guidance , author=. NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications , year=
work page 2021
-
[36]
International Conference on Learning Representations , year=
Adaptive Federated Optimization , author=. International Conference on Learning Representations , year=
-
[37]
IEEE signal processing magazine , volume=
Federated learning: Challenges, methods, and future directions , author=. IEEE signal processing magazine , volume=. 2020 , publisher=
work page 2020
-
[38]
International conference on machine learning , pages=
Improved denoising diffusion probabilistic models , author=. International conference on machine learning , pages=. 2021 , organization=
work page 2021
-
[39]
International Conference on Learning Representations , year=
Score-Based Generative Modeling through Stochastic Differential Equations , author=. International Conference on Learning Representations , year=
-
[40]
Communications of the ACM , volume=
Generative adversarial networks , author=. Communications of the ACM , volume=. 2020 , publisher=
work page 2020
-
[41]
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
Analyzing and improving the image quality of stylegan , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
-
[42]
European Conference on Computer Vision , pages=
Adversarial diffusion distillation , author=. European Conference on Computer Vision , pages=. 2024 , organization=
work page 2024
-
[43]
Proceedings of the IEEE conference on computer vision and pattern recognition , pages=
Deep residual learning for image recognition , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=
-
[44]
Learning multiple layers of features from tiny images , author=. 2009 , institution=
work page 2009
-
[45]
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
Autoaugment: Learning augmentation strategies from data , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
-
[46]
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops , pages=
Randaugment: Practical automated data augmentation with a reduced search space , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops , pages=
-
[47]
Improved Regularization of Convolutional Neural Networks with Cutout
Improved regularization of convolutional neural networks with cutout , author=. arXiv preprint arXiv:1708.04552 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[48]
A survey on image data augmentation for deep learning , author=. Journal of big data , volume=. 2019 , publisher=
work page 2019
-
[49]
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
Fake it till you make it: Learning transferable representations from synthetic imagenet clones , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
-
[50]
Advances in Neural Information Processing Systems , volume=
Stablerep: Synthetic images from text-to-image models make strong visual representation learners , author=. Advances in Neural Information Processing Systems , volume=
-
[51]
Advances in neural information processing systems , volume=
Ensemble distillation for robust model fusion in federated learning , author=. Advances in neural information processing systems , volume=
-
[52]
International conference on machine learning , pages=
Ditto: Fair and robust federated learning through personalization , author=. International conference on machine learning , pages=. 2021 , organization=
work page 2021
-
[53]
International conference on machine learning , pages=
Exploiting shared representations for personalized federated learning , author=. International conference on machine learning , pages=. 2021 , organization=
work page 2021
-
[54]
Personalized Federated Learning: A Meta-Learning Approach
Personalized federated learning: A meta-learning approach , author=. arXiv preprint arXiv:2002.07948 , year=
work page internal anchor Pith review Pith/arXiv arXiv 2002
-
[55]
Advances in neural information processing systems , volume=
Learning imbalanced datasets with label-distribution-aware margin loss , author=. Advances in neural information processing systems , volume=
-
[56]
International Conference on Learning Representations , year=
Decoupling Representation and Classifier for Long-Tailed Recognition , author=. International Conference on Learning Representations , year=
-
[57]
Proceedings of the 2016 ACM SIGSAC conference on computer and communications security , pages=
Deep learning with differential privacy , author=. Proceedings of the 2016 ACM SIGSAC conference on computer and communications security , pages=
work page 2016
-
[58]
International Conference on Learning Representations , year=
Learning Differentially Private Recurrent Language Models , author=. International Conference on Learning Representations , year=
-
[59]
International workshop on trustworthy federated learning , pages=
Federated learning with gan-based data synthesis for non-iid clients , author=. International workshop on trustworthy federated learning , pages=. 2022 , organization=
work page 2022
-
[60]
Chinese Conference on Pattern Recognition and Computer Vision (PRCV) , pages=
Federated learning based on diffusion model to cope with non-iid data , author=. Chinese Conference on Pattern Recognition and Computer Vision (PRCV) , pages=. 2023 , organization=
work page 2023
-
[61]
FedDrip: Federated learning with diffusion-generated synthetic image , author=. IEEE Access , volume=. 2025 , publisher=
work page 2025
-
[62]
IEEE Transactions on Mobile Computing , volume=
Feature matching data synthesis for non-IID federated learning , author=. IEEE Transactions on Mobile Computing , volume=. 2024 , publisher=
work page 2024
-
[63]
Federated Learning Empowered by Generative Content
Federated learning empowered by generative content , author=. arXiv preprint arXiv:2312.05807 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[64]
IEEE Transactions on Mobile Computing , volume=
Filling the missing: Exploring generative AI for enhanced federated learning over heterogeneous mobile edge devices , author=. IEEE Transactions on Mobile Computing , volume=. 2024 , publisher=
work page 2024
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.