Energy-efficient Federated Learning with Dynamic Model Size Allocation

M S Chaitanya Kumar; Sai Satya Narayana J; Steve Drew; Xin Wang; Yunkai Bao

REVIEW 5 major objections 5 minor 9 references

Energy-efficient Federated Learning with Dynamic Model Size Allocation

T0 review · 5 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A federated learning framework that sizes each client's model to its available renewable energy can converge faster and use less energy than fixed-size green scheduling.

desk verdict CAMA is a plausible new combination of FedZero and HeteroFL with a capacity-based allocation algorithm, but its headline energy savings rest on an unvalidated linear energy model while the accuracy results are genuinely measured. read the letter →

arxiv 2411.15481 v1 pith:TAPEX3VC submitted 2024-11-23 cs.DC

M S Chaitanya Kumar , Sai Satya Narayana J , Yunkai Bao , Xin Wang , Steve Drew This is my paper · ORCID

classification cs.DC

keywords federatedlearningcarbonawarenessdynamicmodelsizeordereddropoutrenewableenergyclientselectionefficiencynon-IIDdata

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces CAMA, a carbon-aware federated learning strategy that decides how much of the global model each client trains based on how much excess renewable energy and spare computing capacity that client currently has. Its central claim is that replacing the all-or-nothing green-energy scheduling of the baseline it extends with a dynamic model-size allocation produces both faster convergence and lower cumulative energy use. The paper supports this with 15-round experiments on CIFAR-10 and MNIST using real solar-forecast and hardware-load traces, where the dynamic-size runs report higher final accuracy and lower total energy than the full-model-only baseline while also spreading participation more evenly across clients. If the claim holds, federated learning can be scheduled to follow renewable supply instead of competing with the grid.

What carries the argument

The load-bearing mechanism is dynamic model-size allocation driven by a batch-capacity estimate. The allocation algorithm starts at model rate $m_r = 1$ and repeatedly halves it down to a default of $\mu = 0.0625$ until the number of batches the client can execute with its available energy and spare compute covers the required work; even a client that cannot handle the smallest rate is still given the default tiny model. Ordered dropout makes the submodels nested subsets of the full network, so a layer of size $d_g \times k_g$ is reduced to $(d_g m) \times (k_g m)$, and the server aggregates present elements by a weighted average over the local models. The energy accounting is Equation 3, $E_{c,i} = e_p \times b_c \times m_r$, which scales a client's per-batch energy linearly by the model rate.

What would settle it

Take a GPU class such as the one the paper calls 'small' hardware, train the same model at model rates 1, 0.5, 0.25, and 0.125 on the same batch count, and measure wall-power draw over the run. If measured energy per batch does not scale linearly with $m_r$, the central energy-efficiency result fails.

Watch

Extended reading notes

Core claim

CAMA's central discovery is that a client's contribution to federated learning should be scaled to its energy budget rather than gating participation on being able to run the full model. In each round the server excludes power domains with no surplus energy and clients that have over-participated, estimates how many batches each remaining client can afford, and assigns the largest model rate from $\{1, 0.5, 0.25, 0.125, 0.0625\}$ that the client's batch capacity supports, with a default tiny model for clients below even the smallest threshold. The server then partitions a global layer of dimensions $d_g \times k_g$ into a sublayer of $(d_g m) \times (k_g m)$ for a client with model rate $m$, and aggregates the returned submodels by weighted averaging over the elements each submodel contains. On a Dirichlet-split CIFAR-10 task with ResNet-18, the paper reports a final accuracy of 69.6% at 1.85 kWh after 15 rounds, versus 67.0% at 2.16 kWh for the full-model-only baseline; on MNIST it reports 93.4% at 2.26 kWh versus 91.6% at 2.59 kWh.

Load-bearing premise

The energy savings are computed with Equation 3, which assumes a client's energy per batch falls in direct proportion to the fraction of the model it trains; if the real relationship has fixed overheads, the reported kWh reductions could be wrong.

Editorial extensions

If this is right

Federated learning can include devices whose renewable-energy budget is too small for a full model, instead of dropping them from training.
Communication cost falls for small-model clients, because only the submodel is transmitted in each round.
Training schedules can be tied to solar and wind forecasts, shifting work to moments when surplus green energy exists.
Selection probabilities that penalize recent participation and large-model participation produce more even client involvement over rounds.
The reported accuracy gains suggest that model-size heterogeneity itself can help under non-IID label distributions, not only under energy constraints.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper, the linear energy model in Equation 3 is an assumption, not a measurement; an obvious extension is to verify on real GPUs whether a quarter-size model really costs one quarter of the energy per batch, since fixed overheads would change the reported kWh savings.
Because the paper describes static batch normalization and cumulatively updated global statistics for privacy, a natural follow-up is to combine the dynamic-size aggregation with secure aggregation or differential privacy to protect those statistics.
The same ordered-dropout allocation could be applied to other resource dimensions, such as memory or network bandwidth, suggesting a general resource-aware model-pruning view of federated training.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

CAMA is a sensible combination of FedZero's carbon-aware client selection and HeteroFL's ordered dropout, plus a new capacity-based method for assigning model sizes (Algorithm 2). It ships code and runs real training on MNIST and CIFAR-10, so the accuracy comparison against FedZero is a measured result. The convergence claim has some support: with BN enabled, CAMA reaches about 69.6% final accuracy on CIFAR-10 versus FedZero's 67.0%, and similar on MNIST. The new allocation algorithm is a legitimate, modest extension rather than a new paradigm. Credit is also due for releasing the code and comparing against a strong baseline.

The soft spot is the energy claim, which is the paper's headline. Equation 3 postulates Ec,i = ep × bc × mr, a linear scaling of per-batch energy with model rate. But the paper defines model rates through HeteroFL's hidden channel shrinkage ratio of 0.5, which means a client at mr=0.5 halves both input and output channels. For compute-bound convolutional layers, FLOPs (and therefore energy) scale closer to mr² than mr. Add fixed per-batch overheads like data loading, kernel launches, and communication, and the reported savings of 0.31 kWh over FedZero by round 15 become brittle. The authors never measure real power; every energy number in Table 2 and Figure 3 is arithmetic from Eq. 3. If the true scaling is sublinear, the savings could shrink or even reverse for the smallest model rates. This is load-bearing for the title's promise. Also minor but real: Algorithm 2's model rate sequence has a typo — it should end at 0.0625, not 0.625. The fairness claim is asserted but never quantified; no participation variance or Gini-style metric appears. Table 2 lacks error bars despite the text saying results were averaged over five iterations.

This paper is for people working on green or energy-aware federated learning systems. The combination is plausible and worth discussing, but the empirical evidence is not yet convincing. It deserves a serious referee — not a desk reject — because the system is genuinely new, buildable, and the accuracy results are real. A referee should push for hardware measurements or at least a validated per-device energy model, and for an explicit fairness metric. If I were the editor, I'd send it to review with expectation of major revision.

Referee Report

5 major / 5 minor

Summary. The paper proposes CAMA, a carbon-aware federated learning framework that extends FedZero by dynamically reducing the model size assigned to clients based on their available renewable energy and spare computing capacity. The server estimates each client's batch-processing capability, assigns a model rate using an ordered-dropout partitioning scheme from HeteroFL, and aggregates updates from models of different sizes. Experiments on CIFAR-10 and MNIST with ResNet-18 and a CNN compare CAMA against FedZero over 15 rounds, reporting faster convergence, lower cumulative energy usage, and equitable client participation. The source code is released on GitHub.

Significance. If the central claims hold, CAMA is a practically relevant extension of FedZero: it allows clients with scarce green energy to contribute smaller models rather than being excluded, and the reported accuracy improvements over FedZero are measured directly from training curves and are independent of the energy model. The paper also provides public code, uses real Solcast solar traces, and reports results for two datasets and two model architectures. However, the headline energy savings are computed from an assumed linear energy model rather than measured on hardware, the fairness and scalability claims are never quantified, and there are structural inconsistencies in the definition of model rate. The learning/convergence result appears plausible, but the energy-efficiency claim requires validation before the paper's central message can be accepted.

major comments (5)

[Section 3, Eq. (3)] The energy-efficiency claim rests entirely on the unvalidated linear relation Ec,i = ep × bc × mr. The authors never report direct power or energy measurements on any hardware; Table 2 and Figure 3 are computed from this equation. The paper's own model construction in Section 2.2 shrinks both input and output channels by a factor of 0.5 per complexity step, so for convolutional and linear layers the FLOP count, and hence compute-bound energy, scales approximately as the square of the per-dimension model rate, not linearly. This is compounded by fixed per-batch overheads such as data loading, kernel launches, and communication, which Eq. (3) omits. Because Algorithm 2 chooses the smallest model rate precisely to minimize ep×bc×mr, the reported 0.31 kWh saving over FedZero in Table 2 is partly an artifact of the assumed model rather than an empirical result. The authors should either measure device energy at the model-rate levels they use or re-derive the savings from a validated scaling law and quantify the impact of fixed overheads.
[Section 2.2 and Algorithm 2] The definition of model rate is internally inconsistent. The text lists complexity levels with model rates {1, 0.5, 0.25, 0.125, 0.625}, but Algorithm 2 halves mr at each step, so the fourth halving returns 0.0625, not 0.625; the default value µ = 0.0625 later in the section confirms that 0.625 is likely a typo. Moreover, Section 2.2 says class b trains on '50% of the global model's parameters', but ordered dropout sends a subnetwork with layer dimensions (dg×m) × (kg×m), so m = 0.5 corresponds to 25% of the parameters in that layer, not 50%. This ambiguity is not merely cosmetic: if mr denotes a parameter fraction, Eq. (3) is inconsistent with the channel-shrinkage implementation; if mr denotes the channel ratio, the linear energy scaling in Eq. (3) is still unjustified. The manuscript must fix the rate list and define mr precisely.
[Section 2.1 and Section 3] The claims of equitable client participation and fair selection are never evaluated. Section 2.1 and the abstract assert that CAMA 'ensures equitable client participation', but no experiment reports participation counts, selection frequencies, or any fairness metric such as Jain's index or the Gini coefficient. The only reported outcomes are accuracy and energy. Either add a quantitative fairness evaluation with the actual selection distributions or remove the fairness claim from the abstract and contributions.
[Section 3 and Section 4] The scalability claim is unsupported by the experiments. The abstract and conclusion state that CAMA 'scales efficiently to handle large numbers of clients', but the experiments fix the number of clients at 100 with a maximum selection fraction of 0.1, i.e., about 10 selected clients per round. There is no study varying the client count, no measurement of server-side aggregation cost, and no wall-clock time or communication-volume comparison. Please add a scaling experiment or weaken the claim accordingly.
[Algorithm 1] Algorithm 1 is not reproducible as written. The termination condition 'until |clients| > nand count1 > 2' appears to be missing the operator 'and' (or is malformed), and line 7 contains the unparsed expression 'model size(Pd t=0 min(mspare c,t , rp,t δc ), c)' with undefined notation. The algorithm also refers to C′ without defining it. These issues make it impossible to reconstruct the client-selection procedure from the paper alone and should be corrected before publication.

minor comments (5)

[Abstract and Section 1] There are language errors: 'Despite of' should be 'Despite', and 'integratged' should be 'integrated'.
[Table 2 and Table 3] The cumulative energy for CAMA (BN True) at round 15 in Table 2 is 1.9226 kWh, while Table 3 reports a total energy usage of 1.85 kWh for the same configuration; please reconcile these numbers and state whether Table 2 is a single run or an average across iterations.
[Equations (1) and (2)] The selection probability P(c) and the statistical utility σc use p(c) inconsistently; p(c) is never defined, and the condition p(c) ≥ 1 in Eq. (2) is unclear.
[Figure 1] Figure 1 is referenced before it is formally introduced, and its caption does not explain the meaning of the power-domain and client icons; consider labeling the panels.
[Table 1] The table entry 'labels per user 2(for balanced non-iid [1])' is awkwardly formatted; it should read '2 (balanced non-IID, following [1])'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the convergence claim is independently measured and the energy model is explicitly stated, not a hidden input.

full rationale

The paper's primary novel learning claim—faster convergence and higher accuracy under non-IID data—is evaluated by actually training CNN/ResNet-18 in simulation and comparing accuracy against FedZero (Tables 3–4, Figures 2–4). Those accuracy numbers are not derived from the energy model and therefore carry independent content. The energy-efficiency figures are computed from the explicitly stated cost model in Eq. 3, Ec,i = ep × bc × mr, with hardware classes taken transparently from FedZero; the paper does not present these as direct power measurements. Because Eq. 3 is disclosed as the calculation basis, the reported energy reductions are a direct consequence of the assumed linear scaling plus Algorithm 2's model-size assignment rather than a circular derivation in which the conclusion is hidden in the premise. The legitimate weakness—that linear scaling in mr is unvalidated and may conflict with actual FLOPs scaling (~mr^2 for channel shrinkage) and fixed per-batch overheads—is a correctness/validation risk, not circularity. No load-bearing self-citation appears: HeteroFL, FedZero, and Oort are external prior works. Hence the circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper relies on several domain assumptions inherited from FedZero and HeteroFL, plus a linear energy scaling assumption that is central to the energy results. No new physical entities are introduced. The free parameters are mostly inherited or hand-chosen thresholds.

free parameters (4)

Default model size µ = 0.0625
Returned when a client cannot handle even the smallest model size; hand-chosen threshold in Algorithm 2.
Model size levels = {1, 0.5, 0.25, 0.125, 0.0625}
Five complexity levels with hidden channel shrinkage ratio 0.5, inherited from HeteroFL; the paper lists 0.625, likely a typo.
Client selection alpha = 1
User-defined parameter controlling selection probability decay, from FedZero; set to 1 in experiments.
Exclusion factor = 1
Number of rounds a client is excluded after participation, from FedZero; set to 1 in experiments.

assumptions (4)

domain assumption Energy consumption per batch scales linearly with model size
Eq. 3 defines Ec,i = ep × bc × mr; the energy-saving claims are computed from this model.
domain assumption Clients within a power domain can be powered by that domain's excess renewable energy
Taken from FedZero; the client selection filters power domains with rp,t > 0.
domain assumption Ordered dropout and the masking trick produce a valid global model from heterogeneous local models
Adopted from HeteroFL [1]; not independently validated in this paper.
domain assumption The server can accurately estimate each client's spare computing capacity and required batch count
Algorithm 1 line 7 assumes mspare,c and δc are known and accurate.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Energy-efficient Federated Learning with Dynamic Model Size Allocation." pith.science (2026). https://pith.science/paper/TAPEX3VC

@misc{pith2026241115481,
  author       = {Pith},
  title        = {Pith review of: Energy-efficient Federated Learning with Dynamic Model Size Allocation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TAPEX3VC}},
  note         = {Machine review of arXiv:2411.15481}
}

read the original abstract

Federated Learning (FL) presents a paradigm shift towards distributed model training across isolated data repositories or edge devices without explicit data sharing. Despite of its advantages, FL is inherently less efficient than centralized training models, leading to increased energy consumption and, consequently, higher carbon emissions. In this paper, we propose CAMA, a carbon-aware FL framework, promoting the operation on renewable excess energy and spare computing capacity, aiming to minimize operational carbon emissions. CAMA introduces a dynamic model adaptation strategy which adapts the model sizes based on the availability of energy and computing resources. Ordered dropout is integratged to enable the aggregation with varying model sizes. Empirical evaluations on real-world energy and load traces demonstrate that our method achieves faster convergence and ensures equitable client participation, while scaling efficiently to handle large numbers of clients. The source code of CAMA is available at https://github.com/denoslab/CAMA.

Figures

Figures reproduced from arXiv: 2411.15481 by the authors.

**Figure 2.** Accuracy for Dirichlet distribution on CIFAR10 dataset. CAMAFL with batch normalization obtains a better average performance than FedZero [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

9 extracted references · 7 canonical work pages

[1]

E. Diao, J. Ding, and V. Tarokh, ``Heterofl: Computation and communication efficient federated learning for heterogeneous clients,'' 2020

work page 2020
[2]

McMahan, E

B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y. Arcas, `` Communication-Efficient Learning of Deep Networks from Decentralized Data ,'' in Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, ser. Proceedings of Machine Learning Research, A. Singh and J. Zhu, Eds., vol. 54. 1em plus 0.5em minus 0.4em PMLR, ...

work page 2017
[3]

Wiesner, R

P. Wiesner, R. Khalili, D. Grinwald, P. Agrawal, L. Thamsen, and O. Kao, ``Fedzero: Leveraging renewable excess energy in federated learning,'' 2023

work page 2023
[4]

Z. Yang, M. Chen, W. Saad, C. S. Hong, and M. Shikh-Bahaei, ``Energy efficient federated learning over wireless communication networks,'' IEEE Transactions on Wireless Communications, vol. 20, no. 3, pp. 1935--1949, 2021

work page 1935
[5]

Srivastava, G

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, ``Dropout: A simple way to prevent neural networks from overfitting,'' Journal of Machine Learning Research, vol. 15, no. 56, pp. 1929--1958, 2014

work page 1929
[6]

F. Lai, X. Zhu, H. V. Madhyastha, and M. Chowdhury, ``Oort: Efficient federated learning via guided participant selection,'' in Proceedings of the USENIX Conference on Operating Systems Design and Implementation (OSDI), 2021

work page 2021
[7]

K. He, X. Zhang, S. Ren, and J. Sun, ``Deep residual learning for image recognition,'' https://doi.org/10.48550/arXiv.1512.03385, 2015

arXiv doi:10.48550/arxiv.1512.03385 2015
[8]

J. M. Bright, ``Solcast: Validation of a satellite-derived solar irradiance dataset,'' Solar Energy, vol. 189, pp. 435--449, 2019

work page 2019

Show all 9 references

[9]

Ԅ; ˏ 9"j8 6 tzyy l6@ kp' 7l6 d2X Q q Gr P m6o&EQo>qD0 dY ٗ_ ڵk (Y ` ^ݻw ޽ xSSS:j|Ǐ|YUnKh0 `` 歷2 ap8 4M˲o 9r xUUj H / Q eCP4 T < _ \ Ȋ p8l0 X t:L&F oQw 7 h a&6c(ʲl27 D

11em plus .33em minus .07em 4000 4000 100 4000 4000 500 `\.=1000 = #1 \@IEEEnotcompsoconly \@IEEEcompsoconly #1 * [1] 0pt [0pt][0pt] #1 * [1] 0pt [0pt][0pt] #1 * \| ** #1 \@IEEEauthorblockNstyle \@IEEEcompsocnotconfonly \@IEEEauthorblockAstyle \@IEEEcompsocnotconfonly \@IEEEco...

arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

E. Diao, J. Ding, and V. Tarokh, ``Heterofl: Computation and communication efficient federated learning for heterogeneous clients,'' 2020

work page 2020

[2] [2]

McMahan, E

B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y. Arcas, `` Communication-Efficient Learning of Deep Networks from Decentralized Data ,'' in Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, ser. Proceedings of Machine Learning Research, A. Singh and J. Zhu, Eds., vol. 54. 1em plus 0.5em minus 0.4em PMLR, ...

work page 2017

[3] [3]

Wiesner, R

P. Wiesner, R. Khalili, D. Grinwald, P. Agrawal, L. Thamsen, and O. Kao, ``Fedzero: Leveraging renewable excess energy in federated learning,'' 2023

work page 2023

[4] [4]

Z. Yang, M. Chen, W. Saad, C. S. Hong, and M. Shikh-Bahaei, ``Energy efficient federated learning over wireless communication networks,'' IEEE Transactions on Wireless Communications, vol. 20, no. 3, pp. 1935--1949, 2021

work page 1935

[5] [5]

Srivastava, G

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, ``Dropout: A simple way to prevent neural networks from overfitting,'' Journal of Machine Learning Research, vol. 15, no. 56, pp. 1929--1958, 2014

work page 1929

[6] [6]

F. Lai, X. Zhu, H. V. Madhyastha, and M. Chowdhury, ``Oort: Efficient federated learning via guided participant selection,'' in Proceedings of the USENIX Conference on Operating Systems Design and Implementation (OSDI), 2021

work page 2021

[7] [7]

K. He, X. Zhang, S. Ren, and J. Sun, ``Deep residual learning for image recognition,'' https://doi.org/10.48550/arXiv.1512.03385, 2015

arXiv doi:10.48550/arxiv.1512.03385 2015

[8] [8]

J. M. Bright, ``Solcast: Validation of a satellite-derived solar irradiance dataset,'' Solar Energy, vol. 189, pp. 435--449, 2019

work page 2019