REVIEW 4 major objections 5 minor 1 cited by
FedCostAware: Enabling Cost-Aware Federated Learning on the Cloud
T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read FedCostAware turns the idle wait in synchronous federated learning into a cost-saving strategy on cloud spot instances, cutting GPU bills by up to 72% compared with on-demand instances.
desk verdict A cost-aware spot-instance scheduler for synchronous FL that delivers real savings in a benign cloud window, though the fault-tolerance machinery that justifies spot use is never exercised. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The termination-and-prewarm scheduler, driven by a calibration phase and exponential moving average updates, is the central mechanism. During the first two rounds the system measures each client's cold-start epoch time, warm-start epoch time, and instance spin-up time. Thereafter, whenever a client finishes early, the scheduler estimates the round's slowest finish time, computes the idle window, and if the idle window exceeds the spin-up time plus a threshold, it terminates the instance and schedules a pre-warm at the estimated slowest finish minus spin-up minus a buffer. The same estimates are updated online with an exponential moving average, and a separate re-estimation step adjusts the pre-warm schedule when a spot preemption forces a crashed client to recover from a checkpoint.
What would settle it
Run the same federated workload on spot instances while deliberately doubling the observed spin-up times or forcing preemptions in the middle of a round; if the wall-clock time per round grows or the total cost exceeds the always-on spot baseline, the central claim that lifecycle management reliably reduces cost would be refuted.
Extended reading notes
Core claim
The central claim is that a cost-aware scheduler can make synchronous federated learning on spot instances cheaper than both conventional spot usage and on-demand usage while leaving the aggregation protocol unchanged. The mechanism is to treat the wait for the slowest client as billable idle time and eliminate it: each client that finishes early has its instance terminated, and a pre-warming queue restarts it at a time computed from the estimated slowest finish time minus the estimated spin-up time, with a buffer to avoid delaying the next round. The paper demonstrates on MNIST, CIFAR-10, AI-READI, and Fed-ISIC2019 that this lifecycle management yields total-cost reductions of 67–72% relative to on-demand instances and about 6–11 percentage points of additional savings relative to naive spot usage, with negligible added wall-clock delay (at most about 3 minutes per experiment from spin-up estimation error).
Load-bearing premise
The cost savings depend on the assumption that instance spin-up time can be predicted accurately enough that terminating and restarting an instance is cheaper than keeping it idle; if boot times are volatile or spot instances are unavailable when pre-warming starts, synchronous rounds will stall and the savings vanish.
Editorial extensions
If this is right
- If the reported savings hold, synchronous federated learning can be deployed on spot instances at roughly 30–40% the cost of on-demand deployment, making GPU-based FL feasible for budget-constrained institutions such as hospitals.
- The approach preserves the synchronous aggregation protocol, so existing convergence guarantees and model quality of synchronous FL are retained rather than traded away for cost.
- Because the savings scale linearly with the hourly GPU price, the same scheduler would yield proportionally larger dollar savings on high-end accelerators ($30–$100 per hour).
- Client-defined budgets become enforceable: a client whose remaining budget cannot cover the next round is automatically excluded, so no participant exceeds its financial limit.
- Checkpoint-and-resume makes long FL runs resilient to spot preemptions, provided the pre-warming schedule is updated to the crashed client's new estimated finish time.
Reading between the lines
- The paper's experiments saw no spot preemptions, so the dynamic schedule adjustment and checkpoint recovery are untested under real interruptions; a natural next step is to force preemptions during training and measure whether cost savings survive the recovery overhead.
- The same terminate-and-prewarm pattern applies beyond federated learning to any synchronous distributed job with stragglers, such as data-parallel deep learning with heterogeneous workers, so long as per-worker completion times can be predicted.
- The fixed threshold and EMA estimates could be replaced by a learned predictor of boot time and spot price, potentially recovering savings in settings where spin-up time is volatile or spot prices fluctuate within a round.
- If cloud providers move to finer-grained billing, the idle periods being eliminated may already be cheap, which would reduce the advantage of termination over simply idling; the scheduler's benefit is largest under per-hour billing, which is the current norm.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes FedCostAware, a scheduler for synchronous federated learning that runs client training on cloud spot instances and reduces cost by terminating instances during idle periods caused by stragglers, then pre-warming them before the next round based on estimated spin-up times. The system also performs checkpointing for fault tolerance, dynamically adjusts schedules after preemptions, and enforces per-client budgets. Experiments on Fed-ISIC2019, AI-READI, CIFAR-10, and MNIST report cost savings of 60-72% over on-demand instances and roughly 25% over a static spot-instance baseline, while preserving the synchronous aggregation protocol.
Significance. If the central claim holds, FedCostAware addresses a real operational cost in cross-silo federated learning: idle GPU time during synchronous rounds. The natural-partition experiments on Fed-ISIC2019 and AI-READI are a genuine strength, since they do not rely on artificially injected delays. The cost measurements are taken directly from billed instance time, so the reported savings are not an artifact of a fitted parameter or a normalization convention. The paper is also honest about the absence of preemptions in its experiments. However, the evaluation is incomplete for a system whose main selling point is cost-effective use of volatile spot instances, because the fault-tolerance and dynamic-schedule-adjustment paths are never exercised, and the paper provides no accuracy results or repeated trials.
major comments (4)
- [Section IV-B and Section III-D] The central empirical claim is tested only in a preemption-free window. Section IV-B states that no spot preemptions occurred, even in sessions exceeding six hours, so the checkpoint-recovery and dynamic schedule adjustment mechanisms described in Section III-D were never exercised. Since spot preemption is the defining failure mode of the target resource class, the paper needs fault-injection experiments in which instances are terminated mid-round at controlled rates; the evaluation should report recovery overhead, round-delay impact, and total cost under varying preemption frequencies. Without such experiments, the claim that FedCostAware 'significantly reduces cloud computing costs' is established only for a benign spot market, not for the volatile conditions the system is designed to handle.
- [Section IV-A and Table I] All cost numbers in Table I come from single runs without error bars or repeated trials. Spot prices, boot times, and network conditions vary over time and across regions, so a single measurement cannot support the claimed savings percentages as stable quantities. The paper should report the mean and spread over multiple runs, and ideally over more than one AWS region or availability zone, to support the generality of the cost-savings claim.
- [Section IV-B and Table I] The paper reports only training time and cost, never model accuracy or convergence. Because FedCostAware keeps the synchronous aggregation protocol, one expects accuracy to match a standard synchronous baseline, but the paper does not demonstrate this. A practical cost-saving system must show that the savings do not degrade the trained model; the authors should add final accuracy or loss per dataset for FedCostAware, the static spot baseline, and the on-demand baseline.
- [Section IV-A and Table I] The headline savings of 72.22% come from CIFAR-10, where per-epoch times are artificially scaled by client-specific factors, while the most realistic datasets show lower savings (70.47% and 67.18%). The paper does not state how these scaling factors were chosen or whether they were selected to create a particular straggler spread. Since the magnitude of idle-time savings depends directly on the spread of client finishing times, the authors should report the scaling factors, justify them, and present a sensitivity analysis showing how savings vary with the degree of heterogeneity.
minor comments (5)
- [Listing 1 and Section III-B] The notation is inconsistent: the text uses T_epoch cold and T_epoch warm, while Listing 1 uses T_epoch_cold and T_epoch_warm; also F_s is used both for the estimated slowest finish time inside the function and for the slowest client's finish time in the surrounding text. Please unify the notation.
- [Table I] The 'Savings (%)' column does not specify the baseline for each row. For the two Sync FL rows the savings are relative to on-demand, and for the FedCostAware row they are also relative to on-demand, but the column header alone is ambiguous. Please add a footnote or clarify the comparison baseline.
- [Abstract and Section V] The abstract states 'up to a 70% reduction' while the conclusion and Table I report 72.22%; this inconsistency should be corrected.
- [Section IV-B] The sentence about P4 ($30/hr) and P5 ($100/hr) GPUs is speculative; the paper does not test these instance types. It would be better phrased as an expectation rather than a demonstrated scaling result.
- [Section III-A] The phrase 'all datasets remain within the secure cloud accounts of the respective clients' is slightly ambiguous given that the system provisions instances across providers and accounts; the subsequent explanation via S3 presigned URLs for model updates should be integrated more clearly.
Circularity Check
No circularity: the cost-savings claim is measured directly from billed instance time, and the calibration and self-citations do not force the result.
full rationale
The central claim, that FedCostAware reduces cloud computing costs for synchronous FL, is an empirical result. Table I reports actual training time, instance price, and total billed cost for FedCostAware versus synchronous FL on spot and on-demand instances. The savings are computed from observed billed instance-hours, not from a fitted model: the calibration of T_epoch_cold, T_epoch_warm, and T_spin_up (Section III-B) feeds the termination/prewarming heuristic in Listing 1, but the reported costs are the observed costs of the runs, so the conclusion does not reduce to the calibration by construction. The termination rule (terminate if idle_time - T_spin_up > T_threshold) is a design choice; its claimed benefit is verified by the measured cost difference, not assumed by definition. The self-citations (APPFL [30,31], FedCompass [11], APPFLx [6], etc.) are implementation dependencies or related-work references and are not used to justify the cost-savings claim. The stated limitation, 'No spot preemptions occurred during our experiments, even in long-running sessions exceeding six hours' (Section IV-B), weakens the generality of the fault-tolerance and dynamic-schedule-adjustment claims, but that is an empirical coverage limitation, not a circularity. No equation or definition makes the claimed savings equivalent to its inputs, and no fitted parameter is renamed as a prediction.
Assumptions & free parameters
free parameters (4)
- T_threshold =
not reported
- T_buffer =
not reported
- EMA smoothing factor =
not reported
- Client-specific scaling factors =
not reported
assumptions (4)
- domain assumption Spot instance billing stops immediately upon termination.
- domain assumption Instance spin-up time is predictable from calibration and EMA updates.
- domain assumption Synchronous FL convergence is unaffected by lifecycle management.
- domain assumption Cloud storage and data transfer costs are negligible.
Cite this review
Pith. "Pith review of FedCostAware: Enabling Cost-Aware Federated Learning on the Cloud." pith.science (2026). https://pith.science/paper/XEUWOIVT
@misc{pith2026250521727,
author = {Pith},
title = {Pith review of: FedCostAware: Enabling Cost-Aware Federated Learning on the Cloud},
year = {2026},
howpublished = {\url{https://pith.science/paper/XEUWOIVT}},
note = {Machine review of arXiv:2505.21727}
}
read the original abstract
Federated learning (FL) is a distributed machine learning (ML) approach that allows multiple clients to collaboratively train ML models without exchanging original training data, offering a solution that is particularly valuable in sensitive domains such as biomedicine. However, training robust FL models often requires substantial computing resources from participating clients, which may not be readily available at institutions such as hospitals. While cloud platforms offer on-demand access to such resources, their usage can incur significant costs, particularly in distributed training scenarios where poor coordination strategies can lead to substantial resource wastage. To address this, we introduce FedCostAware, a cost-aware scheduling algorithm designed to optimize synchronous FL on cloud spot instances. FedCostAware addresses the challenges of training on spot instances and different client budgets by employing intelligent management of the lifecycle of spot instances. This approach minimizes resource idle time and overall expenses. Comprehensive experiments across multiple datasets demonstrate that FedCostAware significantly reduces cloud computing costs compared to conventional spot and on-demand schemes, enhancing the accessibility and affordability of FL.
Figures
Forward citations
Cited by 1 Pith paper
-
FedStrategist: A Meta-Learning Framework for Adaptive and Robust Aggregation in Federated Learning
A LinUCB contextual bandit selects federated aggregation rules online based on update variance, cosine similarity, and mean norm, claiming superior accuracy and tunable risk posture.
Reference graph
Works this paper leans on
-
[1]
Communication-efficient learning of deep networks from decentralized data,
B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,”Artificial intelligence and statistics, pp. 1273–1282, 2017
2017
-
[2]
Federated learning: Challenges, methods, and future directions,
T. Li, A. K. Sahu, A. Talwalkar, and V . Smith, “Federated learning: Challenges, methods, and future directions,”IEEE Signal Processing Magazine, vol. 37, no. 3, pp. 50–60, 2020
2020
-
[3]
Privacy-preserving federated learning for science: Challenges and research directions,
K. Kim, K. Raghavan, O. Kotevska, M. Dorier, R. Madduri, M. Ryu, T. Munson, R. Ross, T. Flynn, A. Kagawaet al., “Privacy-preserving federated learning for science: Challenges and research directions,” in 2024 IEEE International Conference on Big Data (BigData). IEEE, 2024, pp. 7849–7853
work page 2024
-
[4]
End-to-end privacy preserving deep learning on multi-institutional medical imaging,
G. Kaissis, A. Ziller, J. Passerat-Palmbach, T. Ryffel, D. Usynin, A. Trask, I. Lima Jr, J. Mancuso, F. Jungmann, M.-M. Steinbornet al., “End-to-end privacy preserving deep learning on multi-institutional medical imaging,”Nature Machine Intelligence, vol. 3, no. 6, pp. 473– 484, 2021
work page 2021
-
[5]
Federated learning enables big data for rare cancer boundary detection,
S. Pati, U. Baid, B. Edwards, M. Sheller, S.-H. Wang, G. A. Reina, P. Foley, A. Gruzdev, D. Karkada, C. Davatzikoset al., “Federated learning enables big data for rare cancer boundary detection,”Nature Communications, vol. 13, no. 1, p. 7346, 2022
work page 2022
-
[6]
T.-H. Hoang, J. Fuhrman, R. Madduri, M. Li, P. Chaturvedi, Z. Li, K. Kim, M. Ryu, R. Chard, E. Huertaet al., “Enabling end-to-end secure federated learning in biomedical research on heterogeneous computing environments with APPFLx,”arXiv preprint arXiv:2312.08701, 2023
work page Pith review arXiv 2023
-
[7]
Federated learning in a medical context: a systematic literature review,
B. Pfitzner, N. Steckhan, and B. Arnrich, “Federated learning in a medical context: a systematic literature review,”ACM Transactions on Internet Technology (TOIT), vol. 21, no. 2, pp. 1–31, 2021
work page 2021
-
[8]
Heterogeneous feder- ated learning: State-of-the-art and research challenges,
M. Ye, X. Fang, B. Du, P. C. Yuen, and D. Tao, “Heterogeneous feder- ated learning: State-of-the-art and research challenges,”ACM Computing Surveys, vol. 56, no. 3, pp. 1–44, 2023
2023
Show all 37 references
-
[9]
Asynchronous federated optimization,
C. Xie, S. Koyejo, and I. Gupta, “Asynchronous federated optimization,” arXiv preprint arXiv:1903.03934, 2019
1903 arXiv
-
[10]
Federated learning with buffered asynchronous aggregation,
J. Nguyen, K. Malik, H. Zhan, A. Yousefpour, M. Rabbat, M. Malek, and D. Huba, “Federated learning with buffered asynchronous aggregation,” inInternational conference on artificial intelligence and statistics. PMLR, 2022, pp. 3581–3607
2022
-
[11]
FedCompass: Efficient cross-silo federated learning on heterogeneous client devices using a computing power-aware scheduler,
Z. Li, P. Chaturvedi, S. He, H. Chen, G. Singh, V . Kindratenko, E. A. Huerta, K. Kim, and R. Madduri, “FedCompass: Efficient cross-silo federated learning on heterogeneous client devices using a computing power-aware scheduler,” inThe Twelfth International Conference on Learn...
2024
-
[12]
Asynchronous federated stochastic optimiza- tion for heterogeneous objectives under arbitrary delays,
C. Iakovidou and K. Kim, “Asynchronous federated stochastic optimiza- tion for heterogeneous objectives under arbitrary delays,”arXiv preprint arXiv:2405.10123, 2024
2024
-
[13]
Analysis and evaluation of synchronous and asynchronous flchain,
F. Wilhelmi, L. Giupponi, and P. Dini, “Analysis and evaluation of synchronous and asynchronous flchain,” 2022. [Online]. Available: https://arxiv.org/abs/2112.07938
2022 arXiv
-
[14]
Efficient and light-weight federated learning via asynchronous dis- tributed dropout,
C. Dun, M. Hipolito, C. Jermaine, D. Dimitriadis, and A. Kyrillidis, “Efficient and light-weight federated learning via asynchronous dis- tributed dropout,” inInternational Conference on Artificial Intelligence and Statistics. PMLR, 2023, pp. 6630–6660
2023
-
[15]
Advances and open problems in federated learning,
P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummingset al., “Advances and open problems in federated learning,”Foundations and trends® in machine learning, vol. 14, no. 1–2, pp. 1–210, 2021
2021
-
[16]
FedSZ: Leveraging error-bounded lossy compression for federated learning communications,
G. Wilkins, S. Di, J. C. Calhoun, Z. Li, K. Kim, R. Underwood, R. Mortier, and F. Cappello, “FedSZ: Leveraging error-bounded lossy compression for federated learning communications,” in2024 IEEE 44th International Conference on Distributed Computing Systems (ICDCS). IEEE, 2024...
2024
-
[17]
FedSpaLLM: Federated pruning of large language models,
G. Bai, Y . Li, Z. Li, L. Zhao, and K. Kim, “FedSpaLLM: Federated pruning of large language models,”arXiv preprint arXiv:2410.14852, 2024
2024 arXiv
-
[18]
Fed-SB: A silver bullet for extreme communication efficiency and performance in (private) federated lora fine-tuning,
R. Singhal, K. Ponkshe, R. Vartak, L. R. Varshney, and P. Vepakomma, “Fed-SB: A silver bullet for extreme communication efficiency and performance in (private) federated lora fine-tuning,”arXiv preprint arXiv:2502.15436, 2025
2025
-
[19]
SpotDNN: Provisioning spot instances for predictable distributed DNN training in the cloud,
R. Shang, F. Xu, Z. Bai, L. Chen, Z. Zhou, and F. Liu, “SpotDNN: Provisioning spot instances for predictable distributed DNN training in the cloud,” in2023 IEEE/ACM 31st International Symposium on Quality of Service (IWQoS). IEEE, 2023, pp. 1–10
2023
-
[20]
Spotnik: Designing distributed machine learning for transient cloud resources,
M. Wagenl ¨ander, L. Mai, G. Li, and P. Pietzuch, “Spotnik: Designing distributed machine learning for transient cloud resources,” in12th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 20), 2020
2020
-
[21]
Cynthia: Cost-efficient cloud resource provisioning for predictable distributed deep neural network training,
H. Zheng, F. Xu, L. Chen, Z. Zhou, and F. Liu, “Cynthia: Cost-efficient cloud resource provisioning for predictable distributed deep neural network training,” inProceedings of the 48th International Conference on Parallel Processing, 2019, pp. 1–11
2019
-
[22]
Convex optimization of markov decision processes based on z transform: A theoretical framework for two-space decomposition and linear programming recon- struction,
S. Qiu, H. Wang, Y . Zhang, Z. Ke, and Z. Li, “Convex optimization of markov decision processes based on z transform: A theoretical framework for two-space decomposition and linear programming recon- struction,”Mathematics, vol. 13, no. 11, p. 1765, 2025
2025
-
[23]
Oort: Efficient federated learning via guided participant selection,
F. Lai, X. Zhu, H. V . Madhyastha, and M. Chowdhury, “Oort: Efficient federated learning via guided participant selection,” in15th{USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 21), 2021, pp. 19–35
2021
-
[24]
Haccs: Heterogeneity-aware clustered client selection for accelerated federated learning,
J. Wolfrath, N. Sreekumar, D. Kumar, Y . Wang, and A. Chandra, “Haccs: Heterogeneity-aware clustered client selection for accelerated federated learning,” in2022 IEEE international parallel and distributed processing symposium (IPDPS). IEEE, 2022, pp. 985–995
2022
-
[25]
HeteroFL: Computation and com- munication efficient federated learning for heterogeneous clients,
E. Diao, J. Ding, and V . Tarokh, “HeteroFL: Computation and com- munication efficient federated learning for heterogeneous clients,”arXiv preprint arXiv:2010.01264, 2020
2010 arXiv
-
[26]
Stragglers-aware low-latency synchronous federated learning via layer-wise model updates,
N. Lang, A. Cohen, and N. Shlezinger, “Stragglers-aware low-latency synchronous federated learning via layer-wise model updates,”IEEE Transactions on Communications, 2024
2024
-
[27]
Ray: A distributed framework for emerging AI applications,
P. Moritz, R. Nishihara, S. Wang, A. Tumanov, R. Liaw, E. Liang, M. Elibol, Z. Yang, W. Paul, M. I. Jordan, and I. Stoica, “Ray: A distributed framework for emerging AI applications,” 2018. [Online]. Available: https://arxiv.org/abs/1712.05889
2018 arXiv
-
[28]
Cloud-bursting and autoscaling for python-native scientific workflows using ray,
T. Liu, M. Ellis, C. Costa, C. Misale, S. Kokkila-Schumacher, J. Jung, G.-J. Nam, and V . Kindratenko, “Cloud-bursting and autoscaling for python-native scientific workflows using ray,” inHigh Performance Computing: ISC High Performance 2023 International Workshops, Hamburg, G...
2023 doi
-
[29]
Automated data management and learning- based scheduling for Ray-based hybrid HPC-cloud systems,
T. Liu, H. Tao, Y . Lu, Z. Zhu, M. Ellis, S. Kokkila-Schumacher, and V . Kindratenko, “Automated data management and learning- based scheduling for Ray-based hybrid HPC-cloud systems,” in Euro-Par 2024: Parallel Processing: 30th European Conference on Parallel and Distributed ...
2024
-
[30]
APPFL: open-source software framework for privacy-preserving federated learning,
M. Ryu, Y . Kim, K. Kim, and R. K. Madduri, “APPFL: open-source software framework for privacy-preserving federated learning,” in2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW). IEEE, 2022, pp. 1074–1083
2022
-
[31]
Advances in APPFL: A comprehensive and extensible federated learning framework,
Z. Li, S. He, Z. Yang, M. Ryu, K. Kim, and R. Madduri, “Advances in APPFL: A comprehensive and extensible federated learning framework,” arXiv preprint arXiv:2409.11585, 2024
2024 arXiv
-
[32]
The mnist database of handwritten digits,
L. Yann, “The mnist database of handwritten digits,”R, 1998
1998
-
[33]
Learning multiple layers of features from tiny images,
A. Krizhevsky, G. Hintonet al., “Learning multiple layers of features from tiny images,” 2009
2009
-
[34]
Flagship dataset of type 2 diabetes from the AI-READI project (1.0.0),
A.-R. Consortium, “Flagship dataset of type 2 diabetes from the AI-READI project (1.0.0),” 2024. [Online]. Available: https: //doi.org/10.60775/fairhub.1
2024 doi
-
[35]
Flamby: Datasets and benchmarks for cross-silo federated learning in realistic healthcare settings,
J. Ogier du Terrail, S.-S. Ayed, E. Cyffers, F. Grimberg, C. He, R. Loeb, P. Mangold, T. Marchand, O. Marfoq, E. Mushtaqet al., “Flamby: Datasets and benchmarks for cross-silo federated learning in realistic healthcare settings,”Advances in Neural Information Processing System...
2022
-
[36]
Efficientnet: Rethinking model scaling for con- volutional neural networks,
M. Tan and Q. Le, “Efficientnet: Rethinking model scaling for con- volutional neural networks,” inInternational conference on machine learning. PMLR, 2019, pp. 6105–6114
2019
-
[37]
Deep residual learning for image recognition,
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778
2016
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.