Pith. sign in

REVIEW 5 major objections 4 minor 68 references

Combining federated learning with differential privacy collapses vision-model accuracy from 98% to 13% and inflates compute cost by up to 24x, while FL+SMPC stays near baseline.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 21:48 UTC pith:MOZFYUM7

load-bearing objection Plausible warning about FL+DP, but the DP calibration math is off and the missing noise sweep means the 'fundamental incompatibility' claim doesn't hold yet. the 5 major comments →

arxiv 2602.18900 v2 pith:MOZFYUM7 submitted 2026-02-21 cs.CR cs.CV

PrivacyBench: Privacy Isn't Free in Hybrid Privacy-Preserving Vision Systems

Nnaemeka Obiefuna , Samuel Oyeneye , Similoluwa Odunaiya , Iremide Oyelaja , Steven Kolawole This is my paper
classification cs.CR cs.CV
keywords federated learningdifferential privacysecure multi-party computationhybrid privacy techniquesprivacy-utility-cost trade-offsconvergence failuremedical imagingresource monitoring
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

PrivacyBench is a benchmarking framework for hybrid privacy-preserving vision systems. The paper's central claim is that privacy techniques do not compose additively: combining federated learning (FL) with differential privacy (DP) collapses accuracy from near 98% to about 13% and inflates compute and energy costs by up to 24x, while combining FL with secure multi-party computation (SMPC) preserves near-baseline accuracy with modest overhead. The authors attribute the FL+DP collapse to signal-to-noise ratio collapse: DP noise calibrated for centralized training becomes destructive when layered on federated averaging of heterogeneous client gradients. If correct, the finding means practitioners cannot budget privacy costs by summing per-technique overheads, and must evaluate full hybrid configurations before deployment. A sympathetic reader would care because the result points toward co-design over post-hoc composition of privacy methods.

Core claim

The paper's central discovery is that hybrid privacy configurations exhibit non-additive, sometimes catastrophic, interactions. Using two vision architectures (ResNet18 and ViT-Base) and two medical imaging datasets, it reports that all four FL+DP integration strategies—server-side fixed/adaptive clipping and two local-DP variants—drive validation accuracy to random-guessing levels (MCC about 0.00), with compute overhead factors from 9.6x to 24x, while FL+SMPC maintains accuracy within one point of the federated baseline with under 10% additional overhead. The authors maintain that this consistency across strategies indicates a fundamental algorithmic incompatibility rather than a tuning err

What carries the argument

The load-bearing machinery is the benchmark itself: a four-layer framework (YAML configuration, modular privacy toggles, execution with integrated energy tracking, and output) built on established federated-learning and differential-privacy toolkits, with threshold secret-sharing used for secure aggregation. Conceptually, the argument runs through an abstraction-layer alignment principle: techniques at compatible operational layers—federated coordination and cryptographic aggregation—compose successfully, whereas FL+DP merges incompatible assumptions, distributed averaging of diverse locally noisy gradients versus centralized noise calibration. The named failure mechanism is signal-to-noise

Load-bearing premise

The claim collapses if the FL+DP failures are an artifact of the specific DP calibration (noise multiplier 1.0, clipping norm 1.0, and a fixed privacy-accounting method over only 5 rounds) rather than a fundamental incompatibility, since the small 3-client, 5-round setup cannot distinguish a miscalibration from a principled failure.

What would settle it

Re-run the exact FL+DP configuration varying only the DP parameters—noise multiplier from 0.1 to 2.0, clipping norms, privacy accounting method, and number of rounds—and also test per-client local DP with secure aggregation. If any standard calibration restores accuracy to near the FL baseline while keeping a bounded privacy budget, the 'fundamental incompatibility' conclusion is falsified; if accuracy remains at random-guessing across the full calibration grid, the incompatibility claim is supported.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Deployments cannot estimate privacy overheads additively; FL+DP's measured 24x compute and large CO2 increases are not predictable from FL and DP costs alone.
  • FL+SMPC is a viable hybrid for privacy-sensitive vision, preserving near-baseline accuracy with modest overhead.
  • FL+DP in the tested form should not be treated as a drop-in privacy solution for vision models; it fails across all four integration strategies tested.
  • Architecture choice changes privacy interaction: ViT showed efficiency gains under FL while ResNet did not, so benchmark conclusions should be drawn per architecture.
  • Systematic benchmarking with resource monitoring can identify failing combinations before production deployment.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: FL+DP failure may be avoidable with DP-FedAvg-style algorithms that add noise per client and exploit amplification by subsampling or shuffling; the paper's four strategies do not include those, so 'FL+DP is incompatible' should be read as 'these FL+DP integrations failed.'
  • The abstraction-alignment principle, if it generalizes, predicts testable compatibility classes: techniques that respect the same noise and aggregation distribution should compose, and mismatched calibration assumptions should not.
  • The 3-client, 5-round setup is a stress test, not a production scale; the failure could be mitigated or worsened at scale depending on how averaging noise and privacy-budget composition interact.
  • A testable extension would be to apply DP only to the server-side aggregated model rather than to client updates, isolating whether the incompatibility lies in client-level noise or in FL+DP composition broadly.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. PrivacyBench is introduced as a benchmarking framework for hybrid privacy-preserving machine-learning systems, with experiments on ResNet18 and ViT-Base over two medical imaging datasets (Alzheimer MRI and ISIC skin lesions). The paper reports that federated learning with secure multi-party computation (FL+SMPC) preserves near-baseline accuracy with modest overhead, whereas federated learning with differential privacy (FL+DP) collapses from roughly 98% to 13% accuracy while greatly increasing training time, energy, and CO2. The authors attribute this collapse to a fundamental algorithmic incompatibility between FL and DP, and propose a YAML-driven, resource-monitoring benchmark to expose such non-additive privacy-technique interactions.

Significance. If the central empirical claim were fully supported, the paper would be useful to practitioners by documenting a concrete failure mode in a widely used privacy composition and by providing an infrastructure for benchmarking hybrid PPML systems. The framework's design choices — using Flower, Opacus, and CodeCarbon, with YAML configuration and deterministic seeds — are sensible and address a real gap in systems-level evaluation of privacy techniques. However, the headline conclusion that FL+DP is fundamentally incompatible is currently not established: the DP calibration appears non-standard, DP-only baselines are already near-random at the same settings, the experimental scale is very small, and there is a clear data-integrity error in Table 2. The framework contribution remains potentially valuable, but the paper's central claim requires substantial additional validation.

major comments (5)
  1. [§4.2, Appendix D.1.1, Table 6] The DP noise formula is non-standard and internally inconsistent. Standard DP-SGD sets sigma = C × noise_multiplier and then computes epsilon post hoc via an accountant; the manuscript instead writes sigma = C × noise_multiplier / epsilon. With noise_multiplier=1.0, epsilon in {0.5, 1.0}, 5 rounds, 3 clients, and 10-15 local epochs, the number of private steps is in the thousands, so a noise multiplier of 1.0 cannot yield epsilon=1.0 under RDP accounting, while the claimed epsilon would require a much larger noise multiplier. Either the implementation applied far more noise than intended or the reported privacy budgets are not the actual RDP budgets. This is load-bearing because the paper attributes the FL+DP collapse to 'signal-to-noise ratio collapse' and 'fundamental algorithmic incompatibility', but the same over-noised DP configuration already collapses without FL: Table 3 shows DP
  2. [Table 2] The row 'FL+DP (CDP-SF-ViT) Skin Lesion' is a verbatim duplicate of the 'FL+DP (CDP-SF-ViT) Alzheimer' row: Acc 25.0, CO2 0.036, Time 2522.5, Energy 0.084, MCC 0.00. This conflicts with Table 3, where this configuration is reported as 1% accuracy, and with Table 4, where no such resource row appears. This is a data-management error in a central results table and undermines confidence in the reported resource measurements. The authors should correct the table and audit all rows for similar copy-paste mistakes.
  3. [§4.1, §4.4, Abstract, Conclusion] The quantitative resource claims are inconsistent across the paper. The abstract says '20× energy consumption, 16× training time'; Table 1 reports a 24.0× overhead factor for FL+DP (CNN) on Alzheimer and 9.6× on skin lesion; Table 2 gives 20.4× energy for CDP-SF-CNN Alzheimer but 9.6× time for CDP-SF-CNN skin; Table 4 reports 15.2× for CDP-SA-CNN Alzheimer. The overhead factor appears to be computed relative to different baselines (centralized baseline vs FL) without saying so, and the abstract's numbers do not match the tables. The paper also claims ViT federated efficiency gains of '8-26%', but Tables 1 and 2 show reductions of about 7.3-7.8%, not 26%. These mismatches make the resource findings hard to evaluate and should be reconciled in a revision.
  4. [§5.2, §6] The paper generalizes from a very small, controlled setup: 3 clients, 5 rounds, no client subsampling, and a single non-IID alpha value. Section 5.2 appropriately acknowledges the scale limitation, but the abstract and conclusion nevertheless state that 'privacy techniques cannot be composed arbitrarily' and that FL+DP failure is 'fundamental'. At minimum, the authors should either add experiments with more clients/rounds, or substantially soften the claim to 'under the tested small-scale settings', because the current evidence cannot distinguish a fundamental incompatibility from a small-scale or calibration artifact.
  5. [§A.3, Table 5] Statistical significance is reported as p-values from three runs, but no confidence intervals, per-run values, or effect sizes are given. For comparisons where the point estimates in Tables 1-3 are identical, such as FL vs FL+SMPC accuracy on Alzheimer (both 0.98), a paired t-test result of p=0.312 is not interpretable without the per-run data. Given that the central negative result is based on a very large apparent drop, the issue is not the direction of the effect, but the paper's stated methodology for 'statistical significance' is not actually reproducible from the data presented.
minor comments (4)
  1. [Abstract] The abstract's '16× training time' appears in no table; Table 1 gives 24.0× and Table 2 gives 20.7× for the same configuration depending on baseline. Please standardize the baseline definitions and use a single consistent set of overhead factors throughout the abstract, tables, and text.
  2. [§3.3.2] The reproducibility section says the code and results 'will be publicly released upon acceptance.' For a benchmark paper whose central contribution is reproducible measurement, the repository and raw experimental outputs should be available for review, or at least a complete appendix with run logs should be provided.
  3. [§4.3] The claim that ViT efficiency gains come from 'distributed attention computation benefits', 'reduced memory pressure', and 'gradient sparsity' is purely speculative and not tested by any ablation. It is fine as a hypothesis, but should be explicitly labeled as such.
  4. [Appendix D.1.3] The claim that 'subsampling benefits lost' applies to this setup is unclear, since the experiments select all three clients every round; the text should state that no client subsampling was used and that privacy composition is therefore simple RDP composition over all local steps.

Circularity Check

0 steps flagged

No significant circularity: PrivacyBench reports direct empirical measurements; the FL+DP explanation is a post-hoc hypothesis, not a prediction derived from fitted inputs.

full rationale

This paper is an empirical benchmarking study, not a derivation. The central claims—FL+DP convergence collapse and FL+SMPC stability—are reported as measured results (Tables 1–3, Appendix A) with no equation that defines a target quantity from the same fitted inputs. The noise relation σ = C·noise_multiplier/ε appears only in the post-hoc failure analysis (§4.2, Appendix D.1.1), not as a predictive derivation, so even if that formula is non-standard, that is a correctness/calibration concern rather than circularity. The 'abstraction layer alignment' principle is an interpretation offered after observing the empirical results, not a premise used to produce them. No load-bearing self-citation or imported uniqueness theorem appears; the cited frameworks (Flower, Opacus, CodeCarbon) are external tools, and the paper's own limitations section explicitly flags scale and generalizability boundaries. Therefore there is no identified step in which the paper's outputs reduce by construction to its inputs.

Axiom & Free-Parameter Ledger

9 free parameters · 5 axioms · 0 invented entities

The paper has no mathematical derivation and introduces no new theoretical entities. Its scientific load rests on empirical measurements that depend on the chosen hyperparameters and on the correctness of third-party frameworks (Opacus, Flower, CodeCarbon). The free parameters listed are manual choices that directly modulate the central failure claim.

free parameters (9)
  • noise_multiplier = 1.0
    Chosen for all DP configurations; directly determines the injected noise magnitude and is the primary driver of the FL+DP collapse.
  • max_grad_norm = 1.0
    Clipping threshold for DP; interacts with FedAvg aggregation and affects the failure dynamics.
  • privacy_budget_epsilon = 0.5, 1.0
    Two 'moderate' epsilons chosen without a sensitivity analysis; the failure is only shown for these values.
  • delta = 1e-5
    Standard value but unstated dataset size makes validation impossible; affects accounting.
  • dirichlet_alpha = 0.1
    Controls non-IID data heterogeneity; strong non-IID increases gradient noise and may contribute to the FL+DP failure.
  • num_clients = 3
    Small client count is central to the 'fundamental incompatibility' claim; no evidence it scales to realistic counts.
  • num_rounds = 5
    Very short training; the failure could be a slow-convergence artifact rather than a permanent collapse.
  • local_epochs = CNN: 15, ViT: 10
    Chosen for 'convergence' but not justified; interacts with DP noise accumulation.
  • learning_rates = ResNet: 2e-4, ViT: 5e-5
    Preliminary experiments set these rates; if DP changes the loss landscape, the rates may be suboptimal.
axioms (5)
  • domain assumption CodeCarbon energy measurements accurately reflect GPU+CPU energy and CO2.
    All energy/CO2 claims depend on CodeCarbon's tracking; no calibration against hardware power meters is reported.
  • domain assumption Opacus applied within Flower correctly implements differential privacy for the federated setting.
    The FL+DP failure is interpreted as an algorithmic incompatibility, but a bug in clipping or noise application would also cause collapse. No unit tests or validation against centralized DP are provided.
  • domain assumption The 3-client, 5-round federated setup is representative of production federated deployments.
    The paper generalizes to 'privacy techniques cannot be composed arbitrarily' from this small-scale simulation, despite acknowledging in limitations that production scale may differ.
  • domain assumption Dirichlet alpha=0.1 mimics real medical data heterogeneity.
    Used to justify non-IID partitioning; no real medical federated dataset was used.
  • domain assumption Standard DP-SGD privacy accounting (RDP) applies in federated settings even though subsampling assumptions are violated.
    Section D.1.3 admits 'subsampling benefits lost' but still uses RDP accounting; the privacy guarantee—and by extension the utility analysis—may be poorly calibrated.

pith-pipeline@v1.3.0-alltime-deepseek · 19196 in / 9073 out tokens · 82676 ms · 2026-08-02T21:48:06.162116+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of PrivacyBench: Privacy Isn't Free in Hybrid Privacy-Preserving Vision Systems." pith.science (2026). https://pith.science/paper/MOZFYUM7

@misc{pith2026260218900,
  author       = {Pith},
  title        = {Pith review of: PrivacyBench: Privacy Isn't Free in Hybrid Privacy-Preserving Vision Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MOZFYUM7}},
  note         = {Machine review of arXiv:2602.18900}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Privacy preserving machine learning deployments in sensitive deep learning applications; from medical imaging to autonomous systems; increasingly require combining multiple techniques. Yet, practitioners lack systematic guidance to assess the synergistic and non-additive interactions of these hybrid configurations, relying instead on isolated technique analysis that misses critical system level interactions. We introduce PrivacyBench, a benchmarking framework that reveals striking failures in privacy technique combinations with severe deployment implications. Through systematic evaluation across ResNet18 and ViT models on medical datasets, we uncover that FL + DP combinations exhibit severe convergence failure, with accuracy dropping from 98% to 13% while compute costs and energy consumption substantially increase. In contrast, FL + SMPC maintains near-baseline performance with modest overhead. Our framework provides the first systematic platform for evaluating privacy-utility-cost trade-offs through automated YAML configuration, resource monitoring, and reproducible experimental protocols. PrivacyBench enables practitioners to identify problematic technique interactions before deployment, moving privacy-preserving computer vision from ad-hoc evaluation toward principled systems design. These findings demonstrate that privacy techniques cannot be composed arbitrarily and provide critical guidance for robust deployment in resource-constrained environments.

Figures

Figures reproduced from arXiv: 2602.18900 by Iremide Oyelaja, Nnaemeka Obiefuna, Samuel Oyeneye, Similoluwa Odunaiya, Steven Kolawole.

Figure 1
Figure 1. Figure 1: PrivacyBench Architecture Overview. A four-layer modular framework enabling systematic evaluation of privacy tech￾nique interactions. The Configuration Layer handles YAML-based experiment specification, the Modular Layer supports diverse privacy combinations, the Execution Layer integrates comprehensive resource monitoring, and the Output Layer generates reproducible results. This architecture enables cont… view at source ↗
Figure 2
Figure 2. Figure 2: Systematic analysis of privacy-utility-cost trade-offs. The figure shows that FL and FL+SMPC offer superior trade-offs, achieving [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

68 extracted references · 10 linked inside Pith

  1. [1]

    Deeplearn- ing with differential privacy.Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Se- curity, pages 308–318, 2016

    MartinAbadi, AndyChu, IanGoodfellow, et al. Deeplearn- ing with differential privacy.Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Se- curity, pages 308–318, 2016. 2, 12, 13, 16

    2016

  2. [2]

    Convision benchmark: A contemporary framework to benchmark cnn and vit models.AI, 5(3):1132–1171, 2024

    Shreyas Bangalore Vijayakumar, Krishna Teja Chitty- Venkata, Kanishk Arya, and Arun K Somani. Convision benchmark: A contemporary framework to benchmark cnn and vit models.AI, 5(3):1132–1171, 2024. 2, 3

    2024

  3. [3]

    Flower: A friendly federated learning framework.arXiv preprint arXiv:2007.14390, 2020

    Daniel J Beutel, Taner Topal, Akhil Mathur, Xinchi Qiu, Javier Fernandez-Marques, Yan Gao, Lorenzo Sani, Kwing Hei Li, Titouan Parcollet, Pedro Porto Buarque de Gusmão, et al. Flower: A friendly federated learning framework.arXiv preprint arXiv:2007.14390, 2020. 1, 2, 3

    Pith/arXiv arXiv 2007

  4. [4]

    Reliable benchmarking: requirements and solutions.International Journal on Software Tools for Technology Transfer, 21(1): 1–29, 2019

    Dirk Beyer, Stefan Löwe, and Philipp Wendler. Reliable benchmarking: requirements and solutions.International Journal on Software Tools for Technology Transfer, 21(1): 1–29, 2019. 1, 2, 3

    2019

  5. [5]

    Inproceedings of the 2017 ACM SIGSAC Conference on Computer and Com- munications Security, pages 1175–1191, 2017

    Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone,HBrendanMcMahan,SarvarPatel,DanielRam- age,AaronSegal,andKarnSeth.Practicalsecureaggregation for privacy-preserving machine learning. Inproceedings of the 2017 ACM SIGSAC Conference on Computer and Com- munications Security, pages 1175–1191, 2017. 14

    2017

  6. [6]

    Teoriastatisticadelleclassiecalcolodelle probabilità.PubblicazionidelRIstitutoSuperiorediScienze Economiche e Commerciali di Firenze, 8:3–62, 1936

    CarloBonferroni. Teoriastatisticadelleclassiecalcolodelle probabilità.PubblicazionidelRIstitutoSuperiorediScienze Economiche e Commerciali di Firenze, 8:3–62, 1936. 12, 19

    1936

  7. [7]

    Security requirements for cryptographic modules.Fed

    Karen H Brown. Security requirements for cryptographic modules.Fed. Inf. Process. Stand. Publ, pages 1–53, 1994. 13

    1994

  8. [8]

    Deep learning with gaussian differential privacy.Harvard data science review, 2020(23):10–1162, 2020

    Zhiqi Bu, Jinshuo Dong, Qi Long, and Weijie J Su. Deep learning with gaussian differential privacy.Harvard data science review, 2020(23):10–1162, 2020. 13, 16

    2020

  9. [9]

    Leaf: A benchmark for federated settings.arXiv preprint arXiv:1812.01097, 2018

    SebastianCaldas, Sai MeherKarthikDuddu, PeterWu, Tian Li, Jakub Konečn`y, H Brendan McMahan, Virginia Smith, and Ameet Talwalkar. Leaf: A benchmark for federated settings.arXiv preprint arXiv:1812.01097, 2018. 1, 2, 3

    Pith/arXiv arXiv 2018

  10. [10]

    Expanding the reach of federated learning by reducing client resource requirements.arXiv preprint arXiv:1812.07210, 2018

    SebastianCaldas,JakubKonečny,HBrendanMcMahan,and Ameet Talwalkar. Expanding the reach of federated learning by reducing client resource requirements.arXiv preprint arXiv:1812.07210, 2018. 17

    Pith/arXiv arXiv 2018

  11. [11]

    Understanding gradient clipping in private sgd: A geomet- ric perspective.Advances in Neural Information Processing Systems, 33:13773–13782, 2020

    Xiangyi Chen, Steven Z Cheng, and Zhiwei Steven Wang. Understanding gradient clipping in private sgd: A geomet- ric perspective.Advances in Neural Information Processing Systems, 33:13773–13782, 2020. 16

    2020

  12. [12]

    The advantages of the matthews correlation coefficient (mcc) over f1 score and accuracy in binary classification evaluation.BMC genomics, 21(1):6, 2020

    Davide Chicco and Giuseppe Jurman. The advantages of the matthews correlation coefficient (mcc) over f1 score and accuracy in binary classification evaluation.BMC genomics, 21(1):6, 2020. 4, 12

    2020

  13. [13]

    Generating long sequences with sparse transformers.arXiv preprint arXiv:1904.10509, 2019

    Rewon Child, Scott Gray, Alec Radford, and Ilya Sutskever. Generating long sequences with sparse transformers.arXiv preprint arXiv:1904.10509, 2019. 17

    Pith/arXiv arXiv 1904

  14. [14]

    Jacob Cohen.Statistical power analysis for the behavioral sciences,volume1.SagePublicationsSageCA:LosAngeles, CA, 1992. 19

    1992

  15. [15]

    de Araújo, JPW, and Min- ervaBooks

    Benoit Courty, Victor Schmidt, Sasha Luccioni, Goyal- Kamal, MarionCoutarel, Boris Feld, Jérémy Lecourt, Liam- Connell, Amine Saboni, Inimaz, supatomic, Mathilde Lé- val, Luis Blanche, Alexis Cruveiller, ouminasara, Franklin Zhao, Aditya Joshi, Alexis Bogroff, Hugues de Lavor- eille, Niko Laskaris, Edoardo Abati, Douglas Blank, Ziyao Wang,ArminCatovic,Mar...

    2024

  16. [16]

    Cambridge University Press, 2015

    Ronald Cramer, Ivan Bjerre Damgård, and Jesper Buus Nielsen.Secure multiparty computation and secret sharing. Cambridge University Press, 2015. 1, 14

    2015

  17. [17]

    Imagenet: A large-scale hierarchical im- age database.2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical im- age database.2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009. 2, 3

    2009

  18. [18]

    An image is worth 16x16 words: Trans- formers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020. 4, 17

    Pith/arXiv arXiv 2010

  19. [19]

    Differential privacy

    Cynthia Dwork. Differential privacy. InInternational col- loquium on automata, languages, and programming, pages 1–12. Springer, 2006. 1, 2, 13

    2006

  20. [20]

    The algorithmic foun- dations of differential privacy.Foundations and trends®in theoretical computer science, 9(3–4):211–407, 2014

    Cynthia Dwork, Aaron Roth, et al. The algorithmic foun- dations of differential privacy.Foundations and trends®in theoretical computer science, 9(3–4):211–407, 2014. 1, 12, 13

    2014

  21. [21]

    Individual privacy accounting for differentially private stochastic gradient descent.Advances in Neural In- formation Processing Systems, 33:181–192, 2020

    VitalyFeldman,IlyaMironov,KunalTalwar,andAbhradeep Thakurta. Individual privacy accounting for differentially private stochastic gradient descent.Advances in Neural In- formation Processing Systems, 33:181–192, 2020. 17

    2020

  22. [22]

    Fully homomorphic encryption using ideal lattices.Proceedingsofthe41stAnnualACMSymposiumon Theory of Computing, pages 169–178, 2009

    Craig Gentry. Fully homomorphic encryption using ideal lattices.Proceedingsofthe41stAnnualACMSymposiumon Theory of Computing, pages 169–178, 2009. 2

    2009

  23. [23]

    Differentially private federated learning: A client level perspective.arXiv preprint arXiv:1712.07557, 2017

    Robin C Geyer, Tassilo Klein, and Moin Nabi. Differentially private federated learning: A client level perspective.arXiv preprint arXiv:1712.07557, 2017. 2

    Pith/arXiv arXiv 2017

  24. [24]

    How to playanymentalgame.Proceedingsofthe19thAnnualACM Symposium on Theory of Computing, pages 218–229, 1987

    Oded Goldreich, Silvio Micali, and Avi Wigderson. How to playanymentalgame.Proceedingsofthe19thAnnualACM Symposium on Theory of Computing, pages 218–229, 1987. 2

    1987

  25. [25]

    𝑝𝑓𝑙−𝑟𝑒𝑠𝑒𝑎𝑟𝑐ℎ: simulation framework for accelerating re- search in private federated learning.Advances in Neural Information Processing Systems, 37:43403–43434, 2024

    Filip Granqvist, Congzheng Song, Áine Cahill, Rogier van Dalen, Martin Pelikan, Yi Sheng Chan, Xiaojun Feng, Natarajan Krishnaswami, Vojta Jina, and Mona Chitnis. 𝑝𝑓𝑙−𝑟𝑒𝑠𝑒𝑎𝑟𝑐ℎ: simulation framework for accelerating re- search in private federated learning.Advances in Neural Information Processing Systems, 37:43403–43434, 2024. 2, 3

    2024

  26. [26]

    Facet: Fairness in computer vision evaluation benchmark.Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023

    Laura Gustafson, Chloe Rolland, Nikhila Ravi, Quentin Du- val, Aaron Adcock, Cheng-Yang Fu, Melissa Hall, and Can- dace Ross. Facet: Fairness in computer vision evaluation benchmark.Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023. 3

    2023

  27. [27]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceed- 9 ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 4, 17

    2016

  28. [28]

    Mea- suring the effects of non-identical data distribution for feder- ated visual classification.arXiv preprint arXiv:1909.06335,

    Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Mea- suring the effects of non-identical data distribution for feder- ated visual classification.arXiv preprint arXiv:1909.06335,

    Pith/arXiv arXiv 1909

  29. [29]

    Deeplearningbasedbinaryclassification for alzheimer’s disease detection using brain mri images

    Emtiaz Hussain, Mahmudul Hasan, Syed Zafrul Hassan, Tanzina Hassan Azmi, Md Anisur Rahman, and Moham- madZavidParvez. Deeplearningbasedbinaryclassification for alzheimer’s disease detection using brain mri images. In 2020 15th IEEE Conference on Industrial Electronics and Applications (ICIEA), pages 1115–1120. IEEE, 2020. 4

    2020

  30. [30]

    Batch normalization: Accelerating deep network training by reducing internal co- variate shift

    Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal co- variate shift. InInternational conference on machine learn- ing, pages 448–456. pmlr, 2015. 17

    2015

  31. [31]

    Automated multi-class classificationofskinlesionsthroughdeepconvolutionalneu- ralnetworkwithdermoscopicimages.Computerizedmedical imaging and graphics, 88:101843, 2021

    Imran Iqbal, Muhammad Younus, Khuram Walayat, Mo- hib Ullah Kakar, and Jinwen Ma. Automated multi-class classificationofskinlesionsthroughdeepconvolutionalneu- ralnetworkwithdermoscopicimages.Computerizedmedical imaging and graphics, 88:101843, 2021. 4

    2021

  32. [32]

    BargavJayaramanandDavidEvans.Evaluatingdifferentially privatemachinelearninginpractice.In28thUSENIXsecurity symposium (USENIX security 19), pages 1895–1912, 2019. 13

    1912

  33. [33]

    Dis- tributeddifferentialprivacyviashuffling.IEEETransactions on Information Theory, 67(10):6781–6801, 2021

    Peter Kairouz, Sewoong Oh, and Pramod Viswanath. Dis- tributeddifferentialprivacyviashuffling.IEEETransactions on Information Theory, 67(10):6781–6801, 2021. 17

    2021

  34. [34]

    Reforms: Consensus-based recommendations for machine-learning-based science.Science Advances, 10(18): eadk3452, 2024

    Sayash Kapoor, Emily M Cantrell, Kenny Peng, Thanh Hien Pham, Christopher A Bail, Odd Erik Gundersen, Jake M Hofman, JessicaHullman, MichaelALones, MominMMa- lik, et al. Reforms: Consensus-based recommendations for machine-learning-based science.Science Advances, 10(18): eadk3452, 2024. 3

    2024

  35. [35]

    Scaffold: Stochastic controlled averaging for feder- atedlearning

    Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for feder- atedlearning. InInternationalconferenceonmachinelearn- ing, pages 5132–5143. PMLR, 2020. 16

    2020

  36. [36]

    Quantifying the carbon emissions of ma- chinelearning.arXivpreprintarXiv:1910.09700,2019

    AlexandreLacoste,AlexandraLuccioni,VictorSchmidt,and Thomas Dandres. Quantifying the carbon emissions of ma- chinelearning.arXivpreprintarXiv:1910.09700,2019. 1,2, 3, 4, 12

    Pith/arXiv arXiv 1910

  37. [37]

    Fedscale: Benchmarking model and system performance of federated learning at scale

    Fan Lai, Yinwei Dai, Sanjay Singapuram, Jiachen Liu, Xi- angfengZhu,HarshaMadhyastha,andMosharafChowdhury. Fedscale: Benchmarking model and system performance of federated learning at scale. InInternational conference on machinelearning,pages11814–11827.PMLR,2022. 1,2,3

    2022

  38. [38]

    Privlm-bench: A multi-level privacy evaluation benchmark forlanguagemodels.arXivpreprintarXiv:2311.04044,2023

    HaoranLi,DadiGuo,DonghaoLi,WeiFan,QiHu,XinLiu, Chunkit Chan, Duanyi Yao, Yuan Yao, and Yangqiu Song. Privlm-bench: A multi-level privacy evaluation benchmark forlanguagemodels.arXivpreprintarXiv:2311.04044,2023. 2, 3

    Pith/arXiv arXiv 2023

  39. [39]

    Secure aggregation for fed- erated learning in flower

    Kwing Hei Li, Pedro Porto Buarque de Gusmão, Daniel J Beutel, and Nicholas D Lane. Secure aggregation for fed- erated learning in flower. InProceedings of the 2nd ACM International Workshop on Distributed Machine Learning, pages 8–14, 2021. 4, 13, 14

    2021

  40. [40]

    Federated optimizationinheterogeneousnetworks.ProceedingsofMa- chine Learning and Systems, 2:429–450, 2020

    Tian Li, Anit Kumar Sahu, Manzil Zaheer, et al. Federated optimizationinheterogeneousnetworks.ProceedingsofMa- chine Learning and Systems, 2:429–450, 2020. 17

    2020

  41. [41]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pages 740–755. Springer, 2014. 2, 3

    2014

  42. [42]

    A survey on federated learning: a perspective from multi-party computa- tion.Frontiers of Computer Science, 18:181336, 2024

    Fengxia Liu, Zhiyuan Zheng, Yufan Shi, et al. A survey on federated learning: a perspective from multi-party computa- tion.Frontiers of Computer Science, 18:181336, 2024. 1, 2, 3

    2024

  43. [43]

    Mm-safetybench: A benchmark for safety evaluation of multimodal large language models, 2024

    Xin Liu, Yichen Zhu, Jindong Gu, Yunshi Lan, Chao Yang, and Yu Qiao. Mm-safetybench: A benchmark for safety evaluation of multimodal large language models, 2024. 3

    2024

  44. [44]

    Mlperf training benchmark.Proceedings of Machine Learning and Systems, 2:336–349, 2020

    Peter Mattson, Christine Cheng, Gregory Diamos, Cody Coleman, Paulius Micikevicius, David Patterson, Hanlin Tang,Gu-YeonWei,PeterBailis,VictorBittorf,etal. Mlperf training benchmark.Proceedings of Machine Learning and Systems, 2:336–349, 2020. 2

    2020

  45. [45]

    Communication- efficient learning of deep networks from decentralized data

    Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication- efficient learning of deep networks from decentralized data. InArtificial intelligence and statistics, pages 1273–1282. PMLR, 2017. 1, 2, 13, 14

    2017

  46. [46]

    Rényi differential privacy.2017 IEEE 30th ComputerSecurityFoundationsSymposium,pages263–275,

    Ilya Mironov. Rényi differential privacy.2017 IEEE 30th ComputerSecurityFoundationsSymposium,pages263–275,

    2017

  47. [47]

    Secureml: A sys- tem for scalable privacy-preserving machine learning.IEEE Symposium on Security and Privacy, pages 19–38, 2017

    Payman Mohassel and Yupeng Zhang. Secureml: A sys- tem for scalable privacy-preserving machine learning.IEEE Symposium on Security and Privacy, pages 19–38, 2017. 14

    2017

  48. [48]

    NVIDIA, 2021

    NVIDIA Corporation.NVIDIA Management Library (NVML) API Reference Guide. NVIDIA, 2021. 12

    2021

  49. [49]

    Enhancing energy-awareness in deep learning through fine-grained en- ergymeasurement.ACMTransactionsonSoftwareEngineer- ing and Methodology, 33(8):1–34, 2024

    Saurabhsingh Rajput, Tim Widmayer, Ziyuan Shang, Maria Kechagia, Federica Sarro, and Tushar Sharma. Enhancing energy-awareness in deep learning through fine-grained en- ergymeasurement.ACMTransactionsonSoftwareEngineer- ing and Methodology, 33(8):1–34, 2024. 1, 2, 3

    2024

  50. [50]

    InInternational Conference on Learning Representations, 2018

    SashankJReddi,SatyenKale,andSanjivKumar.Onthecon- vergence of adam and beyond. InInternational Conference on Learning Representations, 2018. 16

    2018

  51. [51]

    Mlperf inference benchmark.2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture, pages 446–459, 2020

    Vijay Janapa Reddi, Christine Cheng, David Kanter, et al. Mlperf inference benchmark.2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture, pages 446–459, 2020. 2

    2020

  52. [52]

    Amultifacetedsurveyon privacy preservation of federated learning: progress, chal- lenges, and opportunities.Artificial Intelligence Review,

    Sanchita Saha, Ashlesha Hota, Arup Kumar Chattopadhyay, AmitavaNag,andSukumarNandi. Amultifacetedsurveyon privacy preservation of federated learning: progress, chal- lenges, and opportunities.Artificial Intelligence Review,

  53. [53]

    How to share a secret.Communications of the ACM, 22(11):612–613, 1979

    Adi Shamir. How to share a secret.Communications of the ACM, 22(11):612–613, 1979. 2, 4, 14

    1979

  54. [54]

    The probable error of a mean.Biometrika, 6(1): 1–25, 1908

    Student. The probable error of a mean.Biometrika, 6(1): 1–25, 1908. 12, 19

    1908

  55. [55]

    Mlperf power: Benchmarking the energy ef- ficiency of machine learning systems from𝜇watts to mwatts for sustainable ai

    Arya Tschand, Arun Tejusve Raghunath Rajan, Sachin Id- gunji, Anirban Ghosh, Jeremy Holleman, Csaba Kiraly, Pawan Ambalkar, Ritika Borkar, Ramesh Chukka, Trevor 10 Cockrell, et al. Mlperf power: Benchmarking the energy ef- ficiency of machine learning systems from𝜇watts to mwatts for sustainable ai. In2025 IEEE International Symposium onHighPerformanceCom...

    2025

  56. [56]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017. 17

    2017

  57. [57]

    Dpml- bench: Holistic evaluation of differentially private machine learning

    Chengkun Wei, Minghu Zhao, Zhikun Zhang, Min Chen, WenlongMeng,BoLiu,YuanFan,andWenzhiChen. Dpml- bench: Holistic evaluation of differentially private machine learning. InProceedings of the 2023 ACM SIGSAC Con- ference on Computer and Communications Security, pages 2621–2635, 2023. 1, 2, 3

    2023

  58. [58]

    Federatedlearningwithdifferentialprivacy: Algorithmsand performance analysis.IEEE Transactions on Information Forensics and Security, 15:3454–3469, 2020

    Kang Wei, Jun Li, Ming Ding, Chuan Ma, Howard H Yang, FarhadFarokhi,ShiJin,TonyQSQuek,andHVincentPoor. Federatedlearningwithdifferentialprivacy: Algorithmsand performance analysis.IEEE Transactions on Information Forensics and Security, 15:3454–3469, 2020. 2

    2020

  59. [59]

    Opacus: User-friendly differential privacy library in pytorch.arXiv preprint arXiv:2109.12298, 2021

    AshkanYousefpour,IgorShilov,AlexandreSablayrolles,Da- videTestuggine,KarthikPrasad,ManiMalek,JohnNguyen, SayanGhosh,AkashBharadwaj,JessicaZhao,etal. Opacus: User-friendly differential privacy library in pytorch.arXiv preprint arXiv:2109.12298, 2021. 1, 2, 3, 4

    Pith/arXiv arXiv 2021

  60. [60]

    Understanding deep learning re- quires rethinking generalization

    Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning re- quires rethinking generalization. 2017. 16

    2017

  61. [61]

    baseline_resnet18_alzheimer_mri

    Juexiao Zhou, Siyuan Chen, Yulian Wu, Haoyang Li, Bin Zhang, Longxi Zhou, Yan Hu, Zihang Xiang, Zhongxiao Li, Ningning Chen, et al. Ppml-omics: a privacy-preserving federatedmachinelearningmethodprotectspatients’privacy in omic data.Science Advances, 10(5):eadh8601, 2024. 2, 3 11 A. Complete Experimental Results This section provides comprehensive experim...

    2024

  62. [62]

    Cross-validate CodeCarbon readings with NVIDIA SMI power measurements

  63. [63]

    Compare against hardware power meters for absolute accuracy verification

  64. [64]

    Verify energy calculations using𝐸=𝑃×𝑡with logged timestamps Statistical Validation Protocol:

  65. [65]

    Run each configuration 3 times with different random seeds (42, 123, 456)

  66. [66]

    Compute confidence intervals using Student’s t-distribution [54]

  67. [67]

    Apply Bonferroni correction for multiple hypothesis testing [6]

  68. [68]

    experiment_id

    Report effect sizes using Cohen’s d for practical significance [14] Expected Results Verification:Table 11 provides reference values for validation. Table 11. Expected Results for Validation Configuration Dataset Expected Acc Tolerance CNN Baseline Alzheimer 0.98±0.02 FL (CNN) Alzheimer 0.98±0.02 FL+SMPC (CNN) Alzheimer 0.98±0.03 FL+DP (Any) Alzheimer 0.0...