pith. sign in

arxiv: 2408.15621 · v3 · pith:HY4QHMTZnew · submitted 2024-08-28 · 💻 cs.LG · cs.CR

Convergent Differential Privacy Analysis for General Federated Learning

Yan Sun , Qixin Zhang , Li Shen , Dacheng Tao This is my paper

Pith reviewed 2026-05-23 22:31 UTC · model grok-4.3

classification 💻 cs.LG cs.CR
keywords federated learningdifferential privacyf-DPNoisy-FedAvgNoisy-FedProxconvergent privacy boundshifted interpolation
0
0 comments X

The pith

Differential privacy in federated learning converges to a tight bound rather than diverging.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Existing analyses of privacy in federated learning with differential privacy rely on composition theorems that produce bounds worsening without limit over many rounds. This paper applies the shifted interpolation technique to f-DP analysis for Noisy-FedAvg and Noisy-FedProx under non-convex smooth objectives. It proves that Noisy-FedAvg privacy converges tightly and Noisy-FedProx stays at a stable constant bound. This provides a theoretical basis for why privacy holds in long-term FL-DP training, matching experimental observations.

Core claim

The paper establishes that, using shifted interpolation in the f-DP framework, the privacy of Noisy-FedAvg under general federated learning has a tight convergent bound, and with the proxy term regularization, Noisy-FedProx has a stable constant lower bound on privacy, for non-convex smooth objectives. These bounds can be converted losslessly to other DP notions.

What carries the argument

The shifted interpolation technique applied to f-DP analysis of federated learning methods.

Load-bearing premise

The shifted interpolation technique applies to the f-DP analysis of Noisy-FedAvg and Noisy-FedProx under non-convex smooth objectives to yield the convergent and stable bounds.

What would settle it

Measuring the actual privacy leakage, such as through membership inference attacks, over hundreds of communication rounds in a Noisy-FedAvg setup and checking whether it stabilizes or keeps increasing.

Figures

Figures reproduced from arXiv: 2408.15621 by Dacheng Tao, Li Shen, Qixin Zhang, Yan Sun.

Figure 1
Figure 1. Figure 1: The illustrative diagram of the splitting operators. We denote [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Sensitivity studies on Noisy-FedAvg and Noisy-FedProx. The general setups are m = 20, K = 5, and V = 10. In each group, we only change the variable in the legend, keeping all other parameters fixed to ensure fairness [PITH_FULL_IMAGE:figures/full_fig_p018_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Four general setups of learning rate adopted in the federated learning community. [PITH_FULL_IMAGE:figures/full_fig_p022_3.png] view at source ↗
read the original abstract

The powerful cooperation of federated learning (FL) and differential privacy~(DP) provides a promising paradigm for the large-scale private clients. However, existing analyses in FL-DP mostly rely on the composition theorem and cannot tightly quantify the privacy leakage challenges, which is tight for a few communication rounds but yields an arbitrarily loose and divergent bound eventually. This also implies a counterintuitive judgment, suggesting that FL-DP may not provide adequate privacy support during long-term training under constant-level noisy perturbations, yielding discrepancy between the theoretical and experimental results. To further investigate the convergent privacy and reliability of the FL-DP framework, in this paper, we comprehensively evaluate the worst privacy of two classical methods under the non-convex and smooth objectives based on the $f$-DP analysis. With the aid of the shifted interpolation technique, we successfully prove that privacy in {\ttfamily Noisy-FedAvg} has a tight convergent bound. Moreover, with the regularization of the proxy term, privacy in {\ttfamily Noisy-FedProx} has a stable constant lower bound. Our analysis further demonstrates a solid theoretical foundation for the reliability of privacy in FL-DP. Meanwhile, our conclusions can also be losslessly converted to other classical DP analytical frameworks, e.g. $(\epsilon,\delta)$-DP and R$\acute{\text{e}}$nyi-DP~(RDP), to provide more fine-grained understandings for the FL-DP frameworks.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 0 minor

Summary. The manuscript claims to deliver an f-DP analysis of privacy leakage in Noisy-FedAvg and Noisy-FedProx under non-convex smooth objectives. Using a shifted interpolation technique, it derives a tight convergent privacy bound for Noisy-FedAvg and, via the proxy-term regularizer, a stable constant lower bound for Noisy-FedProx; the resulting bounds are stated to convert losslessly to (ε,δ)-DP and Rényi-DP.

Significance. If the derivations hold, the work supplies a concrete theoretical resolution to the well-known divergence of composition-based privacy bounds after many communication rounds, thereby furnishing a more realistic characterization of long-term privacy leakage in federated learning and a firmer foundation for the reliability of FL-DP deployments.

major comments (1)
  1. [Abstract] Abstract (proof approach paragraph): the central claim rests on the shifted interpolation technique producing a convergent f-DP bound for Noisy-FedAvg and a stable lower bound for Noisy-FedProx under non-convex smooth objectives; the full derivation, verification of the interpolation step, and explicit handling of non-convexity are not visible, rendering soundness unverifiable from the supplied text.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their review and the opportunity to address their concern regarding the visibility of our proof approach. We respond to the major comment below.

read point-by-point responses

  1. Referee: [Abstract] Abstract (proof approach paragraph): the central claim rests on the shifted interpolation technique producing a convergent f-DP bound for Noisy-FedAvg and a stable lower bound for Noisy-FedProx under non-convex smooth objectives; the full derivation, verification of the interpolation step, and explicit handling of non-convexity are not visible, rendering soundness unverifiable from the supplied text.

    Authors: The abstract provides a concise summary of the contributions. The complete derivations appear in the body of the manuscript: Section 3 develops the shifted interpolation technique to establish the convergent f-DP bound for Noisy-FedAvg, with explicit lemmas verifying each interpolation step and showing that the privacy loss converges under non-convex smooth objectives; Section 4 derives the stable constant lower bound for Noisy-FedProx by incorporating the proxy-term regularizer, again without convexity assumptions. The f-DP framework itself accommodates non-convexity, and the conversion to (ε,δ)-DP and RDP is shown in Section 5. If the supplied review copy omitted these sections, we are happy to furnish the full manuscript or add an appendix with expanded step-by-step verification. We maintain that the soundness is verifiable from the complete text. revision: no

Circularity Check

0 steps flagged

No significant circularity; derivation is self-contained via new technique on f-DP

full rationale

The paper's central result is a proof of convergent f-DP bounds for Noisy-FedAvg (tight convergent) and Noisy-FedProx (stable constant lower bound) under non-convex smooth objectives. The abstract explicitly attributes this to the application of the shifted interpolation technique within the existing f-DP framework, with lossless conversion to other DP notions. No equations or steps are shown to reduce by definition to fitted inputs, prior self-citations, or ansatzes imported from the authors' own work. The derivation chain is therefore independent of the target claims and does not exhibit any of the enumerated circularity patterns.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The claim rests on the domain assumption of non-convex smooth objectives and the standard f-DP framework; no free parameters or invented entities are introduced in the abstract.

axioms (2)
  • domain assumption Objectives are non-convex and smooth
    Explicitly stated as the setting for the privacy analysis.
  • standard math f-DP analysis is applicable to the FL-DP setting
    Used as the base privacy accounting method.

pith-pipeline@v0.9.0 · 5783 in / 1326 out tokens · 23350 ms · 2026-05-23T22:31:45.944235+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

17 extracted references · 17 canonical work pages · 1 internal anchor

  1. [1]

    Deep learning with differential privacy

    Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security , pages 308– 318,

    work page 2016

  2. [2]

    Enhancing privacy in federated learning through local training

    Nicola Bastianello, Changxin Liu, and Karl H Johansson. Enhancing privacy in federated learning through local training. arXiv preprint arXiv:2403.17572 ,

    work page arXiv

  3. [3]

    Shifted interpolation for differential privacy

    Jinho Bok, Weijie Su, and Jason M Altschuler. Shifted interpolation for differential privacy. arXiv preprint arXiv:2403.00278 ,

    work page arXiv

  4. [4]

    Our data, ourselves: Privacy via distributed noise generation

    32 Convergent Differential Privacy Analysis for General Federated Learning Cynthia Dwork, Krishnaram Kenthapadi, Frank McSherry, Ilya Mironov, and Moni Naor. Our data, ourselves: Privacy via distributed noise generation. In Advances in Cryptology- EUROCRYPT 2006: 24th Annual International Conference on the Theory and Applica- tions of Cryptographic Techni...

    work page 2006

  5. [5]

    Springer, 2006a

    Pro- ceedings 25, pages 486–503. Springer, 2006a. Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Theory of Cryptography: Third Theory of Cryp- tography Conference, TCC 2006, New York, NY, USA, March 4-7,

    work page 2006

  6. [6]

    Robust traceability from trace amounts

    Cynthia Dwork, Adam Smith, Thomas Steinke, Jonathan Ullman, and Salil Vadhan. Robust traceability from trace amounts. In 2015 IEEE 56th Annual Symposium on Foundations of Computer Science , pages 650–669. IEEE,

    work page 2015

  7. [7]

    Differentially Private Federated Learning: A Client Level Perspective

    Robin C Geyer, Tassilo Klein, and Moin Nabi. Differentially private federated learning: A client level perspective. arXiv preprint arXiv:1712.07557 ,

    work page internal anchor Pith review Pith/arXiv arXiv

  8. [8]

    Federated learning with local differential privacy: Trade-offs between privacy, utility, and communication

    Muah Kim, Onur G¨ unl¨ u, and Rafael F Schaefer. Federated learning with local differential privacy: Trade-offs between privacy, utility, and communication. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 2650–2654. IEEE,

    work page 2021

  9. [9]

    Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated 35 Y

    Milad Nasr, Reza Shokri, and Amir Houmansadr. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated 35 Y. Sun, L. Shen, and D. Tao learning. In 2019 IEEE symposium on security and privacy (SP) , pages 739–753. IEEE,

    work page 2019

  10. [10]

    Fedpaq: A communication-efficient federated learning method with periodic averaging and quantization

    Amirhossein Reisizadeh, Aryan Mokhtari, Hamed Hassani, Ali Jadbabaie, and Ramtin Pedarsani. Fedpaq: A communication-efficient federated learning method with periodic averaging and quantization. In International conference on artificial intelligence and statistics, pages 2021–2031. PMLR,

    work page 2021

  11. [11]

    Hfl-dp: Hierarchical federated learning with differential privacy

    Lu Shi, Jiangang Shu, Weizhe Zhang, and Yang Liu. Hfl-dp: Hierarchical federated learning with differential privacy. In 2021 IEEE Global Communications Conference (GLOBE- COM), pages 1–7. IEEE,

    work page 2021

  12. [12]

    Local sgd converges fast and communicates little

    Sebastian Urban Stich. Local sgd converges fast and communicates little. In ICLR 2019- International Conference on Learning Representations,

    work page 2019

  13. [13]

    Dynamic regularized sharpness aware minimization in federated learning: Approaching global consistency and smooth landscape

    Yan Sun, Li Shen, Shixiang Chen, Liang Ding, and Dacheng Tao. Dynamic regularized sharpness aware minimization in federated learning: Approaching global consistency and smooth landscape. In International Conference on Machine Learning, pages 32991–33013. PMLR, 2023a. Yan Sun, Li Shen, Tiansheng Huang, Liang Ding, and Dacheng Tao. Fedspeed: Larger local in...

    work page arXiv

  14. [14]

    Fedadmm: A federated primal-dual algorithm allowing partial participation

    36 Convergent Differential Privacy Analysis for General Federated Learning Han Wang, Siddartha Marella, and James Anderson. Fedadmm: A federated primal-dual algorithm allowing partial participation. In 2022 IEEE 61st Conference on Decision and Control (CDC), pages 287–294. IEEE,

    work page 2022

  15. [15]

    Personalized federated learning with fea- ture alignment and classifier collaboration

    Jian Xu, Xinyi Tong, and Shao-Lun Huang. Personalized federated learning with fea- ture alignment and classifier collaboration. In The Eleventh International Conference on Learning Representations. Ge Yang, Shaowei Wang, and Haijie Wang. Federated learning with personalized local differential privacy. In 2021 IEEE 6th International Conference on Computer ...

    work page 2021

  16. [16]

    Towards faster and better federated learning: A feature fusion approach

    Xin Yao, Tianchi Huang, Chenglei Wu, Ruixiao Zhang, and Lifeng Sun. Towards faster and better federated learning: A feature fusion approach. In 2019 IEEE International Conference on Image Processing (ICIP) , pages 175–179. IEEE,

    work page 2019

  17. [17]

    Federated learning for non-iid data via unified feature learning and optimization objective alignment

    Lin Zhang, Yong Luo, Yan Bai, Bo Du, and Ling-Yu Duan. Federated learning for non-iid data via unified feature learning and optimization objective alignment. In Proceedings of the IEEE/CVF international conference on computer vision , pages 4420–4428, 2021a. Meng Zhang, Ermin Wei, and Randall Berry. Faithful edge federated learning: Scalability and privac...

    work page 2022