pith. sign in

arxiv: 2605.15648 · v1 · pith:A4JGR5TQnew · submitted 2026-05-15 · 💻 cs.CR

Rethinking the Security of DP-SGD: A Corrected Analysis of Differentially Private Machine Learning

Wenhao Wang , Shujie Cui , Hui Cui , Xingliang Yuan This is my paper

Pith reviewed 2026-05-20 18:11 UTC · model grok-4.3

classification 💻 cs.CR
keywords differential privacyDP-SGDprivacy leakagesubsampled Gaussian mechanismmembership inferenceOpacusmachine learning security
0
0 comments X

The pith

DP-SGD implementations with batch normalization leak more than standard SGM analysis reports.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that common DP-SGD code applies an extra normalization step after adding noise to the gradient sum, producing either the Expected-Averaged SGM or Batch-Averaged SGM rather than the pure Subsampled Gaussian Mechanism used in most privacy proofs. Under these more accurate models the derived privacy curves show higher leakage for the same noise and sampling parameters, so an adversary can sometimes infer record membership with greater success than the published guarantee claims. A reader cares because libraries and deployed systems base their privacy claims on the older SGM curves, leaving training data exposed to higher risk than advertised.

Core claim

Re-analysis of DP-SGD under the EASGM and ASGM formulations shows that their privacy guarantees can be weaker than the standard SGM-based guarantee, so that the true privacy leakage may exceed the reported guarantee in some regimes. Empirical audits of four state-of-the-art implementations, including successive versions of Meta's Opacus library, confirm membership-inference leakage beyond the SGM curves, and a corrected privacy guarantee is supplied for the latest Opacus release.

What carries the argument

The Expected-Averaged Subsampled Gaussian Mechanism (EASGM) and Batch-Averaged SGM (ASGM) that arise when the noisy gradient sum is divided by expected or actual batch size instead of being left as a sum.

If this is right

  • Reported privacy budgets based on SGM underestimate the actual membership-inference risk when normalization is present.
  • Libraries must switch to the corrected EASGM or ASGM accounting to issue accurate guarantees.
  • In regimes with small or variable batch sizes the gap between reported and actual leakage grows.
  • Any DP-SGD analysis that omits the normalization step cannot be treated as a tight upper bound on leakage.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same mismatch between summed and averaged formulations may appear in other gradient-based private mechanisms that normalize after noise addition.
  • Implementers could reduce the extra leakage by adjusting noise scales once the corrected accounting is adopted.
  • The result underscores the need to verify that theoretical models match the exact arithmetic performed in released code.

Load-bearing premise

The security game and privacy-curve definitions stay valid when the underlying mechanism changes from SGM to EASGM or ASGM.

What would settle it

Run a membership-inference attack on models trained with the audited Opacus version and measure whether the observed false-positive versus false-negative trade-off matches the EASGM privacy curve rather than the SGM curve.

Figures

Figures reproduced from arXiv: 2605.15648 by Hui Cui, Shujie Cui, Wenhao Wang, Xingliang Yuan.

Figure 1
Figure 1. Figure 1: The left figure illustrates the procedure of the DP game, while the right figure presents the correct mechanism [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Audited privacy leakage 𝛿 (𝜖) evaluated at 𝜖 = 0, with the output dimension fixed at𝑛 = 100. The x-axis indicates the number of data points 𝑁, and the y-axis reports the audited leakage 𝛿 (𝜖). (a)ASGM, q=0.7 (b)EASGM, q=0.005 [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Audited leakage using the (𝜖, 𝛿)-trade-off function compared against the privacy guarantee claimed under SGM, with 𝛿 = 10−5 and 𝑁 = 4 fixed. we can see that, regardless of what 𝑞 is, the audited privacy leak￾age show a generally decrease trend as the number of data points increases for both EASGM and ASGM, which follows the trends predicted by our theoretical analysis in Section 3.6. In particular, for ASG… view at source ↗
Figure 4
Figure 4. Figure 4: Audited privacy leakage of real-world DP [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Theoretical upper bounds on the per-round privacy guarantees of real-world implementations, with each privacy [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Audited privacy leakage of the real-world Opacus [PITH_FULL_IMAGE:figures/full_fig_p016_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Theoretical upper bounds on the 200-round privacy guarantees of the real-world Opacus FEASGM implementations, [PITH_FULL_IMAGE:figures/full_fig_p017_7.png] view at source ↗
read the original abstract

Differentially Private Stochastic Gradient Descent (DP-SGD) is widely used to protect training data in machine learning. Its privacy guarantee is commonly analyzed through a security game in which an adversary infers whether a target record is included in the training dataset from the mechanism output. The resulting privacy leakage is characterized by a privacy curve, which reports the false negative rate as a function of the false positive rate. We identify a mismatch between this formal analysis and common DP-SGD implementations. Existing analyses often model DP-SGD and its variants as the Subsampled Gaussian Mechanism (SGM), where Gaussian noise is added to the sum of clipped gradients computed from a Poisson-sampled batch. In practice, however, many implementations apply an additional normalization step: the noisy gradient sum is divided either by the expected batch size or by the sampled batch size. These mechanisms are therefore better formalized as the Expected-Averaged SGM (EASGM) or the Batch-Averaged SGM (ASGM), respectively. We re-analyze the privacy guarantees of DP-SGD under the EASGM and ASGM formulations. Our theoretical results show that these guarantees can be weaker than the standard SGM-based guarantee, implying that the true privacy leakage may exceed the reported guarantee in some regimes. We further audit four state-of-the-art DP-SGD implementations, including Meta's Opacus library, and observe empirical leakage beyond the SGM-based guarantees. Finally, we audit Opacus versions v0.9.0 to v1.5.4 and derive a corrected privacy guarantee for the latest implementation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper claims that standard DP-SGD privacy analyses model the mechanism as the Subsampled Gaussian Mechanism (SGM), but common implementations add a normalization step by expected batch size (EASGM) or sampled batch size (ASGM). Re-deriving the privacy curves for EASGM and ASGM shows weaker guarantees than SGM in some regimes, implying higher actual leakage than reported. Empirical audits of four libraries (including Meta's Opacus) confirm excess leakage beyond SGM bounds, and a corrected privacy guarantee is derived for Opacus v0.9.0–v1.5.4.

Significance. If the re-derivations and audits hold, the result is significant for the field: it indicates that widely deployed DP-SGD systems may be providing weaker privacy than claimed, affecting compliance, risk assessment, and trust in privacy-preserving ML. The library audits add practical weight by grounding the theoretical gap in real code.

major comments (2)
  1. [ASGM privacy analysis] ASGM formulation (privacy-curve section): the analysis models the output as (sum of clipped gradients + noise) / B with B Poisson-sampled, but does not derive from first principles how the data-dependent distribution of B (which differs under neighboring datasets) affects the effective variance sigma^2/B^2 in the security game. Standard SGM tail bounds or RDP composition cannot be applied directly without showing that the scaling factor can be pulled out or bounded independently of the inclusion bit; this step is load-bearing for the claimed gap between SGM and ASGM leakage.
  2. [Empirical audits] Empirical audit of Opacus (audit section): the reported excess leakage beyond SGM guarantees is presented as evidence that true privacy may exceed reported bounds, but the manuscript must specify the exact number of trials, the precise false-positive/false-negative rates used to construct the empirical privacy curve, and controls for post-hoc parameter choices in the audit; without these, it is difficult to confirm the leakage is not an artifact of the experimental setup.
minor comments (2)
  1. [Introduction] Notation for EASGM vs. ASGM should be introduced with explicit equations early in the manuscript to avoid ambiguity when comparing to the standard SGM definition.
  2. [Opacus audit] The corrected privacy guarantee for Opacus v1.5.4 should be stated explicitly (including the numerical privacy curve or RDP parameters) rather than only described qualitatively.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their thoughtful and constructive review of our manuscript. The comments raise important points about the rigor of our theoretical analysis and the reproducibility of our empirical audits. We address each major comment below and indicate the revisions we will make to improve clarity and completeness.

read point-by-point responses

  1. Referee: [ASGM privacy analysis] ASGM formulation (privacy-curve section): the analysis models the output as (sum of clipped gradients + noise) / B with B Poisson-sampled, but does not derive from first principles how the data-dependent distribution of B (which differs under neighboring datasets) affects the effective variance sigma^2/B^2 in the security game. Standard SGM tail bounds or RDP composition cannot be applied directly without showing that the scaling factor can be pulled out or bounded independently of the inclusion bit; this step is load-bearing for the claimed gap between SGM and ASGM leakage.

    Authors: We appreciate the referee drawing attention to this subtlety in the ASGM model. In the manuscript we define ASGM as the mechanism that outputs (sum of clipped per-example gradients + Gaussian noise) divided by the realized batch size B, where each example is included independently with probability q (Poisson sampling). While the marginal distribution of B is identical under neighboring datasets for large n, the inclusion of the target record makes B and the gradient sum statistically dependent; this correlation affects the conditional variance of the output. Our privacy-curve computation is performed directly from the definition by enumerating the output distributions under the two worlds rather than invoking SGM tail bounds or RDP. To make this explicit, we will add a first-principles derivation in the revised privacy-curve section that (i) writes the output random variable explicitly as a function of the inclusion bit, (ii) shows the resulting mixture over possible B values, and (iii) computes the privacy loss via numerical integration or Monte-Carlo estimation of the likelihood ratio. This will confirm that the scaling cannot be pulled out independently of the inclusion bit and that the resulting privacy curve lies above the SGM curve in the regimes we report. revision: yes

  2. Referee: [Empirical audits] Empirical audit of Opacus (audit section): the reported excess leakage beyond SGM guarantees is presented as evidence that true privacy may exceed reported bounds, but the manuscript must specify the exact number of trials, the precise false-positive/false-negative rates used to construct the empirical privacy curve, and controls for post-hoc parameter choices in the audit; without these, it is difficult to confirm the leakage is not an artifact of the experimental setup.

    Authors: We agree that these experimental details are essential for reproducibility and for ruling out artifacts. In the revised audit section we will report: (a) the exact number of independent trials (10,000 Monte-Carlo simulations per (epsilon, delta) point), (b) the precise construction of the empirical privacy curve (thresholds swept to achieve FPR values from 10^{-4} to 0.5, with FNR measured at each threshold), and (c) the controls employed (fixed random seeds across all library versions, pre-specified audit hyperparameters before any runs, identical clipping and noise parameters for SGM vs. library comparisons, and verification that no post-hoc adjustment of thresholds occurred after observing results). These additions will allow readers to replicate the excess-leakage finding and confirm it is not an artifact. revision: yes

Circularity Check

0 steps flagged

No circularity: privacy bounds re-derived from standard DP definitions for newly modeled mechanisms

full rationale

The paper starts from the established differential privacy security game and privacy curve definitions, then identifies that common DP-SGD implementations include an extra normalization step leading to EASGM or ASGM formulations. It re-derives the corresponding privacy guarantees using standard RDP composition and tail bounds applied to these mechanisms. No central result such as a privacy curve or leakage bound is obtained by fitting a parameter to data inside the paper, nor is any quantity defined in terms of itself. The empirical audits of libraries like Opacus serve as external validation rather than inputs to the derivation. The claimed weaker guarantees follow directly from applying existing analysis techniques to the corrected mechanism definitions.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The paper rests on standard differential privacy axioms and the definition of the membership-inference security game; no new free parameters or invented entities are introduced beyond the two averaged mechanism formulations.

axioms (1)
  • standard math Standard definitions of differential privacy via the security game that measures false-positive and false-negative rates for membership inference.
    The privacy curve is defined directly from this game applied to the mechanism output.

pith-pipeline@v0.9.0 · 5828 in / 1243 out tokens · 55091 ms · 2026-05-20T18:11:44.576001+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

54 extracted references · 54 canonical work pages · 1 internal anchor

  1. [1]

    B., Mironov, I., Talwar, K., and Zhang, L.Deep learning with differential privacy

    Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., and Zhang, L.Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security (2016), pp. 308–318. Rethinking the Security of DP-SGD: A Corrected Analysis of Differentially Private Machine Learning

    work page 2016

  2. [2]

    InProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security (New York, NY, USA, 2024), CCS ’24, Association for Computing Machinery, p

    Aerni, M., Zhang, J., and Tramèr, F.Evaluations of machine learning privacy defenses are misleading. InProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security (New York, NY, USA, 2024), CCS ’24, Association for Computing Machinery, p. 1271–1284

    work page 2024

  3. [3]

    Annamalai, M. S. M. S., Balle, B., Hayes, J., and De Cristofaro, E.To shuffle or not to shuffle: Auditing dp-sgd with shuffling. arXiv preprint arXiv:2411.10614 (2024)

    work page arXiv 2024

  4. [4]

    Annamalai, M. S. M. S., Balle, B., Hayes, J., Kaissis, G., and De Cristofaro, E.The hitchhiker’s guide to efficient, end-to-end, and tight dp auditing. arXiv preprint arXiv:2506.16666 (2025)

    work page arXiv 2025

  5. [5]

    what do you want from theory alone?

    Annamalai, M. S. M. S., Ganev, G., and De Cristofaro, E." what do you want from theory alone?" experimenting with tight auditing of differentially private synthetic data generation. In33rd USENIX Security Symposium (USENIX Security 24) (2024), pp. 4855–4871

    work page 2024

  6. [6]

    arXiv preprint arXiv:2302.02936 (2023)

    Bie, A., Kamath, G., and Zhang, G.Private gans, revisited. arXiv preprint arXiv:2302.02936 (2023)

    work page arXiv 2023

  7. [7]

    J.Deep learning with gaussian differential privacy

    Bu, Z., Dong, J., Long, Q., and Su, W. J.Deep learning with gaussian differential privacy. Harvard data science review 2020, 23 (2020), 10–1162

    work page 2020

  8. [8]

    J.Deep learning with gauss- ian differential privacy

    Bu, Z., Dong, J., Long, Q., and Su, W. J.Deep learning with gauss- ian differential privacy. Harvard Data Science Review 2, 3 (2020). Full PDF available at: https://www.researchgate.net/profile/Weijie- Su/publication/343329517_Deep_Learning_with_Gaussian_Differential_ Privacy/links/60386b16299bf1cc26efb705/Deep-Learning-with-Gaussian- Differential-Privacy...

    work page arXiv 2020

  9. [9]

    arXiv preprint arXiv:2602.17454 (2026)

    Cebere, T., Erb, D., Desfontaines, D., Bellet, A., and Fitzsimons, J.Privacy in theory, bugs in practice: Grey-box auditing of differential privacy libraries. arXiv preprint arXiv:2602.17454 (2026)

    work page arXiv 2026

  10. [10]

    8904–8918

    Chua, L., Ghazi, B., Kamath, P., Kumar, R., Manurangsi, P., Sinha, A., and Zhang, C.How private are dp-sgd implementations? In Proceedings of the 41st International Conference on Machine Learning (2024), pp. 8904–8918

    work page 2024

  11. [11]

    J., and Pearson, E

    Clopper, C. J., and Pearson, E. S.The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika 26, 4 (1934), 404–413

    work page 1934

  12. [12]

    Gaussian Differential Privacy

    Dong, J., Roth, A., and Su, W. J.Gaussian differential privacy. arXiv preprint arXiv:1905.02383 (2019)

    work page internal anchor Pith review Pith/arXiv arXiv 1905

  13. [13]

    Du, Y., Li, J., Chen, Y., Zhang, K., Yuan, Z., Xiao, H., Ribeiro, B., and Li, N.Cas- cading and proxy membership inference attacks.arXiv preprint arXiv:2507.21412 (2025)

    work page arXiv 2025

  14. [14]

    Handcrafted-DP: requirements.txt, line 37

    ftramer. Handcrafted-DP: requirements.txt, line 37. GitHub repository. Avail- able: https://github.com/ftramer/Handcrafted-DP/blame/main/requirements.txt# L37, 2021. Accessed: Apr. 25, 2026

    work page 2021

  15. [15]

    Proceedings of the VLDB Endowment 17, 6 (2024), 1200–1213

    Fu, J., Ye, Q., Hu, H., Chen, Z., W ang, L., W ang, K., and Ran, X.Dpsur: Accel- erating differentially private stochastic gradient descent using selective update and release. Proceedings of the VLDB Endowment 17, 6 (2024), 1200–1213

    work page 2024

  16. [16]

    Differential privacy semantics for on-device per- sonalization (odp)

    Google Privacy Sandbox. Differential privacy semantics for on-device per- sonalization (odp). https://privacysandbox.google.com/protections/on-device- personalization/differential-privacy-semantics-for-odp, 2025. Accessed: 2025- 10-10

    work page 2025

  17. [17]

    VaultGemma: The world’s most capable differentially pri- vate llm

    Google Research. VaultGemma: The world’s most capable differentially pri- vate llm. https://research.google/blog/vaultgemma-the-worlds-most-capable- differentially-private-llm/, 2025. Accessed: 2026-04-24

    work page 2025

  18. [18]

    T., and Wutschitz, L.Numerical composition of differential privacy

    Gopi, S., Lee, Y. T., and Wutschitz, L.Numerical composition of differential privacy. Advances in Neural Information Processing Systems 34 (2021), 11631– 11642

    work page 2021

  19. [19]

    S., and Ryzhik, I

    Gradshteyn, I. S., and Ryzhik, I. M. Table of integrals, series, and products. Academic press, 2014

    work page 2014

  20. [20]

    DPSUR: Dpsur.py

    JeffffffFu. DPSUR: Dpsur.py. GitHub repository. Available: https://github.com/ JeffffffFu/DPSUR/blame/main/algorithm/DPSUR.py#L146, 2023. Accessed: Apr. 25, 2026

    work page 2023

  21. [21]

    DPSUR GDP accountant implementation

    JeffffffFu. DPSUR GDP accountant implementation. https: //github.com/JeffffffFu/DPSUR/blame/main/privacy_analysis/GDP/compute_ gdp.py#L25C11-L25C11, 2023. Accessed: Apr. 27, 2026

    work page 2023

  22. [22]

    arXiv preprint arXiv:2406.08918 (2024)

    Kaissis, G., Kolek, S., Balle, B., Hayes, J., and Rueckert, D.Beyond the calibration point: Mechanism comparison in differential privacy. arXiv preprint arXiv:2406.08918 (2024)

    work page arXiv 2024

  23. [23]

    https://arxiv.org/pdf/2406.04827, 2024

    Koskela, A., and Mohammadi, J.Auditing differential privacy guarantees using density estimation. https://arxiv.org/pdf/2406.04827, 2024. arXiv version. Accessed: Apr. 27, 2026

    work page arXiv 2024

  24. [24]

    A.Auditing differential privacy guarantees using density estimation

    Koskela, A., and Mohammadi, J. A.Auditing differential privacy guarantees using density estimation. In 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) (2025), IEEE, pp. 1007–1026

    work page 2025

  25. [25]

    A.Auditing differential privacy guarantees using density estimation

    Koskela, A., and Mohammadi, J. A.Auditing differential privacy guarantees using density estimation. In 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) (2025), pp. 1007–1026

    work page 2025

  26. [26]

    G., and Parks, H

    Krantz, S. G., and Parks, H. R. A primer of real analytic functions. Springer Science & Business Media, 2002

    work page 2002

  27. [27]

    J., Regehr, M., Kamath, G., and Steinke, T.Avoiding pitfalls for privacy accounting of subsampled mechanisms under composition

    Lebeda, C. J., Regehr, M., Kamath, G., and Steinke, T.Avoiding pitfalls for privacy accounting of subsampled mechanisms under composition. In 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) (2025), IEEE, pp. 996–1006

    work page 2025

  28. [28]

    In 2025 IEEE Symposium on Security and Privacy (SP) (2025), pp

    Li, K., Gong, C., Li, X., Zhao, Y., Hou, X., and Wang, T.From easy to hard: Building a shortcut for differentially private image synthesis. In 2025 IEEE Symposium on Security and Privacy (SP) (2025), pp. 3988–4006

    work page 2025

  29. [29]

    In 33rd USENIX Security Symposium (USENIX Security 24) (Philadelphia, PA, Aug

    Li, K., Gong, C., Li, Z., Zhao, Y., Hou, X., and Wang, T.PrivImage: Differen- tially private synthetic image generation using diffusion models with Semantic- Aware pretraining. In 33rd USENIX Security Symposium (USENIX Security 24) (Philadelphia, PA, Aug. 2024), USENIX Association, pp. 4837–4854

    work page 2024

  30. [30]

    In 33rd USENIX Security Symposium (USENIX Security

    Li, K., Gong, C., Li, Z., Zhao, Y., Hou, X., and W ang, T.{PrivImage}: Differen- tially private synthetic image generation using diffusion models with {Semantic- Aware} pretraining. In 33rd USENIX Security Symposium (USENIX Security

    work page

  31. [31]

    4837–4854

    (2024), pp. 4837–4854

    work page 2024

  32. [32]

    arXiv preprint arXiv:1908.10530 (2019)

    Mironov, I., Talwar, K., and Zhang, L.R \’enyi differential privacy of the sampled gaussian mechanism. arXiv preprint arXiv:1908.10530 (2019)

    work page arXiv 1908

  33. [33]

    In 32nd USENIX Security Symposium (USENIX Security 23) (2023), pp

    Nasr, M., Hayes, J., Steinke, T., Balle, B., Tramèr, F., Jagielski, M., Carlini, N., and Terzis, A.Tight auditing of differentially private machine learning. In 32nd USENIX Security Symposium (USENIX Security 23) (2023), pp. 1631–1648

    work page 2023

  34. [34]

    In 2021 IEEE Symposium on security and privacy (SP) (2021), IEEE, pp

    Nasr, M., Song, S., Thakurta, A., Papernot, N., and Carlini, N.Adversary instantiation: Lower bounds for differentially private machine learning. In 2021 IEEE Symposium on security and privacy (SP) (2021), IEEE, pp. 866–882

    work page 2021

  35. [35]

    A., Ganesh, A., Jagielski, M., Hayes, J., Thakurta, A

    Nasr, M., Steinke, T., Balle, B., Choqette-Choo, C. A., Ganesh, A., Jagielski, M., Hayes, J., Thakurta, A. G., Smith, A., and Terzis, A.The last iterate advan- tage: Empirical auditing and principled heuristic analysis of differentially private sgd. In The Thirteenth International Conference on Learning Representations (2025)

    work page 2025

  36. [36]

    Opacus: privacy_engine.py sgm implementation

    Opacus Team. Opacus: privacy_engine.py sgm implementation. GitHub reposi- tory. Available: https://github.com/meta-pytorch/opacus/blame/v1.5.4/opacus/ privacy_engine.py#L488, 2021. Commit hash: 9b1d19c. Accessed: Apr. 25, 2026

    work page 2021

  37. [37]

    Opacus GDP accountant implementation

    Opacus Team. Opacus GDP accountant implementation. https://github.com/ meta-pytorch/opacus/blame/v1.5.4/opacus/accountants/gdp.py#L23, 2022. Ac- cessed: Apr. 27, 2026. [37]Rudin, W.Real and complex analysis

    work page 2022

  38. [38]

    Advances in Neural Information Processing Systems 36 (2023), 49268– 49280

    Steinke, T., Nasr, M., and Jagielski, M.Privacy auditing with one (1) train- ing run. Advances in Neural Information Processing Systems 36 (2023), 49268– 49280

    work page 2023

  39. [39]

    https://arxiv.org/pdf/2305.08846, 2023

    Steinke, T., Nasr, M., and Jagielski, M.Privacy auditing with one (1) training run. https://arxiv.org/pdf/2305.08846, 2023. arXiv version. Accessed: Apr. 27, 2026

    work page arXiv 2023

  40. [40]

    DP-ImaGen: Opacus readme

    SunnierLee. DP-ImaGen: Opacus readme. https://github.com/SunnierLee/DP- ImaGen/blob/main/src/opacus/README.md, 2023. Accessed: Apr. 25, 2026

    work page 2023

  41. [41]

    DP-FETA RDP accountant implementation

    SunnierLee. DP-FETA RDP accountant implementation. https://github.com/ SunnierLee/DP-FETA/tree/main/opacus/opacus/accountants, 2024. Accessed: Apr. 27, 2026

    work page 2024

  42. [42]

    DP-FETA: Opacus version

    SunnierLee. DP-FETA: Opacus version. GitHub repository. Available: https: //github.com/SunnierLee/DP-FETA/tree/main/opacus, 2025. Accessed: Apr. 25, 2026

    work page 2025

  43. [43]

    In 33rd USENIX Security Symposium (USENIX Security 24) (2024), pp

    Thudi, A., Jia, H., Meehan, C., Shumailov, I., and Papernot, N.Gradients look alike: Sensitivity is often overestimated in {DP-SGD}. In 33rd USENIX Security Symposium (USENIX Security 24) (2024), pp. 973–990

    work page 2024

  44. [44]

    arXiv preprint arXiv:2011.11660 (2020)

    Tramer, F., and Boneh, D.Differentially private learning needs better features (or much more data). arXiv preprint arXiv:2011.11660 (2020)

    work page arXiv 2011

  45. [45]

    Opacus GitHub Issue #571: Privacy Leakage at Low Sample Size

    tudorcebere. Opacus GitHub Issue #571: Privacy Leakage at Low Sample Size. https://github.com/meta-pytorch/opacus/issues/571#issue-1609246665, 2023. Ac- cessed: Apr. 27, 2026

    work page 2023

  46. [46]

    Advances in Neural Information Processing Systems 36 (2023), 55051–55063

    W ang, C., Su, B., Ye, J., Shokri, R., and Su, W.Unified enhancement of privacy bounds for mixture mechanisms via 𝑓 -differential privacy. Advances in Neural Information Processing Systems 36 (2023), 55051–55063

    work page 2023

  47. [47]

    In 33rd USENIX Security Symposium (USENIX Security 24) (2024), pp

    W ang, H., Pang, S., Lu, Z., Rao, Y., Zhou, Y., and Xue, M.dp-promise: Differen- tially private diffusion probabilistic models for image synthesis. In 33rd USENIX Security Symposium (USENIX Security 24) (2024), pp. 1063–1080

    work page 2024

  48. [48]

    In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security (2022), pp

    Wei, J., Bao, E., Xiao, X., and Yang, Y.Dpis: An enhanced mechanism for differentially private sgd with importance sampling. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security (2022), pp. 2885–2899

    work page 2022

  49. [49]

    Xiang, Z., Wang, T., and Wang, D.Privacy audit as bits transmission:(im) possibilities for audit by one run

    work page

  50. [50]

    arXiv preprint arXiv:2509.08704 (2025)

    Xiang, Z., Wang, T., Xiao, H., Tian, Y., and Wang, D.Tight privacy audit in one run. arXiv preprint arXiv:2509.08704 (2025)

    work page arXiv 2025

  51. [51]

    In NeurIPS 2021 Workshop on Privacy in Machine Learning (2021)

    Yousefpour, A., Shilov, I., Sablayrolles, A., Testuggine, D., Prasad, K., Malek, M., Nguyen, J., Ghosh, S., Bharadwaj, A., Zhao, J., et al.Opacus: User-friendly differential privacy library in pytorch. In NeurIPS 2021 Workshop on Privacy in Machine Learning (2021)

    work page 2021

  52. [52]

    In The Thirteenth International Conference on Learning Representations (2025)

    Zhang, X., Bu, Z., Balle, B., Hong, M., Razaviyayn, M., and Mirrokni, V.DiSK: Differentially private optimizer with simplified kalman filter for noise reduction. In The Thirteenth International Conference on Learning Representations (2025)

    work page 2025

  53. [53]

    In International Conference on Wenhao Wang, Shujie Cui, Hui Cui, and Xingliang Yuan Machine Learning (2020), PMLR, pp

    Zheng, Q., Dong, J., Long, Q., and Su, W.Sharp composition bounds for gaussian differential privacy via edgeworth expansion. In International Conference on Wenhao Wang, Shujie Cui, Hui Cui, and Xingliang Yuan Machine Learning (2020), PMLR, pp. 11420–11435. A Privacy Analysis of FEASGM To understand the privacy behavior of FEASGM, we analyze its pri- vacy ...

    work page 2020

  54. [54]

    Then, according to Lemma 2.6, we have lim𝑛→∞ (𝑓 ∗) ⊗ (𝑛−1) (𝛼)= 0for all 𝛼∈ ( 0, 1]

    Note that 𝑓 ∗ ≠ID . Then, according to Lemma 2.6, we have lim𝑛→∞ (𝑓 ∗) ⊗ (𝑛−1) (𝛼)= 0for all 𝛼∈ ( 0, 1]. Then, by Property 1, we have(𝑓 ∗) ⊗ (𝑛−1) = 𝑓. Therefore,lim 𝑛→∞ 𝑓(𝛼)=0,∀𝛼∈ (0,1]. D.10 Proof of Theorem 3.10 Let 𝑓 1 𝐵 and 𝑓 2 𝐵 be defined in Eqs. 9 and 10, respectively, and let 𝑓 1 ∅ =ID and 𝑓 2 ∅ =𝑇(𝐺, 𝐺+ 𝑔𝑁+1 ). We first show that 𝑇𝑃 ⊗ (𝑛−1) , 𝑄 ...

    work page