AdaCliP: Adaptive Clipping for Private SGD

Ananda Theertha Suresh; Felix X. Yu; Sanjiv Kumar; Sashank J. Reddi; Venkatadheeraj Pichapati

REVIEW 3 major objections 4 minor 6 cited by

AdaCliP: Adaptive Clipping for Private SGD

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Adaptive per-coordinate clipping gives the same differential-privacy guarantee with substantially less added noise than standard L2 clipping.

desk verdict A clean per-coordinate clipping idea with a real privacy-accounting gap; the sqrt(B) issue is genuine but runs in the authors' favor here, so the paper deserves a careful revision rather than rejection. read the letter →

arxiv 1908.07643 v2 pith:ARKW4MTF submitted 2019-08-20 cs.LG cs.CRstat.ML

Venkatadheeraj Pichapati , Ananda Theertha Suresh , Felix X. Yu , Sashank J. Reddi , Sanjiv Kumar This is my paper · ORCID

classification cs.LGcs.CRstat.ML

keywords differentialprivacystochasticgradientdescentadaptiveclippingcoordinate-wiseGaussianmechanismmomentsaccountantnoiseMNIST

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AdaCliP is a differentially private SGD algorithm whose central claim is that the usual way of bounding gradient sensitivity—clipping the whole gradient vector's norm—adds more noise than necessary. The paper derives the optimal coordinate-wise scaling: after centering each gradient by its estimated mean, each coordinate is scaled in proportion to the square root of that coordinate's standard deviation, so coordinates with little signal receive very little noise. Under this scaling the expected squared added noise becomes proportional to $(\sum_i s_i^t)^2/\gamma$ rather than the $d\sum_i (s_i^t)^2/\gamma$ of L2 clipping, a factor of $d$ smaller when only a few coordinates are informative. On MNIST logistic regression and a one-hidden-layer network, AdaCliP reports higher accuracy than L2 clipping at the same $(\epsilon,\delta)$ budgets, with gains from about 0.2 to 1.6 percentage points.

What carries the argument

The load-bearing object is the per-coordinate transformation $w^t = (g^t - a^t)/b^t$ with clipping $\hat{w}^t = w^t/\max(1,\|w^t\|)$ before adding Gaussian noise $N(0,\sigma^2 I)$. The optimal scale comes from the constrained program $\min_{a,b}\sum_i b_i^2$ subject to $\sum_i (s_i^2 + (m_i-a_i)^2)/b_i^2 \le \gamma$; Cauchy's inequality gives equality when $a_i = m_i$ and $b_i^2 = s_i S_t/\gamma$. Here $\gamma$ is a parameter bounding the expected squared norm of the transformed gradient, which controls how often clipping occurs. The algorithm estimates the mean $m$ and standard deviation $s$ by exponential moving averages of private noisy gradients, with a debiasing correction that subtracts the known noise variance $(b_i^t)^2\sigma^2$ from each squared deviation.

What would settle it

Run Algorithm 1 exactly as written and feed its per-step noisy gradients to a moments accountant, comparing the resulting $\epsilon$ with the values in Tables 1 and 2; a discrepancy would show the per-example-versus-mean noise equivalence fails at the implemented scale. Separately, on the paper's regression example with $d=1000$ and one informative coordinate, measure the empirical average of $\|b_t\|^2\sigma^2$; if the injected noise grows with $d$, the predicted factor-$d$ reduction is absent.

Watch

Extended reading notes

Core claim

The paper's central claim is that in the family of element-wise linear gradient transformations—subtract a vector, divide coordinate-wise, clip to unit norm, add Gaussian noise, then rescale—the choice that minimizes injected noise is not the usual L2-norm clipping and not classical whitening. Writing $s_i^t$ for the standard deviation of coordinate $i$ of the stochastic gradient at iteration $t$, the optimal shift is $a_i^t = \mathbb{E}g_i^t$ and the optimal scale is $b_i^t = \sqrt{s_i^t S_t/\gamma}$, where $S_t = \sum_j s_j^t$ and $\gamma$ bounds the expected squared norm of the transformed gradient. The expected squared added noise then equals $\sigma^2 S_t^2/\gamma$, compared with $\sigma^2 d\sum_i (s_i^t)^2/\gamma$ for L2 clipping, so the advantage grows with the number of coordinates that carry no signal. On MNIST, AdaCliP reports accuracy improvements of 0.2 to 1.6 percentage points over L2 clipping at the same $(\epsilon,\delta)$ budgets, consistent with the smaller injected noise.

Load-bearing premise

If adding Gaussian noise to each per-example gradient and then averaging is not the same as adding one Gaussian perturbation to the averaged gradient at the scale the moments accountant assumes, then the reported $\epsilon$ does not describe the algorithm actually executed.

Editorial extensions

If this is right

At a fixed $(\epsilon,\delta)$ budget, AdaCliP's coordinate-adaptive clipping yields lower expected injected noise than L2 norm clipping whenever coordinate standard deviations are unequal, because $(\sum_i s_i)^2 \le d\sum_i s_i^2$.
On sparse-gradient problems such as the paper's regression example, the noise saving is a factor of the dimension $d$, and the optimization error no longer grows with the number of uninformative coordinates.
AdaCliP needs no public data or precomputed Lipschitz bounds; it estimates the per-coordinate mean and standard deviation from the same private noisy gradients used for updates.
The privacy-utility tradeoff in deep models improves consistently on MNIST: with $(\epsilon,10^{-5})$-differential privacy, accuracy rises by 0.2 to 1.6 percentage points at matched $\epsilon$ compared with L2 clipping.
The same moments accountant used for standard private SGD applies, so AdaCliP can be inserted into existing private-training pipelines without changing the privacy accounting method.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct consequence the paper leaves implicit: if the per-user noise in Algorithm 1 is not scaled by $1/\sqrt{B}$ before averaging, the effective noise-to-sensitivity ratio on the batch mean is not the $\sigma$ used in the moments accountant, so the reported $\epsilon$ values should be rechecked against the exact per-user algorithm.
The debiased variance estimator subtracts $(b_i^t)^2\sigma^2$ under the assumption that clipping did not occur, while the analysis assumes clipping happens with constant probability; an unbiased variance estimator robust to clipping would make the theory and implementation consistent.
The same per-coordinate scaling idea could be tested in federated learning settings, where client gradient variances are naturally heterogeneous; one could check whether AdaCliP's advantage over L2 clipping grows with the degree of heterogeneity across clients.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The thing to know about this paper is that the core optimization is genuinely new and clean: instead of clipping the whole gradient or adapting a single L2 threshold, AdaCliP chooses per-coordinate scales by minimizing the expected squared noise subject to a variance budget. Theorem 1 is a neat Holder-inequality argument, and the regression example makes the factor-d improvement vivid. The MNIST results are consistent and modest (0.2-1.6% accuracy gains), and the measured noise ratio backs the theory. I was not expecting to like it as much as I did.

The soft spots are real but not fatal. Section 5 says adding Gaussian noise to each per-user gradient and adding it to the mean is 'essentially equivalent.' That is not a proof. As the stress-test note says, for batch size B the averaged per-user noise has covariance b b^T sigma^2 / B, while the accountant is fed sigma for a sensitivity-1 mechanism. So the accountant input does not match the mechanism. However, the direction of the discrepancy is in the authors' favor: with B=600, the effective per-step noise-to-sensitivity ratio is sigma sqrt(B)/2, which is much larger than sigma, so the reported epsilon is conservative. The privacy claim is likely salvageable, but only after the authors actually write down the per-step conditional sensitivity and the resulting z, rather than hand-waving equivalence.

The second gap is theory-to-algorithm transfer. Theorem 2 assumes a_t equals the true gradient mean; the algorithm estimates it from noisy gradients, and the variance estimator subtracts b_i^2 sigma^2 under an assumption that clipping does not happen. That conflicts with the analysis, which relies on clipping occurring with constant probability. This is worth a proper derivation or at least an explicit statement of what is heuristic.

Also minor: h2 is tuned but never reported, no code is released, and the experiments omit the adaptive L2 clipping baselines the authors themselves cite (van der Veen et al., Thakkar et al.). For a paper whose claim is 'adds less noise than previous methods,' those comparisons matter.

The citation pattern is fine; Lemma 2 is self-cited but standard.

Bottom line: this is a serious piece of work, the central idea is sound, but the privacy accounting needs to be rewritten before the headline claim is established. I would send it to a good referee and ask for a revision.

Referee Report

3 major / 4 minor

Summary. The paper proposes AdaCliP, a differentially private SGD method with coordinate-wise adaptive clipping. In the proposed framework, a stochastic gradient is transformed by subtracting a vector a_t and dividing by a vector b_t, clipped to unit L2 norm, and perturbed with Gaussian noise; Theorem 1 states that under a constraint E||w_t||^2 <= gamma, the optimal choices are a_t equal to the true gradient mean and b_t proportional to coordinate standard deviations, yielding noise whose squared norm scales as the square of the sum of standard deviations rather than d times the L2-clipping noise. Algorithm 1 estimates these quantities from noisy gradients and averages per-user noisy gradients over a minibatch. Experiments on a synthetic regression and on MNIST for logistic regression and a neural network report that AdaCliP achieves better accuracy than L2 clipping and noise proportional to the gradient norm bound for the same reported (epsilon,delta).

Significance. The core optimization idea is attractive and the derivation in Theorem 1 is transparent: if the privacy mechanism is genuinely the one analyzed in Section 4.1, coordinate-wise adaptive clipping provably reduces added noise relative to uniform clipping, and the empirical gains on MNIST would be valuable. The paper also gives a concrete algorithm and an explicit experimental protocol, which are assets. However, the advertised privacy guarantee is not established as written, because the noise accounting in Section 5 does not match the mechanism actually run in Algorithm 1, and because the convergence theorem is proved for an idealized variant with batch size 1 and a_t equal to the true gradient mean. The central claim is therefore currently conditional on a repair of the privacy analysis.

major comments (3)

[Section 5, Algorithm 1] The sentence 'adding Gaussian noise to the individual user processed gradient and to the mean processed gradient is essentially equivalent' is not correct as stated. If each per-user transformed gradient receives N(0,sigma^2 I), the averaged noisy gradient has noise covariance (b_t b_t^T) sigma^2 / B, whereas the framework of Section 4.1 assumes the mean receives N(0,sigma^2 I). Conditioned on the history, so that b_t is fixed, the per-step mechanism has L2 sensitivity 2||b_t||/B and noise standard deviation ||b_t|| sigma / sqrt(B), giving an effective Gaussian-mechanism noise scale z = sigma sqrt(B)/2 (up to the usual factor of 2 in the sensitivity convention) rather than sigma. The moments accountant is therefore invoked with the wrong per-step privacy parameter, so the reported (epsilon,delta) is not derived from the algorithm as written and the claimed 'provably adds less noise under (epsilon,delta)-DP' is not established. For the experimental batch size B=600 the corrected z is larger and the reported epsilon would be conservative, but for B<4 the opposite holds. Please either modify Algorithm 1 to scale the per-user noise by sqrt(B), or add the noise directly to the mean with scale sigma, and in either case provide a conditional-privacy analysis that accounts for the data-dependent b_t.
[Section 5, variance estimator] The debiasing identity used for the variance estimate, deriving E[(g_t^i - m_t^i)^2] from E[(~g_t^i - m_t^i)^2] - (b_t^i)^2 sigma^2, is derived under the assumption that clipping does not take place, i.e., ~g_t = g_t + b_t N_t. This directly conflicts with the design premise of the paper, which states that E||w_t||^2 <= gamma ensures that w_t is clipped with constant probability. Consequently the variance estimate is biased precisely in the operating regime the algorithm targets. In addition, if per-user noise is averaged over a batch, the correction should involve sigma^2 / B rather than sigma^2. This should be acknowledged as a heuristic, or the estimator should be analyzed in the presence of clipping.
[Section 4.2, Theorem 2] Theorem 2 is stated for batch size 1 and for a_t equal to the true expectation E[g_t], while Algorithm 1 uses a minibatch of size B and replaces a_t by the exponential moving average m_t of noisy gradients defined in Eq. (4). The convergence bound therefore does not apply to the algorithm whose accuracy is reported in Section 6. Since the theoretical motivation for AdaCliP rests on this bound, the paper should either extend the analysis to the estimated a_t and b_t (with batch size B), or explicitly state Theorem 2 as an ideal-population analysis and provide a separate, weaker guarantee for the implemented variant.

minor comments (4)

[Section 4, Theorem 1 and Theorem 2] The notation s_t^i = Σ E[(g_t^i - E g_t^i)^2] appears to be a typo; it should denote the coordinate variance E[(g_t^i - E g_t^i)^2] or its square root, since the proof uses (s_t^i)^2 in the numerator. Please make the definition of s_t^i unambiguous.
[Section 5, Eq. (5)] The expression for b_t^i in Algorithm 1 is rendered ambiguously in the text; the intended formula b_t^i = sqrt(s_t^i / gamma) * sqrt(sum_j s_t^j) should be written explicitly with parentheses so that the derivation in Theorem 1 is reproducible.
[Appendix B] The text contains 'deferentially private momentum optimization'; this should be 'differentially private'.
[Section 6, Figure 3] Figure 3 reports an average noise ratio around 0.8, but no error bars or number of independent runs are given; since this figure directly supports the claim of reduced added noise, some measure of variability would be helpful.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular reduction in the noise-minimization core; only a minor, non-load-bearing self-citation in the convergence lemma.

full rationale

The paper's central derivation is not circular. Theorem 1 (Section 4.1) is a self-contained constrained optimization: it minimizes the expected squared L2-norm of the added Gaussian noise, sum_i (b_t^i)^2, subject to the constraint E||w_t||^2 ≤ gamma, and the Holder-inequality argument yields b_t^i proportional to sqrt(s_t^i) with squared noise proportional to (sum_i sqrt(s_t^i))^2/gamma. This result does not assume the conclusion; it derives it from the stated objective. Section 4.3 then evaluates the L2-clipping choice a=0, b=C as a feasible point and compares objective values, which is a legitimate feasible-point comparison rather than an input-output circularity. The convergence analysis (Theorem 2 and Appendix A) relies on Lemma 2, which is cited from [42] by overlapping authors; however, Lemma 2 is a standard SGD variance/bias bound that is not the paper's target privacy-or-noise claim, so under the hard rules this self-citation does not constitute load-bearing circularity. Two passages are worth flagging but are not circular: Section 5's statement that 'since the sum of Gaussian noises is also Gaussian noise, adding Gaussian noise to the individual user processed gradient and to the mean processed gradient is essentially equivalent' is a scale-sensitive accounting assertion (off by a sqrt(B) factor for the released batch mean), and Equation (5)'s debiasing step E(g_i - m_i)^2 = E(g_tilde_i - m_i)^2 - (b_i)^2 sigma^2 explicitly assumes clipping does not occur while the analysis assumes it occurs with constant probability. These are correctness or privacy-accounting gaps, not self-referential derivations. Accordingly, no prediction in the paper reduces to its input by construction, and the score of 2 reflects only the minor non-load-bearing self-citation in the convergence lemma.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central noise-minimization derivation is self-contained. The main load-bearing assumptions are about the privacy mechanism's composition and about the validity of the variance estimator; the latter two are stated or implied but not fully justified.

free parameters (2)

h2 = not reported
Upper clamp in the variance estimate update (5); the authors state h2 is the only hyperparameter tuned in experiments but do not report its value.
gamma (variance budget) = 1 (implicit)
The transformed gradient is clipped at norm 1, which fixes the bound E||w||^2 <= gamma in Theorem 1 to gamma = 1. This choice is not discussed as tunable, but it sets the overall scale of the added noise.

assumptions (5)

standard math Gaussian mechanism gives (epsilon, delta)-DP for noise scale sigma (Lemma 1)
Standard differential privacy background; used to translate noise scale into privacy parameters.
standard math SGD convergence bound from Reddi et al. (Lemma 2)
Used to motivate the variance and bias tradeoff; the cited theorem is by overlapping authors, including two authors of this paper.
domain assumption E||w_t||^2 <= gamma limits clipping in the analysis
The paper limits the optimization to transformed gradients whose expected squared norm is bounded; clipping is shown to occur with constant probability under this assumption.
ad hoc to paper Adding noise to each per-example gradient is equivalent to adding noise to the mean
Section 5 relies on this to present AdaCliP's noise addition as equivalent to standard DP-SGD; the equivalence requires batch-size scaling that is not specified.
ad hoc to paper Variance estimator assumes no clipping
The correction (g_i - m_i)^2 - b_i^2 sigma^2 is derived under the assumption that the noisy gradient equals the true gradient plus noise, though the analysis predicts clipping with constant probability.

how reviews work

0 comments

Cite this review

Pith. "Pith review of AdaCliP: Adaptive Clipping for Private SGD." pith.science (2026). https://pith.science/paper/ARKW4MTF

@misc{pith2026190807643,
  author       = {Pith},
  title        = {Pith review of: AdaCliP: Adaptive Clipping for Private SGD},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ARKW4MTF}},
  note         = {Machine review of arXiv:1908.07643}
}

read the original abstract

Privacy preserving machine learning algorithms are crucial for learning models over user data to protect sensitive information. Motivated by this, differentially private stochastic gradient descent (SGD) algorithms for training machine learning models have been proposed. At each step, these algorithms modify the gradients and add noise proportional to the sensitivity of the modified gradients. Under this framework, we propose AdaCliP, a theoretically motivated differentially private SGD algorithm that provably adds less noise compared to the previous methods, by using coordinate-wise adaptive clipping of the gradient. We empirically demonstrate that AdaCliP reduces the amount of added noise and produces models with better accuracy.

Figures

Figures reproduced from arXiv: 1908.07643 by the authors.

**Figure 3.** Average noise per gradient vs for (, 10−5 )-DP neural network Mean estimate: Since there is no direct access to stochastic gradients at time t, mt is approximated by exponential average of previous noisy gradients g˜ t (momentum style approach) mt = β1mt−1 + (1 − β1)˜g t , (4) where β1 is a decay parameter of the exponential moving average. Variance estimate: For variance, we need to estimate E(g t i −mt i ) 2 an… view at source ↗

**Figure 4.** Comparison of SGD with various momentum factors [PITH_FULL_IMAGE:figures/full_fig_p018_4.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

StraightDP: Geometry-Aware Differential Privacy for Rectified-Flow Transformers
cs.LG 2026-07 conditional novelty 6.0 of 10

StraightDP releases a few DP class-conditioned moments to define the noise-end velocity of a rectified flow, then uses DP-SGD only on the sample-specific part, improving strong-privacy generation accuracy.
Private training in quantum machine learning
quant-ph 2026-06 unverdicted novelty 6.0 of 10

Hybrid QML models trained with classical DP-SGD retain higher accuracy than classical models under fixed privacy budgets on synthetic and image-classification tasks.
Efficient Differentially Private Fine-Tuning of LLMs via Reinforcement Learning
cs.LG 2025-07 reject novelty 6.0 of 10

RLDP uses a soft actor-critic policy to adapt per-adapter clipping and noise during DP-SGD fine-tuning of LLMs, claiming utility gains and faster convergence, but the privacy proof is internally inconsistent.
Structure-Preference Enabled Graph Embedding Generation under Differential Privacy
stat.ML 2025-01 reject novelty 5.0 of 10

A private graph embedding method that claims to preserve user-chosen node proximities, though its central proof and privacy analysis contain serious gaps.
AdaDPIGU: Differentially Private SGD with Adaptive Clipping and Importance-Based Gradient Updates for Deep Neural Networks
cs.LG 2025-07 reject novelty 4.0 of 10

A DP-SGD variant using top-60% gradient sparsification and coordinate-wise adaptive clipping is proposed; its privacy guarantee is not established for the actual algorithm because the mask comes from private data.
Fast Fourier Transform-Based Spectral and Temporal Gradient Filtering for Differential Privacy
cs.LG 2025-05 reject novelty 4.0 of 10

FFTKF applies an FFT low-pass mask and a Kalman filter to privatized gradients, reporting modest accuracy gains over DP-SGD and DiSK on several image benchmarks, but with incomplete privacy accounting.

Reference graph

Works this paper leans on

43 extracted references · 33 canonical work pages · cited by 6 Pith papers

[1]

Delving deep into rectiﬁers: Surpassing human-level performance on imagenet classiﬁcation

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectiﬁers: Surpassing human-level performance on imagenet classiﬁcation. InProceedings of the IEEE international conference on computer vision, pages 1026–1034, 2015

work page 2015
[2]

Imagenet classiﬁcation with deep convolutional neural networks

Alex Krizhevsky, Ilya Sutskever, and Geoﬀrey E Hinton. Imagenet classiﬁcation with deep convolutional neural networks. InAdvances in neural information processing systems, pages 1097–1105, 2012

work page 2012
[3]

Recurrent neural network based language model

Tomáš Mikolov, Martin Karaﬁát, Lukáš Burget, Jan Černock` y, and Sanjeev Khudanpur. Recurrent neural network based language model. In Eleventh Annual Conference of the International Speech Communication Association, 2010

work page 2010
[4]

Grammar as a foreign language

Oriol Vinyals, Łukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoﬀrey Hinton. Grammar as a foreign language. InAdvances in Neural Information Processing Systems, pages 2773–2781, 2015

work page 2015
[5]

The netﬂix challenge.WIRED-SAN FRANCISCO-, 16(3):114, 2008

Jordan Ellenberg. The netﬂix challenge.WIRED-SAN FRANCISCO-, 16(3):114, 2008

work page 2008
[6]

Federated learning for mobile keyboard prediction

Andrew Hard, Kanishka Rao, Rajiv Mathews, Françoise Beaufays, Sean Augenstein, Hubert Eichner, Chloé Kiddon, and Daniel Ramage. Federated learning for mobile keyboard prediction. arXiv preprint arXiv:1811.03604, 2018

arXiv 2018
[7]

Lattice rescoring strategies for long short term memory language models in speech recognition

Shankar Kumar, Michael Nirschl, Daniel Holtmann-Rice, Hank Liao, Ananda Theertha Suresh, and Felix Yu. Lattice rescoring strategies for long short term memory language models in speech recognition. InAutomatic Speech Recognition and Understanding Workshop (ASRU), 2017 IEEE, pages 165–172. IEEE, 2017

work page 2017
[8]

Calibrating noise to sensitivity in private data analysis

Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Theory of cryptography conference, pages 265–284. Springer, 2006

2006

Show all 43 references

[9]

The algorithmic foundations of diﬀerential privacy.Foun- dations and TrendsR©in Theoretical Computer Science, 9(3–4):211–407, 2014

Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations of diﬀerential privacy.Foun- dations and TrendsR©in Theoretical Computer Science, 9(3–4):211–407, 2014

2014
[10]

Privacy-preserving logistic regression

Kamalika Chaudhuri and Claire Monteleoni. Privacy-preserving logistic regression. InAdvances in Neural Information Processing Systems, pages 289–296, 2009

2009
[11]

Diﬀerentially private empirical risk minimization

Kamalika Chaudhuri, Claire Monteleoni, and Anand D Sarwate. Diﬀerentially private empirical risk minimization. Journal of Machine Learning Research, 12(Mar):1069–1109, 2011

2011
[12]

Bolt- on diﬀerential privacy for scalable stochastic gradient descent-based analytics

Xi Wu, Fengan Li, Arun Kumar, Kamalika Chaudhuri, Somesh Jha, and Jeﬀrey Naughton. Bolt- on diﬀerential privacy for scalable stochastic gradient descent-based analytics. InProceedings of the 2017 ACM International Conference on Management of Data, pages 1307–1322. ACM, 2017

2017
[13]

Privacy-preserving deep learning

Reza Shokri and Vitaly Shmatikov. Privacy-preserving deep learning. InProceedings of the 22nd ACM SIGSAC conference on computer and communications security, pages 1310–1321. ACM, 2015. 12

2015
[14]

Deep learning with diﬀerential privacy

Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with diﬀerential privacy. InProceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pages 308–318. ACM, 2016

2016
[15]

Learning diﬀerentially private language models without losing accuracy.arXiv preprint arXiv:1710.06963, 2017

H Brendan McMahan, Daniel Ramage, Kunal Talwar, and Li Zhang. Learning diﬀerentially private language models without losing accuracy.arXiv preprint arXiv:1710.06963, 2017

2017 arXiv
[16]

Diﬀerentially private empirical risk minimization: Eﬃcient algorithms and tight error bounds.arXiv preprint arXiv:1405.7085, 2014

Raef Bassily, Adam Smith, and Abhradeep Thakurta. Diﬀerentially private empirical risk minimization: Eﬃcient algorithms and tight error bounds.arXiv preprint arXiv:1405.7085, 2014

2014 arXiv
[17]

Our data, ourselves: Privacy via distributed noise generation

Cynthia Dwork, Krishnaram Kenthapadi, Frank McSherry, Ilya Mironov, and Moni Naor. Our data, ourselves: Privacy via distributed noise generation. InAnnual International Conference on the Theory and Applications of Cryptographic Techniques, pages 486–503. Springer, 2006

2006
[18]

The optimal noise-adding mechanism in diﬀerential privacy

Quan Geng and Pramod Viswanath. The optimal noise-adding mechanism in diﬀerential privacy. IEEE Transactions on Information Theory, 62(2):925–951, 2016

2016
[19]

Optimal noise-adding mechanism in additive diﬀerential privacy.arXiv preprint arXiv:1809.10224, 2018

Quan Geng, Wei Ding, Ruiqi Guo, and Sanjiv Kumar. Optimal noise-adding mechanism in additive diﬀerential privacy.arXiv preprint arXiv:1809.10224, 2018

2018 arXiv
[20]

Privacy, accuracy, and consistency too: a holistic solution to contingency table release

Boaz Barak, Kamalika Chaudhuri, Cynthia Dwork, Satyen Kale, Frank McSherry, and Kunal Talwar. Privacy, accuracy, and consistency too: a holistic solution to contingency table release. In Proceedings of the twenty-sixth ACM SIGMOD-SIGACT-SIGART symposium on Principles of databa...

2007
[21]

When random sampling preserves privacy

Kamalika Chaudhuri and Nina Mishra. When random sampling preserves privacy. InAnnual International Cryptology Conference, pages 198–213. Springer, 2006

2006
[22]

Mechanism design via diﬀerential privacy

Frank McSherry and Kunal Talwar. Mechanism design via diﬀerential privacy. InFoundations of Computer Science, 2007. FOCS’07. 48th Annual IEEE Symposium on, pages 94–103. IEEE, 2007

2007
[23]

Localprivacyandstatisticalminimax rates

JohnCDuchi, MichaelIJordan, andMartinJWainwright. Localprivacyandstatisticalminimax rates. In Foundations of Computer Science (FOCS), 2013 IEEE 54th Annual Symposium on, pages 429–438. IEEE, 2013

2013
[24]

A stability-based validation procedure for diﬀeren- tially private machine learning

Kamalika Chaudhuri and Staal A Vinterbo. A stability-based validation procedure for diﬀeren- tially private machine learning. InAdvances in Neural Information Processing Systems, pages 2652–2660, 2013

2013
[25]

Learning in a large func- tion space: Privacy-preserving mechanisms for svm learning.arXiv preprint arXiv:0911.5708, 2009

Benjamin IP Rubinstein, Peter L Bartlett, Ling Huang, and Nina Taft. Learning in a large func- tion space: Privacy-preserving mechanisms for svm learning.arXiv preprint arXiv:0911.5708, 2009

2009 arXiv
[26]

Functional mechanism: regression analysis under diﬀerential privacy.Proceedings of the VLDB Endowment, 5(11):1364–1375, 2012

Jun Zhang, Zhenjie Zhang, Xiaokui Xiao, Yin Yang, and Marianne Winslett. Functional mechanism: regression analysis under diﬀerential privacy.Proceedings of the VLDB Endowment, 5(11):1364–1375, 2012

2012
[27]

Some methods of speeding up the convergence of iteration methods.USSR Computational Mathematics and Mathematical Physics, 4(5):1–17, 1964

Boris T Polyak. Some methods of speeding up the convergence of iteration methods.USSR Computational Mathematics and Mathematical Physics, 4(5):1–17, 1964. 13

1964
[28]

Adaptive subgradient methods for online learning and stochastic optimization.Journal of Machine Learning Research, 12(Jul):2121–2159, 2011

John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization.Journal of Machine Learning Research, 12(Jul):2121–2159, 2011

2011
[29]

Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014

2014 arXiv
[30]

Practical secure aggregation for privacy-preserving machine learning

Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. Practical secure aggregation for privacy-preserving machine learning. InProceedings of the 2017 ACM SIGSAC Conference on Computer and Co...

2017
[31]

Yu, Sanjiv Kumar, and Brendan McMahan

Naman Agarwal, Ananda Theertha Suresh, Felix X. Yu, Sanjiv Kumar, and Brendan McMahan. cpSGD: Communication-eﬃcient and diﬀerentially-private distributed SGD. InProceedings of NeurIPS, pages 7575–7586, 2018

2018
[32]

Privacy integrated queries: an extensible platform for privacy-preserving data analysis

Frank D McSherry. Privacy integrated queries: an extensible platform for privacy-preserving data analysis. InProceedings of the 2009 ACM SIGMOD International Conference on Manage- ment of data, pages 19–30. ACM, 2009

2009
[33]

Boosting and diﬀerential privacy

Cynthia Dwork, Guy N Rothblum, and Salil Vadhan. Boosting and diﬀerential privacy. In2010 IEEE 51st Annual Symposium on Foundations of Computer Science, pages 51–60. IEEE, 2010

2010
[34]

Diﬀerential privacy and robust statistics

Cynthia Dwork and Jing Lei. Diﬀerential privacy and robust statistics. InProceedings of the forty-ﬁrst annual ACM symposium on Theory of computing, pages 371–380. ACM, 2009

2009
[35]

Concentrated diﬀerential privacy: Simpliﬁcations, extensions, and lower bounds

Mark Bun and Thomas Steinke. Concentrated diﬀerential privacy: Simpliﬁcations, extensions, and lower bounds. InTheory of Cryptography Conference, pages 635–658. Springer, 2016

2016
[36]

The composition theorem for diﬀerential privacy

Peter Kairouz, Sewoong Oh, and Pramod Viswanath. The composition theorem for diﬀerential privacy. IEEE Transactions on Information Theory, 63(6):4037–4049, 2017

2017
[37]

Diﬀerentially private model publishing for deep learning.arXiv preprint arXiv:1904.02200, 2019

Lei Yu, Ling Liu, Calton Pu, Mehmet Emre Gursoy, and Stacey Truex. Diﬀerentially private model publishing for deep learning.arXiv preprint arXiv:1904.02200, 2019

1904 arXiv
[38]

P3sgd: Patient privacy preserving sgd for regularizing deep cnns in pathological image classiﬁcation

Bingzhe Wu, Shiwan Zhao, Guangyu Sun, Xiaolu Zhang, Zhong Su, Caihong Zeng, and Zhihong Liu. P3sgd: Patient privacy preserving sgd for regularizing deep cnns in pathological image classiﬁcation. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p...

2019
[39]

Diﬀerentially private releasing via deep generative model (technical report).arXiv preprint arXiv:1801.01594, 2018

Xinyang Zhang, Shouling Ji, and Ting Wang. Diﬀerentially private releasing via deep generative model (technical report).arXiv preprint arXiv:1801.01594, 2018

2018 arXiv
[40]

Three tools for practical diﬀerential privacy.arXiv preprint arXiv:1812.02890, 2018

Koen Lennart van der Veen, Ruben Seggers, Peter Bloem, and Giorgio Patrini. Three tools for practical diﬀerential privacy.arXiv preprint arXiv:1812.02890, 2018

2018 arXiv
[41]

Diﬀerentially private learning with adaptive clipping

Om Thakkar, Galen Andrew, and H Brendan McMahan. Diﬀerentially private learning with adaptive clipping. arXiv preprint arXiv:1905.03871, 2019

1905 arXiv
[42]

Stochastic variance reduction for nonconvex optimization

Sashank J Reddi, Ahmed Hefny, Suvrit Sra, Barnabas Poczos, and Alex Smola. Stochastic variance reduction for nonconvex optimization. InInternational conference on machine learning, pages 314–323, 2016. 14

2016
[43]

Gradient-based learning applied to document recognition.Proceedings of the IEEE, 86(11):2278–2324, 1998

Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haﬀner. Gradient-based learning applied to document recognition.Proceedings of the IEEE, 86(11):2278–2324, 1998. 15 Appendix - AdaCliP: Adaptive Clipping for Private SGD A AdaCliP Convergence Analysis Theorem. Suppose the fun...

1998

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Delving deep into rectiﬁers: Surpassing human-level performance on imagenet classiﬁcation

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectiﬁers: Surpassing human-level performance on imagenet classiﬁcation. InProceedings of the IEEE international conference on computer vision, pages 1026–1034, 2015

work page 2015

[2] [2]

Imagenet classiﬁcation with deep convolutional neural networks

Alex Krizhevsky, Ilya Sutskever, and Geoﬀrey E Hinton. Imagenet classiﬁcation with deep convolutional neural networks. InAdvances in neural information processing systems, pages 1097–1105, 2012

work page 2012

[3] [3]

Recurrent neural network based language model

Tomáš Mikolov, Martin Karaﬁát, Lukáš Burget, Jan Černock` y, and Sanjeev Khudanpur. Recurrent neural network based language model. In Eleventh Annual Conference of the International Speech Communication Association, 2010

work page 2010

[4] [4]

Grammar as a foreign language

Oriol Vinyals, Łukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoﬀrey Hinton. Grammar as a foreign language. InAdvances in Neural Information Processing Systems, pages 2773–2781, 2015

work page 2015

[5] [5]

The netﬂix challenge.WIRED-SAN FRANCISCO-, 16(3):114, 2008

Jordan Ellenberg. The netﬂix challenge.WIRED-SAN FRANCISCO-, 16(3):114, 2008

work page 2008

[6] [6]

Federated learning for mobile keyboard prediction

Andrew Hard, Kanishka Rao, Rajiv Mathews, Françoise Beaufays, Sean Augenstein, Hubert Eichner, Chloé Kiddon, and Daniel Ramage. Federated learning for mobile keyboard prediction. arXiv preprint arXiv:1811.03604, 2018

arXiv 2018

[7] [7]

Lattice rescoring strategies for long short term memory language models in speech recognition

Shankar Kumar, Michael Nirschl, Daniel Holtmann-Rice, Hank Liao, Ananda Theertha Suresh, and Felix Yu. Lattice rescoring strategies for long short term memory language models in speech recognition. InAutomatic Speech Recognition and Understanding Workshop (ASRU), 2017 IEEE, pages 165–172. IEEE, 2017

work page 2017

[8] [8]

Calibrating noise to sensitivity in private data analysis

Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Theory of cryptography conference, pages 265–284. Springer, 2006

2006