Pith. sign in

REVIEW 4 major objections 5 minor 54 references

Lower Bounds for Public-Private Learning under Distribution Shift

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proves that when the distribution shift between public and private data is large, public data provides no benefit for differentially private Gaussian mean estimation or linear regression, and the full sample-complexity burden…

desk verdict New public-private lower bounds under distribution shift, but the headline large-shift dichotomy is unproven because the residual public-data term is not shown negligible; the step 'alpha >~ tau' is a non sequitur. read the letter →

arxiv 2507.17895 v1 pith:2HBMVW56 submitted 2025-07-23 cs.LG cs.CR

Amrith Setlur , Pratiksha Thaker , Jonathan Ullman This is my paper
classification cs.LGcs.CR MSC 68P2762F1562J05
keywords differentialprivacypublicdatadistributionshiftfingerprintingmethodsamplecomplexitylowerboundsGaussianmeanestimationlinearregressionBayesian
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a separate source of public data can make differentially private learning easier when the public data come from a shifted distribution, the situation most deployments face. For two canonical problems, Gaussian mean estimation with a shifted mean and Gaussian linear regression with a shifted regression vector, it argues that the answer is no in a strong sense: either the public data alone are enough to reach the target accuracy $\alpha$, or the private data alone must reach the known private-only sample complexity. When the shift $\tau$ is large relative to $\alpha$ and grows faster than the public-data noise scale $\sqrt{d/m}$, the public samples contribute nothing, and the full burden $n=\Omega(d/(\alpha\varepsilon)+d/\alpha^2)$ falls on the private data. The lower bounds use a Bayesian fingerprinting argument with conjugate Gaussian priors, which lets the analysis absorb distribution shift and correlated label noise.

What carries the argument

The load-bearing object is the reweighted fingerprinting statistic: for each private point $Z_i=\langle M(X)-\mu_{\mathrm{priv}}, X_i-\mu_{\mathrm{priv}}\rangle$, and for each public point the same correlation is divided by $m\tau^2/d+1$, so its weight vanishes as the shift grows. Privacy bounds the expected sum from above by roughly $n\varepsilon\alpha$ plus a public-data term; an accuracy-based fingerprinting lemma bounds the same sum from below by $\Omega(d)$. The distribution shift is handled by putting a Gaussian prior $v\sim\mathcal{N}(0,\tau^2 I_d/d)$ on the shift vector so that the private parameter conditioned on all data remains jointly Gaussian, and for linear regression the shifted labels are re-expressed as non-identically distributed Gaussian noise with covariance $\Sigma=\sigma^2 I_N+(\tau^2/d)PP^\top$, which brings generalized least-squares machinery into the proof.

What would settle it

Recompute the final comparison in Appendix A.2: the displayed inequality supports roughly $\alpha\gtrsim\tau\sqrt{m/d}$ (or $\alpha\gtrsim\tau\sqrt{m}$ after resolving typos), not simply $\alpha\gtrsim\tau$. For an asymptotic family with $\tau=2\sqrt{d/m}$ and $\alpha$ between the two thresholds, check whether the public-data term is actually negligible; if it is not, the dichotomy in Theorems 4.2 and 5.2 fails.

Watch

Extended reading notes

Core claim

The central claim is that complementarity between public and private data does not appear even under distribution shift. For Gaussian mean estimation with $\mathbb{P}=\mathcal{N}(\mu_P,I_d)$ and $\mathbb{Q}=\mathcal{N}(\mu_Q,I_d)$, $\|\mu_P-\mu_Q\|_2\le\tau$, and for Gaussian linear regression with $\|\beta_P-\beta_Q\|_2\le\tau$, any $(\varepsilon,\delta)$-differentially private learner accurate to $\alpha$ must satisfy either $m=\Omega(d/\alpha^2)$ or $n=\Omega(d/(\alpha\varepsilon)+d/\alpha^2)$; the second condition is forced when $\tau=\omega(\sqrt{d/m})$ and $\tau\ge\alpha$, so in the large-shift regime no amount of public data reduces the private sample requirement. This is stated as Theorems 4.2 and 5.2.

Load-bearing premise

The load-bearing step is that once the shift is large relative to the target accuracy and the public-sample noise, the public samples' contribution to the correlation measure used in the proof disappears; the appendix's final comparison appears to support a different threshold, so the large-shift dichotomy rests on that comparison being corrected.

Editorial extensions

If this is right

  • When the shift is small relative to the target accuracy, the lower bounds show that public and private data are not complementary: either $m=\Omega(d/\alpha^2)$ public samples suffice, or $n+m=\Omega(d/(\alpha\varepsilon)+d/\alpha^2)$ total samples are needed.
  • When the shift is large, $\tau=\omega(\sqrt{d/m})$ and $\tau\ge\alpha$, the required private sample size is $n=\Omega(d/(\alpha\varepsilon)+d/\alpha^2)$, independent of $m$; public data cannot reduce it.
  • The results are minimax lower bounds and therefore apply to every $(\varepsilon,\delta)$-differentially private mechanism, not just to natural estimators such as the empirical mean or ordinary least squares.
  • The Bayesian fingerprinting proof also recovers the known no-shift public-private mean estimation lower bound and supplies the first public-private lower bound for Gaussian linear regression without or with parameter shift.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A practical reading of the large-shift regime is that deployment teams should not credit public data toward the private sample budget once the measured shift exceeds both the target error and roughly $\sqrt{d/m}$.
  • A corollary of the threshold is that increasing the public sample size shrinks the regime in which a fixed shift is 'small', so in this model more public data can move the problem toward the no-benefit regime rather than away from it.
  • The conjugate-prior trick is specific to Gaussian problems; extending the dichotomy to logistic regression or heavy-tailed distributions would require a new way to keep the posterior tractable, since joint normality fails.
  • The result does not rule out public-data value through feature learning or dimensionality reduction, the kinds of gains shown in positive results, because those operate before the final estimation step; the lower bounds target the estimation problem once the representation is fixed.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies differentially private estimation with public and private samples drawn from different Gaussian distributions. Its main claims are lower bounds for public-private Gaussian mean estimation and Gaussian linear regression under distribution shift: when the shift is small, either the public sample size or the combined sample size must already suffice on its own; when the shift is large, the entire burden falls on private samples and public data is of no benefit. The proofs use the Bayesian fingerprinting framework, placing Gaussian priors on the private parameter and the shift vector and then comparing upper and lower bounds on a fingerprinting statistic.

Significance. If the main claims were established, the paper would substantially extend the public-private lower bounds of Bassily et al. and Ullah et al. to distribution shift and would give the first public-private lower bounds for Gaussian linear regression. The Bayesian-fingerprinting reformulation is attractive, and the reduction from minimax risk to Bayes risk under Gaussian priors is legitimate and unconditional. However, the central large-shift claim is not established by the current proofs: the final step of the large-shift argument is a non sequitur, and there are additional false or unproved identities in the no-shift linear regression proof and in the posterior-concentration lemma for mean estimation. The paper is interesting in conception but not sound in its present form.

major comments (4)
  1. [Appendix A.2, Theorem A.10 and final paragraph] The derivation of the large-shift case is invalid. From Corollary A.12 and Lemma A.15, the public contribution is bounded by [α√(md) + ατ√(md)] / (1 + mτ²/d). When τ = ω(√(d/m)), this residue is approximately α d^{3/2} / (√m · τ), which is o(d) only if τ = ω(α√(d/m)), not whenever τ = ω(√(d/m)) and α = O(√d). The displayed sentence "The above holds when α >~ τ" is not a consequence of the displayed inequality; solving the inequality gives a different threshold. Consequently the large-shift dichotomy in Theorems 4.2 and A.10, which is the paper's headline claim, is unproven as stated.
  2. [Appendix A.1, Lemma A.7 and Corollary A.9] Lemma A.7 is false as stated. With µ_priv ∼ N(0, σ²I_d) and ¯µ | µ ∼ N(µ, I_d/N), one has E∥¯µ∥² = σ²d + d/N, not O(d/N). Since the posterior mean is ¯µ · Nσ²/(Nσ² + 1), the squared distance E∥E[µ|X] − ¯µ∥² is Θ(d/N²) for fixed σ², not O(d/N³). This invalidates the bound used in Eq. (9) and hence the proof of Corollary A.9 and Theorem A.1. The no-shift lower bound may be recoverable with corrected constants, but the proof as written is unsound.
  3. [Appendix C, Eq. (42)] The identity used to pass from Eq. (41) to Eq. (42) is false for λ > 0. Since ˆβ = (XᵀX + λI)⁻¹Xᵀy, we have Xᵀy = XᵀXˆβ + λˆβ, so XᵀX(β − ˆβ) = XᵀXβ − Xᵀy + λˆβ. The missing λˆβ term is not analyzed. This breaks the proof of Theorem C.5 (the lower bound E[Σ_i Z_i] = Ω(d)) and therefore the proof of Theorem 5.1 for no-shift linear regression. If λ = 0 is intended, this must be stated and invertibility of XᵀX justified.
  4. [Appendix D, final large-shift analysis after Eq. (205)] The large-shift analysis for linear regression repeats the same error. For κ = mτ²/d + 1, one has md − m²τ²/κ = md/κ. Under τ = ω(√(d/m)) this is o(md), not o(1). The public contribution is then α√(md/κ) ≈ αd/τ (for α = O(1)), which is o(d) only when τ = ω(α), a condition not implied by τ = ω(√(d/m)) when m is large. Thus Theorem 5.2's large-shift case, and the analogous informal claim in Theorem 1.3, are not established by the supplied derivation.
minor comments (5)
  1. [Theorem A.10 and Appendix A.2] The first bullet states τ = O(d/√m), but the proof and all other statements use τ = O(√(d/m)); this appears to be a typo. In the proof of the first case, "m = Ω(d²/α)" also looks like a typo for m = Ω(d/α²).
  2. [Theorems 1.1, 1.3, 4.2, 5.2] The informal theorems split the regime on τ ≲ α versus τ ≳ α, while the formal theorems split on τ = O(√(d/m)) versus τ = ω(√(d/m)). These conditions are not equivalent, and the relationship between α and τ in the formal statements should be stated precisely.
  3. [Proposition A.11] The bound on the public sum is written as O(α√(md) + ατ√(md)); the natural concentration bound is α(√(md) + √m · τ). The displayed bound is a valid overestimate for d ≥ 1, but it should be corrected or justified, because it is later used in the large-shift simplification.
  4. [Theorems 1.2 and 1.3] In the informal statements, the phrase "enough total data to solve the mean estimation problem with differential privacy" should read "the linear regression problem" for the linear regression theorems.
  5. [Appendix A, Lemma A.14] The weighted average ¯X is introduced in the lemma and redefined in Eq. (22); the notation should be set once, before the statement of the lemma, to avoid confusion with the ordinary empirical mean.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular reasoning: the lower bounds are proved by a Bayes-risk/fingerprinting reduction and do not assume their conclusions.

full rationale

The paper's central lower bounds are derived within a minimax/Bayes framework: a prior is placed on the private parameter and the shift vector, a fingerprinting statistic is defined, the expected statistic is upper bounded using differential privacy and lower bounded using posterior calculations, and the resulting inequalities are combined. None of the theorem conclusions are assumed in the setup or in the cited fingerprinting lemmas. The cited works (Narayanan 2023, Cai et al. 2021, Bassily et al. 2020) supply proof techniques or previously established lower bounds, and the present paper re-derives the needed lemmas rather than invoking the target result. The Bayes-risk reduction is unconditional because minimax risk dominates Bayes risk under any prior. Self-citations appear only as references to the standard fingerprinting method, which is an external proof framework and not a loaded assumption. The visible weakness in the final algebraic step of Appendix A.2, where 'The above holds when α ≳ τ' does not follow from the displayed inequality, is a correctness gap in the threshold derivation rather than a circularity: the needed inequality is not equivalent to an input by construction, and no fitted parameter is relabeled as a prediction. Therefore no enumerated circular step is present, and the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper's lower bounds are minimax; the Gaussian priors (sigma^2 on the private mean, tau^2/d on the shift, b on the regression vector) are proof devices whose values do not enter the final bounds. There are no numbers fitted to data and no new postulated entities. The main assumptions are the Gaussian model and the borrowed Bayesian fingerprinting framework.

assumptions (3)
  • domain assumption Public and private data are Gaussian with identity covariance; in the shifted setting the shift is bounded by tau in l2 norm.
    Stated in Sections 4 and 5; the results are for these canonical distributions, not general distributions.
  • domain assumption The Bayesian fingerprinting framework of Narayanan 2023 provides valid lower bounds via priors and posterior calculations.
    The paper relies on this framework (Section 2) to convert accuracy and privacy into bounds on the fingerprinting statistic.
  • standard math Standard concentration inequalities for Gaussian random matrices (Vershynin, Wainwright).
    Used in Lemmas B.1-B.7 and throughout the appendices.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Lower Bounds for Public-Private Learning under Distribution Shift." pith.science (2026). https://pith.science/paper/2HBMVW56

@misc{pith2026250717895,
  author       = {Pith},
  title        = {Pith review of: Lower Bounds for Public-Private Learning under Distribution Shift},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2HBMVW56}},
  note         = {Machine review of arXiv:2507.17895}
}
read the original abstract

The most effective differentially private machine learning algorithms in practice rely on an additional source of purportedly public data. This paradigm is most interesting when the two sources combine to be more than the sum of their parts. However, there are settings such as mean estimation where we have strong lower bounds, showing that when the two data sources have the same distribution, there is no complementary value to combining the two data sources. In this work we extend the known lower bounds for public-private learning to setting where the two data sources exhibit significant distribution shift. Our results apply to both Gaussian mean estimation where the two distributions have different means, and to Gaussian linear regression where the two distributions exhibit parameter shift. We find that when the shift is small (relative to the desired accuracy), either public or private data must be sufficiently abundant to estimate the private parameter. Conversely, when the shift is large, public data provides no benefit.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

54 extracted references · 38 canonical work pages

  1. [1]

    Deep learning with differential privacy

    Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pages 308--318, 2016

    2016

  2. [2]

    IV.---On least squares and linear combination of observations

    Alexander C Aitken. IV.---On least squares and linear combination of observations . Proceedings of the Royal Society of Edinburgh, 55: 0 42--48, 1936

    work page 1936

  3. [3]

    Privacy in Metalearning and Multitask Learning: Modeling and Separations

    Maryam Aliakbarpour, Konstantina Bairaktari, Adam Smith, Marika Swanberg, and Jonathan Ullman. Privacy in metalearning and multitask learning: Modeling and separations. arXiv preprint arXiv:2412.12374, 2024

    work page Pith review arXiv 2024

  4. [4]

    Public data-assisted mirror descent for private model training

    Ehsan Amid, Arun Ganesh, Rajiv Mathews, Swaroop Ramaswamy, Shuang Song, Thomas Steinke, Vinith M Suriyakumar, Om Thakkar, and Abhradeep Thakurta. Public data-assisted mirror descent for private model training. In International Conference on Machine Learning, pages 517--535. PMLR, 2022

    work page 2022

  5. [5]

    Can foundation models help us achieve perfect secrecy? arXiv preprint arXiv:2205.13722, 2022

    Simran Arora and Christopher R \'e . Can foundation models help us achieve perfect secrecy? arXiv preprint arXiv:2205.13722, 2022

    arXiv 2022

  6. [6]

    Information complexity of stochastic convex optimization: Applications to generalization and memorization

    Idan Attias, Gintare Karolina Dziugaite, Mahdi Haghifam, Roi Livni, and Daniel M Roy. Information complexity of stochastic convex optimization: Applications to generalization and memorization. arXiv preprint arXiv:2402.09327, 2024

    arXiv 2024

  7. [7]

    The power of the hybrid model for mean estimation

    Brendan Avent, Yatharth Dubey, and Aleksandra Korolova. The power of the hybrid model for mean estimation. Proceedings on Privacy Enhancing Technologies, 4: 0 48--68, 2020

    work page 2020

  8. [8]

    Private empirical risk minimization: Efficient algorithms and tight error bounds

    Raef Bassily, Adam Smith, and Abhradeep Thakurta. Private empirical risk minimization: Efficient algorithms and tight error bounds. In 2014 IEEE 55th annual symposium on foundations of computer science, pages 464--473. IEEE, 2014

    work page 2014

Show all 54 references
  1. [9]

    Private query release assisted by public data

    Raef Bassily, Albert Cheu, Shay Moran, Aleksandar Nikolov, Jonathan Ullman, and Steven Wu. Private query release assisted by public data. In International Conference on Machine Learning, pages 695--703. PMLR, 2020

    2020

  2. [10]

    Private estimation with public data

    Alex Bie, Gautam Kamath, and Vikrant Singhal. Private estimation with public data. Advances in Neural Information Processing Systems, 35: 0 18653--18666, 2022

    2022

  3. [11]

    Differentially private optimization on large model at small cost

    Zhiqi Bu, Yu-Xiang Wang, Sheng Zha, and George Karypis. Differentially private optimization on large model at small cost. In International Conference on Machine Learning, pages 3192--3218. PMLR, 2023

    2023

  4. [12]

    Fingerprinting codes and the price of approximate differential privacy

    Mark Bun, Jonathan Ullman, and Salil Vadhan. Fingerprinting codes and the price of approximate differential privacy. In Proceedings of the forty-sixth annual ACM symposium on Theory of computing, pages 1--10, 2014

    2014

  5. [13]

    The cost of privacy: Optimal rates of convergence for parameter estimation with differential privacy

    T Tony Cai, Yichen Wang, and Linjun Zhang. The cost of privacy: Optimal rates of convergence for parameter estimation with differential privacy. The Annals of Statistics, 49 0 (5): 0 2825--2850, 2021

    2021

  6. [14]

    Score attack: A lower bound technique for optimal differentially private learning

    T Tony Cai, Yichen Wang, and Linjun Zhang. Score attack: A lower bound technique for optimal differentially private learning. arXiv preprint arXiv:2303.07152, 2023

    2023 arXiv

  7. [15]

    Unlocking high-accuracy differentially private image classification through scale

    Soham De, Leonard Berrada, Jamie Hayes, Samuel L Smith, and Borja Balle. Unlocking high-accuracy differentially private image classification through scale. arXiv preprint arXiv:2204.13650, 2022

    2022 arXiv

  8. [16]

    Calibrating noise to sensitivity in private data analysis

    Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Theory of cryptography conference, pages 265--284. Springer, 2006

    2006

  9. [17]

    Analyze gauss: optimal bounds for privacy-preserving principal component analysis

    Cynthia Dwork, Kunal Talwar, Abhradeep Thakurta, and Li Zhang. Analyze gauss: optimal bounds for privacy-preserving principal component analysis. In Proceedings of the forty-sixth annual ACM symposium on Theory of computing, pages 11--20, 2014

    2014

  10. [18]

    Robust traceability from trace amounts

    Cynthia Dwork, Adam Smith, Thomas Steinke, Jonathan Ullman, and Salil Vadhan. Robust traceability from trace amounts. In 2015 IEEE 56th Annual Symposium on Foundations of Computer Science, pages 650--669. IEEE, 2015

    2015

  11. [19]

    Joint selection: Adaptively incorporating public information for private synthetic data

    Miguel Fuentes, Brett C Mullins, Ryan McKenna, Gerome Miklau, and Daniel Sheldon. Joint selection: Adaptively incorporating public information for private synthetic data. In International Conference on Artificial Intelligence and Statistics, pages 2404--2412. PMLR, 2024

    2024

  12. [20]

    Why is public pretraining necessary for private model training? In International Conference on Machine Learning, pages 10611--10627

    Arun Ganesh, Mahdi Haghifam, Milad Nasr, Sewoong Oh, Thomas Steinke, Om Thakkar, Abhradeep Guha Thakurta, and Lun Wang. Why is public pretraining necessary for private model training? In International Conference on Machine Learning, pages 10611--10627. PMLR, 2023

    2023

  13. [21]

    Submix: Practical private prediction for large-scale language models

    Antonio Ginart, Laurens van der Maaten, James Zou, and Chuan Guo. Submix: Practical private prediction for large-scale language models. arXiv preprint arXiv:2201.00971, 2022

    2022 arXiv

  14. [22]

    Mixed differential privacy in computer vision

    Aditya Golatkar, Alessandro Achille, Yu-Xiang Wang, Aaron Roth, Michael Kearns, and Stefano Soatto. Mixed differential privacy in computer vision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8376--8386, 2022

    2022

  15. [23]

    Preventing false discovery in interactive data analysis is hard

    Moritz Hardt and Jonathan Ullman. Preventing false discovery in interactive data analysis is hard. In 2014 IEEE 55th annual symposium on foundations of computer science, pages 454--463. IEEE, 2014

    2014

  16. [24]

    Exploring the limits of differentially private deep learning with group-wise clipping

    Jiyan He, Xuechen Li, Da Yu, Huishuai Zhang, Janardhan Kulkarni, Yin Tat Lee, Arturs Backurs, Nenghai Yu, and Jiang Bian. Exploring the limits of differentially private deep learning with group-wise clipping. In The Eleventh International Conference on Learning Representations, 2022

    2022

  17. [25]

    Auditing differentially private machine learning: How private is private sgd? Advances in Neural Information Processing Systems, 33: 0 22205--22216, 2020

    Matthew Jagielski, Jonathan Ullman, and Alina Oprea. Auditing differentially private machine learning: How private is private sgd? Advances in Neural Information Processing Systems, 33: 0 22205--22216, 2020

    2020

  18. [26]

    (nearly) dimension independent private erm with adagrad via publicly estimated subspaces

    Peter Kairouz, Monica Ribero Diaz, Keith Rush, and Abhradeep Thakurta. (nearly) dimension independent private erm with adagrad via publicly estimated subspaces. In Conference on Learning Theory, pages 2717--2746. PMLR, 2021

    2021

  19. [27]

    Privately learning high-dimensional distributions

    Gautam Kamath, Jerry Li, Vikrant Singhal, and Jonathan Ullman. Privately learning high-dimensional distributions. In Conference on Learning Theory, pages 1853--1902. PMLR, 2019

    1902

  20. [28]

    New lower bounds for private estimation and a generalized fingerprinting lemma

    Gautam Kamath, Argyris Mouzakis, and Vikrant Singhal. New lower bounds for private estimation and a generalized fingerprinting lemma. Advances in neural information processing systems, 35: 0 24405--24418, 2022

    2022

  21. [29]

    On the convergence of differentially-private fine-tuning: To linearly probe or to fully fine-tune? arXiv preprint arXiv:2402.18905, 2024

    Shuqi Ke, Charlie Hou, Giulia Fanti, and Sewoong Oh. On the convergence of differentially-private fine-tuning: To linearly probe or to fully fine-tune? arXiv preprint arXiv:2402.18905, 2024

    2024

  22. [30]

    Toward training at imagenet scale with differential privacy

    Alexey Kurakin, Shuang Song, Steve Chien, Roxana Geambasu, Andreas Terzis, and Abhradeep Thakurta. Toward training at imagenet scale with differential privacy. arXiv preprint arXiv:2201.12328, 2022

    2022 arXiv

  23. [31]

    Large language models can be strong differentially private learners

    Xuechen Li, Florian Tramer, Percy Liang, and Tatsunori Hashimoto. Large language models can be strong differentially private learners. In International Conference on Learning Representations, 2021

    2021

  24. [32]

    Leveraging public data for practical private query release

    Terrance Liu, Giuseppe Vietri, Thomas Steinke, Jonathan Ullman, and Steven Wu. Leveraging public data for practical private query release. In International Conference on Machine Learning, pages 6968--6977. PMLR, 2021

    2021

  25. [33]

    Optimal differentially private model training with public data

    Andrew Lowy, Zeman Li, Tianjian Huang, and Meisam Razaviyayn. Optimal differentially private model training with public data. In Forty-first International Conference on Machine Learning, 2024

    2024

  26. [34]

    Scalable differential privacy with sparse network finetuning

    Zelun Luo, Daniel J Wu, Ehsan Adeli, and Li Fei-Fei. Scalable differential privacy with sparse network finetuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5059--5068, 2021

    2021

  27. [35]

    Fingerprinting codes meet geometry: Improved lower bounds for private query release and adaptive data analysis

    Xin Lyu and Kunal Talwar. Fingerprinting codes meet geometry: Improved lower bounds for private query release and adaptive data analysis. arXiv preprint arXiv:2412.14396, 2024

    2024 arXiv

  28. [36]

    Large scale transfer learning for differentially private image classification

    Harsh Mehta, Abhradeep Thakurta, Alexey Kurakin, and Ashok Cutkosky. Large scale transfer learning for differentially private image classification. arXiv preprint arXiv:2205.02973, 2022

    2022 arXiv

  29. [37]

    Kevin P. Murphy. Probabilistic Machine Learning: An introduction. MIT Press, 2022. URL http://probml.github.io/book1

    2022

  30. [38]

    Better and simpler lower bounds for differentially private statistical estimation

    Shyam Narayanan. Better and simpler lower bounds for differentially private statistical estimation. arXiv preprint arXiv:2310.06289, 2023

    2023 arXiv

  31. [39]

    Tight and robust private mean estimation with few users

    Shyam Narayanan, Vahab Mirrokni, and Hossein Esfandiari. Tight and robust private mean estimation with few users. In International Conference on Machine Learning, pages 16383--16412. PMLR, 2022

    2022

  32. [40]

    Making the shoe fit: Architectures, initializations, and tuning for learning with privacy

    Nicolas Papernot, Steve Chien, Shuang Song, Abhradeep Thakurta, and Ulfar Erlingsson. Making the shoe fit: Architectures, initializations, and tuning for learning with privacy. 2019

    2019

  33. [41]

    Smooth lower bounds for differentially private algorithms via padding-and-permuting fingerprinting codes

    Naty Peter, Eliad Tsfadia, and Jonathan Ullman. Smooth lower bounds for differentially private algorithms via padding-and-permuting fingerprinting codes. In The Thirty Seventh Annual Conference on Learning Theory, pages 4207--4239. PMLR, 2024

    2024

  34. [42]

    Lower bounds for private estimation of gaussian covariance matrices under all reasonable parameter regimes

    Victor S Portella and Nick Harvey. Lower bounds for private estimation of gaussian covariance matrices under all reasonable parameter regimes. arXiv preprint arXiv:2404.17714, 2024

    2024 arXiv

  35. [43]

    Between pure and approximate differential privacy

    Thomas Steinke and Jonathan Ullman. Between pure and approximate differential privacy. arXiv preprint arXiv:1501.06095, 2015 a

    2015 arXiv

  36. [44]

    Interactive fingerprinting codes and the hardness of preventing false discovery

    Thomas Steinke and Jonathan Ullman. Interactive fingerprinting codes and the hardness of preventing false discovery. In Conference on learning theory, pages 1588--1628. PMLR, 2015 b

    2015

  37. [45]

    Tight lower bounds for differentially private selection

    Thomas Steinke and Jonathan Ullman. Tight lower bounds for differentially private selection. In 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS), pages 552--563. IEEE, 2017

    2017

  38. [46]

    Differentially private learning needs better features (or much more data)

    Florian Tramer and Dan Boneh. Differentially private learning needs better features (or much more data). In International Conference on Learning Representations, 2020

    2020

  39. [47]

    Considerations for differentially private learning with large-scale public pretraining

    Florian Tram \`e r, Gautam Kamath, and Nicholas Carlini. Considerations for differentially private learning with large-scale public pretraining. arXiv preprint arXiv:2212.06470, 2022

    2022 arXiv

  40. [48]

    Public-data assisted private stochastic optimization: Power and limitations

    Enayat Ullah, Michael Menart, Raef Bassily, Crist \'o bal Guzm \'a n, and Raman Arora. Public-data assisted private stochastic optimization: Power and limitations. arXiv preprint arXiv:2403.03856, 2024

    2024 arXiv

  41. [49]

    Answering n^ 2+o(1) counting queries with differential privacy is hard

    Jonathan Ullman. Answering n^ 2+o(1) counting queries with differential privacy is hard. In Proceedings of the forty-fifth annual ACM symposium on Theory of computing, pages 361--370, 2013

    2013

  42. [50]

    The limits of post-selection generalization

    Jonathan Ullman, Adam Smith, Kobbi Nissim, Uri Stemmer, and Thomas Steinke. The limits of post-selection generalization. Advances in Neural Information Processing Systems, 31, 2018

    2018

  43. [51]

    High-dimensional probability: An introduction with applications in data science, volume 47

    Roman Vershynin. High-dimensional probability: An introduction with applications in data science, volume 47. Cambridge university press, 2018

    2018

  44. [52]

    High-dimensional statistics: A non-asymptotic viewpoint, volume 48

    Martin J Wainwright. High-dimensional statistics: A non-asymptotic viewpoint, volume 48. Cambridge university press, 2019

    2019

  45. [53]

    Differentially private fine-tuning of language models

    Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, et al. Differentially private fine-tuning of language models. In International Conference on Learning Representations, 2021 a

    2021

  46. [54]

    Large scale private learning via low-rank reparametrization

    Da Yu, Huishuai Zhang, Wei Chen, Jian Yin, and Tie-Yan Liu. Large scale private learning via low-rank reparametrization. In International Conference on Machine Learning, pages 12208--12218. PMLR, 2021 b

    2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.