REVIEW 21 cited by
Unlocking High-Accuracy Differentially Private Image Classification through Scale
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Differential Privacy (DP) provides a formal privacy guarantee preventing adversaries with access to a machine learning model from extracting information about individual training points. Differentially Private Stochastic Gradient Descent (DP-SGD), the most popular DP training method for deep learning, realizes this protection by injecting noise during training. However previous works have found that DP-SGD often leads to a significant degradation in performance on standard image classification benchmarks. Furthermore, some authors have postulated that DP-SGD inherently performs poorly on large models, since the norm of the noise required to preserve privacy is proportional to the model dimension. In contrast, we demonstrate that DP-SGD on over-parameterized models can perform significantly better than previously thought. Combining careful hyper-parameter tuning with simple techniques to ensure signal propagation and improve the convergence rate, we obtain a new SOTA without extra data on CIFAR-10 of 81.4% under (8, 10^{-5})-DP using a 40-layer Wide-ResNet, improving over the previous SOTA of 71.7%. When fine-tuning a pre-trained NFNet-F3, we achieve a remarkable 83.8% top-1 accuracy on ImageNet under (0.5, 8*10^{-7})-DP. Additionally, we also achieve 86.7% top-1 accuracy under (8, 8 \cdot 10^{-7})-DP, which is just 4.3% below the current non-private SOTA for this task. We believe our results are a significant step towards closing the accuracy gap between private and non-private image classification.
Forward citations
Cited by 21 Pith papers
-
Sequential Auditing for f-Differential Privacy
A new sequential auditor for f-differential privacy adaptively chooses its sample size, detects violations across the whole privacy tradeoff curve, and holds a user-set false-rejection rate.
-
Correlating Cross-Iteration Noise for DP-SGD using Model Curvature
Using Hessian eigenvalues from public data to design correlated noise for DP-SGD improves accuracy by 1–4% over current DP-MF methods.
-
Machine Learning with Privacy for Protected Attributes
Feature differential privacy is a relaxation of DP that guards selected features only, and the paper's two-batch algorithm recovers subsampling amplification and improves utility over standard DP when public features exist.
-
Adapting to Linear Separable Subsets with Large-Margin in Differentially Private Learning
A private halfspace learner achieves empirical and population zero-one error O(1/(γ²εn) + |S_out|/(γn)) for the best separable subset S_in = S\S_out with margin γ, without knowing γ or S_out in advance.
-
Scaling Laws for Differentially Private Language Models
Differentially private language models obey scaling laws in which compute-optimal models are roughly 10-50x smaller than non-private Chinchilla-optimal models, with large batch sizes and rapid saturation of compute.
-
Private Direct Preference Optimization for LLM Alignment
PrivDPO perturbs the DPO objective with an unbiased randomized rescaling to enforce epsilon-preference privacy, achieving near-DPO utility on three benchmarks and three LLM families up to 32B.
-
End-to-End Differential Privacy in Training Deep Neural Network Classifiers
Perturbing softmax outputs with the Dirichlet mechanism during training yields input-private, label-public classifiers that beat prior differentially private training accuracy on five image benchmarks.
-
Differentially Private Natural Gradient Descent
DP-NGD enables second-order optimization under differential privacy by decoupling curvature estimation onto public data, performing isotropic DP operations in a whitened space, and dynamically clamping curvature eigen...
-
An Interactive Framework for Finding the Optimal Trade-off in Differential Privacy
Modeling the privacy-accuracy Pareto front with sigmoid curves and letting users pick points on hypothetical fronts finds the optimal DP privacy budget with fewer queries and evaluations than pairwise baselines.
-
Lower Bounds for Public-Private Learning under Distribution Shift
For Gaussian mean estimation and linear regression with distribution shift, the paper claims that public data never provides complementary value: either public data alone suffices, or (for large shifts) private data a...
-
Optimizing Canaries for Privacy Auditing with Metagradient Descent
Optimized canary examples, crafted by metagradient descent on a small non-private model, more than double empirical epsilon lower bounds in black-box DP-SGD privacy audits on CIFAR-10.
-
Implementing Adaptations for Vision AutoRegressive Model
Fine-tuned Vision AutoRegressive models mostly beat a strong diffusion baseline on downstream image generation, but DP fine-tuning yields poor FID scores.
-
UniAud: A Unified Auditing Framework for High Auditing Power and Utility with One Training Run
UniAud uses synthetic uncorrelated canaries and self-comparison inference to reach near-optimal empirical epsilon lower bounds in one black-box DP audit run, while UniAud++ improves the utility-auditing trade-off via ...
-
Hyperparameters in Score-Based Membership Inference Attacks
A new shadow-model hyperparameter selection method (KL-LiRA) makes membership inference attacks nearly as effective without knowing target hyperparameters, and training-data-based hyperparameter tuning shows no detect...
-
Beyond the Crawl: Unmasking Browser Fingerprinting in Real User Interactions
An automated crawl of the same 14,895 pages missed 211 of the 471 websites flagged as fingerprinting in sessions by 30 instructed volunteers, about 45%.
-
Balls-and-Bins Sampling for DP-SGD
Balls-and-Bins sampling for DP-SGD has a tight privacy analysis: as private as Poisson at large epsilon, with shuffle-comparable utility, verified by Monte Carlo accounting.
-
Leveraging Programmatically Generated Synthetic Data for Differentially Private Diffusion Training
A stage-wise diffusion training method that substitutes synthetic images in coarse and cleaning steps to cut the privacy noise in DP image generation.
-
The pretraining domain outweighs the training objective in setting the privacy-utility trade-off of differentially private medical image analysis
In DP-SGD chest X-ray classification, MIMIC-CXR supervised pretraining beats ImageNet and DINOv3 initializations, but the study cannot cleanly separate pretraining domain from objective because key comparison arms are...
-
Mitigating Disparate Impact of Differentially Private Learning through Bounded Adaptive Clipping
Bounded adaptive clipping puts a floor under the adaptive clipping threshold in DP-SGD and improves worst-class accuracy on skewed and Fashion MNIST.
-
The Impact of Generalization Techniques on the Interplay Among Privacy, Utility, and Fairness in Image Classification
Combining DP-SAT with De et al.'s generalization techniques reaches 81.11% accuracy under (8, 10^-5)-DP on CIFAR-10, while amplifying bias on skewed datasets.
-
Enhancing One-run Privacy Auditing with Quantile Regression-Based Membership Inference
Applying quantile regression based membership inference scoring to one-run black-box DP-SGD auditing yields tighter empirical privacy lower bounds on CIFAR-10, though the gains are uneven and the efficiency claim is o...
Discussion (0). Continue with ORCID to comment.