The dithered Gaussian mechanism discretizes the output of the Gaussian mechanism via a randomly shifted grid, inheriting Gaussian privacy guarantees while reducing private randomness to a constant per coordinate.
hub
Opacus: User-friendly differential privacy library in pytorch
24 Pith papers cite this work. Polarity classification is still indexing.
24
Pith papers citing it
abstract
We introduce Opacus, a free, open-source PyTorch library for training deep learning models with differential privacy (hosted at opacus.ai). Opacus is designed for simplicity, flexibility, and speed. It provides a simple and user-friendly API, and enables machine learning practitioners to make a training pipeline private by adding as little as two lines to their code. It supports a wide variety of layers, including multi-head attention, convolution, LSTM, GRU (and generic RNN), and embedding, right out of the box and provides the means for supporting other user-defined layers. Opacus computes batched per-sample gradients, providing higher efficiency compared to the traditional "micro batch" approach. In this paper we present Opacus, detail the principles that drove its implementation and unique features, and benchmark it against other frameworks for training models with differential privacy as well as standard PyTorch.
hub tools
citation-role summary
background 2
method 1
citation-polarity summary
representative citing papers
Fair fine-tuning under Equalized Odds yields a tight bound Adv(A, M_f) ≤ Δ_EO · W on adversarial advantage in distribution inference attacks, with empirical reductions below detection threshold across six datasets.
A new framework is introduced for end-to-end provable robustness against backdoor attacks by composing randomized smoothing with differentially private training via privacy profiles.
FML-Bench shows a simple greedy hill-climber nearly matches tree search on dense-opportunity tasks while an adaptive agent that broadens search on stagnation outperforms six baselines across 18 tasks.
PACZero achieves zero mutual information privacy in LLM fine-tuning via sign-quantized subset-aggregated ZO gradients, delivering near non-private accuracy on SST-2 at I=0.
AS-LoRA adaptively chooses which LoRA factor to update per layer and round using a curvature-aware second-order score, eliminating reconstruction error floors and improving performance in DP federated learning.
FiBeR adds a closed-form filter-aware correction A(ω)σ_w² to the second-moment term for temporally filtered DP gradients, improving adaptive optimization performance.
DP-GCL improves differentially private contrastive learning by bounding group-level contributions through batch partitioning and intra-group augmentation, delivering 5.6% higher image classification accuracy and 20.1% higher retrieval accuracy than existing approaches.
DPrivBench is a new benchmark for evaluating LLMs on differential privacy reasoning, with results showing good performance on textbook mechanisms but substantial failures on advanced algorithms.
DPQuant uses epoch-wise probabilistic layer rotation and DP loss sensitivity to quantize only a changing subset of layers, reducing accuracy degradation from quantization noise in DP-SGD and delivering up to 2.21x throughput gains with under 2% accuracy drop.
Identifies output label space as a privacy side-channel in DP continual learning, formalizes DP for CL, and demonstrates two mitigation methods yielding higher accuracy than prior work.
A federated system combining autoencoders, FedAvg, Renyi DP-SGD, and INT8 quantization matches centralized AUROC performance (0.782 for ConvAE) on PTB-XL while halving model size and cutting edge latency by up to 44% with independent privacy and quantization penalties.
CE-FedGNN enables federated GNN training on coupled distributed graphs via infrequent aggregated representation exchange, moving-average estimation for staleness, and metric-DP, with O(1/sqrt(T)) convergence and O(T^{3/4}) communication.
DP-SGD-RC applies Hutchinson and Hutch++ estimators to approximate per-sample gradient norms for clipping in DP-SGD, claiming competitive privacy noise multipliers and utility on Llama 3.2-1B with reduced memory.
A differentially private fine-tuning method that constructs a quadratic utility function to allow exact sampling from a multivariate normal distribution while providing theoretical privacy guarantees.
A stabilized DP training and offline distillation protocol prevents collapse to single-class predictors in private speech classification under strong privacy while releasing only an audio-only model.
A differentially private pipeline using node-level DP summaries to fit ERGMs or SBMs, generate synthetic networks, and simulate SIS disease spread on ARTNet sexual contact data produces incidence, prevalence, and intervention effect sizes close to non-private versions.
FedVideoMAE combines VideoMAE pretraining, LoRA adaptation, client DP-SGD and secure aggregation to cut federated communication 28x while reaching 65-66% accuracy under strong privacy on RWF-2000 with 40 clients.
Add/remove adjacency in DP overstates attribute privacy relative to substitute adjacency; new auditing attacks confirm inconsistency with add/remove reports but consistency with substitute accounting.
CA-ADP adjusts differential privacy noise per mini-batch class composition to improve F-scores by 3.3-8.5% over standard DP on three fall-detection datasets while claiming formal (ε,δ) guarantees.
Post-processing via random selection or linear combination of differentially private models allows meeting arbitrary target privacy parameters without additional training.
Three optimized MPC protocols for privacy-preserving vertical federated learning that support global and global-local updates while reducing computation versus naive full-MPC delegation.
Empirical study of DP transfer learning reveals that larger clipping bounds outperform under tight privacy and cumulative DP noise explains batch-size effects better than existing heuristics.
citing papers explorer
-
Dithered Gaussian Mechanism for Randomness-Efficient Differential Privacy
The dithered Gaussian mechanism discretizes the output of the Gaussian mechanism via a randomly shifted grid, inheriting Gaussian privacy guarantees while reducing private randomness to a constant per coordinate.
-
Fair Finetuning Mitigates Distribution Inference Attacks
Fair fine-tuning under Equalized Odds yields a tight bound Adv(A, M_f) ≤ Δ_EO · W on adversarial advantage in distribution inference attacks, with empirical reductions below detection threshold across six datasets.
-
Provable Robustness against Backdoor Attacks via the Primal-Dual Perspective on Differential Privacy
A new framework is introduced for end-to-end provable robustness against backdoor attacks by composing randomized smoothing with differentially private training via privacy profiles.
-
FML-bench: A Controlled Study of AI Research Agent Strategies from the Perspective of Search Dynamics
FML-Bench shows a simple greedy hill-climber nearly matches tree search on dense-opportunity tasks while an adaptive agent that broadens search on stagnation outperforms six baselines across 18 tasks.
-
PACZero: PAC-Private Fine-Tuning of Language Models via Sign Quantization
PACZero achieves zero mutual information privacy in LLM fine-tuning via sign-quantized subset-aggregated ZO gradients, delivering near non-private accuracy on SST-2 at I=0.
-
Adaptive Selection of LoRA Components in Privacy-Preserving Federated Learning
AS-LoRA adaptively chooses which LoRA factor to update per layer and round using a curvature-aware second-order score, eliminating reconstruction error floors and improving performance in DP federated learning.
-
FIBER: A Differentially Private Optimizer with Filter-Aware Innovation Bias Correction
FiBeR adds a closed-form filter-aware correction A(ω)σ_w² to the second-moment term for temporally filtered DP gradients, improving adaptive optimization performance.
-
Differentially Private Contrastive Learning via Bounding Group-level Contribution
DP-GCL improves differentially private contrastive learning by bounding group-level contributions through batch partitioning and intra-group augmentation, delivering 5.6% higher image classification accuracy and 20.1% higher retrieval accuracy than existing approaches.
-
DPrivBench: Benchmarking LLMs' Reasoning for Differential Privacy
DPrivBench is a new benchmark for evaluating LLMs on differential privacy reasoning, with results showing good performance on textbook mechanisms but substantial failures on advanced algorithms.
-
DPQuant: Efficient and Differentially-Private Model Training via Dynamic Quantization Scheduling
DPQuant uses epoch-wise probabilistic layer rotation and DP loss sensitivity to quantize only a changing subset of layers, reducing accuracy degradation from quantization noise in DP-SGD and delivering up to 2.21x throughput gains with under 2% accuracy drop.
-
Privacy Leakage via Output Label Space and Differentially Private Continual Learning
Identifies output label space as a privacy side-channel in DP continual learning, formalizes DP for CL, and demonstrates two mitigation methods yielding higher accuracy than prior work.
-
Privacy-Preserving Federated Autoencoder for ECG Anomaly Detection on Edge Devices
A federated system combining autoencoders, FedAvg, Renyi DP-SGD, and INT8 quantization matches centralized AUROC performance (0.782 for ConvAE) on PTB-XL while halving model size and cutting edge latency by up to 44% with independent privacy and quantization penalties.
-
Provably Communication-Efficient and Privacy-Preserving Federated Graph Neural Networks
CE-FedGNN enables federated GNN training on coupled distributed graphs via infrequent aggregated representation exchange, moving-average estimation for staleness, and metric-DP, with O(1/sqrt(T)) convergence and O(T^{3/4}) communication.
-
Efficient DP-SGD for LLMs with Randomized Clipping
DP-SGD-RC applies Hutchinson and Hutch++ estimators to approximate per-sample gradient norms for clipping in DP-SGD, claiming competitive privacy noise multipliers and utility on Llama 3.2-1B with reduced memory.
-
An exponential mechanism based on quadratic approximations for fine-tuning machine learning models with privacy guarantees
A differentially private fine-tuning method that constructs a quadratic utility function to allow exact sampling from a multivariate normal distribution while providing theoretical privacy guarantees.
-
Private Speech Classification without Collapse: Stabilized DP Training and Offline Distillation
A stabilized DP training and offline distillation protocol prevents collapse to single-class predictors in private speech classification under strong privacy while releasing only an audio-only model.
-
Differentially Private Modeling of Disease Transmission within Human Contact Networks
A differentially private pipeline using node-level DP summaries to fit ERGMs or SBMs, generate synthetic networks, and simulate SIS disease spread on ARTNet sexual contact data produces incidence, prevalence, and intervention effect sizes close to non-private versions.
-
FedVideoMAE: Efficient Privacy-Preserving Federated Video Moderation
FedVideoMAE combines VideoMAE pretraining, LoRA adaptation, client DP-SGD and secure aggregation to cut federated communication 28x while reaching 65-66% accuracy under strong privacy on RWF-2000 with 40 clients.
-
Beyond Membership: Limitations of Add/Remove Adjacency in Differential Privacy
Add/remove adjacency in DP overstates attribute privacy relative to substitute adjacency; new auditing attacks confirm inconsistency with add/remove reports but consistency with substitute accounting.
-
Class-Aware Adaptive Differential Privacy in Deep Learning for Sensor-Based Fall Detection
CA-ADP adjusts differential privacy noise per mini-batch class composition to improve F-scores by 3.3-8.5% over standard DP on three fall-detection datasets while claiming formal (ε,δ) guarantees.
-
Differentially Private Model Merging
Post-processing via random selection or linear combination of differentially private models allows meeting arbitrary target privacy parameters without additional training.
-
Secure and Privacy-Preserving Vertical Federated Learning
Three optimized MPC protocols for privacy-preserving vertical federated learning that support global and global-local updates while reducing computation versus naive full-MPC delegation.
-
On Optimal Hyperparameters for Differentially Private Deep Transfer Learning
Empirical study of DP transfer learning reveals that larger clipping bounds outperform under tight privacy and cumulative DP noise explains batch-size effects better than existing heuristics.
- Fundamental Limitations of Favorable Privacy-Utility Guarantees for DP-SGD