REVIEW 3 cited by
Fine-Tuning Language Models with Differential Privacy through Adaptive Noise Allocation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Language models are capable of memorizing detailed patterns and information, leading to a double-edged effect: they achieve impressive modeling performance on downstream tasks with the stored knowledge but also raise significant privacy concerns. Traditional differential privacy based training approaches offer robust safeguards by employing a uniform noise distribution across all parameters. However, this overlooks the distinct sensitivities and contributions of individual parameters in privacy protection and often results in suboptimal models. To address these limitations, we propose ANADP, a novel algorithm that adaptively allocates additive noise based on the importance of model parameters. We demonstrate that ANADP narrows the performance gap between regular fine-tuning and traditional DP fine-tuning on a series of datasets while maintaining the required privacy constraints.
Forward citations
Cited by 3 Pith papers
-
Dual-Priv Pruning : Efficient Differential Private Fine-Tuning in Multimodal Large Language Models
A framework for DP fine-tuning of MLLMs that prunes visual tokens before training and selectively applies noisy gradient updates to blocks with the largest norms, reporting modest utility and memory gains over DP-SGD.
-
PPFL-RDSN: Privacy-Preserving Federated Learning-based Residual Dense Spatial Networks for Encrypted Lossy Image Reconstruction
A federated, privacy-preserving RDSN framework for encrypted image reconstruction whose local differential privacy mechanism is not actually differentially private because it releases low-frequency DCT coefficients wi...
-
Inclusive Federated Learning Through Compliance-Weighted Noise Allocation in Healthcare AI
Compliance-weighted noise allocation in federated healthcare learning claims no accuracy loss versus uniform noise, but its differential privacy guarantee applies only to the aggregator dataset, not client data.
Discussion (0). Sign in to comment.