REVIEW 9 cited by
The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
This paper describes a testing methodology for quantitatively assessing the risk that rare or unique training-data sequences are unintentionally memorized by generative sequence models---a common type of machine-learning model. Because such models are sometimes trained on sensitive data (e.g., the text of users' private messages), this methodology can benefit privacy by allowing deep-learning practitioners to select means of training that minimize such memorization. In experiments, we show that unintended memorization is a persistent, hard-to-avoid issue that can have serious consequences. Specifically, for models trained without consideration of memorization, we describe new, efficient procedures that can extract unique, secret sequences, such as credit card numbers. We show that our testing strategy is a practical and easy-to-use first line of defense, e.g., by describing its application to quantitatively limit data exposure in Google's Smart Compose, a commercial text-completion neural network trained on millions of users' email messages.
Forward citations
Cited by 9 Pith papers
-
Adversarial Machine Learning Attacks on Financial Reporting via Maximum Violated Multi-Objective Attack
MVMO, a new weighted multi-objective attack, can inflate earnings and lower fraud scores in about 50 to 66 percent of firm-years, versus under 14 percent for standard attacks.
-
How much do language models memorize?
A compression-based measurement puts GPT-style model memorization capacity at roughly 3.6 bits per parameter, with membership inference success following a sigmoid in the dataset-to-capacity ratio.
-
lmgame-Bench: How Good are LLMs at Playing Games?
lmgame-Bench turns six classic games into a scaffolded LLM evaluation suite, ranks 13 models, detects contamination, and reports RL transfer from Sokoban or Tetris to unseen games and planning tasks.
-
Preempting Text Sanitization Utility in Resource-Constrained Privacy-Preserving LLM Interactions
A local small language model can predict when a differentially private sanitized prompt will still yield useful LLM output, saving up to 20% of wasted API calls, and an exact-nearest-neighbor implementation of the dX-...
-
Privacy Accounting and Quality Control in the Sage Differentially Private ML Platform
Sage enforces a global differential privacy guarantee over a growing data stream using block-level composition, and adds privacy-adaptive training with SLAed validation to maintain model quality.
-
Context Reasoner: Incentivizing Reasoning Capability for Contextualized Privacy and Safety Compliance via Reinforcement Learning
A Context Reasoner pipeline that cold-starts LLMs on distilled legal reasoning and applies PPO with a rule-based compliance reward improves performance on CI-based legal compliance benchmarks and transfers to general ...
-
Towards the Anonymization of the Language Modeling
Authors introduce MLM and CLM specialization methods that avoid memorizing identifiers in sensitive training data while aiming for a privacy-utility tradeoff on medical datasets.
-
The FACTS of Technology-Assisted Sensitivity Review
The paper outlines the impact of FACTS issues on technology-assisted sensitivity review for government documents and identifies areas for future research.
-
Synthetic Data Privacy Metrics
This preprint reviews existing privacy metrics for synthetic data and privacy-enhancing techniques, and argues that the field lacks standardization.
Discussion (0). Continue with ORCID to comment.