REVIEW 8 cited by
Knowledge Sanitization of Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We explore a knowledge sanitization approach to mitigate the privacy concerns associated with large language models (LLMs). LLMs trained on a large corpus of Web data can memorize and potentially reveal sensitive or confidential information, raising critical security concerns. Our technique efficiently fine-tunes these models using the Low-Rank Adaptation (LoRA) method, prompting them to generate harmless responses such as ``I don't know'' when queried about specific information. Experimental results in a closed-book question-answering task show that our straightforward method not only minimizes particular knowledge leakage but also preserves the overall performance of LLMs. These two advantages strengthen the defense against extraction attacks and reduces the emission of harmful content such as hallucinations.
Forward citations
Cited by 8 Pith papers
-
Forget Narrowly, Retain Broadly: Unlearning as an Asymmetric Generalization Problem
SUITE defines the forget-retain boundary at semantic, syntactic and lexical levels; training on it plus JensUn++ yields near-complete forgetting with minimal retain and utility loss.
-
Distribution Preference Optimization: A Fine-grained Perspective for LLM Unlearning
DiPO is a distribution-level unlearning method that constructs preference distributions from the model's own high-confidence logits and achieves state-of-the-art forget quality on TOFU while preserving utility.
-
Towards Evaluation for Real-World LLM Unlearning
DCUE evaluates LLM unlearning by comparing core-token confidence score distributions of the unlearned model and the original model, corrected by a validation set, using the Kolmogorov-Smirnov test.
-
Enhancing Safe and Controllable Protein Generation via Knowledge Preference Optimization
A knowledge-graph-guided preference optimization framework that fine-tunes protein language models to generate fewer sequences similar to known harmful proteins.
-
SoK: Machine Unlearning for Large Language Models
A new taxonomy for LLM unlearning distinguishes removal-intended from suppression-intended methods, and argues that gradient ascent methods functionally behave like suppression.
-
A Survey on Model Extraction Attacks and Defenses for Large Language Models
A taxonomy of model extraction attacks and defenses for large language models, with proposed evaluation metrics and future research directions.
-
Lifting Data-Tracing Machine Unlearning to Knowledge-Tracing for Foundation Models
A position paper urging a shift from data-tracing to knowledge-tracing machine unlearning for foundation models, supported by a CLIP case study that shows current methods struggle to generalize.
-
UCD: Unlearning in LLMs via Contrastive Decoding
UCD steers an LLM away from forget-set content at inference time by mixing in the difference between forget-tuned and retain-tuned small models.
Discussion (0). Sign in to comment.