REVIEW 20 cited by
Guardrail Baselines for Unlearning in LLMs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Recent work has demonstrated that finetuning is a promising approach to 'unlearn' concepts from large language models. However, finetuning can be expensive, as it requires both generating a set of examples and running iterations of finetuning to update the model. In this work, we show that simple guardrail-based approaches such as prompting and filtering can achieve unlearning results comparable to finetuning. We recommend that researchers investigate these lightweight baselines when evaluating the performance of more computationally intensive finetuning methods. While we do not claim that methods such as prompting or filtering are universal solutions to the problem of unlearning, our work suggests the need for evaluation metrics that can better separate the power of guardrails vs. finetuning, and highlights scenarios where guardrails expose possible unintended behavior in existing metrics and benchmarks.
Forward citations
Cited by 20 Pith papers
-
LEMUR: Latent Entropy-aware Multimodal Unlearning via Visual-anchored Reasoning Redirection
An inference-time unlearning method uses token entropy to detect when RL-trained multimodal reasoning models are about to recite private facts in their chain of thought, then redirects decoding with a visual anchor.
-
Measure, Don't Optimize: Forecasting Recovery in LLM Unlearning
A Jacobian-lens audit predicts model-level relearning recovery in LLM unlearning but cannot pick which facts return and backfires when used as a training penalty.
-
How Context Attribution Handles What the Model Already Knows
Context attribution methods cannot disentangle in-context from in-weight knowledge and assign unfaithful scores under overlap; new metrics and WMDP-Cyber++ quantify the failure.
-
Distribution Preference Optimization: A Fine-grained Perspective for LLM Unlearning
DiPO is a distribution-level unlearning method that constructs preference distributions from the model's own high-confidence logits and achieves state-of-the-art forget quality on TOFU while preserving utility.
-
What Should LLMs Forget? Quantifying Personal Data in LLMs for Right-to-Be-Forgotten Requests
WikiMem, a Wikidata-derived canary dataset and a calibrated NLL-ranking metric, identifies which human-fact associations an LLM has memorized, with higher rates for famous people and larger models.
-
Learning-Time Encoding Shapes Unlearning in LLMs
How knowledge is encoded during LLM fine-tuning strongly affects later unlearning: paraphrased training data helps unlearning, while entangled chunks hinder selective forgetting.
-
Unlearning Isn't Invisible: Detecting Unlearning Traces in LLMs from Model Outputs
Unlearning in LLMs leaves detectable 'fingerprints' that let a simple classifier distinguish an unlearned model from its original, even on unrelated prompts.
-
Towards Lifecycle Unlearning Commitment Management: Measuring Sample-level Unlearning Completeness
IAM interpolates between an original model and a shadow model to score each sample's unlearning completeness, achieving top AUC for exact unlearning and top correlation for approximate unlearning, and exposing under- ...
-
GUARD: Generation-time LLM Unlearning via Adaptive Restriction and Detection
GUARD performs inference-time unlearning by classifying prompts, retrieving original answers, and penalizing token matches during beam search, preserving utility but with forget quality that collapses on larger TOFU f...
-
Layered Unlearning for Adversarial Relearning
Layered Unlearning sequentially forgets growing subsets of data to create fold-specific inhibition, blocking recovery of early folds by adversarial relearning.
-
A mean teacher algorithm for unlearning of language models
A mean teacher optimizer that approximates slow natural gradient descent, paired with a new negative log-unlikelihood loss, reduces memorization and privacy leakage on MUSE-News and MUSE-Books, with the strongest vari...
-
Agents Are All You Need for LLM Unlearning
A four-agent pipeline, Vanilla, AuditErase, Critic, and Composer, filters target references out of LLM responses, claiming robust and scalable inference-time unlearning without weight updates.
-
SEUF: Is Unlearning One Expert Enough for Mixture-of-Experts LLMs?
SEUF restricts MoE LLM unlearning to the top-1 most relevant expert and adds a router anchor loss, improving utility by up to 35% while keeping forget efficacy roughly comparable.
-
GROM: Gradient-Free Rapid One-Shot Machine Unlearning
A single closed-form ridge update to selected MLP layers removes targeted knowledge from LLMs in seconds, with state-of-the-art forgetting-utility trade-offs and quantization robustness.
-
Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks
A system-first taxonomy and literature synthesis of multimodal unlearning across vision, language, video, and audio, with datasets, benchmarks, metrics, applications, and open challenges.
-
Exploring Criteria of Loss Reweighting to Enhance LLM Unlearning
The authors propose SatImp, a product of a saturation weight and an importance weight, and show it improves the unlearn-retain trade-off on TOFU, WMDP, and MUSE.
-
Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond
Adding sharpness-aware minimization to NPO unlearning slows relearning attacks on WMDP and MUSE benchmarks.
-
Generative Model Unlearning: A Survey through Target Events, Unlearning Operators, and Evaluation Protocols
A survey and framework that categorizes generative model unlearning by point-wise versus concept-wise objectives, parameter-based versus non-parametric methods, and completeness/utility/efficiency evaluation.
-
Step-by-Step Reasoning Attack: Revealing 'Erased' Knowledge in Large Language Models
Step-by-step reasoning prompts can recover purportedly erased facts from unlearned LLMs, but the paper's quantitative evidence is internally inconsistent.
-
UCD: Unlearning in LLMs via Contrastive Decoding
UCD steers an LLM away from forget-set content at inference time by mixing in the difference between forget-tuned and retain-tuned small models.
Discussion (0). Continue with ORCID to comment.