MDU minimizes forward KL divergence from prompt-conditional to prompt-masked unconditional predictions at masked positions to unlearn knowledge in MDLMs while trading off privacy and utility via temperature scaling.
hub
Simplicity prevails: Rethinking negative preference optimization for llm unlearning
17 Pith papers cite this work. Polarity classification is still indexing.
17
Pith papers citing it
hub tools
citation-role summary
background 1
citation-polarity summary
roles
background 1polarities
background 1representative citing papers
RASLIK uses randomized antipodal search on linearized influence kernels to achieve data Pareto improvement in LLM unlearning, outperforming baselines with sublinear complexity and double gains in quality and efficiency.
CURE applies contrastive unlearning to reduce deprecated API usage in code LLMs and improve correct replacements on a benchmark dataset while preserving general performance.
Adaptive Unlearning suppresses package hallucinations in code-generating LLMs by 81% while preserving benchmark performance, using model-generated data and no human labels.
REGLU guides LoRA-based unlearning via representation subspaces and orthogonal regularization to outperform prior methods on forget-retain trade-off in LLM benchmarks.
Treating retention as the dominant task and using constructive gradient synthesis like SAGO allows LLM unlearning to achieve higher general performance recovery without weakening the forgetting effect.
WIN-U delivers a retain-free unlearning update that approximates the gold-standard retrained model via a Woodbury-informed Newton step using only forget-set curvature information.
ULS provides minimax-optimal estimation of remaining-data parameters in machine unlearning with limited access and decomposes error into oracle plus unlearning cost terms.
MPU is a framework that achieves privacy-preserving unlearning for LLMs by distributing perturbed model copies for local client-side unlearning followed by server-side aggregation with harmonic denoising.
Downgrading optimizers to lower-information variants during LLM unlearning yields more robust forgetting on MUSE and WMDP benchmarks by converging to harder-to-perturb loss basins.
MAST ranks attention-projection tensors by off-principal energy, update magnitude, and forget-gradient coupling to selectively unlearn RLVR-induced reasoning, achieving significant forgetting on MATH while preserving GSM8K and retain MATH unlike full-parameter updates.
DPOP augments DPO with a conditional penalty on reference-greedy responses and reports 5.3% and 4.4% relative win-rate gains on AlpacaEval 2.0 for two models.
Machine unlearning should be restricted to dataset-defined deletion achieving retraining equivalence, while other LLM tasks require separate terminology and evaluation baselines.
A six-dimensional MathVerifier supplies hard negatives and per-sample weights that improve DPO performance on math reasoning for a 1.5B Qwen2.5 model over standard SFT and unweighted DPO.
OFMU is a penalty-based bi-level optimizer for machine unlearning that alternates between a gradient-ascent forgetting step and a gradient-descent utility-restoration step, with a similarity penalty between forget and retain gradients.
MSA performs data unlearning in LLMs by arithmetic operations on prior model checkpoints to remove targeted datapoint influence, with experiments showing competitive or better results than existing unlearning methods.
citing papers explorer
-
Machine Unlearning for Masked Diffusion Language Models
MDU minimizes forward KL divergence from prompt-conditional to prompt-masked unconditional predictions at masked positions to unlearn knowledge in MDLMs while trading off privacy and utility via temperature scaling.
-
Randomized Antipodal Search Done Right for Data Pareto Improvement of LLM Unlearning
RASLIK uses randomized antipodal search on linearized influence kernels to achieve data Pareto improvement in LLM unlearning, outperforming baselines with sublinear complexity and double gains in quality and efficiency.
-
Towards Knowledge Alignment in Code LLMs: Contrastive Unlearning for Evolving APIs
CURE applies contrastive unlearning to reduce deprecated API usage in code LLMs and improve correct replacements on a benchmark dataset while preserving general performance.
-
LLM Ghostbusters: Surgical Hallucination Suppression via Adaptive Unlearning
Adaptive Unlearning suppresses package hallucinations in code-generating LLMs by 81% while preserving benchmark performance, using model-generated data and no human labels.
-
Representation-Guided Parameter-Efficient LLM Unlearning
REGLU guides LoRA-based unlearning via representation subspaces and orthogonal regularization to outperform prior methods on forget-retain trade-off in LLM benchmarks.
-
Modeling LLM Unlearning as an Asymmetric Two-Task Learning Problem
Treating retention as the dominant task and using constructive gradient synthesis like SAGO allows LLM unlearning to achieve higher general performance recovery without weakening the forgetting effect.
-
WIN-U: Woodbury-Informed Newton-Unlearning as a retain-free Machine Unlearning Framework
WIN-U delivers a retain-free unlearning update that approximates the gold-standard retrained model via a Woodbury-informed Newton step using only forget-set curvature information.
-
Efficient machine unlearning with minimax optimality
ULS provides minimax-optimal estimation of remaining-data parameters in machine unlearning with limited access and decomposes error into oracle plus unlearning cost terms.
-
MPU: Towards Secure and Privacy-Preserving Knowledge Unlearning for Large Language Models
MPU is a framework that achieves privacy-preserving unlearning for LLMs by distributing perturbed model copies for local client-side unlearning followed by server-side aggregation with harmonic denoising.
-
Downgrade to Upgrade: Optimizer Simplification Enhances Robustness in LLM Unlearning
Downgrading optimizers to lower-information variants during LLM unlearning yields more robust forgetting on MUSE and WMDP benchmarks by converging to harder-to-perturb loss basins.
-
Mechanism-Guided Selective Unlearning for RLVR-Induced Reasoning
MAST ranks attention-projection tensors by off-principal energy, update magnitude, and forget-gradient coupling to selectively unlearn RLVR-induced reasoning, achieving significant forgetting on MATH while preserving GSM8K and retain MATH unlike full-parameter updates.
-
Boosting Direct Preference Optimization with Penalization
DPOP augments DPO with a conditional penalty on reference-greedy responses and reports 5.3% and 4.4% relative win-rate gains on AlpacaEval 2.0 for two models.
-
Position: The Term "Machine Unlearning" Is Overused in LLMs
Machine unlearning should be restricted to dataset-defined deletion achieving retraining equivalence, while other LLM tasks require separate terminology and evaluation baselines.
-
Hard Negative Sample-Augmented DPO Post-Training for Small Language Models
A six-dimensional MathVerifier supplies hard negatives and per-sample weights that improve DPO performance on math reasoning for a 1.5B Qwen2.5 model over standard SFT and unweighted DPO.
-
OFMU: Optimization-Driven Framework for Machine Unlearning
OFMU is a penalty-based bi-level optimizer for machine unlearning that alternates between a gradient-ascent forgetting step and a gradient-descent utility-restoration step, with a similarity penalty between forget and retain gradients.
-
Revisiting the Past: Data Unlearning with Model State History
MSA performs data unlearning in LLMs by arithmetic operations on prior model checkpoints to remove targeted datapoint influence, with experiments showing competitive or better results than existing unlearning methods.
- RepSelect: Robust LLM Unlearning via Representation Selectivity