REVIEW 4 cited by
ReLearn: Unlearning via Learning for Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Current unlearning methods for large language models usually rely on reverse optimization to reduce target token probabilities. However, this paradigm disrupts the subsequent tokens prediction, degrading model performance and linguistic coherence. Moreover, existing evaluation metrics overemphasize contextual forgetting while inadequately assessing response fluency and relevance. To address these challenges, we propose ReLearn, a data augmentation and fine-tuning pipeline for effective unlearning, along with a comprehensive evaluation framework. This framework introduces Knowledge Forgetting Rate (KFR) and Knowledge Retention Rate (KRR) to measure knowledge-level preservation, and Linguistic Score (LS) to evaluate generation quality. Our experiments show that ReLearn successfully achieves targeted forgetting while preserving high-quality output. Through mechanistic analysis, we further demonstrate how reverse optimization disrupts coherent text generation, while ReLearn preserves this essential capability. Code is available at https://github.com/zjunlp/unlearn.
Forward citations
Cited by 4 Pith papers
-
Model Unlearning via Sparse Autoencoder Subspace Guided Projections
SSPU uses SAE-derived subspaces to guide weight updates, lowering WMDP-Cyber accuracy by 3.22% more than RMU while largely preserving MMLU, TruthfulQA, and GSM8K performance.
-
Representation-Aware Unlearning via Activation Signatures: From Suppression to Entity-Signature Erasure
KIF reports near-oracle entity unlearning on TOFU (FQ=0.99, MU=0.62) by suppressing subject-specific activation directions and distilling the behavior into LoRA, but its 'internal attenuation' evidence rests on an out...
-
SoK: Machine Unlearning for Large Language Models
A new taxonomy for LLM unlearning distinguishes removal-intended from suppression-intended methods, and argues that gradient ascent methods functionally behave like suppression.
-
Editing as Unlearning: Are Knowledge Editing Methods Strong Baselines for Large Language Model Unlearning?
WISE and AlphaEdit, two knowledge editing methods, are competitive unlearning baselines when unlearning is framed as editing a model's answer into a refusal.
Discussion (0). Continue with ORCID to comment.