How knowledge is encoded during LLM fine-tuning strongly affects later unlearning: paraphrased training data helps unlearning, while entangled chunks hinder selective forgetting.
Atyaephyra at SemEval-2025 Task 4: Low-Rank Negative Preference Optimization
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
We present a submission to the SemEval 2025 shared task on unlearning sensitive content from LLMs. Our approach employs negative preference optimization using low-rank adaptation. We show that we can utilize this combination to efficiently compute additional regularization terms, which help with unlearning stabilization. The results of our approach significantly exceed the shared task baselines.
citation-role summary
background 1
citation-polarity summary
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
Learning-Time Encoding Shapes Unlearning in LLMs
How knowledge is encoded during LLM fine-tuning strongly affects later unlearning: paraphrased training data helps unlearning, while entangled chunks hinder selective forgetting.