REVIEW 4 cited by
GRU: Mitigating the Trade-off between Unlearning and Retention for LLMs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
GRU: Mitigating the Trade-off between Unlearning and Retention for LLMs
read the original abstract
Large language model (LLM) unlearning has demonstrated its essential role in removing privacy and copyright-related responses, crucial for their legal and safe applications. However, the pursuit of complete unlearning often comes with substantial costs due to its compromises in their general functionality, leading to a notorious trade-off between unlearning and retention. It motivates this paper to explore enhanced unlearning schemes that can mitigate this trade-off. Specifically, we propose Gradient Rectified Unlearning (GRU), an improved framework that regulates the directions of gradient updates during the unlearning procedure such that their side impacts on other, unrelated responses can be minimized. GRU is easy and general to implement, demonstrating practical effectiveness across a variety of well-established unlearning benchmarks.
Forward citations
Cited by 4 Pith papers
-
HermesHFL: Incentive-Compatible Hierarchical Federated Unlearning for Dynamic LLM Fine-Tuning
HermesHFL plus Neogen jointly optimize incentives, client–edge association, and gradient-ascent unlearning so hierarchical LoRA fine-tuning remains useful after clients leave and rejoin.
-
Null Space Constrained Contrastive Visual Forgetting for MLLM Unlearning
A contrastive visual forgetting technique constrained to the null space of retained knowledge enables targeted unlearning of visual concepts in MLLMs while preserving non-target visual and all textual knowledge.
-
HermesHFL: Incentive-Compatible Hierarchical Federated Unlearning for Dynamic LLM Fine-Tuning
HermesHFL couples LoRA-based hierarchical federated fine-tuning with gradient-ascent unlearning, client rejoin, and incentive allocation, solved by a CMA-ES/CHC bilevel evolutionary optimizer with a neural surrogate.
-
HermesHFL: Incentive-Compatible Hierarchical Federated Unlearning for Dynamic LLM Fine-Tuning
HermesHFL adds incentive contracts and gradient-ascent unlearning to hierarchical federated LoRA fine-tuning for leave-unlearn-rejoin clients, but its claimed superiority is contradicted by its own experiments.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.