Pith. sign in

REVIEW 6 cited by

LUME: LLM Unlearning with Multitask Evaluations

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2502.15097 v3 pith:F242K7D3 submitted 2025-02-20 cs.CL cs.LG

Anil Ramakrishna , Yixin Wan , Xiaomeng Jin , Kai-Wei Chang , Zhiqi Bu , Bhanukiran Vinzamuri , Volkan Cevher , Mingyi Hong
show 1 more author
Rahul Gupta
This is my paper · ORCID
classification cs.CLcs.LG
keywords unlearningunlearnbiographiesevaluationsllmslumemodelssensitive
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Unlearning aims to remove copyrighted, sensitive, or private content from large language models (LLMs) without a full retraining. In this work, we develop a multi-task unlearning benchmark (LUME) which features three tasks: (1) unlearn synthetically generated creative short novels, (2) unlearn synthetic biographies with sensitive information, and (3) unlearn a collection of public biographies. We further release two fine-tuned LLMs of 1B and 7B parameter sizes as the target models. We conduct detailed evaluations of several recently proposed unlearning algorithms and present results on carefully crafted metrics to understand their behavior and limitations.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Forget Narrowly, Retain Broadly: Unlearning as an Asymmetric Generalization Problem

    cs.LG 2026-07 accept novelty 7.0 of 10

    SUITE defines the forget-retain boundary at semantic, syntactic and lexical levels; training on it plus JensUn++ yields near-complete forgetting with minimal retain and utility loss.

  2. Leak@$k$: Unlearning Does Not Make LLMs Forget Under Probabilistic Decoding

    cs.LG 2025-11 reject novelty 6.0 of 10

    LLM unlearning methods that pass greedy-decoding benchmarks leak forgotten facts when the model is sampled repeatedly, and the new leak@k metric quantifies this.

  3. SoK: Machine Unlearning for Large Language Models

    cs.LG 2025-06 conditional novelty 5.0 of 10

    A new taxonomy for LLM unlearning distinguishes removal-intended from suppression-intended methods, and argues that gradient ascent methods functionally behave like suppression.

  4. Not Every Token Needs Forgetting: Selective Unlearning to Limit Change in Utility in Large Language Model Unlearning

    cs.CL 2025-06 reject novelty 5.0 of 10

    SU uses two assistant models trained on different data splits to score tokens, then unlearns only tokens whose scores diverge, claiming better retain-set utility with comparable forget quality.

  5. Lacuna Inc. at SemEval-2025 Task 4: LoRA-Enhanced Influence-Based Unlearning for LLMs

    cs.CL 2025-06 conditional novelty 4.0 of 10

    LIBU combines a Fisher-diagonal-weighted gradient update on the forget set with Sophia second-order fine-tuning to unlearn sensitive content from OLMo models, reaching 0.483 MMLU in its best-reported setup.

  6. iShumei-Chinchunmei at SemEval-2025 Task 4: A balanced forgetting and retention multi-task framework using effective unlearning loss

    cs.CL 2025-07 conditional novelty 3.0 of 10

    The authors propose Effective Unlearning Loss, the inverse of the standard next-token prediction loss, within a multi-task framework, and report a 5th-place finish at SemEval-2025 Task 4.

Pith tools