Pith. sign in

REVIEW 11 cited by

MU-Bench: A Multitask Multimodal Benchmark for Machine Unlearning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2406.14796 v2 pith:JZC2VSO3 submitted 2024-06-21 cs.LG cs.AI

Jiali Cheng , Hadi Amiri This is my paper
classification cs.LGcs.AI
keywords modelsmu-benchtrainedunlearningadvancementsapproachesbenchmarkcomprehensive
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Recent advancements in Machine Unlearning (MU) have introduced solutions to selectively remove certain training samples, such as those with outdated or sensitive information, from trained models. Despite these advancements, evaluation of MU methods have been inconsistent, employing different trained models and architectures, and sample removal strategies, which hampers accurate comparison. In addition, prior MU approaches have mainly focused on singular tasks or modalities, which is not comprehensive. To address these limitations, we develop MU-Bench, the first comprehensive benchmark for MU that (i) unifies the sets of deleted samples and trained models, and (ii) provides broad coverage of tasks and data modalities, including previously unexplored domains such as speech and video classification. Our evaluation show that RandLabel and SalUn are the most effective general MU approaches on MU-Bench, and BadT and SCRUB are capable of achieving random performance on the deletion set. We analyze several under-investigated aspects of unlearning, including scalability, the impacts of parameter-efficient fine-tuning and curriculum learning, and susceptibility to dataset biases. MU-Bench provides an easy-to-use package that includes dataset splits, models, and implementations, together with a leader board to enable unified and scalable MU research.

Discussion (0). Sign in to comment.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. REMEDI: A Benchmark for Retention and Unlearning Evaluation in Multi-label Clinical Disease Inference

    cs.LG 2026-06 unverdicted novelty 7.0 of 10

    REMEDI is a new benchmark for evaluating machine unlearning in multi-label clinical disease inference on MIMIC-III data that reveals trade-offs in existing methods.

  2. TRACER: Token ReAssignment for Concept ERasure in Generative Recommendation

    cs.IR 2026-06 unverdicted novelty 7.0 of 10

    TRACER uses token reassignment for concept-related items plus a coherence regularizer to unlearn specific concepts in generative recommendation while preserving utility better than baselines.

  3. ICU-Bench:Benchmarking Continual Unlearning in Multimodal Large Language Models

    cs.AI 2026-05 unverdicted novelty 7.0 of 10

    ICU-Bench is a new continual unlearning benchmark for MLLMs using 1000 privacy profiles, 9500 images, and 100 forget tasks, showing existing methods fail to balance forgetting, utility, and scalability.

  4. CURE:Circuit-Aware Unlearning for LLM-based Recommendation

    cs.IR 2026-04 unverdicted novelty 7.0 of 10

    CURE disentangles LLM recommendation circuits into forget-specific, retain-specific, and task-shared modules with tailored update rules to achieve more effective unlearning than weighted baselines.

  5. Understanding Machine Unlearning Through the Lens of Mode Connectivity

    cs.LG 2026-07 unverdicted novelty 6.0 of 10

    Unlearned models usually connect to their originals by smooth low-loss paths, and the smoothness of that path can predict how hard the unlearning task was.

  6. Understanding Machine Unlearning Through the Lens of Mode Connectivity

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Unlearned models often lie in connected low-loss basins, and that geometry can predict unlearning difficulty and improve robustness to relearning attacks.

  7. ICU-Bench:Benchmarking Continual Unlearning in Multimodal Large Language Models

    cs.AI 2026-05 conditional novelty 6.0 of 10

    Existing continual unlearning methods fail to maintain forgetting, retained knowledge, and utility over 100 sequential privacy-deletion tasks in multimodal document models.

  8. A Mechanistic Perspective and Circuit-Guided Difficulty Metric for Unlearning

    cs.LG 2026-01 conditional novelty 6.0 of 10

    A circuit-similarity score predicts which samples an LLM unlearning method will fail to erase, with hard samples relying on deeper, output-facing pathways.

  9. FUTURE: Flexible Unlearning for Tree Ensemble

    cs.LG 2025-08 conditional novelty 6.0 of 10

    FUTURE forgets training samples from tree ensembles by optimizing sigmoid-smoothed split thresholds and copying them back to the original discrete trees.

  10. Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks

    cs.LG 2026-07 conditional novelty 5.0 of 10

    A system-first taxonomy and literature synthesis of multimodal unlearning across vision, language, video, and audio, with datasets, benchmarks, metrics, applications, and open challenges.

  11. A Numerical PDEs Approach to Evolution Equations in Shape Analysis Based on Regularized Morphoelasticity

    math.NA 2026-04 unverdicted novelty 4.0 of 10

    Regularized morphoelasticity yields a high-order elliptic system for continuous shape evolution that is solved by mixed finite elements in FEniCSx within an LDDMM-style optimal-control growth model.

Pith tools