Pith. sign in

REVIEW 5 cited by

Protecting Privacy in Multimodal Large Language Models with MLLMU-Bench

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.22108 v2 pith:5JUVCTM4 submitted 2024-10-29 cs.CL cs.AI

Zheyuan Liu , Guangyao Dou , Mengzhao Jia , Zhaoxuan Tan , Qingkai Zeng , Yongle Yuan , Meng Jiang This is my paper · ORCID
classification cs.CLcs.AI
keywords multimodalunlearninglanguagelargemodelsalgorithmsbenchmarkmllmu-bench
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Generative models such as Large Language Models (LLM) and Multimodal Large Language models (MLLMs) trained on massive web corpora can memorize and disclose individuals' confidential and private data, raising legal and ethical concerns. While many previous works have addressed this issue in LLM via machine unlearning, it remains largely unexplored for MLLMs. To tackle this challenge, we introduce Multimodal Large Language Model Unlearning Benchmark (MLLMU-Bench), a novel benchmark aimed at advancing the understanding of multimodal machine unlearning. MLLMU-Bench consists of 500 fictitious profiles and 153 profiles for public celebrities, each profile feature over 14 customized question-answer pairs, evaluated from both multimodal (image+text) and unimodal (text) perspectives. The benchmark is divided into four sets to assess unlearning algorithms in terms of efficacy, generalizability, and model utility. Finally, we provide baseline results using existing generative model unlearning algorithms. Surprisingly, our experiments show that unimodal unlearning algorithms excel in generation and cloze tasks, while multimodal unlearning approaches perform better in classification tasks with multimodal inputs.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Toward Fine-Grained Forgetting:Attribute Unlearning for Multimodal Large Language Models

    cs.AI 2026-08 reject novelty 6.0 of 10

    The paper defines attribute-level MLLM unlearning and proposes CLRP, but the method's headline forgetting gains on cloze are partly produced by test-time logit subtraction applied only to the forget and test sets.

  2. Understanding Machine Unlearning Through the Lens of Mode Connectivity

    cs.LG 2026-07 unverdicted novelty 6.0 of 10

    Unlearned models usually connect to their originals by smooth low-loss paths, and the smoothness of that path can predict how hard the unlearning task was.

  3. POPS: Recovering Unlearned Multi-Modality Knowledge in MLLMs with Prompt-Optimized Parameter Shaking

    cs.CR 2026-07 conditional novelty 6.0 of 10

    Prompt-optimized suffixes plus synthetic fine-tuning recover ~82% of knowledge that multimodal unlearning methods claim to erase from MLLMs.

  4. Towards Benign Memory Forgetting for Selective Multimodal Large Language Model Unlearning

    cs.AI 2025-11 conditional novelty 6.0 of 10

    An MLLM unlearning method and benchmark that aim to erase targeted private facts while preserving image understanding.

  5. Temporal Working Memory: Query-Guided Segment Refinement for Enhanced Multimodal Understanding

    cs.CV 2025-02 conditional novelty 5.0 of 10

    A query-guided frame and audio segment selection module, called TWM, improves nine multimodal models on AVQA, video captioning, and video-text retrieval in the reported tables.

Pith tools