Pith. sign in

REVIEW 15 cited by

UnlearnCanvas: Stylized Image Dataset for Enhanced Machine Unlearning Evaluation in Diffusion Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2402.11846 v4 pith:QSH7WNR6 submitted 2024-02-19 cs.CV

Yihua Zhang , Chongyu Fan , Yimeng Zhang , Yuguang Yao , Jinghan Jia , Jiancheng Liu , Gaoyuan Zhang , Gaowen Liu
show 3 more authors
Ramana Rao Kompella Xiaoming Liu Sijia Liu
This is my paper · ORCID
classification cs.CV
keywords unlearningdatasetevaluationapplicationsbenchmarkcapabilitiesdiffusiongeneration
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

The technological advancements in diffusion models (DMs) have demonstrated unprecedented capabilities in text-to-image generation and are widely used in diverse applications. However, they have also raised significant societal concerns, such as the generation of harmful content and copyright disputes. Machine unlearning (MU) has emerged as a promising solution, capable of removing undesired generative capabilities from DMs. However, existing MU evaluation systems present several key challenges that can result in incomplete and inaccurate assessments. To address these issues, we propose UnlearnCanvas, a comprehensive high-resolution stylized image dataset that facilitates the evaluation of the unlearning of artistic styles and associated objects. This dataset enables the establishment of a standardized, automated evaluation framework with 7 quantitative metrics assessing various aspects of the unlearning performance for DMs. Through extensive experiments, we benchmark 9 state-of-the-art MU methods for DMs, revealing novel insights into their strengths, weaknesses, and underlying mechanisms. Additionally, we explore challenging unlearning scenarios for DMs to evaluate worst-case performance against adversarial prompts, the unlearning of finer-scale concepts, and sequential unlearning. We hope that this study can pave the way for developing more effective, accurate, and robust DM unlearning methods, ensuring safer and more ethical applications of DMs in the future. The dataset, benchmark, and codes are publicly available at https://unlearn-canvas.netlify.app/.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 15 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. You Don't Need All That Attention: Surgical Memorization Mitigation in Text-to-Image Diffusion Models

    cs.CV 2026-02 conditional novelty 7.0 of 10

    A per-prompt cross-attention spike detector plus repulsive-attractive guidance (GUARD) substantially reduces verbatim and template memorization in Stable Diffusion at inference time.

  2. Edit Away and My Face Will not Stay: Personal Biometric Defense against Malicious Generative Editing

    cs.CV 2024-11 conditional novelty 7.0 of 10

    FaceLock perturbs portraits so diffusion-based edits destroy face-recognition similarity, and it evaluates success with the same face model that it attacks.

  3. LU-500: A Logo Benchmark for Concept Unlearning

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A new 500-company benchmark shows current concept-erasure methods cannot remove small logos from generated images without also changing unrelated content.

  4. GUDA: Counterfactual Group-wise Training Data Attribution for Diffusion Models via Unlearning

    cs.LG 2026-01 reject novelty 6.0 of 10

    GUDA approximates leave-one-group-out counterfactual models with unlearning and ranks group influence by ELBO differences.

  5. SPQR: A Multi-Dimensional Benchmark for Safety Alignment under Benign Model Adaptation

    cs.CR 2025-11 conditional novelty 6.0 of 10

    SPQR is a benchmark that scores safety, prompt adherence, quality, and post-fine-tuning robustness of text-to-image safety methods, and it shows benign fine-tuning often collapses safety alignment.

  6. Unlearning Isn't Invisible: Detecting Unlearning Traces in LLMs from Model Outputs

    cs.LG 2025-06 conditional novelty 6.0 of 10

    Unlearning in LLMs leaves detectable 'fingerprints' that let a simple classifier distinguish an unlearned model from its original, even on unrelated prompts.

  7. Rethinking Machine Unlearning in Image Generation Models

    cs.AI 2025-06 conditional novelty 6.0 of 10

    A new taxonomy and multi-aspect evaluation framework for image generation unlearning, with a curated dataset, shows that ten existing unlearning methods perform poorly on preservation and robustness.

  8. AdvAnchor: Enhancing Diffusion Model Unlearning with Adversarial Anchors

    cs.LG 2024-12 conditional novelty 6.0 of 10

    AdvAnchor generates adversarial anchors, embeddings perturbed to be dissimilar from the target concept, and fine-tunes the model toward them, improving the erasure-preservation trade-off in diffusion model unlearning.

  9. Targeted Therapy in Data Removal: Object Unlearning Based on Scene Graphs

    cs.CV 2024-11 conditional novelty 6.0 of 10

    A scene graph-based framework for object-level machine unlearning that removes a specific object from a generative image model while keeping other objects and samples intact.

  10. Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks

    cs.LG 2026-07 conditional novelty 5.0 of 10

    A system-first taxonomy and literature synthesis of multimodal unlearning across vision, language, video, and audio, with datasets, benchmarks, metrics, applications, and open challenges.

  11. Automating Evaluation of Diffusion Model Unlearning with (Vision-) Language Model World Knowledge

    cs.LG 2025-07 conditional novelty 5.0 of 10

    A new evaluation tool uses (vision-)language model world knowledge to rank nearby concepts and craft adversarial prompts, showing that diffusion unlearning is incomplete and that semantic similarity correlates with co...

  12. Few-Shot Concept Unlearning with Low Rank Adaptation

    cs.LG 2025-05 reject novelty 5.0 of 10

    The authors combine few-shot unlearning with low-rank adaptation on the CLIP text encoder to erase concepts from Stable Diffusion v2 in under a minute, reporting low forget-CLIP scores and detection rates on three concepts.

  13. MUNBa: Machine Unlearning via Nash Bargaining

    cs.CV 2024-11 conditional novelty 5.0 of 10

    MUNBa is a machine unlearning method that uses Nash bargaining to balance forgetting and preservation gradients, improving unlearning quality, generalization, and robustness in image classification and generation.

  14. Generative Model Unlearning: A Survey through Target Events, Unlearning Operators, and Evaluation Protocols

    cs.LG 2025-07 conditional novelty 4.0 of 10

    A survey and framework that categorizes generative model unlearning by point-wise versus concept-wise objectives, parameter-based versus non-parametric methods, and completeness/utility/efficiency evaluation.

  15. FameBias: Embedding Manipulation Bias Attack in Text-to-Image Models

    cs.CV 2024-12 conditional novelty 4.0 of 10

    FameBias linearly combines a famous person's embedding with a trigger word's embedding to make text-to-image models generate that person, reaching 53% bias success without training.

Pith tools