Pith. sign in

REVIEW 3 major objections 5 minor 66 references

Existing MLLM unlearning methods can cut private leakage yet often damage public figures and landmarks that share the same photo as the forgotten person.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 06:16 UTC pith:XI6P5WW7

load-bearing objection PPE-Bench is a useful, well-scoped fix for a real gap in MLLM unlearning evaluation: private targets co-occurring with public figures/landmarks, with clear evidence that standard methods damage the public side and can relearn private answers. the 3 major comments →

arxiv 2607.02897 v1 pith:XI6P5WW7 submitted 2026-07-03 cs.CR cs.AIcs.CV

PPE-Bench: A Benchmark for Evaluating MLLM Unlearning under Private-Public Entanglement

Xianren Zhang , Delvin Ce Zhang , Dongwon Lee , Suhang Wang This is my paper
classification cs.CR cs.AIcs.CV
keywords MLLM unlearningprivate-public entanglementmachine unlearningvisual privacymultimodal benchmarksright to be forgottenadjacent set
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Multimodal large language models trained on web data can memorize private details from photos, so machine unlearning is used to remove that knowledge without full retraining. Prior unlearning tests mostly use simplified single-person images and keep the forget and retain data fully separate. This paper argues those tests miss a common real-world case: a private person often appears with a public celebrity or in front of a famous landmark, so scrubbing the private target can also harm public context that should stay. PPE-Bench builds images that place each fictitious private individual next to a public figure and a landmark, then scores models on three linked goals—forgetting private attributes, keeping unrelated utility, and preserving answers about the co-occurring public content. Across standard unlearning methods the private answers drop, but accuracy on the adjacent public questions falls sharply as well, and private facts can reappear after later fine-tuning only on public data. Two simple remedies—an adjacent-set preservation loss and target-guided image blurring—raise public retention, at some cost to forgetting strength.

Core claim

When private individuals, public celebrities, and landmarks share the same image, standard MLLM unlearning methods reduce answers about the private target but substantially degrade accuracy on questions about the co-occurring public figure and landmark; the forgotten private knowledge can also re-emerge after subsequent fine-tuning on public information alone.

What carries the argument

PPE-Bench: a multimodal unlearning benchmark whose every image entangles a forget-target individual with a public figure and landmark, then splits queries into forget, retain, and adjacent sets so private removal and public preservation are measured on the same visual scene.

Load-bearing premise

Synthetic images that place fictitious faces next to a fixed pool of celebrities and landmarks are realistic enough that measured drops on adjacent public questions will transfer to real social-media photos and real identities.

What would settle it

Train and evaluate a strong unlearning method on PPE-Bench-style entangled images: if forget-set keyword accuracy falls near zero while adjacent public-figure and landmark accuracy stays close to the vanilla model, and private attributes remain suppressed after public-only relearning, the claim that existing methods substantially harm adjacent public information would not hold for that method.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces PPE-Bench to evaluate multimodal LLM unlearning when private target individuals are visually co-present with public celebrities and landmarks in the same images. It constructs 1,800 synthetic entangled images (from MLLMU-Bench profiles + Gemini generation) paired with forget/retain/adjacent QA splits, then shows that standard methods (GA, NPO, KL-Min, GD) reduce forget-set keyword accuracy (and BLEU/ROUGE-L) on both seen and unseen images while also degrading adjacent public-information accuracy; private answers can reappear after subsequent public-only finetuning. Two mitigation objectives (PIP with adjacent/retain losses, and target-guided blurring via masks Tt/Tc) are proposed and shown to improve public retention at some cost to forgetting strength. Experiments use Qwen3-VL-4B and LLaVA-1.5-7B.

Significance. If the reported trade-off and relearning vulnerability hold, PPE-Bench supplies a practically relevant evaluation axis missing from prior MLLM unlearning suites (CLEAR, MLLMU-Bench, PEBench), which used isolated single-subject images and fully disjoint forget/retain sets. The open data release, dual-model results, seen/unseen splits, and explicit adjacent-set metric constitute concrete, reusable contributions that can drive entanglement-aware unlearning research. The two simple mitigations (PIP and blurring) further illustrate that the failure mode is partially addressable. External validity is limited by the synthetic construction, yet the directional claim is well-supported inside the controlled regime and is therefore useful for the community.

major comments (3)
  1. [§4.3 / Figure 3] §4.3 / Figure 3: The central claim that existing methods “substantially harm adjacent public information” is demonstrated only on the entangled images themselves. A control that unlearns the same private targets on non-entangled (single-subject) images and then measures public-figure/landmark accuracy on clean public images would isolate how much of the observed drop is caused by visual entanglement versus generic side-effects of the unlearning objectives. Without it the entanglement-specific interpretation remains suggestive rather than fully isolated.
  2. [§4.1 / Tables 1, 3] §4.1 Evaluation Metrics and Tables 1/3: Keyword presence accuracy is a coarse proxy for free-form generation; a model that refuses, hallucinates a different date, or produces a near-paraphrase can all receive the same binary score. An error analysis or secondary LLM-as-judge / exact-match metric on a stratified sample would confirm that the large ACC drops truly reflect loss of private attributes rather than surface-form artifacts.
  3. [§3.4.2] §3.4.2 Eqs. (2)–(4): Target-guided blurring presupposes per-image target bounding boxes (or masks) for every forget and adjacent sample. Real right-to-be-forgotten requests typically supply only the identity or a few images, not dense annotations. The paper should either demonstrate an automatic face-detection pipeline that recovers comparable performance or clearly scope the method as an oracle upper bound.
minor comments (5)
  1. Throughout: “LLaV A” is inconsistently spaced; standardize to “LLaVA”.
  2. [Figure 2 / Table 2] Figure 2 caption and §3.1: the total image count is stated as 1,800 (1,500 train + 300 test) yet Table 2 lists 1,500 images; reconcile the numbers.
  3. [§4.6] §4.6 / Figure 6: hyper-parameter sweeps are shown only for PIP; a brief note on whether the same λr range was used for GD/KL would aid reproducibility.
  4. [Appendix A.2] Appendix A.2 prompt: the generation instruction does not constrain lighting, pose diversity or occlusion; a short qualitative failure-mode gallery would help readers judge visual realism.
  5. [References] References: several arXiv preprints (e.g., PEBench, MMUNLEARNER) lack final venue or version dates; update where possible.

Circularity Check

0 steps flagged

No significant circularity: PPE-Bench is an independently constructed evaluation suite whose empirical findings on existing unlearning methods do not reduce to fitted inputs or self-defined quantities.

full rationale

The paper's load-bearing claims are empirical measurements on a newly constructed benchmark (forget-set keyword accuracy drop, adjacent public-info accuracy, retain-set utility, and relearning recovery), obtained by applying standard unlearning objectives (GA, NPO, KL-Min, GD) and two proposed mitigations to two foundation MLLMs, then scoring free-form answers with external keyword match / BLEU / ROUGE-L against held-out templates and unseen images. The benchmark construction (Gemini-generated entangled images from MLLMU-Bench profiles + fixed celebrity/landmark pools, QA templates, forget/retain/adjacent splits) is independent of the unlearning loss functions; success is not defined by re-optimizing a quantity that already encodes the target result. The two mitigation methods (PIP loss term on adjacent set; target-guided blurring) are proposed and evaluated on the same bench, which is ordinary method-paper practice and does not force the negative finding about prior methods. No uniqueness theorem, self-citation chain, or fitted parameter is invoked to declare the observed trade-off inevitable. The derivation chain is therefore self-contained experimental evaluation rather than circular reduction.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 2 invented entities

The central empirical claim rests on synthetic data generation choices, standard unlearning loss forms taken from prior work, and two hand-chosen loss weights. No new physical or mathematical entities are postulated; the free parameters are ordinary training hyper-parameters.

free parameters (3)
  • λr (retain-set loss weight) = 2
    Tuned on the benchmark; selected as 2 after a sensitivity sweep that trades forget-set drop against retain accuracy (Figure 6 left).
  • λa (adjacent-set preservation weight) = 1.6
    Tuned after fixing λr; selected as 1.6 for best joint preservation vs. forgetting trade-off (Figure 6 right).
  • unlearning learning rates per method/model = 1e-5–2e-5
    Method- and model-specific rates listed in Table 4 (1e-5 to 2e-5 range); chosen by authors rather than derived.
axioms (3)
  • domain assumption Synthetic Gemini-generated images that place a fictitious face next to a named celebrity and landmark are a valid proxy for real private-public visual entanglement.
    Invoked throughout Section 3.2; all quantitative claims about adjacent damage rest on this proxy.
  • domain assumption Keyword presence in free-form model output is a sufficient measure of whether private or public knowledge has been retained or forgotten.
    Section 4.1 Evaluation Metrics; used for all ACC numbers in Tables 1/3 and Figures 3–5.
  • standard math Standard unlearning objectives (GA, NPO, GD, KL-Min) are correctly instantiated by the loss formulas in Appendix A.3.
    Taken from cited prior work; no new derivation claimed.
invented entities (2)
  • PPE-Bench (forget / retain / adjacent split with entangled images) no independent evidence
    purpose: Provide an evaluation setting that couples private forgetting with public preservation inside the same image.
    New resource introduced by the paper; independent evidence is the public data release itself, but external validation on real photos is not yet shown.
  • Target-guided image blurring transforms Tt and Tc no independent evidence
    purpose: Disentangle the private face from surrounding public context during unlearning without requiring full adjacent QA annotation.
    Defined by Eqs. (2)–(4); a simple masking technique, not a new physical entity.

pith-pipeline@v1.1.0-grok45 · 20513 in / 2940 out tokens · 25433 ms · 2026-07-12T06:16:30.296696+00:00 · methodology

0 comments
read the original abstract

Multimodal Large Language Models (MLLMs) have shown strong capabilities, but they may memorize private information from web data, raising privacy concerns. Machine unlearning offers a way to remove such private knowledge without retraining from scratch. However, existing MLLM unlearning benchmarks have two major limitations. First, they rely on simplified images that contain only the single target individual, failing to reflect the visual complexity of real-world photos. Second, they typically assume that the forget set and retain set are fully separated, ignoring the fact that private information is often visually entangled with benign public information. For example, a private individual may appear with a public figure or in front of a well-known landmark, where unlearning the private target should not damage the public context. To address these limitations, we propose PPE-Bench, a new benchmark for evaluating MLLM unlearning under private-public entanglement. Each image contains a target individual to be forgotten and public information to be preserved, including public figure and landmark. We further introduce two simple but effective methods to better preserve public information during unlearning. Through experiments, we find that existing unlearning methods can reduce private information leakage, but often substantially harm adjacent public information.

Figures

Figures reproduced from arXiv: 2607.02897 by Delvin Ce Zhang, Dongwon Lee, Suhang Wang, Xianren Zhang.

Figure 1
Figure 1. Figure 1: (a) Existing benchmarks have completely separate forget set and retain set and images are simple where [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of PPE-Bench. (a) We generate visually entangled images containing a fictitious individual, a [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Retain-set accuracy versus the drop in forget [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Forget set accuracy before and after further [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Case study comparing responses from dif [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 6
Figure 6. Figure 6: Left: trade-off between forget set accuracy [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 8
Figure 8. Figure 8: Distribution of public figures (left) and land [PITH_FULL_IMAGE:figures/full_fig_p010_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

66 extracted references · 22 linked inside Pith

  1. [1]

    arXiv preprint arXiv:2408.08632 , year=

    A survey on benchmarks of multimodal large language models , author=. arXiv preprint arXiv:2408.08632 , year=

    Pith/arXiv arXiv

  2. [2]

    ACM Computing Surveys , volume=

    Natural language understanding and inference with MLLM in visual question answering: A survey , author=. ACM Computing Surveys , volume=. 2025 , publisher=

    2025

  3. [3]

    arXiv preprint arXiv:2408.01228 , year=

    The phantom menace: unmasking privacy leakages in vision-language models , author=. arXiv preprint arXiv:2408.01228 , year=

    Pith/arXiv arXiv

  4. [4]

    arXiv preprint arXiv:2403.05156 , year=

    On protecting the data privacy of large language models (llms): A survey , author=. arXiv preprint arXiv:2403.05156 , year=

    Pith/arXiv arXiv

  5. [5]

    Proceedings of the 62nd annual meeting of the association for computational linguistics (volume 1: Long papers) , pages=

    Machine unlearning of pre-trained large language models , author=. Proceedings of the 62nd annual meeting of the association for computational linguistics (volume 1: Long papers) , pages=

  6. [6]

    arXiv preprint arXiv:2407.20516 , year=

    Machine unlearning in generative ai: A survey , author=. arXiv preprint arXiv:2407.20516 , year=

    Pith/arXiv arXiv

  7. [7]

    arXiv preprint arXiv:2503.12545 , year=

    PEBench: A Fictitious Dataset to Benchmark Machine Unlearning for Multimodal Large Language Models , author=. arXiv preprint arXiv:2503.12545 , year=

    Pith/arXiv arXiv

  8. [8]

    arXiv preprint arXiv:2411.12103 , year=

    Does unlearning truly unlearn? a black box evaluation of llm unlearning methods , author=. arXiv preprint arXiv:2411.12103 , year=

    Pith/arXiv arXiv

  9. [9]

    arXiv preprint arXiv:2404.05868 , year=

    Negative preference optimization: From catastrophic collapse to effective unlearning , author=. arXiv preprint arXiv:2404.05868 , year=

    Pith/arXiv arXiv

  10. [10]

    Advances in Neural Information Processing Systems , volume=

    Soft prompt threats: Attacking safety alignment and unlearning in open-source llms through the embedding space , author=. Advances in Neural Information Processing Systems , volume=

  11. [11]

    arXiv preprint arXiv:2407.00106 , year=

    Ununlearning: Unlearning is not sufficient for content regulation in advanced generative ai , author=. arXiv preprint arXiv:2407.00106 , year=

    Pith/arXiv arXiv

  12. [12]

    arXiv preprint arXiv:2409.18025 , year=

    An adversarial perspective on machine unlearning for ai safety , author=. arXiv preprint arXiv:2409.18025 , year=

    Pith/arXiv arXiv

  13. [13]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Figstep: Jailbreaking large vision-language models via typographic visual prompts , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  14. [14]

    arXiv preprint arXiv:2407.21659 , year=

    Cross-modality information check for detecting jailbreaking in multimodal large language models , author=. arXiv preprint arXiv:2407.21659 , year=

    Pith/arXiv arXiv

  15. [15]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    Clear: Character unlearning in textual and visual modalities , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  16. [16]

    First Conference on Language Modeling , year =

    TOFU: A Task of Fictitious Unlearning for LLMs , author=. First Conference on Language Modeling , year =

  17. [17]

    Protecting privacy in multimodal large language models with mllmu-bench , author=. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages=

    2025

  18. [18]

    International Conference on Learning Representations , year=

    Towards Deep Learning Models Resistant to Adversarial Attacks , author=. International Conference on Learning Representations , year=

  19. [19]

    Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    Defense against adversarial attacks using high-level representation guided denoiser , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

  20. [20]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    Demystifying Verbatim Memorization in Large Language Models , author=. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    2024

  21. [21]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Towards robust knowledge unlearning: An adversarial framework for assessing and improving unlearning robustness in large language models , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  22. [22]

    arXiv preprint arXiv:2209.02299 , year=

    A survey of machine unlearning , author=. arXiv preprint arXiv:2209.02299 , year=

    Pith/arXiv arXiv

  23. [23]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Photomaker: Customizing realistic human photos via stacked id embedding , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  24. [24]

    First Conference on Language Modeling , year =

    Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning , author=. First Conference on Language Modeling , year =

  25. [25]

    Conference on Lifelong Learning Agents , pages=

    Continual learning and private unlearning , author=. Conference on Lifelong Learning Agents , pages=. 2022 , organization=

    2022

  26. [26]

    arXiv preprint arXiv:2402.02309 , year=

    Jailbreaking attack against multimodal large language model , author=. arXiv preprint arXiv:2402.02309 , year=

    Pith/arXiv arXiv

  27. [27]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Visual adversarial examples jailbreak aligned large language models , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  28. [28]

    Journal of Machine Learning Research , volume=

    An optimal algorithm for bandit and zero-order convex optimization with two-point feedback , author=. Journal of Machine Learning Research , volume=

  29. [29]

    International conference on machine learning , pages=

    Learning transferable visual models from natural language supervision , author=. International conference on machine learning , pages=. 2021 , organization=

    2021

  30. [30]

    Text summarization branches out , pages=

    Rouge: A package for automatic evaluation of summaries , author=. Text summarization branches out , pages=

  31. [31]

    Advances in Neural Information Processing Systems , volume=

    Judging llm-as-a-judge with mt-bench and chatbot arena , author=. Advances in Neural Information Processing Systems , volume=

  32. [32]

    arXiv preprint arXiv:2402.09910 , year=

    De-cop: Detecting copyrighted content in language models training data , author=. arXiv preprint arXiv:2402.09910 , year=

    Pith/arXiv arXiv

  33. [33]

    International Conference on Learning Representations , year =

    LoRA: Low-Rank Adaptation of Large Language Models , author=. International Conference on Learning Representations , year =

  34. [34]

    IEEE transactions on image processing , volume=

    Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising , author=. IEEE transactions on image processing , volume=. 2017 , publisher=

    2017

  35. [35]

    Advances in Neural Information Processing Systems , volume=

    Single image unlearning: Efficient machine unlearning in multimodal large language models , author=. Advances in Neural Information Processing Systems , volume=

  36. [36]

    arXiv preprint arXiv:2503.14604 , year=

    Image Captioning Evaluation in the Age of Multimodal LLMs: Challenges and Future Perspectives , author=. arXiv preprint arXiv:2503.14604 , year=

    Pith/arXiv arXiv

  37. [37]

    arXiv preprint arXiv:2410.15067 , year=

    A survey on all-in-one image restoration: Taxonomy, evaluation and future trends , author=. arXiv preprint arXiv:2410.15067 , year=

    Pith/arXiv arXiv

  38. [38]

    Advances in neural information processing systems , volume=

    Flamingo: a visual language model for few-shot learning , author=. Advances in neural information processing systems , volume=

  39. [39]

    Advances in neural information processing systems , volume=

    Visual instruction tuning , author=. Advances in neural information processing systems , volume=

  40. [40]

    Advances in Neural Information Processing Systems , volume=

    Interpreting clip with sparse linear concept embeddings (splice) , author=. Advances in Neural Information Processing Systems , volume=

  41. [41]

    Advances in Neural Information Processing Systems , volume=

    What matters when building vision-language models? , author=. Advances in Neural Information Processing Systems , volume=

  42. [42]

    ICLR , year=

    Catastrophic Failure of LLM Unlearning via Quantization , author=. ICLR , year=

  43. [43]

    Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages=

    Bleu: a method for automatic evaluation of machine translation , author=. Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages=

  44. [44]

    Advances in Neural Information Processing Systems , volume=

    Membership inference attacks against large vision-language models , author=. Advances in Neural Information Processing Systems , volume=

  45. [45]

    arXiv preprint arXiv:2411.03350 , year=

    A comprehensive survey of small language models in the era of large language models: Techniques, enhancements, applications, collaboration with llms, and trustworthiness , author=. arXiv preprint arXiv:2411.03350 , year=

    Pith/arXiv arXiv

  46. [46]

    arXiv preprint arXiv:2506.00359 , year=

    Keeping an Eye on LLM Unlearning: The Hidden Risk and Remedy , author=. arXiv preprint arXiv:2506.00359 , year=

    arXiv

  47. [47]

    arXiv preprint arXiv:2410.12207 , year=

    Divide-Verify-Refine: Aligning LLM Responses with Complex Instructions , author=. arXiv preprint arXiv:2410.12207 , year=

    Pith/arXiv arXiv

  48. [48]

    arXiv preprint arXiv:2502.18387 , year=

    How Far are LLMs from Real Search? A Comprehensive Study on Efficiency, Completeness, and Inherent Capabilities , author=. arXiv preprint arXiv:2502.18387 , year=

    Pith/arXiv arXiv

  49. [49]

    arXiv preprint arXiv:2502.17823 , year=

    A general framework to enhance fine-tuning-based llm unlearning , author=. arXiv preprint arXiv:2502.17823 , year=

    arXiv

  50. [50]

    arXiv preprint arXiv:2502.12359 , year=

    Lanp: Rethinking the impact of language priors in large vision-language models , author=. arXiv preprint arXiv:2502.12359 , year=

    Pith/arXiv arXiv

  51. [51]

    arXiv preprint arXiv:2307.15043 , year=

    Universal and transferable adversarial attacks on aligned language models , author=. arXiv preprint arXiv:2307.15043 , year=

    Pith/arXiv arXiv

  52. [52]

    Advances in Neural Information Processing Systems , volume=

    Large language model unlearning , author=. Advances in Neural Information Processing Systems , volume=

  53. [53]

    Information & communications technology law , volume=

    The European Union general data protection regulation: what it is and what it means , author=. Information & communications technology law , volume=. 2019 , publisher=

    2019

  54. [54]

    The california consumer privacy act: Towards a european-style privacy regime in the united states , author=. J. Tech. L. & Pol'y , volume=. 2018 , publisher=

    2018

  55. [55]

    International Conference on Advances in Digital Science , pages=

    Right to be forgotten in the age of machine learning , author=. International Conference on Advances in Digital Science , pages=. 2021 , organization=

    2021

  56. [56]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    Mmunlearner: Reformulating multimodal machine unlearning in the era of multimodal large language models , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  57. [57]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Analyzing and improving the image quality of stylegan , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  58. [58]

    European Conference on Computer Vision , pages=

    Arc2face: A foundation model for id-consistent human faces , author=. European Conference on Computer Vision , pages=. 2024 , organization=

    2024

  59. [59]

    Image editing in Gemini just got a major upgrade , year =

  60. [60]

    Advances in Neural Information Processing Systems , volume=

    Variational bayesian unlearning , author=. Advances in Neural Information Processing Systems , volume=

  61. [61]

    arXiv preprint arXiv:2511.21631 , year=

    Qwen3-vl technical report , author=. arXiv preprint arXiv:2511.21631 , year=

    Pith/arXiv arXiv

  62. [62]

    Proceedings of the 41st International Conference on Machine Learning , pages=

    The WMDP benchmark: measuring and reducing malicious use with unlearning , author=. Proceedings of the 41st International Conference on Machine Learning , pages=

  63. [63]

    Advances in neural information processing systems , volume=

    Direct preference optimization: Your language model is secretly a reward model , author=. Advances in neural information processing systems , volume=

  64. [64]

    International Conference on Learning Representations , volume=

    Can sensitive information be deleted from llms? objectives for defending against extraction attacks , author=. International Conference on Learning Representations , volume=

  65. [65]

    arXiv e-prints , pages=

    Who's Harry Potter? Approximate Unlearning in LLMs , author=. arXiv e-prints , pages=

  66. [66]

    arXiv preprint arXiv:2410.21276 , year=

    Gpt-4o system card , author=. arXiv preprint arXiv:2410.21276 , year=

    Pith/arXiv arXiv