{"id":"71f63c13-819e-4b5d-8bc2-7b4325f51d43","arxiv_id":"2412.00782","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Erased concepts in text-to-image diffusion models can still be generated from high-likelihood latent seeds recovered by diffusion inversion, across nine ablation methods and six concepts.","lead":"This paper asks whether concept erasure methods for diffusion models actually erase concepts, by inverting erased-concept images back into the model's latent space. It finds that state-of-the-art ablated models still contain high-likelihood latent seeds that reconstruct the erased concepts with high quality.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Missing concept-free control: high-PSNR inversion may be a universal property of diffusion models, not evidence that ablated concepts persist.","rationale":"The reader identified the same missing control, and I agree that it is the most load-bearing gap. The paper's operational definition of memory (Sec. 3.1) is 'a high-likelihood seed exists that generates the ablated image'; under inversion, this condition can often be satisfied for arbitrary inputs because the optimizer searches over the full latent space and any point in the typical set is 'likely.' Thus the existence result alone is not separable from the power of inversion. Other weaknesses (five query images for the many-seeds claim, fixed Gaussian prior, no error bars) weaken quantitative generalization but do not threaten the qualitative interpretation as directly. If the proposed no-concept control fails to change the numbers, the paper still offers a useful attack/evaluation protocol, but its central memory/intractability conclusion should be explicitly downgraded to a statement about inversion-based recovery rather than persistent concept memory.","tokens_in":20865,"tokens_out":7231,"duration_ms":86796,"concrete_test":"Run the identical Renoise protocol (Sec. 3.1) on a diffusion model that provably lacks the target concept—e.g., an SD v1.4 model with the concept's textual embedding removed and cross-attention to that token masked, or a same-architecture model trained from scratch on a dataset without the concept (e.g., a class-conditional LDM trained without 'tench'). Invert the same erased-set images using the same captions and compute mean PSNR, d_N, and NLL. If the no-concept control also yields PSNR ≥ 25 dB and d_N within the Fig. 5 range, the paper's memory interpretation is unsupported and the central claim reduces to a statement about inversion. If the control yields clearly lower PSNR and higher d_N, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Abstract; Sec. 3.2, Fig. 5) is that erased concepts persist because diffusion inversion recovers seed latents that are high-likelihood and reconstruct erased-concept images with PSNR roughly 23–34 dB and d_N roughly 0.42–2.49. This inference is only valid if such recoverable seeds are not a generic property of the inversion procedure applied to any diffusion model. The paper's only control is an unablated 'Vanilla' model (Fig. 5), which shares the base weights and still has the concept; it is not a model from which the concept is genuinely absent. A model retrained without the concept (or otherwise guaranteed not to represent it) could plausibly produce the same numbers because diffusion inversion is specifically designed to find a seed for a given image, and the seed prior is the same N(0,I) used by all LDMs. The Limitations section (Sec. 4) partially addresses inversion's power with a scrambled-image control, showing lower NLL and PSNR, but scrambled images are not a positive control for absence of concept knowledge; they only show that arbitrary pixel patterns are harder to invert. Without a no-concept baseline, the conclusion that 'fully erasing concept information may be intractable' is not established: the result may characterize diffusion inversion, not residual memory.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a latent-space evaluation protocol for concept erasure in text-to-image diffusion models. For each erased-concept image, the authors use diffusion inversion (Renoise) to recover a latent seed, then measure the seed's likelihood under the model's Gaussian prior and the reconstruction quality (PSNR) of the generated image. Across nine erasure methods and six concepts, they report mean PSNR values around 23-34 dB and relative distance values d_N between 0.42 and 2.49, which they interpret as evidence that erased concepts remain accessible. They further introduce a Sequential Inversion Block to retrieve multiple distinct, likely seeds for a single query image and conclude that fully erasing concept information may be intractable.","tokens_in":21121,"tokens_out":4598,"duration_ms":42799,"significance":"If the interpretation holds, the paper provides a broadly applicable evaluation tool and a concrete demonstration that current concept erasure methods may be circumvented through latent-space inversion. The breadth of methods and concepts is commendable, and the inversion-based analysis is an original angle compared with prior image-space or text-space evaluations. The main value is empirical: it surfaces a specific attack surface for ablated LDMs. The strength of the central conclusion, however, depends critically on whether the observed reconstructions reflect true persistence of erased concepts or simply the general power of diffusion inversion, and the manuscript currently lacks the control needed to separate these explanations.","major_comments":[{"comment":"The central claim that erased concepts persist rests on the absence of a positive control. The only baseline is the unablated 'Vanilla' model, which shares the base weights and still contains the target concept. A model that genuinely lacks the concept (e.g., retrained on a dataset without the concept) could plausibly yield the same high PSNR and overlapping likelihood values, because diffusion inversion is designed to find a seed for any given image. The scrambled-image control in Sec. 4 shows lower NLL and PSNR, but scrambled images are out-of-distribution, not a no-concept baseline. Without such a control, the inference that erased concepts persist is not established; the result may characterize diffusion inversion rather than residual memory.","section":"Sec. 3.2, Fig. 5"},{"comment":"The likelihood comparison against N(0,I) is likely non-informative. Inversion methods such as Renoise are explicitly designed to produce latents near the typical set of the prior, so overlap between NLL→zT(E), NLL→zT(R), and NLL(N) is expected by construction for any diffusion model, ablated or not. The scrambled-image example in Sec. 4 still yields a seed with NLL 23.89K, which is within the high-likelihood region. The d_N metric therefore does not provide evidence that the erased set is specifically more plausible than the reference set; it may simply reflect the inversion procedure's constraint. A more discriminative test would compare inversion of erased-concept images against inversion of control images on a model that is known to lack the concept.","section":"Eq. (5), Fig. 2"},{"comment":"The paper reports mean PSNR and d_N without error bars, sample counts, or per-image distributions. The abstract claims 'for every image from the erased concept set' seeds can be found, but the experimental section does not state how many images are in each concept set or whether the reported means reflect the worst case. Without such statistics, the strength of the existence claim cannot be assessed. Please provide the number of query images, per-image results (e.g., box plots or scatter plots), and confidence intervals.","section":"Sec. 3.2, Fig. 5"}],"minor_comments":[{"comment":"The histograms and the Gaussian fits are not labeled with sample sizes, and the symbol 'N' is overloaded (normal distribution vs. sample count). Please clarify.","section":"Fig. 2 and Fig. 3"},{"comment":"The phrase 'a value smaller than 1 ... indicates that something is wrong with the model' is vague; please specify what 'wrong' means in the context of the hypothesis.","section":"Sec. 3.1"},{"comment":"References [30] and [31] point to the same paper (Rombach et al.) and are cited separately; please consolidate.","section":"References"},{"comment":"The x-axis label 'Tench' appears with a space in the figure caption; please fix the typo.","section":"Fig. 5"},{"comment":"The choice of 5 renoising steps is justified in the Supplementary, but the main text does not mention sensitivity of the results to this parameter; consider a brief robustness statement.","section":"Supplementary Sec. E"}],"recommendation":"major_revision","confidential_remarks":"The missing positive control is the primary technical issue. The paper's contribution is still valuable if reframed as an audit or attack method rather than a proof that erasure is intractable. The scope is appropriate for a computer vision venue, and the experimental breadth is a strength. The authors should be encouraged to add a control that separates inversion power from residual memory, or to soften the central claim accordingly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper establishes a real vulnerability—current concept erasure methods leave the erased concepts accessible through latent-space inversion—and introduces a relative likelihood metric worth adopting. The strong conclusion that erasure is intractable, however, is not backed up by the right control.\n\nWhat's new: a systematic latent-space inversion analysis across nine erasure methods and six concepts, plus a Sequential Inversion Block that finds multiple distinct seeds for the same image. The reconstruction PSNRs (23–34 dB) and the relative distance values (mostly around 1–2) credibly show that ablated models still contain seeds that generate the erased concept.\n\nThe main soft spot is the missing positive control. The only comparisons are an unablated Vanilla model, which still has the concept, and a scrambled-image condition, which only shows that unnatural images are harder to invert. Neither tells you whether a model that genuinely lacks the concept would also invert to high-likelihood seeds. If it would, the paper's finding is about inversion power, not memory. Since church images are natural images, this is plausible, so 'fully erasing may be intractable' is not established.\n\nSmaller issues: no error bars in the main figures, the multi-seed experiment uses five query images per concept, and the likelihood overlap with N(0,I) is expected because the latent prior is unchanged. These are fixable. The math is straightforward, and the citation pattern is honest—they clearly distinguish their latent-space approach from Pham et al. [26] and Zhang et al. [48].\n\nI would send this to peer review. The evaluation methodology is a contribution, and the control issue is addressable with a no-concept baseline or a careful rewrite of the conclusion. Not a desk reject.","headline":"A real vulnerability in concept erasure, but the 'intractable' conclusion needs a no-concept control to separate memory from inversion power.","tokens_in":21663,"tokens_out":5171,"would_cite":true,"duration_ms":47126,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Concept erasure methods for text-to-image diffusion models do not erase concepts: diffusion inversion recovers high-likelihood latent seeds that regenerate erased images with high fidelity across all nine tested methods.","keywords":["concept erasure","diffusion models","machine unlearning","diffusion inversion","latent space","text-to-image generation","model safety","likelihood analysis"],"falsifier":"Run the same inversion and likelihood protocol on a diffusion model that was retrained from scratch without the target concept in its training data: if that model also yields high-PSNR reconstructions of the concept and a $d_N(E,R)$ near 1, the paper's central inference fails, whereas poor reconstructions or a large $d_N(E,R)$ would confirm that the failure is specific to ablation methods. A second check is to invert random reference images and compare their PSNR distribution to erased images under the same ablated model.","tokens_in":20671,"feed_emoji":"🧠","tokens_out":9326,"duration_ms":76005,"temperature":0.7,"pith_summary":"The paper asks whether concept-erasure methods actually remove a concept from a text-to-image diffusion model, or merely make it harder to summon with text. Its answer, tested on nine published ablation methods and six concepts, is that the concept remains accessible: inverting an erased image back to a seed latent yields a seed that is both likely under the model's Gaussian prior and capable of regenerating the image at high fidelity (mean PSNR around 23 to 34 dB). On the paper's relative-likelihood measure, erased-concept seeds land as close to the normal prior as seeds of ordinary non-erased images. The paper concludes that fully erasing concept information from such models may be intractable, and that current ablation evaluations miss a whole class of latent routes to forbidden content.","feed_headline":"Erased concepts stay reachable in diffusion models","feed_subtitle":"Diffusion inversion finds high-likelihood seeds that regenerate erased images across nine ablation methods.","key_machinery":"The argument runs on three pieces. First, diffusion inversion (Renoise, an iterative noising procedure) maps a query image to a seed latent $z_T$. Second, likelihood is measured as the negative log-likelihood of $z_T$ under the standard normal prior, and the paper's relative distance $d_N(E,R)$ is the ratio of the Earth Mover's Distance between the erased set's NLL distribution and the normal prior to the same EMD for a reference set; this ratio turns raw likelihood values into a unitless criterion where values near 1 mean erased-concept seeds are as plausible as reference seeds, and values below 1 mean they are even more plausible. Third, the Sequential Inversion Block (a decoder inversion that moves an initial latent toward the query image, followed by diffusion inversion) produces many distant high-likelihood seeds for a single erased image.","core_discovery":"The paper's central finding is that, for every image in an erased-concept set and across the nine ablation methods examined, there exists at least one latent seed $z_T$ such that running the ablated model from that seed reconstructs the erased image with high quality, and the seed's negative log-likelihood under the standard normal prior is comparable to seeds of reference images. Across concepts and methods the relative distance $d_N(E,R)$ ranges from 0.42 to 2.49, meaning the erased set is about as likely as the reference set; in several cases it is even closer to the prior than the reference set. The paper also shows that one erased image has many distinct memories: starting from different support images, a sequential inversion procedure produces multiple well-separated seeds, with average pairwise cosine distances of 0.58 and 0.69 in the two concepts shown, that all regenerate the same erased image. The intended conclusion is that ablated models have not forgotten the concept; the concept information persists in latent space.","pith_inferences":["The paper does not include a positive control such as a model retrained from scratch without the concept; until that control is tested, the high-PSNR inversions could reflect a general property of diffusion inversion rather than a specific failure of ablation.","A testable extension would run the same inversion and likelihood protocol on a model trained without the target concept and compare its $d_N(E,R)$ to the ablated models.","The many-memories result suggests that the volume of latent space assigned to a concept is large; future erasure methods could be scored by the volume of reconstructing seeds, not just the likelihood of individual ones."],"forward_implications":["Safety evaluations that only sample text prompts with random seeds undercount the model's ability to produce erased content; a latent-space probe should be part of the evaluation.","The relative-distance metric $d_N(E,R)$ offers a quantitative way to compare erasure methods by how far erased-concept latents sit from the prior, relative to reference images.","Because each erased image has many distant high-likelihood seeds, a successful erasure method must suppress a region of latent space, not a single seed or prompt.","White-box access to a model is enough to regenerate erased content, so erasure guarantees should be assumed limited for open models."],"supporting_citations":[{"why":"Supplies the Renoise inversion procedure used to recover seed latents from query images.","marker":"[9]"},{"why":"Defines the Stable Diffusion v1.4 base model that all ablation methods fine-tune.","marker":"[32]"},{"why":"One of the nine concept-erasure baselines; the paper reproduces its ablated models.","marker":"[7]"},{"why":"Another erasure baseline (EraseDiff) whose ablated models are probed.","marker":"[43]"},{"why":"Supplies the evaluation protocol and dataset collection for erased and reference sets.","marker":"[48]"},{"why":"Supplies the I2P dataset used as the erased set for the Nudity concept.","marker":"[36]"},{"why":"Supplies COCO images used as the reference set of non-erased content.","marker":"[20]"},{"why":"Provides the deterministic DDIM sampling and inversion foundation that the inversion builds on.","marker":"[38]"},{"why":"Defines Earth Mover's Distance used in the relative-distance measure.","marker":"[33]"},{"why":"Provides an alternative null-text inversion used to show the analysis holds beyond Renoise.","marker":"[25]"}],"fun_headline_variants":["Diffusion models keep memories of 'erased' concepts","Ablated concepts regenerate from specific latent seeds","Concept erasure leaves recoverable latent traces","Latent seeds resurrect erased diffusion concepts","Nine ablation methods fail to erase concepts fully"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a model which truly erased a concept would not admit a high-likelihood seed that reconstructs the erased image, a premise the paper invokes when it interprets high reconstruction PSNR at Section 3.2 as evidence of persistence; the study includes no positive control such as a model retrained from scratch without the concept.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion models keep memories of 'erased' concepts","Ablated concepts regenerate from specific latent seeds","Concept erasure leaves recoverable latent traces","Latent seeds resurrect erased diffusion concepts","Nine ablation methods fail to erase concepts fully"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000187,"raw_usage":{"total_tokens":1298,"prompt_tokens":887,"completion_tokens":411,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":503,"completion_tokens_details":{"reasoning_tokens":342}},"tokens_in":503,"tokens_out":411,"duration_ms":4071,"temperature":1.0,"reasoning_tokens":342,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:00:26.394502+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same inversion and likelihood protocol on a diffusion model that was retrained from scratch without the target concept in its training data: if that model also yields high-PSNR reconstructions of the concept and a $d_N(E,R)$ near 1, the paper's central inference fails, whereas poor reconstructions or a large $d_N(E,R)$ would confirm that the failure is specific to ablation methods. A second check is to invert random reference images and compare their PSNR distribution to erased images under the same ablated model.","supporting_citations":[{"cited_title":"High-resolution image synthesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"Defines the Stable Diffusion v1.4 base model that all ablation methods fine-tune."},{"cited_title":"Erasing concepts from diffusion models","cited_arxiv_id":null,"evidence_quote":"One of the nine concept-erasure baselines; the paper reproduces its ablated models."},{"cited_title":"To gener- ate or not? safety-driven unlearned diffusion models are still easy to generate unsafe images","cited_arxiv_id":null,"evidence_quote":"Supplies the evaluation protocol and dataset collection for erased and reference sets."},{"cited_title":"Safe latent diffusion: Mitigating inappro- priate degeneration in diffusion models","cited_arxiv_id":null,"evidence_quote":"Supplies the I2P dataset used as the erased set for the Nudity concept."},{"cited_title":"Microsoft coco: Common objects in context","cited_arxiv_id":null,"evidence_quote":"Supplies COCO images used as the reference set of non-erased content."},{"cited_title":"The earth mover’s distance as a metric for image retrieval","cited_arxiv_id":null,"evidence_quote":"Defines Earth Mover's Distance used in the relative-distance measure."},{"cited_title":"Null-text inversion for editing real im- ages using guided diffusion models","cited_arxiv_id":null,"evidence_quote":"Provides an alternative null-text inversion used to show the analysis holds beyond Renoise."}],"review_version":1}