{"id":"4f2260c3-237f-4e7f-b525-7f00260d3de2","arxiv_id":"2607.06432","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":6,"one_line_summary":"TILDE derives a minimum-deviation, energy-tilted target distribution for concept unlearning in diffusion models and realizes it via residual ∇-GFlowNet training.","lead":"TILDE formulates concept unlearning in diffusion models as a distributional alignment problem, deriving an energy-tilted target that suppresses unwanted concepts while preserving benign generation. It matters because deployed text-to-image systems need to remove copyrighted or unsafe concepts without degrading overall image quality.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"The thresholded CLIP energy is the correct load-bearing concern: without the threshold, retention collapses entirely, and τ is hand-tuned per category with sensitivity tested on only one concept.","rationale":"The reader's verdict of CONDITIONAL is appropriate. The theoretical contribution (Proposition 4, Theorem 1) is sound: the KL-projection derivation is standard and correct, and the residual ∇-GFlowNet machinery is properly adapted from Liu et al. [23]. The optimization caveat (Appendix C.6) — that score-matching loss does not guarantee the integrated ratio form required by Theorem 1 — is a known gap in score-based methods and is honestly disclosed, but it is not the most load-bearing concern for the empirical claims.\n\nThe CLIP threshold concern is more load-bearing because it directly determines whether the target distribution p*_C is well-defined in practice. If CLIP similarity does not separate concept from benign images, the energy in Eq. (3) penalizes benign images, and the 'minimum-deviation' property claimed in D2/D3 fails — the method is no longer preserving benign relative mass but actively distorting it. The paper's own ablation confirms this: without the threshold, RRacc drops from 0.76 to 0.08.\n\nSeveral additional factors keep the verdict at CONDITIONAL rather than ACCEPT: (1) no code or data is released; (2) no error bars or significance tests are reported, and some advantages in Table 1 are modest (e.g., TILDE's object RRacc=0.72 vs. CA's 0.91); (3) ablations (Table 2) and threshold sensitivity (Figure 3) are single-concept only; (4) the FADE evaluation is limited to three style concepts where retain-only references are practical. The per-concept results in Appendix G show meaningful variance — for example, TILDE's RRacc ranges from 0.55 (Brad Pitt) to 0.82 (Banana) — suggesting category-averaged numbers may mask concept-specific failures.\n\nThe paper's framing is genuinely novel and well-motivated: specifying the post-unlearning distribution explicitly via KL projection, rather than leaving it implicit in the update rule, is a clean conceptual contribution. The residual ∇-GFlowNet instantiation is appropriate for the pretrained-times-tilt structure. But the empirical validation needs strengthening — error bars, multi-concept ablations, and an independent concept detector — before the central claim is fully secured.","tokens_in":29884,"tokens_out":3889,"duration_ms":246506,"concrete_test":"For each concept category (objects, characters, styles), compute and plot the CLIP similarity distribution s_C(x) for (a) images generated from forget prompts under the pretrained model and (b) images generated from related-retain prompts under the pretrained model. Report the overlap coefficient (intersection area) between the two distributions at the chosen τ. If the overlap exceeds 15% for any category, the threshold cannot cleanly separate concept from benign images, and the energy tilt is misallocating probability mass for a non-trivial fraction of benign samples. Additionally, re-run the Pikachu unlearning experiment with an independent concept detector (e.g., a fine-tuned binary classifier or a different VLM) replacing CLIP in Eq. (3); if RRacc drops by more than 10 points, the results are contingent on CLIP-specific geometry rather than the distributional alignment principle.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The reader correctly identifies the most load-bearing assumption. The entire method hinges on Eq. (3): the thresholded CLIP energy Ê_C(x) must cleanly partition concept-expressing images from benign ones at the threshold τ. The paper itself demonstrates this is fragile — removing the threshold causes catastrophic retention collapse (RRacc=0.08, GRacc=0.49, Section 5.3 Q4). The threshold is set at the 90–95th percentile of benign CLIP similarities (Appendix E.6), meaning 5–10% of benign images fall above τ and receive energy penalties. For concepts with high semantic overlap with neighbors (e.g., 'cat' vs. other animals, 'Van Gogh' vs. Impressionism), CLIP similarity distributions for concept and benign images likely overlap substantially, making any single threshold a poor separator. The sensitivity analysis (Figure 3) is conducted on Pikachu only, and the per-category thresholds (0.21/0.22/0.20) are close together, suggesting limited tuning range. Additionally, the CLIP model used for the energy (Appendix E.1: openai/clip-vit-large-patch14) is the same model used for the CLIP-score evaluation metrics (Uclip, RRclip, GRclip), creating a shared-embedding-space bias: if CLIP systematically misranks concept evidence in a particular direction, both the training signal and the evaluation metric inherit the same bias. The primary accuracy metrics (UAcc, RRacc, GRacc) use Qwen2.5-VL instead, which mitigates but does not eliminate the concern, since CLIP and VLMs may share conceptual blind spots for ambiguous or compositional concepts. The paper acknowledges this limitation in Section 6 but it underlies every reported result.","agreement_with_reader":"agree"},"referee_report":{"model":"glm-5.2","summary":"The paper proposes TILDE, a concept-unlearning method for text-to-image diffusion models that formulates unlearning as a constrained KL projection of the pretrained conditional distribution onto a forget-energy-tilted target. The target distribution takes the Gibbs form p*_C(x|y) ∝ p_pre(x|y) exp(−βE_C(x))/Z, which suppresses concept-expressing images while preserving benign relative mass. The authors instantiate this target using residual ∇-GFlowNet training (building on Liu et al., ICLR 2025), which learns only the score correction relative to the pretrained model. The forget energy is a thresholded CLIP-based surrogate that applies zero penalty below a similarity threshold τ and an exponential penalty above it. Experiments across objects, characters, and artistic styles show strong forgetting (UAcc 0.97–0.99) with improved retention over baselines (RRacc 0.64–0.76, GRAcc 0.84–0.87), and the method reports favorable FADE and FID scores on style concepts.","tokens_in":30189,"tokens_out":1679,"duration_ms":194900,"significance":"The paper's main conceptual contribution is the explicit formulation of concept unlearning as a distributional alignment problem with a closed-form target, rather than leaving the post-unlearning distribution implicit. The KL projection (Proposition 4, Appendix D.1) is standard but correctly applied, and the connection to GFlowNet proportional sampling is well-motivated. The thresholded energy design is a practical and sensible choice for separating concept-expressing from benign images. The ablation in Table 2 isolates the contributions of the ∇ form, the residual parameterization, and the gϕ correction, which is informative. The paper honestly acknowledges in Appendix C.6 that score-matching loss alone does not guarantee the integrated ratio-form DB condition (Theorem 1), stating only that L=0 implies gradient equality, not full distributional correctness. The evaluation uses VLM-based accuracy metrics (Qwen2.5-VL) alongside CLIP scores and the FADE distributional metric against retain-only references, which is more thorough than many prior works. The method is evaluated across multiple diffusion architectures (Appendix F) and provides per-concept breakdowns (Appendix G).","major_comments":[{"comment":"Section 2.2, Eq. (3), and Figure 3: The thresholded CLIP energy is load-bearing for all results. The paper demonstrates that removing the threshold causes catastrophic retention collapse (RRacc=0.08, GRacc=0.49), and τ is hand-tuned per category (0.21/0.22/0.20 for objects/characters/style). However, the sensitivity analysis (Figure 3) is conducted on Pikachu only. The per-category thresholds are close together, but the paper does not test whether a single threshold generalizes across concepts within a category, nor whether the CLIP similarity distributions for concept vs. benign images overlap substantially for concepts with high semantic overlap (e.g., 'cat' vs. other animals). The paper states (Appendix E.6) that τ is set at the 90–95th percentile of benign CLIP similarities, meaning 5–10% of benign images fall above τ and receive energy penalties. For concepts where the separation is","section":null},{"comment":"Appendix E.1: The same CLIP model (openai/clip-vit-large-patch14) is used for both the forget-energy computation during training and the CLIP-score evaluation metrics (Uclip, RRclip, GRclip). This creates a shared-embedding-space bias: if CLIP systematically misranks concept evidence in a particular direction, both the training signal and the evaluation metric inherit the same bias. The primary accuracy metrics (UAcc, RRacc, GRAcc) use Qwen2.5-VL instead, which mitigates but does not eliminate this concern. The paper should explicitly discuss this potential circularity and consider reporting CLIP-score results with a different CLIP variant, or at minimum acknowledge the limitation more prominently than in Appendix E.1.","section":null},{"comment":"Table 1 and Appendix G: The per-concept results reveal notable variability that the category-level averages partially obscure. For example, on 'Cat' (Table 9), TILDE achieves UAcc=0.96 but RRacc=0.82, while on 'Dog' (Table 10), UAcc=0.94 but RRacc=0.59—a substantial retention gap. On 'Cartoon Style' (Table 8), UAcc drops to 0.82, the lowest among all TILDE results. The paper briefly notes that 'on objects, related-retain remains harder due to tighter semantic coupling' (Section 5.3, Q2), but does not analyze which concept properties predict success or failure. A brief discussion of when the method is expected to struggle would strengthen the paper.","section":null}],"minor_comments":[{"comment":"Section 2.2: The descriptor set Q_C is mentioned with an example for Pikachu, but the paper does not specify how many descriptors are used per concept or how sensitive the method is to the choice of descriptors. Appendix E.3 mentions 'the canonical text descriptor of C' is used unless stated otherwise, but Table 2 and Figure 3 use Pikachu with a 4-descriptor set. Clarify which experiments use single vs. multi-descriptor sets.","section":null},{"comment":"Table 1: The 'Avg. FADE↓' and 'Avg. FID↓' columns are reported only for style concepts. The paper should state this limitation more visibly in the table caption rather than only in the text (Section 5.2).","section":null},{"comment":"Table 1, ESD-x row for Pikachu (Appendix G, Table 14): Uclip=0.0, RRclip=0.0, GRclip=0.0 appears to be a data error or degenerate output. This should be checked.","section":null},{"comment":"Section 3, Eq. (8): The notation ĉ_EC is used in the equation but Ê_C is used in the surrounding text. Standardize the notation.","section":null},{"comment":"Figure 2: The caption references 'hϕ(zt, t, y)' but the figure label uses 'hϕ'. Minor inconsistency.","section":null},{"comment":"Appendix C.5, Proposition 3: The energy is stated as non-differentiable at s_C(x)=τ, with the implementation using the zero subgradient. This is a reasonable practical choice but should be mentioned in the main text (Section 2.2) for completeness, as it affects gradient flow during training.","section":null},{"comment":"References: The paper cites [14] (George et al., ECCV 2026) and [13] (George et al., CVPR 2025), which appear to include the first author of this manuscript. This should be disclosed for transparency.","section":null}],"recommendation":"major_revision","confidential_remarks":"The reader's report and stress-test note correctly identify the thresholded CLIP energy as the most load-bearing concern. I agree this is the right focus. The concern about shared CLIP embedding space for training and evaluation is also valid but partially mitigated by the VLM-based accuracy metrics. The paper is a solid contribution with a clean formulation and honest theoretical caveats, but the limited sensitivity analysis (single concept) and lack of discussion about when the method fails are the main gaps. The per-concept variability (e.g., RRacc=0.59 on Dog) suggests the category-level summary overstates robustness. I recommend major revision primarily to expand the threshold sensitivity analysis beyond Pikachu and to add a per-concept robustness discussion. The theoretical framework is sound; the gaps are empirical."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for a careful and constructive report. The three major comments all identify legitimate gaps: (1) the threshold sensitivity analysis is limited to one concept and does not test single-threshold generalization or high-overlap concepts; (2) the same CLIP model is used for both training and evaluation, creating a potential shared-embedding-space bias; (3) per-concept variability is not analyzed for predictive concept properties. We agree with all three and will revise accordingly. We can partially address comments 1 and 2 with additional experiments and discussion, and fully address comment 3 with added analysis.","responses":[{"response":"The referee is correct that the sensitivity analysis in Figure 3 is limited to Pikachu and that we do not test single-threshold generalization or high-overlap concepts. We will address this in revision as follows. First, we will add threshold-sensitivity sweeps for at least two additional concepts from different categories (e.g., Cat from objects, Van Gogh from styles) to test whether the trade-off curve shape and optimal-tau neighborhood generalize. Second, we will report results using a single shared threshold across all concepts within a category (rather than the slightly different per-category values 0.21/0.22/0.20) to show that the method does not require per-concept tuning. Third, we will add a CLIP-similarity histogram analysis for high-overlap cases such as Cat vs. other animals, showing the degree of separation between concept-expressing and benign images and explicitly discussing cases where overlap is substantial. We acknowledge that for concepts with deep semantic overlap, the thresholded energy may penalize a non-trivial fraction of benign images, and we will state this limitation explicitly. We note that the per-category thresholds are already close together (0.20-0.22), which provides some evidence that a single threshold may suffice, but we agree this must be demonstrated empirically rather than asserted.","revision_made":"partial","referee_comment":"Section 2.2, Eq. (3), Figure 3: Threshold sensitivity analysis conducted on Pikachu only; per-category thresholds close but not tested for single-threshold generalization within a category; no test of CLIP similarity overlap for high-semantic-overlap concepts (e.g., 'cat' vs. other animals); Appendix E.6 notes 5-10% of benign images fall above tau."},{"response":"This is a valid concern. The shared-embedding-space bias means that if CLIP systematically misranks concept evidence, both the training signal and the CLIP-based evaluation metrics inherit the same bias. We will take two steps in revision. First, we will add an explicit discussion of this limitation in the main text (Section 5.2 or the Limitations section), rather than burying it in Appendix E.1. Second, we will report CLIP-score evaluation results using a different CLIP variant (e.g., openai/clip-vit-base-patch32 or laion/CLIP-ViT-H-14) for at least a subset of concepts, to verify that the CLIP-score trends are not artifacts of the shared embedding space. We note that our primary accuracy metrics (UAcc, RRacc, GRAcc) use Qwen2.5-VL, which is architecturally independent of the CLIP model used in training, so the core forgetting/retention claims do not depend on the shared embedding space. However, we agree that the CLIP-score metrics (Uclip, RRclip, GRclip) are potentially affected and that this should be both tested and discussed prominently.","revision_made":"partial","referee_comment":"Appendix E.1: Same CLIP model (openai/clip-vit-large-patch14) used for both forget-energy computation during training and CLIP-score evaluation metrics (Uclip, RRclip, GRclip), creating shared-embedding-space bias. Primary accuracy metrics use Qwen2.5-VL, which mitigates but does not eliminate the concern. Paper should discuss this circularity and consider reporting CLIP-score results with a different CLIP variant."},{"response":"We agree that the per-concept variability deserves explicit analysis. The referee's examples are apt: Dog has substantially lower RRacc (0.59) than Cat (0.82), and Cartoon Style has the lowest UAcc (0.82) among all TILDE results. We will add a discussion paragraph analyzing the concept properties that predict difficulty. Our interpretation, supported by the per-concept tables, is that retention suffers most when the forget concept is semantically coupled to many neighboring concepts in CLIP space (e.g., 'Dog' overlaps with many animal-related prompts, so the thresholded energy may penalize some benign animal images), and forgetting is harder when the concept is visually diffuse or stylistic rather than a concrete object (e.g., 'Cartoon Style' is a broad visual category without a single canonical referent, making the CLIP-based energy less discriminative). We will also note that for 'Dog' specifically, the low RRacc is partly because related-retain concepts for Dog include other animals whose CLIP similarity to 'Dog' descriptors may exceed tau, causing collateral suppression. This analysis will be added to Section 5.3 and connected to the Limitations section's discussion of energy quality.","revision_made":"yes","referee_comment":"Table 1 and Appendix G: Per-concept results reveal notable variability obscured by category-level averages (e.g., Cat: UAcc=0.96, RRacc=0.82 vs. Dog: UAcc=0.94, RRacc=0.59; Cartoon Style: UAcc=0.82, lowest among all TILDE results). Paper does not analyze which concept properties predict success or failure. A brief discussion of when the method is expected to struggle would strengthen the paper."}],"tokens_in":29969,"tokens_out":1230,"duration_ms":253359,"standing_objections":[]},"desk_editor":{"model":"glm-5.2","letter":"Short version: the KL-projection formulation (Proposition 4) is genuinely new and clean — it makes the post-unlearning target distribution explicit rather than leaving it implicit in the update rule. The residual ∇-GFlowNet instantiation is well-motivated and the main results place TILDE on the Pareto frontier. The soft spot is real but bounded: the thresholded CLIP energy is load-bearing, and the paper itself shows what happens when you remove it (RRacc collapses to 0.08). This deserves a serious referee who can push on the empirical gaps.","headline":"Clean KL-projection formulation of concept unlearning; empirical validation needs strengthening but the core idea is sound","tokens_in":30779,"tokens_out":181,"would_cite":true,"duration_ms":47651,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"Energy-tilted distributions remove concepts from image models without collateral damage","keywords":[],"falsifier":"Find a concept where CLIP similarity cannot cleanly separate concept-expressing from benign images (e.g., a highly compositional or abstract concept), and show that no single threshold τ simultaneously achieves high unlearning accuracy and acceptable retention — or that the threshold must be individually tuned per concept with no stable within-category value.","tokens_in":29920,"feed_emoji":"","tokens_out":1078,"duration_ms":72992,"temperature":0.7,"pith_summary":"TILDE argues that concept unlearning in text-to-image diffusion models should be defined by specifying the target post-unlearning distribution before choosing an update rule. The authors derive this target as a constrained KL projection: among all conditional distributions that suppress the unwanted concept, pick the one closest to the pretrained model's distribution. The solution has a clean Gibbs form — the pretrained distribution multiplied by an exponential energy penalty on concept-expressing images, normalized per prompt. This means concept images lose probability mass while benign images keep their pretrained relative proportions, avoiding both the collateral damage of hard suppression methods and the diversity collapse of reward-maximization methods. The authors realize this target in a diffusion model using residual ∇-GFlowNet training, which learns only the score correction (the deviation from the pretrained denoiser) needed to realize the energy-tilted distribution, rather than training a new sampler from scratch. A thresholded CLIP-based forget energy creates a no-penalty zone for benign images below the threshold, so they receive zero gradient and stay anchored to pretrained behavior.","feed_headline":"Forget concepts from image models by tilting distributions, not suppressing them","feed_subtitle":"TILDE derives the minimum-deviation target distribution for concept unlearning and realizes it with residual GFlowNet training, achieving 97","key_machinery":"Constrained KL projection yielding a Gibbs-form tilted target; thresholded CLIP-based forget energy with a no-penalty zone; residual ∇-GFlowNet detailed-balance score matching in diffusion latent space; forward-looking residual parameterization of the continuation score","core_discovery":"The paper's central object is the energy-tilted target distribution p*_C(x|y) ∝ p_pre(x|y) exp(−βE_C(x))/Z, derived as the unique solution to a constrained KL projection that minimizes deviation from the pretrained model subject to a forgetting constraint. The key mechanistic finding is that this target can be realized in diffusion latent space via residual ∇-GFlowNet training: the pretrained model supplies the base distribution, and the learner estimates only the score correction induced by the forget-energy tilt. The thresholded forget energy is load-bearing — without the threshold, every sample receives a penalty and retention collapses (RRacc drops to 0.08), while with it, benign images'","pith_inferences":[],"forward_implications":["If the distributional alignment formulation is correct, then existing unlearning methods can be classified by where they deviate from it: hard suppression corresponds to an extreme tilt, anchor-based methods substitute a manually chosen target distribution, and reward maximization drops the proportional-sampling property that preserves diversity.","The thresholded energy design implies that retain prompts can be mixed into training as ordinary samples with no extra loss term, since below-threshold samples receive zero concept-suppression gradient — a free Pareto improvement when retain supervision is available.","The residual ∇-GFlowNet machinery is energy-agnostic: replacing the CLIP-based forget energy with a stronger concept detector (VLM ensemble, task-specific classifier) would transfer without modification, extending the approach to ambiguous or compositional concepts where CLIP similarity is unreliable.","The framework predicts that distributional fidelity should be measured locally (per-prompt) rather than via coarse aggregate metrics like FID, because collateral damage is easiest to hide in semantically adjacent prompts — motivating metrics like FADE that compare against a retain-only reference."],"fun_headline_variants":["TILDE reframes concept unlearning as distributional alignment, not just erasure","Minimum-deviation unlearning: suppress concepts while keeping benign generation intact","Energy-tilted distributions let diffusion models forget without collapsing retention","Residual GFlowNets learn score corrections for concept erasure in diffusion models","TILDE derives the optimal post-unlearning distribution and trains a GFlowNet to match it"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The method depends on a thresholded CLIP similarity score to cleanly separate concept-expressing images from benign ones. The threshold τ is hand-tuned per category (e.g., 0.21 for objects, 0.22 for characters), and removing the threshold causes retention to collapse catastrophically. If CLIP similarity does not cleanly separate concept from non-concept images for a given concept, the energy tilt misallocates probability mass and the entire approach fails.","fun_headline_variants_meta":{"raw":{"variants":["TILDE reframes concept unlearning as distributional alignment, not just erasure","Minimum-deviation unlearning: suppress concepts while keeping benign generation intact","Energy-tilted distributions let diffusion models forget without collapsing retention","Residual GFlowNets learn score corrections for concept erasure in diffusion models","TILDE derives the optimal post-unlearning distribution and trains a GFlowNet to match it","Forgetting as KL projection: minimize deviation from the pretrained model under constraints","TILDE unlearns concepts by tilting toward a retain-only distributional target","Thresholded forget energy is essential for retention in distributional unlearning"]},"model":"glm-5.2","effort":"high","cost_usd":0.0,"raw_usage":{"total_tokens":761,"prompt_tokens":575,"completion_tokens":186,"prompt_tokens_details":null},"tokens_in":575,"tokens_out":186,"duration_ms":11131,"temperature":1.0,"reasoning_tokens":35,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-08T05:50:58.341358+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"Find a concept where CLIP similarity cannot cleanly separate concept-expressing from benign images (e.g., a highly compositional or abstract concept), and show that no single threshold τ simultaneously achieves high unlearning accuracy and acceptable retention — or that the threshold must be individually tuned per concept with no stable within-category value.","supporting_citations":[],"review_version":1}