{"id":"1512af7c-78c6-41cc-a64a-917131b3f10e","arxiv_id":"2608.12829","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A single steering vector extracted from the middle text layer of a multimodal diffusion transformer can erase concepts like nudity, celebrities, and art styles at inference time, with no training.","lead":"This paper presents a way to stop AI image generators from producing unwanted content, such as nudity, celebrities, or specific art styles, without retraining the model. It works by extracting an internal steering direction from the model's middle layers and nudging the generation away from the unwanted concept at every step.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The t=0.5 steering vector is injected at all timesteps, but the paper never verifies that the direction is timestep-invariant; rectified flow's straightness is a property of latents, not hidden text-branch activations.","rationale":"The paper is genuinely interesting: the block-role analysis via noise injection and attention maps is reasonable, and the method's quantitative results, though lacking error bars and code, span three categories, two models, and adversarial attacks. I am not raising a fabrication concern or even necessarily a fatal flaw; the issue is an unverified load-bearing premise. The central mechanism is singular: one vector, one construction time, injection everywhere. All other assumptions, such as Eq. (4) token-summing, prompt-pair curation, and hyperparameter choices, could in principle be adjusted with more data or tuning, but if the steering direction is not transferable across timesteps, the 'single vector' framework itself breaks. The paper's evidence for temporal transfer is indirect: Table 2 tests construction time but not per-step injection, and Figure 5 shows images from different construction timesteps but all generated with all-timestep injection. No measurement of feature trajectories or per-step contributions is provided. The proposed check is inexpensive and directly targets this premise. Because the issue is testable and the rest of the paper stands or falls on it, the reader's CONDITIONAL verdict is appropriate; I would not move to ACCEPT or REJECT without the check.","tokens_in":15116,"tokens_out":5601,"duration_ms":58760,"concrete_test":"On SDv3.5-medium, for one concept (e.g., Taylor Swift) with fixed paired prompts, construct steering vectors at t=0.1, 0.3, 0.5, 0.7, 0.9 using the same initial noise. Measure (i) pairwise cosine similarity between these vectors, and (ii) GIPHY/LLaVA erasure and Aesthetic scores under three injection policies: A—fixed t=0.5 vector at all timesteps (paper's method); B—vector constructed at current timestep injected only at that timestep; C—fixed vector injected only over the central window t∈[0.3,0.7]. If B materially outperforms A, or C matches A, the fixed-vector-at-all-timesteps assumption is unsupported. Report cosine similarities; values below roughly 0.3 would show the direction is not stable, and the rectified-flow argument would not rescue stability.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that one steering vector—computed at a single intermediate timestep (t=0.5) and block (b=10) from text-branch activations—can be injected at every denoising timestep and into blocks 3–10 (§3.2–§3.3). The stated justification is that rectified flow follows a straight, deterministic trajectory. That guarantees straight paths in latent sample space, not that text-branch activation differences are timestep-invariant. Section 3.3 asserts that 'features evolve in a more coherent and predictable manner' without measuring any feature trajectory. Table 2 only compares vectors constructed at early vs. intermediate vs. late timesteps, and all variants are still injected at all timesteps; it does not test whether a t=0.5 vector is the right direction at t≈0.1 or t≈0.9. If the appropriate semantic direction rotates or scales across timesteps, the reported erasure could be driven mainly by mid-range steps, with early/late injections partly neutral or harmful. The method's headline efficiency and coherence depend on this single-vector transfer; without a per-step decomposition, the central mechanism is unverified. The same transferability question applies across prompt contexts: Eq. (4) averages over n paired prompts and sums only tokens that differ, so syntactic/contextual confounds are not ruled out, but the temporal transfer issue is more fundamental because even a perfect direction fails if it is not valid at the injected timestep.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a training-free concept erasure method for multimodal diffusion transformers (MM-DiTs). The method constructs a steering vector from the difference between text-branch activations of an unwanted-concept prompt and a safe-concept prompt, extracted at an intermediate block and an intermediate denoising timestep, and injects this single vector into a consecutive set of early-to-middle blocks at every denoising timestep. Experiments on Stable Diffusion v3.5 and FLUX.1 cover three concept categories (celebrity, art style, nudity), compare against tuning-free and fine-tuning baselines, and include ablations on construction block, construction timestep, injection block combinations, steering strength, multi-style steering, and adversarial robustness.","tokens_in":15391,"tokens_out":2889,"duration_ms":31074,"significance":"If the claims hold, the paper offers a practical, low-overhead intervention for safety-oriented concept erasure in modern MM-DiTs, with a useful capability of steering toward a specified safe concept rather than merely suppressing the unwanted one. The authors provide extensive qualitative ablations, a multi-style steering demonstration, and adversarial robustness results, and they explicitly target components of the architecture (text-branch sparsity, middle-block semantics, rectified-flow sampling) that are plausible and well motivated. The work is a reasonable fit for the multimedia community and could be a useful baseline for future activation-steering research. However, the central temporal-transfer mechanism is not directly verified, and the quantitative comparisons rest on single-run numbers with hyperparameters selected on the same metrics.","major_comments":[{"comment":"The claim that a single steering vector constructed at t=0.5 can be injected at all denoising timesteps is justified only by the straightness of rectified flow latent trajectories, but that property concerns latent sample paths, not text-branch activation differences. Table 2 varies the construction timestep while still injecting the resulting vector at all timesteps, so it does not establish that the t=0.5 direction is appropriate at t=0.1 or t=0.9. I request either a per-timestep injection ablation (e.g., injecting only into early, middle, or late timestep windows) or a direct measurement of the consistency of the steering direction across timesteps (e.g., cosine similarity between vectors computed at different t), because the method's stated efficiency and coherence depend on this transfer.","section":"§3.3, Table 2"},{"comment":"Table 1 reports each metric for a single run or a single prompt set, with no error bars or significance tests. Since the steering strength l, construction block b, injection block range, and construction timestep t are selected using ablations that report the same erasure and aesthetic metrics as Table 1, the headline comparisons to baselines are at risk of selection bias. Please report results over multiple seeds, state how the test prompts and hyperparameter-selection data were separated, and confirm that the final Table 1 numbers are obtained on a held-out set.","section":"§4.2, Table 1"},{"comment":"The number n of paired prompts used to construct the steering vector is never stated, and no analysis is provided of the variability of the per-pair differences (x+_{b,i} - x-_{b,i}). If the summed token differences are dominated by syntactic or contextual artifacts of the paired prompts rather than by the target concept's semantics, the resulting vector may not transfer across prompt contexts. Please report n, the per-pair direction consistency (e.g., average pairwise cosine similarity), and an experiment varying n to show that the vector saturates as n grows.","section":"Eq. (4), §3.2"}],"minor_comments":[{"comment":"The sensitivity analysis heading contains a typo: 'ł' should be 'l' in both the heading and the surrounding text, and the variable should be consistently typeset as l or \\ell.","section":"§4.3"},{"comment":"The experimental setup does not state how many images are generated per prompt, how many random seeds are used, or what sampling steps (e.g., 50, 28) are employed; these details are needed to interpret the FID and CLIP numbers and to reproduce the results.","section":"§4.1"},{"comment":"The caption of Figure 3 would be clearer if it specified the scale of the injected Gaussian noise and which block outputs are modified, since those choices affect the qualitative conclusions about block roles.","section":"Figure 3"},{"comment":"Some references are incomplete or inconsistent with archival style: [22] points to a general URL without a version, and [9] mixes ICML formatting with a different style; please check all entries for completeness.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper has a promising and well-scoped idea, but the core mechanism of timestep transfer is currently asserted rather than demonstrated, and the quantitative comparisons lack variance estimates. The authors should be asked to supply the additional experiments described in the major comments; if those experiments support the claims, the paper is likely suitable for acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nShort version: this is a plausible tuning-free concept erasure method for MM-DiTs, built on a sensible block-role observation, but the evidence as reported is not as strong as the abstract suggests. The genuinely new bit is using sparse text-branch tokens from a middle block to construct a single steering vector and injecting it into consecutive early-middle blocks across all denoising timesteps. That is a real engineering contribution over LLM-style steering and UNet-specific methods, and the qualitative results on SDv3.5 and FLUX.1 look believable.\n\nWhat the paper does well: the block-role analysis (Gaussian noise injection plus attention-map visualization) is a reasonable way to locate where concepts live; focusing on text tokens rather than 4096 image tokens is sensible; and the method is genuinely training-free and cheap. The ablations for block choice, timestep choice, and injection range are the right ablations to run, and the sensitivity analysis for l shows plateaus, which is reassuring.\n\nNow the reservations, in rough order of seriousness.\n\nFirst, the timestep-invariance claim is not actually tested. Rectified flow gives straight paths for latents; it does not imply text-branch activation differences are constant across t. The paper says features evolve 'coherently' and cites the straight trajectory, but Table 2 only varies construction timestep, always injecting at all steps. That does not measure whether a t=0.5 vector is appropriate at t=0.1 or t=0.9. If the direction rotates or scales, erasure could be driven by mid-range steps, and early/late injections could be neutral or harmful. This is the load-bearing assumption and it remains unverified.\n\nSecond, the evaluation has selection-on-the-metrics issues. The steering strength, construction block/timestep, and injection block range are chosen via ablations on the same erasure and aesthetic metrics reported in Table 1. With single runs and no error bars, the headline numbers are likely optimistic. Also, FLUX-specific hyperparameters (blocks, l, t) are never stated; the sensitivity analysis is only on SDv3.5.\n\nThird, the steering vector construction is underspecified. n in Eq. (4) is not given, the paired-prompt curation is described as 'carefully curated' without examples or counts, and the assumption that summing only differing tokens isolates concept semantics in T5-XXL is asserted, not tested. If the averaged difference is dominated by syntactic artifacts or is not linearly transferable across contexts, the method would degrade.\n\nNone of this kills the method, but it does mean the central causal story is not yet proven. For people working on generative-model safety or activation engineering, this is a useful read and a reasonable baseline. The paper deserves a serious referee; I would send it out. I would ask for code/data, repeated runs with error bars, a per-timestep analysis of the steering direction, and FLUX hyperparameters before relying on the numbers.","headline":"Plausible and cheap tuning-free erasure for MM-DiTs, but the timestep-transfer assumption is unverified and the evaluation is tuned on the same metrics.","tokens_in":15966,"tokens_out":2532,"would_cite":false,"duration_ms":24780,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a single steering vector, computed from the difference between unwanted and safe text representations in the middle block of a multimodal diffusion transformer, can erase a target concept from generated images…","keywords":["concept erasure","multimodal diffusion transformers","steering vectors","inference-time intervention","controllable generation","rectified flow","text-to-image generation","model safety"],"falsifier":"Take paired prompts with the target concept placed in different sentence positions and syntactic roles, construct the steering vector from one subset, then measure erasure on held-out prompts; if erasure collapses when the context changes, the vector is encoding context-specific artifacts rather than concept semantics.","tokens_in":14881,"feed_emoji":"🎨","tokens_out":6114,"duration_ms":58719,"temperature":0.7,"pith_summary":"Multimodal diffusion transformers, the architecture behind many recent text-to-image models, can be made to stop generating unwanted content—celebrities, art styles, nudity—without any fine-tuning. The paper claims that the target concept's semantics are concentrated in the middle transformer blocks, and that a single vector computed from the difference between an unwanted prompt and a safe prompt in the text branch can push generation away from the concept when added to early and middle blocks at every denoising step. Because the vector lives on the sparse text tokens and rectified flow sampling follows a straight trajectory, one vector constructed at one timestep can steer all steps with negligible overhead. If correct, this gives deployed models an inexpensive safety switch that also permits controlled redirection, such as turning one art style into another.","feed_headline":"Steering vectors erase concepts with no retraining of image models","feed_subtitle":"No training needed: steering vectors remove celebrity, style, and nudity while preserving image quality.","key_machinery":"The load-bearing object is the middle-block text-branch steering vector, defined in Eq. (4): $v_b = \\frac{1}{n}\\sum_i (x^+_{b,i} - x^-_{b,i})$, the average difference of summed text-token outputs for safe versus unwanted paired prompts, then normalized and scaled. It works because the text branch is sparse (154 or 333 tokens versus 4096 image tokens), because middle-block text activations are where concept semantics are most salient, and because rectified flow gives a straight, deterministic sampling path so a direction measured at one intermediate timestep remains coherent when added at all timesteps. Injection into consecutive early and middle blocks (in the paper, blocks 3 through 10) lets the bias accumulate without the distortion that early-only injection causes or the ineffectiveness of middle- or late-only injection.","core_discovery":"The central claim is that target-concept semantics in MM-DiTs are localized: early blocks settle global layout, late blocks add fine detail, and middle blocks carry the primary conceptual content, as supported by block-wise noise injection and attention-map visualization. Taking this as a premise, the paper constructs a steering vector $v^{\\mathrm{steer}}_b$ at a middle block $b$ by feeding paired prompts that differ only in the target phrase, summing the text-branch output tokens that differ, averaging the difference over $n$ pairs, normalizing, and scaling by a strength $l$. Injecting this single vector into the output text tokens of several consecutive early and middle blocks at every denoising timestep shifts the unwanted concept toward the chosen safe one. The paper reports state-of-the-art erasure of celebrities, art styles, and nudity on Stable Diffusion-v3.5-medium and FLUX.1 while keeping aesthetic quality and unrelated-image fidelity, with controllable multi-style steering and strong robustness to adversarial prompts.","pith_inferences":["The same construction could be repurposed beyond safety: choosing contrastive pairs for any attribute, such as hairstyle, lighting, era, or object, would yield a single-vector editor, provided that attribute is semantically localized in middle blocks as the paper shows for the tested concepts.","Because the steering vector is built in the text branch and injected additively, multiple concept vectors might be composable or portable across MM-DiTs that share the text encoder; the paper does not test either possibility.","A practical limit the paper does not address: an additive, publicly known steering direction can in principle be inverted and subtracted, so the method is a content filter rather than an unlearning guarantee against a determined adversary."],"forward_implications":["A single forward pass per concept builds the steering vector; erasure then costs only vector additions at inference, making it practical for deployed models.","Erasure is controllable: the user chooses the safe concept, so the same target can be redirected to different outputs, such as turning Van Gogh style into pixel art, watercolor, or ink wash.","Because steering acts on internal representations rather than prompt keywords, it resists adversarial prompts that rename or paraphrase the target concept.","The same vector applied across all denoising steps keeps steering temporally coherent, so image layout and quality are largely preserved.","The construction is conceptually transferable across MM-DiT models, as demonstrated on two different architectures."],"supporting_citations":[{"why":"Supplies the MM-DiT architecture with joint text-image attention that the method intervenes on.","marker":"[9]"},{"why":"The second model family on which the method is evaluated.","marker":"[22]"},{"why":"Provides the rectified-flow straight sampling trajectory that justifies applying one vector across all timesteps.","marker":"[27, 29]"},{"why":"Defines the diffusion transformer backbone underlying the MM-DiT block structure.","marker":"[33]"},{"why":"The sentence-level text encoder whose contextual token representations the steering vector is built from.","marker":"[35]"},{"why":"Prior activation-steering work that motivates extracting semantic directions from internal representation differences.","marker":"[7, 45]"},{"why":"Supports the choice of intermediate timesteps for balanced semantic extraction.","marker":"[31]"}],"fun_headline_variants":["One vector erases concepts in diffusion models","Tuning-free steering removes concepts from images","Steering vectors erase concepts in multimodal diffusion","No-retrain concept erasure via middle-block steering","Semantic steering deletes unwanted content without training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that summing the text-branch token differences between an unwanted and a safe prompt isolates the target concept's semantic direction, and that one scaled version of that direction remains correct for every prompt context and every denoising step.","fun_headline_variants_meta":{"raw":{"variants":["One vector erases concepts in diffusion models","Tuning-free steering removes concepts from images","Steering vectors erase concepts in multimodal diffusion","No-retrain concept erasure via middle-block steering","Semantic steering deletes unwanted content without training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000617,"raw_usage":{"total_tokens":2900,"prompt_tokens":1016,"completion_tokens":1884,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":632,"completion_tokens_details":{"reasoning_tokens":1815}},"tokens_in":632,"tokens_out":1884,"duration_ms":14405,"temperature":1.0,"reasoning_tokens":1815,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:26:14.872625+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take paired prompts with the target concept placed in different sentence positions and syntactic roles, construct the steering vector from one subset, then measure erasure on held-out prompts; if erasure collapses when the context changes, the vector is encoding context-specific artifacts rather than concept semantics.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The second model family on which the method is evaluated."}],"review_version":1}