{"id":"7564c61d-28d5-4731-a6eb-7faf774d32e3","arxiv_id":"2412.18302","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"FameBias linearly combines a famous person's embedding with a trigger word's embedding to make text-to-image models generate that person, reaching 53% bias success without training.","lead":"This paper introduces FameBias, an attack that blends the text embedding of a target public figure with the embedding of a trigger word so that a text-to-image model depicts that figure in otherwise generic scenes. It requires no model fine-tuning and the authors report a 53% bias success rate on Stable Diffusion 2 using an automated vision-language judge.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central mechanism is under-specified: CLIP has no per-word output embedding, so Eq. (1) cannot be applied to a trigger word inside a sentence as written; a reproduction requires a precise implementation choice.","rationale":"The reader's weakest assumption identifies exactly the same load-bearing concern: Eq. (1) in Section 3.2 requires isolating a trigger word's embedding inside CLIP's contextualized sentence encoding, and the paper does not say how this is done. I agree that this is the most consequential gap. The paper's other weaknesses are real but secondary: alpha and beta were selected on the same evaluation set used for the headline numbers, the eight targets were pre-filtered to those SD2 can already render, and LLaVa's agreement with human raters is imperfect. Those issues would inflate or distort the reported success rates, but they do not make the attack's construction undefined. The embedding-manipulation gap does: if the mechanism is not specified, the experimental results cannot be interpreted as validating any particular attack. I would keep the reader's CONDITIONAL verdict rather than moving to ACCEPT or REJECT, because the idea is plausible and could be validated by releasing code or precisely specifying the token-level operation. The requested reproduction is feasible and would settle the concern directly. I do not see evidence of a deeper mathematical inconsistency; the problem is specification and reproducibility, which a conditional acceptance with a request for code and implementation details addresses appropriately.","tokens_in":10320,"tokens_out":2804,"duration_ms":29989,"concrete_test":"Obtain or reconstruct the exact implementation from the authors, then run a minimal reproduction on a fixed subset of the paper's trigger-target pairs: (A) replace only the trigger token's input embedding inside the CLIP text encoder before the Transformer, using er = alpha * e_target + beta * e_trigger, and (B) replace the final pooled sentence embedding of the full prompt with the same linear blend. Generate the same prompts with both variants and compare BSR and TFR. If implementation (A) is the one used and reproduces roughly BSR=53% and TFR=65%, the concern is resolved. If the paper's text describes neither variant or if neither variant reproduces the results, the central mechanism remains undefined.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that blending the CLIP embedding of a trigger word with a target figure's embedding, er = alpha * e_wp + beta * e_wt, biases Stable Diffusion V2 without fine-tuning. Section 3.2 defines e_wt as 'the CLIP text embedding of the trigger word' and states that the adversary modifies this embedding before it reaches the diffusion model. But CLIP's text encoder is a Transformer that outputs a contextualized representation for the full prompt, typically the final [EOS] token representation; it does not expose an isolated, non-contextual embedding for one word inside a multi-word prompt. The paper never specifies whether the manipulation is applied (A) to the token embedding of the trigger at the input to the CLIP Transformer, (B) to the per-token output hidden states after contextualization, or (C) to the final pooled sentence embedding. These choices are not equivalent. If (A), the resulting sentence embedding is not the simple linear blend in Eq. (1) because attention will re-contextualize the modified token with the surrounding words. If (C), there is no separate e_wt to blend, only the whole-prompt embedding, so Eq. (1) is undefined. Since the threat model explicitly grants attackers control of encoder outputs but not of the diffusion model, the entire attack hinges on this unspecified operation. Without a precise, implementable definition of how e_wt is isolated and reinserted, the reported BSR=53% and TFR=65% cannot be reproduced or even checked, and the headline claim that the attack works 'solely on input embedding vectors' is not yet supported. Secondary concerns, such as alpha and beta being tuned on the evaluation set and the LLaVa metric being imperfect, are acknowledged or fixable; the embedding-manipulation gap is the load-bearing one.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces FameBias, an embedding-manipulation attack on text-to-image models. The proposed method linearly combines the CLIP text embedding of a target public figure with that of a trigger word, er = α·e_p + β·e_wt, and replaces the trigger's embedding before it reaches the diffusion model, aiming to make Stable Diffusion V2 generate images of the figure while preserving the trigger's semantic context. The authors evaluate on 8 public figures and 10 job-related triggers, using LLaVA yes/no answers to compute Bias Success Rate (BSR) and Trigger Fidelity Rate (TFR), reporting overall BSR=53% and TFR=65%. They also test UCE concept erasing as a defense and conduct a small human-consistency check. The paper is framed as a preliminary study, with the authors noting that more extensive evaluations are planned.","tokens_in":10542,"tokens_out":7139,"duration_ms":59745,"significance":"If the reported results were validated, the attack would be notable because it requires no fine-tuning of the diffusion model and only manipulates input embeddings. The paper has several strengths: the threat model is clearly stated; the evaluation spans multiple targets, triggers, and prompt templates; and the authors include a human-consistency analysis in Section 5, which is more than many attack papers do. However, the central implementation is under-specified and the reported numbers are in-sample, so the current significance is limited.","major_comments":[{"comment":"The attack is not implementable as written because CLIP's text encoder produces a contextualized representation for the entire prompt and does not expose an isolated embedding for a single word in a multi-word prompt. The paper does not specify whether the blend er = α·e_wp + β·e_wt is applied to the input token embedding of the trigger, to the contextualized per-token hidden state at the trigger's position, or to the pooled sentence embedding; these choices are not equivalent, and for the pooled representation a separate e_wt does not exist. Since the entire method builds on replacing e_wt, this missing implementation detail prevents reproduction and independent verification of the attack.","section":"Section 3.2, Eq. (1)"},{"comment":"The headline success rates are in-sample numbers, not predictive results. The target figures were selected in part because the unmodified SD-v2 model could already generate them clearly (Section 4.1), and the hyperparameters α=1.5 and β=0.3 were chosen by maximizing AII=BSR·TFR on the same evaluation set used to report the overall BSR=53% and TFR=65% (Section 4.3.1). No validation or test split is used, so the reported figures are fitted results. In addition, the parameter search is only two one-dimensional sweeps with the other parameter fixed at an arbitrary value, so the claimed optimality is not established even in-sample.","section":"Sections 4.1, 4.3.1, 4.2"},{"comment":"The paper's own human-evaluation analysis shows that LLaVA is not a reliable judge for several targets: it answered 'NO' for all images in a set that human raters unanimously recognized as Shakira, and for some figures its agreement with human raters is lower than inter-human agreement and sometimes lower than chance. Since all BSR and TFR numbers are computed solely from LLaVA yes/no answers, the reported attack success rates are not a validated measure of whether the target figure appears. The authors should report metrics on a human-validated subset, or validate LLaVA's agreement per target before using it as the sole evaluation instrument.","section":"Section 5 and Sections 4.1-4.2"},{"comment":"The attack's effectiveness is not compared against a baseline. Because the targets were chosen for being clearly generable by the unmodified model, the reported BSR may partly reflect the model's prior tendency to depict these figures for generic 'photo of a {job}' prompts rather than the effect of the embedding manipulation. The paper should report BSR and TFR on the unmodified SD-v2 for the same prompts, and ideally also for a control perturbation, to isolate the contribution of Eq. (1).","section":"Section 4.2 and Section 4.1"}],"minor_comments":[{"comment":"The name of the attack is inconsistent: 'FameBias', 'FAME BIAS', and 'F AME BIAS' are used interchangeably.","section":"Throughout"},{"comment":"The worked example is internally inconsistent: the text mentions a 'savior' prompt and 'modify the original doctor embedding', and Eq. (1) mixes e_{Trump} with e_{savior}; please align the example.","section":"Section 3.2"},{"comment":"The prompt count is confusing: the text says 'we generate four images using the prompts ...' and then lists three prompt templates; clarify that four images are generated per template, yielding 320 images per template.","section":"Section 4.1"},{"comment":"The term 'trigger' is overloaded: in the alternative-trigger experiment it refers to an object (e.g., 'stethoscope'), not the person-noun used in the main experiments; consider using 'object trigger' or 'tool trigger'.","section":"Section 4.3.2"},{"comment":"The description of Figure 6b, 'adds an additional direction of men → women', is vague; specify how the direction is computed and added to the embedding.","section":"Section 5"},{"comment":"Reference [1] contains a typo ('ac sces sible') and should be checked; reference [12] should include the year and venue if available.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"This is a preliminary preprint (v1 dated Dec 2024) and the authors state that 'more extensive evaluations are planned.' For a journal venue, the evaluation protocol needs to be substantially strengthened: a precise implementation of the embedding manipulation, a held-out validation split, and per-target validation of the evaluation model are all necessary before the headline claims can be supported. The relationship to the authors' prior work [12] should also be clarified in terms of the novel contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: the headline numbers (BSR 53%, TFR 65%) are not backed by a reproducible method. The core operation in Eq. (1) is undefined for a single word inside a CLIP-encoded sentence. CLIP's text encoder returns a pooled representation of the whole prompt, not per-token output embeddings. The paper never says whether the attacker edits the input token embedding, a hidden state, or the final sentence embedding. Those are not equivalent. If they edit the input token, attention re-contextualizes it; if they edit the final embedding, there is no separate e_wt to blend. The stress-test note is right, and the reader's skepticism is warranted.\n\nWhat's genuinely useful: the threat model is timely—no-fine-tuning biasing of T2I models is worth studying. The paper is clearly written and unusually honest. It acknowledges LLaVa's inconsistency with human raters, runs a small human evaluation with Fleiss/Cohen kappas, tests a defense (UCE), and tries alternative triggers. Those are real strengths. The linear-arithmetic idea is old (word2vec era), but applying it as a training-free attack on public figures is a small novel step.\n\nThe soft spots beyond the central gap: alpha and beta were selected by maximizing BSR*TFR on the same evaluation set used for the headline results, so the reported numbers are fitted, not predictive. Targets were chosen because SD2 could already generate them clearly—selection on the outcome. Each table cell averages four images, so cells are noisy. And LLaVa agreement is poor for some targets (Shakira, Castro), which they admit.\n\nIf the authors can specify the embedding operation precisely and release code, this could be a decent short paper. As it stands, the method cannot be reproduced or checked. The reader's conditional verdict is fair, maybe generous on novelty but right on soundness. I'd send it to review only if the authors first clarify the mechanism; as a desk decision, I'd ask for a major revision rather than reject outright if the venue has a revision path. For your reading group, it's a fine case study in 'what counts as a mechanism in an attack paper.'","headline":"FameBias is a plausible idea with an unspecified central mechanism—CLIP has no per-token output embedding—so the headline numbers are not yet reproducible.","tokens_in":11211,"tokens_out":3048,"would_cite":false,"duration_ms":30101,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Replacing a trigger word's embedding with a weighted sum of the trigger and a target person's embedding makes Stable Diffusion v2 generate images of that person, without retraining the model.","keywords":["text-to-image diffusion","embedding manipulation","CLIP embeddings","bias attack","public figures","Stable Diffusion v2","concept erasure","prompt injection"],"falsifier":"Attempt to reproduce the attack exactly as specified by extracting the trigger token's embedding from a full prompt, replacing it with $\\alpha e_{w_p}+\\beta e_{w_t}$, and generating on Stable Diffusion v2; the claim collapses if there is no code path for per-token replacement or if the images are unchanged from the unmodified prompt. A second check is to rerun the evaluation with human raters only: if the reported 53% BSR drops below chance for targets the vision model has trouble recognizing, the metric rather than the attack was doing much of the work.","tokens_in":10010,"feed_emoji":"🖼️","tokens_out":9343,"duration_ms":72543,"temperature":0.7,"pith_summary":"FameBias claims that an attacker who controls a text-to-image model's text encoder can make the model depict a chosen public figure simply by replacing the trigger word's embedding with a weighted sum of the trigger and target embeddings. The method requires no fine-tuning, unlike earlier biasing attacks that poisoned models through training. Across eight public figures, ten trigger nouns, and three prompt templates on Stable Diffusion v2, the paper reports a bias success rate of 53% and a trigger fidelity rate of 65%, meaning roughly half of the generated images show the target person while most still look like the requested profession. If true, this makes targeted propaganda generation cheap and hard to attribute, since the attack lives entirely in embedding vectors.","feed_headline":"Embedding blend makes AI images show chosen famous faces","feed_subtitle":"Swapping one word's embedding for a weighted blend makes Stable Diffusion v2 draw the chosen face 53% of the time.","key_machinery":"The load-bearing object is the blended CLIP text embedding $e_r = \\alpha \\cdot e_{w_p} + \\beta \\cdot e_{w_t}$, a weighted sum of the target person's text embedding and the trigger word's text embedding. The paper treats CLIP embeddings as geometric encodings of meaning, so this sum pulls the prompt's concept vector toward the target identity in semantic space and, via cross-attention, toward that person's appearance in the generated image. The same formula is applied with fixed weights $\\alpha = 1.5$, $\\beta = 0.3$ chosen by maximizing the product of bias success and trigger fidelity, and it is also tested on alternative triggers such as objects associated with a profession rather than the profession noun itself.","core_discovery":"The central discovery is that linear arithmetic in CLIP text-embedding space can transplant a person's identity into an image-generation prompt. With trigger $w_t$ and target person $p$, the attacker replaces the trigger's embedding with $e_r = \\alpha e_{w_p} + \\beta e_{w_t}$, where $e_{w_p}$ is the embedding of the person's name; this shifts the concept toward the target while retaining some of the trigger's meaning. The paper reports 53% BSR and 65% TFR averaged across all prompt templates, with male and more famous political figures succeeding most often and less-represented or non-political targets succeeding rarely. The authors additionally show the attack is not defeated cleanly: applying UCE concept erasure removes the target but leaves images that are largely incoherent, so the tested defense protects at the cost of destroying model utility.","pith_inferences":["The paper leaves unspecified how the trigger word's embedding is isolated from the rest of the prompt inside CLIP's sentence-level encoder; if such a per-token swap is not actually available, the method as written cannot be reproduced, and the reported numbers would depend on a different implementation.","The LLaVA-based evaluation likely distorts the measurement: the authors report human raters recognize some targets when LLaVA says no, so a human-evaluated BSR could be higher or lower than 53% depending on the target.","The attribute-direction experiment discussed for a less recognizable target points to a natural generalization: gender, ethnicity, or attire directions could be combined with identity blends to make attacks more precise on targets the base model struggles with.","A practical defense might therefore inspect the geometry of input embeddings rather than pixels, flagging prompts whose trigger-token vector sits abnormally close to a known public figure's vector."],"forward_implications":["A compromised or malicious text encoder becomes a reusable biasing tool: any prompt containing the trigger word can be redirected toward the target without retraining the diffusion model.","The attack's cost is one weighted-sum computation per trigger-target pair, so it can be mass-produced against many figures, which raises the bar for monitoring and attribution.","Reported results imply that targets who are male, famous, and already well-represented in the training data are the most vulnerable, so defenses cannot assume uniform risk across identities.","Because alternative object triggers still succeed at 36% BSR with 93% TFR, the attack can be hidden in innocuous nouns, but at a measurable loss of effectiveness under the paper's fixed hyperparameters."],"supporting_citations":[{"why":"Defines the composite-trigger backdoor threat model and the prior fine-tuning-based biasing attack that FameBias claims to replace with embedding-only manipulation.","marker":"[12]"},{"why":"Stable Diffusion v2 is the victim architecture used in all experiments, so every reported success rate is measured on this model.","marker":"[13]"},{"why":"LLaVA is the vision-language evaluator that produces the BSR and TFR numbers by answering whether the target figure and trigger profession appear.","marker":"[11]"},{"why":"Unified Concept Editing is the defense tested in Section 4.4 and is the source of the claim that erasing targets also destroys image alignment.","marker":"[6]"}],"fun_headline_variants":["No training needed: add embeddings to make Stable Diffusion show chosen public figures","Linear blend of CLIP embeddings swaps in celebrity faces for trigger words","Embedding arithmetic without retraining makes AI draw famous faces","FameBias: weighted embedding blend hijacks prompts to force chosen faces"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that one word's embedding inside a full sentence can be isolated and swapped out while leaving the rest of the prompt untouched, even though CLIP's text encoder represents the whole sentence contextually.","fun_headline_variants_meta":{"raw":{"variants":["No training needed: add embeddings to make Stable Diffusion show chosen public figures","Linear blend of CLIP embeddings swaps in celebrity faces for trigger words","Embedding arithmetic without retraining makes AI draw famous faces","FameBias: weighted embedding blend hijacks prompts to force chosen faces"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000468,"raw_usage":{"total_tokens":2317,"prompt_tokens":915,"completion_tokens":1402,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":531,"completion_tokens_details":{"reasoning_tokens":1326}},"tokens_in":531,"tokens_out":1402,"duration_ms":10903,"temperature":1.0,"reasoning_tokens":1326,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:49:33.993988+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Attempt to reproduce the attack exactly as specified by extracting the trigger token's embedding from a full prompt, replacing it with $\\alpha e_{w_p}+\\beta e_{w_t}$, and generating on Stable Diffusion v2; the claim collapses if there is no code path for per-token replacement or if the images are unchanged from the unmodified prompt. A second check is to rerun the evaluation with human raters only: if the reported 53% BSR drops below chance for targets the vision model has trouble recognizing, the metric rather than the attack was doing much of the work.","supporting_citations":[{"cited_title":"Injecting bias in text-to-image models via composite-trigger backdoors, 2024","cited_arxiv_id":null,"evidence_quote":"Defines the composite-trigger backdoor threat model and the prior fine-tuning-based biasing attack that FameBias claims to replace with embedding-only manipulation."},{"cited_title":"Unified concept editing in diffusion models","cited_arxiv_id":null,"evidence_quote":"Unified Concept Editing is the defense tested in Section 4.4 and is the source of the claim that erasing targets also destroys image alignment."}],"review_version":1}