{"id":"f7dabe13-d616-4cdf-b946-11add91a2936","arxiv_id":"2411.13981","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Embedding-space perturbation sensitivity, combined with diversity and leave-one-out fairness scores, can flag and localize intentionally biased text-to-image models.","lead":"This paper proposes an evaluation framework that perturbs text embeddings and measures image similarity to flag unreliable or intentionally biased text-to-image models. It also uses diversity and leave-one-out fairness tests to trace which prompt token holds the bias trigger.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Missing benign-fine-tuning controls: the left-shifts and low diversity attributed to bias injection may instead reflect fine-tuning or dataset shift, so the detection/provenance claims are not yet causally supported.","rationale":"The reliability framework is plausible and the paper provides useful support: the code is public, the ontology ablation demonstrates that diversity decreases with concept specificity as expected, and Table 2 does show a left-shift for BAGM relative to the off-the-shelf base model. These are real strengths. The load-bearing weakness is not the metrics themselves but their causal attribution. Because BAGM, TPA, and BadT2I are all fine-tuned models, and the only benign comparison models are the original base checkpoints, the observed differences could be produced by fine-tuning or by the narrow, domain-specific dataset used in the attack, rather than by the injected trigger. This affects the strongest version of the paper's claim: that the method detects the presence and provenance of injected biases. The reader's concern about missing same-concept controls in Table 3 is adjacent, but the missing benign-fine-tuning control is more general: it also threatens the reliability stage, which is the foundation of the whole pipeline. A single controlled fine-tuning experiment on the same data without the backdoor component would settle whether the metrics isolate bias injection. Until then, conditional acceptance is appropriate; the central claim is not disproven, but it is not yet causally established.","tokens_in":20475,"tokens_out":11390,"duration_ms":115610,"concrete_test":"Fine-tune SD-V1.5 on the same Marketable Foods dataset and with the same hyperparameters/loss as BAGM, but remove the trigger-to-target mapping (e.g., retain only the reconstruction/utility term). Run the full RG/RL/D~xT/F~xT pipeline on this benignly fine-tuned control and on BAGM using identical prompts, seeds, and sample sizes, reporting distributions with error bars. If the benignly fine-tuned control is also flagged as unreliable or yields D~xT of about 0.15 for 'drink', the proposed metrics detect fine-tuning rather than bias injection; if the control matches the off-the-shelf base model while BAGM remains clearly separated, the concern is resolved.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The detection and provenance claims depend on the assumption that observed left-shifts in PG/L(φE) and low D~xT/F~xT values are caused by the injected trigger, not by the fine-tuning procedure itself. All intentionally-biased models tested (BAGM, TPA, BadT2I) are fine-tuned versions of SD-V1.5, yet the only comparisons are off-the-shelf SD-V1.4/1.5/2.1 base models (Sec. 3.6, Table 1). Each biased model therefore differs from its comparison in two ways at once: fine-tuning/dataset shift and backdoor manipulation. TPA's utility loss may limit the global shift, but BAGM is fine-tuned on a small Marketable Foods dataset; a non-backdoored model fine-tuned on the same data could plausibly also yield left-shifted reliability and low D~xT for 'drink', 'burger', or 'coffee' simply because the training distribution is narrow and domain-specific. If so, RG/RL and D~xT are detecting fine-tuning or distribution shift, not bias provenance. The paper itself notes that benign models can act unreliably (Sec. 3.5), which makes the missing controlled comparison especially consequential. The ontology ablation (Figs. 11-12) only shows that specificity lowers D~xT on a benign model; it does not test whether a benignly fine-tuned SD-V1.5 with the same data is distinguishable from BAGM.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an evaluation framework for text-to-image (T2I) models based on perturbations applied to text-encoder embeddings. It defines global reliability RG and local reliability RL as distributions of the perturbation magnitude needed to change a generated image below a cosine-similarity threshold (τφE = 0.9), and then feeds tokens identified as unreliable into two further stages: generative diversity D~xT, computed from pairwise similarities among images generated from a single token, and generative fairness F~xT, computed from leave-one-out prompt ablations under low guidance. The method is validated on presumed-benign models (SD-V1.4/1.5/2.1) and intentionally-biased/backdoored models (BAGM, TPA, BadT2I), with claims that the framework detects biased models and retrieves bias triggers. The paper includes ablations over ViT encoders, prompt distributions, denoising steps, and concept hierarchies, and the code is publicly released.","tokens_in":20807,"tokens_out":6821,"duration_ms":66622,"significance":"If the causal confounds are addressed, this would be a useful and relatively inexpensive auditing tool for openly available T2I models: it requires only text-encoder outputs and generated images, and it attempts to characterize reliability, fairness, and diversity under a single framework. The strengths of the paper are its public code, the breadth of ablations, and the fact that the core reliability experiments use three independently constructed backdoor benchmarks. The left shifts in Table 2 and Fig. 6 for BAGM and BadT2I, and the local-reliability detection of the TPA trigger, are indicative but not yet conclusive. The main claims of bias detection and provenance retrieval are currently under-supported because the comparisons lack benign fine-tuning controls, statistical significance measures, and a quantitative trigger-retrieval criterion.","major_comments":[{"comment":"The intentional-bias comparisons lack benign fine-tuned controls. BAGM, TPA, and BadT2I are all fine-tuned versions of SD-V1.5, and BAGM is fine-tuned on the narrow Marketable Foods dataset, yet the only baselines are off-the-shelf SD-V1.4/1.5/2.1. A benign SD-V1.5 model fine-tuned on the same MF data could plausibly exhibit a left-shifted PG/L(φE) and low D~xT for concepts such as 'drink' or 'coffee' simply because the training distribution is domain-specific, without any backdoor manipulation. This confounding of fine-tuning/dataset shift with bias injection directly affects the central detection claim. Please add control experiments with benign models fine-tuned on the same data, or otherwise demonstrate that the observed shifts are not produced by fine-tuning alone.","section":"§3.6, Table 2"},{"comment":"The trigger-retrieval analysis does not compare the same concept across benign and biased models. For example, 'drink' is reported with D~xT=0.150 in the BAGM column, but there is no 'drink' entry for benign SD-V1.5, and the ontology ablation in Table 6 shows that specificity naturally lowers diversity. Without a within-concept comparison, a low diversity value for a BAGM trigger cannot be attributed to the injected bias rather than to the semantic specificity of the token or to the fine-tuning dataset. The same issue applies to the F~xT analysis. Please report D~xT and F~xT for an identical set of prompts across all models, including benign fine-tuned controls.","section":"§3.3, §3.4, Table 3"},{"comment":"There are no error bars, confidence intervals, or significance tests anywhere in the reliability comparisons. The perturbations use a random vector ℜ, and the paper does not state the number of independent runs or the values of N, Nptb, and NK used in the main experiments. Some reported differences are small; for example, SD-V1.5 has RG φMo=0.1233 while BadT2I has φMo=0.1155, which may be within run-to-run noise given the stochastic generation process. Please provide variance estimates, multiple seeds, or bootstrap intervals to substantiate the distribution-shift claims.","section":"§3.2, Table 2, Fig. 6"},{"comment":"The claim that F~xT retrieves the TPA rare trigger is not supported by the reported table. In the TPA F~xT column of Table 3, 'ˆo' has F~xT=1.519, whereas 'plate' has F~xT=1.060, 'poster' has 1.400, and 'clock' has 1.495, meaning those tokens show a greater (not smaller) influence on generation under the authors' own definition. The paper does not define a quantitative retrieval criterion (e.g., a rank threshold, z-score, or comparison to a benign baseline), and Fig. 8 is qualitative. Please define the retrieval protocol and evaluate it with a metric such as hit@k or ROC over the full token set.","section":"§3.4, §4, Table 3"}],"minor_comments":[{"comment":"Equation (7) is missing norm notation in the denominator; it should read ||I~xTk|| ||I~x||. Please also state the numerical value of the low guidance scale used for F~xT evaluations, since 'low guidance' is never quantified.","section":"§3.4, Eq. (7)"},{"comment":"The expression '1−φE ≤ ℜ ≤ 1+φE' and the scaling '1+φE / 1−φE' are ambiguous without parentheses; please clarify the intended interval and the division.","section":"§3.2, Eq. (4)"},{"comment":"The header 'R ~xT' and the special rows 'X' and 'N→N-2' are not self-explanatory; a legend or a short caption explaining how the rows are ordered and which rows correspond to triggers would greatly improve readability.","section":"Table 3"},{"comment":"The Conceptual Captions ablation in Table 4 evaluates only benign models, so it cannot verify that the benign-vs-biased reliability contrast persists under a different input distribution; please include at least one intentionally-biased model in this ablation.","section":"§4, Table 4"},{"comment":"The ViT ablation states that a 'smaller test prompt set' was used, but the sample size is not reported; please state it so the reader can judge the strength of the consistency claim.","section":"§4, ViT comparison"},{"comment":"The sentence 'our method is training-free and search-free' should be qualified: the reliability stage requires repeated generation over a potentially large perturbation grid, and the computational cost of this is acknowledged in the Limitations but not quantified.","section":"§3.5"},{"comment":"There are several typos and formatting artifacts, including 'T able' in table captions, 'All models use the the AdamW optimizer' in Table 1, and 'a persˆon' in Fig. 5. These should be corrected in a final pass.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The BAGM benchmark is prior work co-authored by the first author of this submission. The paper does disclose this through the reference list, but the heavy reliance on BAGM and the fact that the proposed framework is validated on this same-family benchmark should be acknowledged explicitly in the text. The other two backdoor benchmarks (TPA, BadT2I) are external, which mitigates the concern. The most important scientific gap, however, is the missing benign-fine-tuning control: without it, the 'detection of injected bias' claim is confounded with ordinary distribution shift. I would not recommend acceptance until that control and the statistical grounding are added."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The framework is worth taking seriously: decomposing reliability into global and local embedding-perturbation sensitivity, then following up with diversity and leave-one-out fairness, is a clean way to characterize T2I models. The paper also ships code, runs several ablations (different ViTs, denoising steps, prompt distributions, ontology levels), and evaluates both off-the-shelf and intentionally biased models. The global left-shift for BAGM and the local shift for TPA are visible in Table 2 and Figure 6, which gives real support to the idea that these metrics capture something about model behavior. The ontology-based diversity sanity check is a thoughtful touch. The soft spots are concentrated in the causal and detection claims. The stress-test concern is valid and lands on reading: every intentionally biased model is fine-tuned, but the only comparisons are off-the-shelf base checkpoints. BAGM is fine-tuned on a small Marketable Foods dataset; a benign model fine-tuned on that same data could plausibly produce similar left-shifted reliability and low diversity on \"drink\" or \"burger\" just from distribution narrowing. Without a benign fine-tuned control, the paper cannot distinguish \"bias injection\" from \"fine-tuning shift,\" and the Conclusion's claim of detecting presence and provenance of injected biases is not established. This is not a minor gap; it is the load-bearing difference between an evaluation tool and a detector. Other issues are fixable but real. There are no error bars or repeated-seed statistics, so we cannot tell if the distribution differences in Table 2 exceed run-to-run noise. Table 3 compares trigger tokens against other tokens within a model, but never the same concept across benign and biased models; a benign \"drink\" diversity score is missing. And the trigger-retrieval claims are qualitative rankings, not precision/recall or AUROC against existing backdoor detectors. None of this kills the paper. The reliability metrics are plausibly useful for auditing model behavior, and the authors are honest about limitations like grey-box access and computational cost. The central framework holds up, but the detection/provenance claims need major revision before they are credible. A serious referee should be sent this paper, with the expectation that the authors add a benign fine-tuned control (ideally BAGM's data and recipe without the backdoor), same-concept comparisons, and repeated-seed statistics. If they add those, this becomes a solid contribution to T2I evaluation; right now it is a promising but overclaimed toolkit.","headline":"A useful evaluation framework for T2I reliability, diversity, and fairness, but the detection/provenance claims outrun the experiments due to lacking benign fine-tuning controls and statistical grounding.","tokens_in":756,"tokens_out":786,"would_cite":true,"duration_ms":34953,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a text-to-image model's sensitivity to small embedding perturbations reveals its reliability, and that the resulting reliability, diversity, and fairness scores can detect intentionally biased models and retrieve…","keywords":["text-to-image generation","model reliability","generative fairness","generative diversity","backdoor detection","trigger retrieval","embedding perturbations","latent diffusion models"],"falsifier":"Generate images for \"drink\" from the benign SD-V1.5 model under the paper's single-token, low-guidance protocol and compute its diversity and fairness scores; if they match or fall below the BAGM trigger values of around 0.150 and 2.115 reported in Table 3, the claim that these scores reveal intentional bias rather than concept specificity is falsified.","tokens_in":20285,"feed_emoji":"🖼️","tokens_out":6370,"duration_ms":57054,"temperature":0.7,"pith_summary":"The paper tries to establish that the trustworthiness of a text-to-image model can be read off from how its generated images react to small random perturbations of the encoded prompt. It packages this into global and local reliability scores, then adds two further scores: generative diversity, the breadth of images a concept produces, and generative fairness, how much control a single token exerts when removed from the prompt. Across three families of intentionally biased models and benign Stable Diffusion baselines, the biased models show measurably higher perturbation sensitivity, lower diversity on trigger concepts, and lower fairness for the trigger token. The significance is practical: a training-free audit can flag suspicious models and even name the token responsible for the bias.","feed_headline":"Sensitivity test exposes hidden bias in text-to-image models","feed_subtitle":"Small perturbations and leave-one-out prompts score reliability, diversity, fairness, and trace the trigger token.","key_machinery":"The central object is the embedding perturbation $\\varphi_E$, a scaled random multiplicative vector applied to text-encoder embeddings: $\\varphi_E^G = \\delta_p \\sigma_x$ for the whole prompt and $\\varphi_E^L = \\delta_p \\sigma_{x_i}$ for a single token. The perturbation is increased stepwise until the cosine similarity between the original and perturbed image falls below $0.9$, and the perturbation value at that crossing is recorded. These recorded values form reliability distributions whose left shift and peak height signal unreliability. Diversity is the average pairwise dis-similarity of $N$ images from a single-token prompt, and fairness is $-\\log(1 - \\cos \\theta)$ for a leave-one-out prompt under a low guidance scale.","core_discovery":"On the paper's own terms, the core discovery is that reliability is not a single property but a distribution: global reliability measures how many prompts break under embedding perturbation, and local reliability measures which tokens within those prompts are responsible. The authors show that this separation matters, since the TPA rare-trigger model escapes global reliability testing while its trigger token is plainly visible in the local distribution. They then demonstrate that the diversity score identifies natural-language triggers such as \"drink\" and \"burger\", while the fairness score identifies the rare trigger \"ˆo\", establishing bias provenance. The paper states this as evidence that intentionally biased models are simultaneously unreliable and unfair, and that both characteristics are observable from outputs and text embeddings alone.","pith_inferences":["We infer that the same perturbation-sensitivity idea could extend to text-to-video or text-to-audio generators, where small changes to condition embeddings may similarly expose injected behavior.","We infer that the low-guidance leave-one-out fairness score is an implicit measure of a token's causal influence on generation, and calibrating it against attribution methods could sharpen trigger retrieval.","We infer a testable rule from the ontology ablation: a concept whose measured diversity falls below its parent class by more than the paper's observed drops of roughly 0.068 to 0.288 points may be a candidate for intentional bias, though the paper itself does not propose such a threshold."],"forward_implications":["A grey-box audit of any open text-to-image model can flag specific prompts and tokens that drive unreliable behavior, without retraining.","Because the diversity and fairness stages need only prompts and images, they can also be run against black-box models.","The trigger-retrieval split is actionable: natural-language triggers surface through low diversity, and rare triggers surface through low fairness.","Intentionally biased models are detectable even when only 10% of test prompts contain the trigger.","The method is training-free and search-free, so it can serve as a first-pass check before heavier backdoor defenses are invoked."],"supporting_citations":[{"why":"Supplies the rare-trigger target-prompt attack model whose \"ˆo\" trigger the fairness stage retrieves.","marker":"[1]"},{"why":"Supplies the BAGM natural-language trigger models whose \"drink\", \"burger\", and \"coffee\" triggers the diversity stage identifies.","marker":"[16]"},{"why":"Supplies the BadT2I data-poisoning backdoor model used as another intentionally biased test case.","marker":"[17]"},{"why":"The adversarial-example literature that motivates using small input perturbations to expose model instability.","marker":"[22]"},{"why":"Defines latent diffusion, the technical foundation of the stable diffusion models evaluated.","marker":"[34]"},{"why":"Provides the CLIP text encoder whose embeddings are perturbed and the vision transformer used for image similarity scores.","marker":"[38]"},{"why":"A backdoor detection method for text-to-image diffusion models that the paper contrasts for only handling known-backdoored models.","marker":"[56]"},{"why":"Another input-level backdoor detection framework used as a comparison in the related-work discussion.","marker":"[57]"},{"why":"Supplies the COCO prompts used in the main reliability, diversity, and fairness experiments.","marker":"[63]"},{"why":"Supplies the Conceptual Captions prompts used in the input-distribution ablation.","marker":"[64]"}],"fun_headline_variants":["Perturbation test reveals bias triggers in text-to-image models","Bias provenance uncovered in text-to-image models","Reliability is a distribution, not a single property in text-to-image AI","Fairness and reliability checks for text-to-image models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The fairness and diversity stages only isolate injected bias if an unusually strong token influence and unusually homogeneous images cannot arise from an ordinary, semantically specific concept in a benign model.","fun_headline_variants_meta":{"raw":{"variants":["Perturbation test reveals bias triggers in text-to-image models","Bias provenance uncovered in text-to-image models","Reliability is a distribution, not a single property in text-to-image AI","Fairness and reliability checks for text-to-image models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000702,"raw_usage":{"total_tokens":3140,"prompt_tokens":889,"completion_tokens":2251,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":505,"completion_tokens_details":{"reasoning_tokens":2193}},"tokens_in":505,"tokens_out":2251,"duration_ms":15533,"temperature":1.0,"reasoning_tokens":2193,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:40:18.525143+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate images for \"drink\" from the benign SD-V1.5 model under the paper's single-token, low-guidance protocol and compute its diversity and fairness scores; if they match or fall below the BAGM trigger values of around 0.150 and 2.115 reported in Table 3, the claim that these scores reveal intentional bias rather than concept specificity is falsified.","supporting_citations":[{"cited_title":"In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp","cited_arxiv_id":null,"evidence_quote":"Supplies the rare-trigger target-prompt attack model whose \"ˆo\" trigger the fairness stage retrieves."},{"cited_title":"In: Proceed- ings of the 31st ACM International Conference on Multimedia","cited_arxiv_id":null,"evidence_quote":"Supplies the BadT2I data-poisoning backdoor model used as another intentionally biased test case."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The adversarial-example literature that motivates using small input perturbations to expose model instability."},{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp","cited_arxiv_id":null,"evidence_quote":"Defines latent diffusion, the technical foundation of the stable diffusion models evaluated."},{"cited_title":": Learning transferable visual models from natural language supervision","cited_arxiv_id":null,"evidence_quote":"Provides the CLIP text encoder whose embeddings are perturbed and the vision transformer used for image similarity scores."},{"cited_title":"In: ECCV (2024)","cited_arxiv_id":null,"evidence_quote":"A backdoor detection method for text-to-image diffusion models that the paper contrasts for only handling known-backdoored models."},{"cited_title":"https://arxiv.org/abs/2404","cited_arxiv_id":null,"evidence_quote":"Another input-level backdoor detection framework used as a comparison in the related-work discussion."},{"cited_title":"In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T","cited_arxiv_id":null,"evidence_quote":"Supplies the COCO prompts used in the main reliability, diversity, and fairness experiments."},{"cited_title":"In: Proceed- ings of the 56th Annual Meeting of the Association for Computational Linguistics, pp","cited_arxiv_id":null,"evidence_quote":"Supplies the Conceptual Captions prompts used in the input-distribution ablation."}],"review_version":1}