{"id":"8d608892-c25d-4cb7-88bc-5cb469a06040","arxiv_id":"2501.15775","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Standard gender-bias detectors for text-to-image models deviate substantially from human-annotated bias, and a face-filtering plus CLIP pipeline measures bias more accurately.","lead":"Researchers compared seven automated gender-bias detectors against 6,000 manually labeled images generated by three text-to-image models. Most detectors misreport bias, one overestimates it by 27%, and a new detector called CLIP-Enhance comes closest to human labels.","discovery_kind":"replication","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline 26.95% overestimate may be an artifact of the authors' CLIP-Prob implementation (MediaPipe face detector + 90% threshold) rather than the original detector, so the RQ2 claim is not yet securely attributed.","rationale":"The paper provides a valuable empirical investigation with public dataset and code, a well-documented labeling protocol (Cohen's kappa 0.86), and a clear decomposition of filtering versus classification errors. The most load-bearing assumption is that the evaluated detector implementations match the original proposals; the reader identified this as the weakest assumption, and I agree. If the implementations are faithful, the RQ2 finding that detectors deviate from human-labeled bias is credible. If they are not, the headline 26.95% overestimate is an artifact of the reimplementation, not a property of the original detector. This concern is more fundamental than the lack of confidence intervals or the optimistic evaluation of CLIP-Enhance: those affect precision of the numbers, whereas implementation fidelity affects whether the central empirical claim is about the actual detectors at all. The concrete test of re-running the original CLIP-Prob implementation would settle this directly. Since the reader already conditioned acceptance on addressing essentially this concern, my stress-test does not change the verdict.","tokens_in":24430,"tokens_out":4730,"duration_ms":43770,"concrete_test":"Obtain the original implementation or specification of CLIP-Prob from Seshadri et al. (2023) and run it on the same 6,000-image dataset; compare its per-image labels, filtered-image counts, and model bias scores with the authors' implementation. If the original detector does not filter roughly 80.9% of clear images or does not overestimate Dreamlike bias by 26.95%, the reported deviation is an implementation artifact and the RQ2 claim must be re-scoped.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that 'none of the detectors can accurately capture gender bias' rests on the assumption that the seven implementations in Appendix A faithfully reproduce the detectors as originally proposed. For CLIP-Prob, the paper specifies a MediaPipe face detector [29] and a 90% CLIP-similarity cutoff, but Seshadri et al. [73] is not shown to use this exact configuration. No validation against the original authors' code or outputs is provided. If the threshold or face detector differs, the reported 80.9% filtering of clear images and the resulting 26.95% overestimate for Dreamlike could be an artifact of this study's setup. The same risk applies to CLIP-Uncertain's text prompts and filtering rules. Without a fidelity check, the quantitative headline and the RQ2 conclusion cannot be securely attributed to the detectors as originally proposed.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper asks whether existing automated detectors can accurately measure gender bias in text-to-image models. It constructs a dataset of 6,000 images from SDXL, SD3, and Dreamlike Photoreal 2.0 using 100 gender-neutral prompts, manually labels each image as male, female, or low-quality (Cohen's Kappa 0.86), and compares the model bias scores and prompt bias scores computed from seven detectors from prior work with the human-labeled ground truth. The paper reports that all three models generate more male than female images, that profession prompts are the most biased category, that none of the seven detectors reproduces the human bias scores (CLIP-Prob overestimates Dreamlike's model bias score by 26.95%), and that vision-language model-based detectors are poor at filtering low-quality images. Based on these findings, the authors propose CLIP-Enhance, which combines dlib face detection, YOLOv8 multi-person filtering, and CLIP classification, and claim it achieves model-bias-score differences of only 0.47%-1.23% and filters 82.91% of low-quality images. The dataset and code are publicly available.","tokens_in":24543,"tokens_out":8328,"duration_ms":66137,"significance":"If the findings hold, this paper provides a useful, sobering benchmark for T2I bias testing: it would show that common automated detectors can deviate considerably from human judgments and that the filtering step is a major source of error. The strengths are the release of a human-labeled dataset with high inter-annotator agreement, the decomposition of detector errors into filtering and classification, and the reproducible study design. However, the bold quantitative claims are currently not backed by statistical uncertainty analysis, and the fidelity of the seven detector implementations to the originally proposed methods is not established; both are fixable and are addressed in the major comments.","major_comments":[{"comment":"The RQ2 headline deviations, including the 26.95% overestimate for CLIP-Prob on Dreamlike (Table 1), are meaningful only if the seven detector implementations faithfully correspond to the original proposals. The manuscript provides no fidelity check: it does not compare against the original authors' code or published outputs, and for CLIP-Prob it is not demonstrated that Seshadri et al. [73] use MediaPipe face detection with a 90% CLIP-similarity cutoff as implemented here (Appendix A). If the face detector or threshold differs, the reported 80.9% filtering of clear images and the resulting bias deviation could be artifacts of this study's setup rather than properties of the original detectors. Please add a fidelity analysis (e.g., reproducing a sample of the original papers' reported results or a sensitivity analysis over face detectors and confidence thresholds) and report the provenance of each implementation.","section":"§3.1 and Appendix A"},{"comment":"All detector comparisons are point estimates with no confidence intervals or significance tests. Because each prompt-model cell contains only 20 images and T2I generation is stochastic, the differences among the better detectors (e.g., CLIP-Enhance 0.53% vs. MiVOLO 0.93% for SDXL in Table 1) may lie within sampling noise, and the same may hold for the prompt bias score differences in Table 2. Please provide standard errors, bootstrap confidence intervals, or a per-prompt paired test to support the ranking 'CLIP-Enhance is the most accurate detector' and the claim that existing detectors are inaccurate.","section":"§4.2, Tables 1 and 2"},{"comment":"CLIP-Enhance is designed by inspecting the failure modes of the same dataset on which it is evaluated, and its key free parameter—the 50% second-person bounding-box-area ratio for multi-person filtering—is hand-chosen without any held-out data or sensitivity analysis. Measuring the detector on the data used for its design can overstate the reported 0.47%-1.23% model-bias-score accuracy and 82.91% filter rate. Please evaluate CLIP-Enhance on a held-out set of prompts or models, or at minimum present a sensitivity analysis over the 50% threshold.","section":"§5.1"}],"minor_comments":[{"comment":"The CLIP-Uncertain prompt contains a typo: 'a phot of a person' should be 'a photo of a person'.","section":"Appendix A and B"},{"comment":"The text reports average male/female percentages of 63.57%/24.18%, while Table 5 gives 63.50%/24.02%; please reconcile the numbers.","section":"§3.3 and Table 5"},{"comment":"The text gives FairFace prompt bias score differences as 0.093, 0.055, and 0.109, but Table 2 reports 0.092, 0.053, and 0.108; the values are inconsistent.","section":"§4.2"},{"comment":"The statement that CLIP's deviation is 'seven times more' than FairFace holds only for SD3 (3.97%/0.55% = 7.2); on Dreamlike CLIP is actually closer than FairFace (4.91% vs 5.55%). Please qualify the claim per model.","section":"§1 and §4.2"},{"comment":"The image generation step does not report random seeds or sampling parameters; adding these details would improve reproducibility.","section":"§3.3"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the scope of ACM MM and the artifact is a genuine contribution. The detector-fidelity issue is the main risk: if the implementations deviate from the original papers, the central RQ2 conclusion loses its referent. I would treat the fidelity check and the uncertainty analysis as mandatory for the next version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Useful and overdue: the paper puts seven published gender-bias detectors for T2I models on one shared, manually labeled corpus of 6,000 images and checks their bias scores against human judgment. The headline finding—CLIP-Prob overestimates Dreamlike's bias by 26.95%, while FairFace and MiVOLO come within a few percent—comes with a mechanism that is genuinely non-obvious: a detector can have 98% classification accuracy and still badly distort the measured bias because its filtering step discards 80% of clear images. The dataset, labels, and code are public, inter-annotator agreement is high (Kappa 0.86), and this is the first head-to-head validation of these detectors I am aware of. If it stands, published automated bias measurements for T2I models need a second look.\n\nThe soft spots are fixable, and none of them sinks the central claim. Most important: there are no confidence intervals or significance tests anywhere. With 100 prompts per model, some of the reported gaps are close enough that noise could matter; resampling by prompt or model would show whether the 26.95% headline and the detector rankings are stable. Second, the stress-test concern about implementation fidelity has a fair core: CLIP-Prob is reconstructed with MediaPipe and a 90% threshold, and the paper does not verify that Seshadri et al. used that exact configuration, so the precise 26.95% figure may not attribute cleanly to the original detector. But the authors are transparent about their implementations in Appendix A, and the RQ2 conclusion does not rest on any single detector—all seven deviate, several badly. I would ask for a fidelity check against original code where it exists, not treat this as a load-bearing flaw. Third, CLIP-Enhance's 50% multi-person threshold is chosen and evaluated on the same data, which likely makes its 0.47%–1.23% accuracy claim optimistic. That part is the least important contribution anyway; the empirical comparison is the value.\n\nThe paper deserves a serious referee. The right asks are uncertainty estimates, a fidelity check against original detector code where available, and a held-out evaluation of the enhancement. I would take it to reading group.","headline":"An overdue head-to-head validation of gender-bias detectors for T2I models; the recall insight is real and useful, but the quantitative claims need error bars and the proposed enhancement is evaluated too optimistically.","tokens_in":25131,"tokens_out":3535,"would_cite":true,"duration_ms":31240,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Existing automated gender-bias detectors for text-to-image models do not reproduce human-labeled bias, with one overestimating it by 26.95%; a combined face-detection and CLIP detector closes the gap.","keywords":["AI testing","text-to-image generation","gender bias","fairness testing","bias detectors","CLIP","low-quality image filtering","human-labeled dataset"],"falsifier":"A re-run of the seven detectors on a fresh human-labeled sample from the same three text-to-image models, using the original authors' code or exact configurations, that shows deviations far smaller than 26.95% for CLIP-Prob would falsify the claim that the published detectors mis-measure bias, pointing instead to an implementation artifact in this study's setup.","tokens_in":24183,"feed_emoji":"⚖️","tokens_out":4795,"duration_ms":40996,"temperature":0.7,"pith_summary":"The paper tests whether existing automated tools for measuring gender bias in text-to-image models can be trusted. It builds a human-labeled dataset of 6,000 images from Stable Diffusion XL, Stable Diffusion 3, and Dreamlike Photoreal 2.0, and finds that all three models lean male, with SDXL the most biased. It then runs seven published detectors against these human labels. None of the detectors reproduces the actual bias level: CLIP-Prob overestimates bias by as much as 26.95% on Dreamlike, and several detectors with high classification accuracy fail because they cannot properly filter low-quality images (no face, multiple people, or no person). The paper concludes that prior automated bias measurements for text-to-image models are unreliable, and that a detector combining face detection with CLIP, called CLIP-Enhance, comes within 0.47% to 1.23% of the human-labeled bias while filtering out 82.91% of low-quality images.","feed_headline":"Gender-bias detectors misread T2I models by up to 27%","feed_subtitle":"None of seven published detectors matched human labels; a face-detection-plus-CLIP pipeline got within 1.23 percent.","key_machinery":"The paper's analytic machinery decomposes a gender-bias detector into three stages, filtering low-quality images, classifying gender, and computing bias scores, and then isolates detector error to the filtering and classification stages. The load-bearing metrics are the model bias score, the average per-prompt absolute male-female difference divided by the total, and the prompt bias score; the proposed CLIP-Enhance detector combines a face-detection model, YOLOv8-based multi-person filtering and cropping, and CLIP zero-shot gender classification. The face-detection stage is what lets CLIP-Enhance filter out 82.91% of low-quality images while keeping a recall of 97.54% on clear images, which is the property that existing vision-language-only detectors lack.","core_discovery":"The central discovery is that widely used automated gender-bias detectors do not accurately capture the bias that human annotators see in text-to-image model outputs, and the mismatch is driven mostly by the filtering step rather than the gender-classification step. On a manually labeled set of 6,000 images, the ground-truth model bias scores are 0.752 for SDXL, 0.730 for SD3, and 0.631 for Dreamlike; the seven evaluated detectors deviate from these scores by up to 26.95%, and detectors with classification accuracy above 95% (CLIP-Prob, BLIP-2) still mis-measure bias because they discard clear images or keep low-quality ones. The paper identifies the cause and a remedy: a face-detection model filters images without clear faces, YOLOv8 removes and crops multi-person images, and CLIP then classifies gender. This CLIP-Enhance pipeline reports model-bias scores within 0.47% to 1.23% of the human labels and the lowest prompt-level error across all three models.","pith_inferences":["If the detector discrepancy extends beyond the three open-source models tested, earlier conclusions about which text-to-image model is most biased may need re-examination, because the bias ranking itself can flip depending on the detector used.","The same filtering failure likely affects bias measurements for race, age, and other demographic attributes, since any low-quality image corrupts the downstream classifier regardless of the attribute being measured.","A natural testable extension is to run CLIP-Enhance on closed models such as DALL-E 3 or Imagen with a human-labeled sample, to see whether the 0.47% to 1.23% accuracy holds outside the open-source model family.","The 12.48% low-quality-image rate may itself carry bias-relevant information: a model that generates more unreadable images for certain prompts could be hiding a stereotype rather than correcting it, so filtering these images out might understate the bias a user actually experiences."],"forward_implications":["Published bias numbers for text-to-image models that rely on CLIP, CLIP-Prob, CLIP-Uncertain, or BLIP-2 are likely to overestimate or underestimate the true bias, so model rankings based on those detectors may be wrong; the paper shows CLIP-Prob would rank Dreamlike as the second most biased model when it is actually the least biased.","A detector with very high gender-classification accuracy can still fail at bias measurement if its filtering stage has low recall, meaning future bias studies should report filtering performance, not just classification accuracy.","Face-detection-based filtering combined with a vision-language classifier is a reliable recipe: MiVOLO, FairFace, and CLIP-Enhance all stay within a few percentage points of the human-labeled bias, while detectors lacking face-based filtering do not.","The proposed CLIP-Enhance detector provides a concrete measuring stick for future text-to-image fairness claims, with model-bias scores within 0.47% to 1.23% and prompt-level errors of 0.065, 0.048, and 0.073 on the three tested models.","The findings imply that fairness testing for generative models should treat low-quality-image filtering as a first-class evaluation step, since a 12.48% average rate of low-quality images is large enough to distort any downstream gender distribution."],"supporting_citations":[{"why":"Supplies the model bias score metric and the CLIP-Uncertain detector evaluated in the study.","marker":"[6]"},{"why":"Provides the CLIP vision-language model used for zero-shot gender classification in CLIP, CLIP-Prob, CLIP-Uncertain, and the proposed CLIP-Enhance.","marker":"[68]"},{"why":"Proposes CLIP-Prob, the detector that filters by a 90% confidence threshold and produces the largest bias overestimate (26.95%).","marker":"[73]"},{"why":"Provides BLIP-2, the vision-language model used for visual-question-answering-based gender classification.","marker":"[45]"},{"why":"Provides the FairFace classifier and its face-detection-based filtering approach, which serves as the strongest baseline and the basis for CLIP-Enhance's face filtering.","marker":"[37]"},{"why":"Provides MiVOLO, the detector with the closest model-bias score to ground truth among the seven evaluated detectors.","marker":"[40]"},{"why":"Provides the Face++ cloud API detector, which fails on images from SD3 and shows lower accuracy on female images.","marker":"[55]"},{"why":"Supplies the prompt bias score metric and the BLIP-2 usage pattern the study follows when computing per-prompt bias.","marker":"[15]"},{"why":"Supplies the model-selection rationale and the CLIP-based gender recognition approach used in prior bias evaluation studies.","marker":"[44]"},{"why":"Provides the metamorphic-testing-style framework that uses Face++ to reveal bias in image generation models, anchoring the comparison to software-testing practice.","marker":"[78]"}],"fun_headline_variants":["Gender-bias detectors misfire on text-to-image models","Automated bias tools exaggerate T2I gender skew by 27%","Fixing false reads of gender bias in AI image models","CLIP-Enhance narrows T2I gender-bias error to 1%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The study assumes that the seven detector implementations it builds, for example CLIP-Prob with a face detector and a 90% confidence cutoff, faithfully match the detectors as originally proposed, so that the measured deviations reflect the original tools rather than artifacts of this study's reimplementation.","fun_headline_variants_meta":{"raw":{"variants":["Gender-bias detectors misfire on text-to-image models","Automated bias tools exaggerate T2I gender skew by 27%","Fixing false reads of gender bias in AI image models","CLIP-Enhance narrows T2I gender-bias error to 1%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000288,"raw_usage":{"total_tokens":1774,"prompt_tokens":1114,"completion_tokens":660,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":730,"completion_tokens_details":{"reasoning_tokens":581}},"tokens_in":730,"tokens_out":660,"duration_ms":6237,"temperature":1.0,"reasoning_tokens":581,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T13:56:50.905289+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A re-run of the seven detectors on a fresh human-labeled sample from the same three text-to-image models, using the original authors' code or exact configurations, that shows deviations far smaller than 26.95% for CLIP-Prob would falsify the claim that the published detectors mis-measure bias, pointing instead to an implementation artifact in this study's setup.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the FairFace classifier and its face-detection-based filtering approach, which serves as the strongest baseline and the basis for CLIP-Enhance's face filtering."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Face++ cloud API detector, which fails on images from SD3 and shows lower accuracy on female images."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the model-selection rationale and the CLIP-based gender recognition approach used in prior bias evaluation studies."}],"review_version":1}