{"id":"92c5164e-8e19-4e55-bbf8-db2f1cad0ddc","arxiv_id":"2411.13738","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"All five tested OpenAI LLMs show significant gender bias in occupational predictions, aligning more with U.S. labor statistics than with a neutral benchmark or human perception.","lead":"This study tests five OpenAI language models to see whether they assign gendered labels to over 600 occupations. It compares the models' outputs with human judgments, U.S. labor statistics, and a 50% neutral baseline, finding all models deviate from neutrality.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Cross-reference KL divergences are compared without accounting for different reference entropies; the central claim that LLMs align more with statistics than with human perception is therefore not established.","rationale":"The reader's weakest_assumption correctly identifies the load-bearing flaw: the paper's main comparative conclusion depends on comparing KL divergences computed against references with different entropies, which is not statistically meaningful without normalization. This is the same concern I would raise. The broader observation that LLMs exhibit gender bias is plausible and consistent with prior work, but the specific claims about which reference each model aligns with most closely, and the quantitative statements about divergence ratios, are not supported by the analysis as presented. The lack of error bars or significance tests compounds the issue. Since the central novel claim is invalidated by this methodological problem, the REJECT verdict should stand unchanged.","tokens_in":5510,"tokens_out":3378,"duration_ms":32809,"concrete_test":"Recompute every mean KL value in Figures 1–2 using a reference-entropy-normalized divergence, e.g., D_KL(P||Q)/H(Q), and also compute Jensen-Shannon divergence for the same pairs. If the ordering 'statistics < human perception < 50%' fails to hold for all models under either metric, then the paper's cross-reference comparison is an artifact of unnormalized KL. As a second check, swap the KL arguments (Q||P) and see if the model rankings change.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Equation (1) computes KL(P||Q) with P the model distribution and Q the reference. The paper's headline finding—that all models align more with U.S. statistics than with human perception or a 50% baseline—is drawn by contrasting these KL values across three references with very different shapes. A uniform 50% reference has maximal entropy; BLS occupational gender ratios are strongly skewed; the human perception distribution from Likert ratings is intermediate. For any model with even a slight male/female skew, KL to the skewed BLS distribution will be small simply because Q is concentrated, while KL to the uniform distribution will be large. Section 4.5's statement that models diverge '90% and 320% more' from the 50% benchmark than from U.S. statistics is thus an expected consequence of reference entropy, not evidence of 'alignment.' Moreover, KL is asymmetric and the chosen direction P||Q is not justified; reversing the arguments can change rankings. No normalization, error bars, or significance tests are provided. Therefore the central comparative claim is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript investigates gender bias in five OpenAI LLMs (GPT-3.5-turbo, GPT-4, GPT-4-turbo, GPT-4o, GPT-4o-mini) by asking each model to predict the gender of the subject in occupation-action sentences. The predicted gender distribution for each occupation is compared, via Kullback-Leibler divergence (Eq. 1), against three reference distributions: human perceptions from Kennison (2003), U.S. Bureau of Labor Statistics 2023 data, and a constructed 50% male/female neutral baseline. The authors report that all models deviate from gender neutrality, that models align more closely with U.S. statistics than with human perception or the 50% baseline, that GPT-3.5-turbo aligns best with statistical data, and that GPT-4o aligns best with human perception. They also claim that their newly generated evaluation set avoids test-set contamination. The central claim is that LLMs reflect real-world statistical gender distributions more than human perceptions or neutrality.","tokens_in":5663,"tokens_out":3525,"duration_ms":33717,"significance":"If the comparative methodology were sound, the paper would provide a useful new evaluation set of over 600 occupation-action sentences and a framework for comparing model gender bias against human and statistical references. The dataset and code are released, which is a positive contribution. However, the headline comparative result rests on a technically invalid use of KL divergence across references with different entropies, and the absence of uncertainty quantification makes the model rankings unsupported. The contamination-free claim is asserted without evidence. These issues are central to the paper's contribution, though they are potentially fixable with re-analysis and more transparent reporting.","major_comments":[{"comment":"The central comparison is invalid as presented. D_KL(P||Q) is computed with the model distribution P and a reference Q, and the values for the three references (human perception, U.S. statistics, 50% baseline) are directly contrasted to rank which reference a model is 'closest' to. This is not meaningful because the reference distributions have very different entropies and shapes: the 50% baseline is uniform with maximal entropy, the BLS distribution is strongly skewed for many occupations, and the human perception distribution is intermediate. A model with even a mild male/female skew will mechanically have a lower KL to the concentrated BLS distribution than to the uniform baseline, regardless of 'alignment'. The percentages reported in §4.5 (90%, 320%, 250%, 177%) are therefore expected entropy artifacts, not evidence about model behavior. The paper should use a symmetric or normalized divergence (e.g., Jensen-Shannon divergence) or compare each KL value against the KL of a trivial/chance model, and should report all three values in a way that separates reference entropy from model fit.","section":"§4, Eq. (1)"},{"comment":"The construction of the model probability distribution P is under-specified. The paper states only that 'We aggregated top 10 log probabilities for male and female tokens to estimate binary gender distributions.' It does not specify the token sets for 'male' and 'female' (e.g., whether 'man', 'woman', 'boy', 'girl' are included), how the 10 log probabilities are combined (sum, average, log-softmax), or how the two groups are normalized to sum to 1. Without this, the reported KL values cannot be reproduced or interpreted, and the comparison across models could reflect token-set choices rather than gender bias. Full details or code for this aggregation must be provided.","section":"§3 (Method)"},{"comment":"The results report mean KL divergences with no error bars, confidence intervals, or significance tests. The models are run at temperature 0.1, but the paper does not state the number of runs or whether the same prompts were repeated; for stochastic models, a single run is insufficient to support claims such as 'GPT-4o stands out as most aligned' or 'GPT-3.5-turbo exhibits the closest alignment.' The observed differences between models may be within run-to-run variability. The paper should include repeated sampling, report dispersion (standard deviation or bootstrap CI), and perform a paired comparison across occupations or another appropriate significance test before making ranking claims.","section":"§4.1, §4.2, §4.4"},{"comment":"The claim that the evaluation set 'prevents data leakage and test set contamination' is not substantiated. The occupation-action sentences are generated from occupation titles and common job descriptions; even if the exact sentences are novel, the models' training data very likely contain similar phrases, and for closed models there is no way to verify absence. The paper provides no test of contamination (e.g., comparing model behavior on the generated sentences vs. paraphrased versions, or checking against known training data). This claim is load-bearing for the paper's novelty and should be either supported with evidence or substantially weakened.","section":"§2, Abstract"}],"minor_comments":[{"comment":"The notation is inconsistent: the paper refers to 'GPT-4.0' in §4.1 and 'GPT-4o' elsewhere; 'GPT-4.0' is ambiguous and should be unified to 'GPT-4o'.","section":"§4.1"},{"comment":"The caption reads '50% benchmark was averaged across both datasets' but the sentence does not explain what 'both datasets' refers to here or why averaging is appropriate; please clarify how the three columns were computed.","section":"Figure 2 caption"},{"comment":"The Limitations section cites 'Kennison & Trofe (2003)' but Reference [7] is listed as a single-author work by Kennison (2003); the citation and reference list must be reconciled.","section":"§7"},{"comment":"The prompt in Appendix A.2 ends with 'Your response is 1 word only¡' containing a stray inverted exclamation mark; there is also an extra period after 'word only' that is inconsistent with the instruction that the answer be one word.","section":"Appendix A.2"},{"comment":"The description of Bender et al. [2] as 'evaluated transformer-based models such as BERT and GPT-3' is inaccurate; that paper is a position paper on the risks of large language models and does not present such an evaluation. Please correct the citation context.","section":"§2"}],"recommendation":"major_revision","confidential_remarks":"The paper is a preliminary empirical study with a useful released dataset. The main problem is not that the authors' conclusions are obviously false, but that the current analysis does not support them: the KL comparison across references with different entropies is a methodological error that, if left uncorrected, would mislead readers. With re-analysis using a divergence that controls for reference entropy, and with proper uncertainty quantification, the paper could be viable for an applied NLP venue. The contamination-free claim also needs to be softened or supported. I recommend major revision rather than rejection because the dataset and framework remain potentially valuable after these corrections."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: the 600-occupation evaluation set is a genuinely useful resource, but the paper's headline comparison—models align more with U.S. statistics than with human perception—is built on a statistically invalid use of KL divergence and shouldn't be taken at face value.\n\nWhat the paper does well: it constructs a new test set by starting from BLS occupation titles and generating one-sentence role descriptions, then asks five GPT models to assign male/female. That labor is real. Thinking about evaluation in terms of contamination is a legitimate concern, and comparing model outputs against both human perception data (Kennison) and official labor statistics is a sensible framing. The figures and heatmaps are clear.\n\nThe soft spot is load-bearing. In Section 4.5 the paper compares mean KL divergences computed against three different reference distributions—human perception, BLS statistics, and a 50% baseline. KL is not a normalized distance. A model with any gender skew will tend to have a small KL to a skewed reference (BLS) and a large KL to a uniform reference because the reference entropies differ. So the claim that models 'diverge 320% more' from the 50% benchmark than from statistics is largely an expected artifact, not evidence of alignment. The direction of KL is never justified either; reversing P and Q can change rankings. No error bars or significance tests are given, so we don't know whether model differences are real. Also, the contamination-free claim is not demonstrated—the items are derived from public sources and sentence generation itself uses an LLM; that doesn't guarantee absence from training data. The Limitations section acknowledges some issues (OpenAI-only, outdated perception data, KL's narrowness) but misses this cross-reference problem, which is the central one.\n\nTo be fair, the basic observation that all models deviate from neutrality is plausible and consistent with prior work. For someone building an occupation-level gender bias benchmark, the dataset is worth a look; for someone citing the alignment finding, it isn't. This deserves a serious referee only if the statistical comparison is repaired—say, with a normalized divergence, calibration curves, a symmetric measure, and bootstrap CIs. As presented, I wouldn't accept it, but I'd encourage the author to revise rather than shelve it.","headline":"Useful new occupation dataset, but the central cross-reference KL comparison is statistically invalid and the headline alignment claim doesn't hold.","tokens_in":6178,"tokens_out":4041,"would_cite":false,"duration_ms":36445,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T50","62B10"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper builds a contamination-free test set of over 600 occupations and shows that five GPT-series models all deviate from gender neutrality, aligning more closely with official U.S. labor statistics than with human perceptions.","keywords":["gender bias","large language models","KL divergence","test set contamination","occupational gender stereotypes","GPT-4o","human perception","labor statistics"],"falsifier":"Recompute alignment scores with a symmetric divergence such as Jensen–Shannon, or normalize each model's KL value by the entropy of its reference distribution. If the model ordering changes—say, GPT-4o no longer beats GPT-3.5-turbo on the perception benchmark, or models no longer align more with statistics than with perception—the paper's central ranking fails. A second check is to rerun the GPT-4o versus GPT-4o-mini comparison at temperature 0 with many repeated prompts; if the near-doubled divergence gap shrinks or vanishes, the mini-model result is sampling noise rather than a stable property.","tokens_in":5268,"feed_emoji":"🤖","tokens_out":9433,"duration_ms":72825,"temperature":0.7,"pith_summary":"The paper attempts to establish that gender bias in large language models is real rather than an artifact of contaminated benchmarks, and that current models remain far from gender-neutral behavior. It introduces a contamination-free evaluation set of more than 600 occupation-specific sentences, built from a human perception dataset and official U.S. labor statistics, then probes five GPT-series models for single-word gender judgments. Using the Kullback-Leibler divergence (a standard measure of how one probability distribution differs from another), it compares model outputs to three references: male and female human respondents, U.S. labor statistics, and a 50 percent neutrality baseline. The reported results are that every tested model deviates substantially from neutrality, all models align more closely with the statistical distribution than with human perception, and the alignment ordering differs by reference: GPT-4o is closest to human perception, GPT-3.5-turbo to U.S. statistics, and GPT-4o-mini is almost twice as divergent as GPT-4o.","feed_headline":"All five LLMs fail a 50/50 gender-neutrality test","feed_subtitle":"GPT-4o tracks human views best; GPT-3.5-turbo tracks U.S. stats best, yet both stay far from neutral.","key_machinery":"Three components carry the argument. First, the contamination-free evaluation set: occupations from a human perception study and from U.S. labor statistics are turned into singular occupation-plus-action sentences (e.g., 'Accountant maintains financial records and prepares budgets'), which the paper argues are unlikely to appear verbatim in training corpora. Second, the measurement probe: each model is forced to answer 'male' or 'female' in one word, and the top-ten log probabilities for those two tokens are aggregated into a binary probability distribution. Third, the Kullback-Leibler divergence, $D_{KL}(P \\parallel Q)$, is the identity that converts each model distribution into a scalar 'alignment' score against each reference distribution; the paper's rankings and percentage comparisons are all orderings of these mean divergences. The central analytical move is to treat lower KL divergence as closer agreement and to compare those values across the three references.","core_discovery":"On the paper's own terms, the central discovery is a negative and a positive claim paired together. The negative claim: five GPT-series models—gpt-3.5-turbo, gpt-4, gpt-4-turbo, gpt-4o, and gpt-4o-mini—all produce gender predictions for occupational roles that deviate significantly from a 50/50 no-bias benchmark when measured by KL divergence, so none of them is gender-neutral. The positive claim: the same model outputs align more with the 2023 U.S. labor statistics than with human perceptions, with GPT-3.5-turbo best matching the statistics and GPT-4o best matching human perception. The paper further reports that GPT-4o-mini shows nearly twice the mean KL divergence of its larger sibling GPT-4o, a gap it links to questions about model size and cost advantages. These findings are presented as evidence that bias persists in the newest model generations and persists in a test setting designed to rule out training-data leakage.","pith_inferences":["The cross-benchmark ranking rests on directly comparing KL divergences whose reference distributions have different entropies; normalizing by reference entropy or using a symmetric measure such as Jensen–Shannon divergence would test whether the 'closer to statistics than to human perception' ordering is an artifact of asymmetry.","If official labor statistics are taken as the normative target, the paper's ethical tension becomes concrete: realistic gender ratios today encode historical segregation, so a model that aligns to statistics is not necessarily less biased—it may just be calibrated to an unequal present.","The methodology could be extended to non-binary gender categories or to gender-neutral pronouns; the forced 'male or female' choice likely compresses and partly manufactures the binary signal the paper measures.","A direct follow-up experiment suggested by the data: fine-tune GPT-4o-mini on GPT-4o's probability outputs for the same occupation sentences and check whether the divergence gap narrows, which would implicate capacity or distillation rather than training data."],"forward_implications":["Bias numbers reported on benchmarks that overlap LLM training data should be treated with suspicion; the paper's contamination-free protocol gives a way to measure intrinsic bias instead of memorized benchmark answers.","If GPT-4o-mini's near-doubled divergence is characteristic of smaller or distilled variants, then applications defaulting to cheap, fast models will inherit stronger occupational gender stereotypes than the flagship model.","All five models being closer to U.S. workforce statistics than to a 50/50 ideal means deployment in hiring or content tools will tend to reproduce current occupational segregation, not correct it.","The sentence-construction procedure is replicable and can be applied to other model families, other languages, or other protected attributes without relying on published benchmark sets."],"supporting_citations":[{"why":"Supplies the human gender-perception ratings that are converted into probability distributions for the perception benchmark.","marker":"[7]"},{"why":"Supplies the official U.S. workforce gender ratios that serve as the statistical distribution benchmark.","marker":"[10]"},{"why":"Motivates the need for a contamination-free test set by documenting how benchmark overlap with training data hides model behavior.","marker":"[2]"},{"why":"Cited as the basis for avoiding test-set contamination in assessing foundation-model properties, including bias.","marker":"[4]"},{"why":"Provides the word-embedding bias measurement that the paper extends from static embeddings to generative model predictions.","marker":"[1]"},{"why":"Documents the persistence of gender stereotypes in learned representations, the background expectation the paper tests in newer models.","marker":"[3]"}],"fun_headline_variants":["All five GPT models flunk 50/50 gender neutrality","GPT models skew gender roles mirroring U.S. labor stats","GPT-4o-mini shows double the gender bias of GPT-4o","None of five LLMs hit 50/50 gender-neutral predictions","GPTs align with official stats, not human views, on gender"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline ranking assumes that KL divergences computed against three different reference distributions (human perception, U.S. statistics, and a 50/50 baseline) can be compared directly, even though these references have different entropies and KL divergence is asymmetric.","fun_headline_variants_meta":{"raw":{"variants":["All five GPT models flunk 50/50 gender neutrality","GPT models skew gender roles mirroring U.S. labor stats","GPT-4o-mini shows double the gender bias of GPT-4o","None of five LLMs hit 50/50 gender-neutral predictions","GPTs align with official stats, not human views, on gender"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000533,"raw_usage":{"total_tokens":2541,"prompt_tokens":896,"completion_tokens":1645,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":512,"completion_tokens_details":{"reasoning_tokens":1553}},"tokens_in":512,"tokens_out":1645,"duration_ms":10228,"temperature":1.0,"reasoning_tokens":1553,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:56:15.649323+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute alignment scores with a symmetric divergence such as Jensen–Shannon, or normalize each model's KL value by the entropy of its reference distribution. If the model ordering changes—say, GPT-4o no longer beats GPT-3.5-turbo on the perception benchmark, or models no longer align more with statistics than with perception—the paper's central ranking fails. A second check is to rerun the GPT-4o versus GPT-4o-mini comparison at temperature 0 with many repeated prompts; if the near-doubled divergence gap shrinks or vanishes, the mini-model result is sampling noise rather than a stable property.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the human gender-perception ratings that are converted into probability distributions for the perception benchmark."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the official U.S. workforce gender ratios that serve as the statistical distribution benchmark."}],"review_version":1}