{"id":"b93afdb2-0c55-4625-addc-f1d633e4a94d","arxiv_id":"2412.03537","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Intrinsic and prompt-adapted gender biases in causal LLMs are highly correlated (Pearson rho >= 0.94) on WinoBias, contrary to earlier fine-tuning results.","lead":"This study tests whether gender biases in large language models carry over when the models are adapted with prompts instead of fine-tuning. It reports that the bias rankings of Mistral, Falcon and Llama models under zero-shot and few-shot prompting stay strongly correlated with their pre-trained biases, even with fairness or bias-inducing prompts.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's headline claim is about pre-trained models, but Section 3.1 says all evaluations use instruction-tuned versions; the reported rho therefore does not yet establish transfer from pre-training.","rationale":"I read the paper in good faith. The empirical study is careful in several ways: it covers multiple model families, five random seeds, randomized option ordering, and it directly contrasts with earlier MLM fine-tuning results. The reported correlations are high and consistent across neutral, fairness-inducing, and bias-inducing prompts, which is real evidence that prompt-adapted behavior tracks something stable in the model. However, the abstract's 'pre-trained' language is load-bearing because the paper's advertised contribution is to the bias-transfer hypothesis for pre-trained causal models. Section 3.1 explicitly says the experiments use 'instruction fine-tuned versions,' and no later section reports a separate analysis of base checkpoints. That is an internal mismatch between the experiment and the conclusion. A rerun on base checkpoints would settle whether the claim is about pre-training or about instruction-tuned alignment. I therefore keep the reader's conditional verdict, but the condition should include checkpoint family, not only metric definition. The secondary ambiguity about signed versus absolute Selection Bias is worth fixing, but the checkpoint mismatch is the concern I would hang the verdict on. The reader's weakest assumption concerned metric alignment; that is related but not the same as the checkpoint identity issue I identify here.","tokens_in":11552,"tokens_out":8596,"duration_ms":89973,"concrete_test":"Obtain the exact base (non-instruction-tuned) checkpoints for Mistral-7B, Falcon-40B, Llama-3-8B, and Llama-3-70B (or the exact model identifiers from the authors) and rerun the Section 4.1 intrinsic, zero-shot, and few-shot protocol and Table 2 correlations on the same WinoBias data using the same signed per-occupation Selection Bias convention as Figure 2. If the base-checkpoint correlations remain at or above the claimed thresholds, the concern is resolved; if they drop below, the headline claim must be restricted to instruction-tuned models.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim—that intrinsic biases in pre-trained Mistral, Falcon, and Llama models predict zero- and few-shot prompted biases—requires that the 'intrinsic' measurements come from the pre-trained checkpoints. Section 3.1 instead states: 'we investigate fairness in adaptations using the instruction fine-tuned versions of highly performant LLMs.' Instruction-tuned models have already undergone a task-specific alignment stage; their next-token-generation biases are not necessarily the pre-training biases. If the reported rho values were obtained on instruction-tuned models, the abstract's 'pre-trained' claim is not supported by these experiments, and the paper's implication that pre-training fairness determines prompt-adapted fairness does not follow. The table and figure captions do not identify exact model revisions or state whether base or instruct checkpoints were used, so the mismatch is unresolved. This is an internal inconsistency between the experiment and the conclusion, not a disagreement with an external consensus.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies whether gender bias transfers from pre-trained causal language models to prompt-adapted versions. Using WinoBias pronoun co-reference resolution, the authors define an intrinsic next-token-generation evaluation and zero- and few-shot multiple-choice prompting evaluations, and measure a per-occupation Selection Bias (SB) metric. They report strong Pearson correlations (rho >= 0.94) between intrinsic and prompted biases for Llama 3 8B/70B, Falcon 40B, and Mistral 7B, and correlations remain high (rho >= 0.92) when models are pre-prompted with fairness- or bias-inducing prompts and when few-shot composition is varied. The paper concludes that pre-trained model fairness is a strong predictor of prompt-adapted bias, in contrast to prior findings for fine-tuned masked language models.","tokens_in":11735,"tokens_out":5419,"duration_ms":56595,"significance":"If the central claim were established for base pre-trained checkpoints, the result would be an important contribution to the bias transfer literature: it would show that, unlike fine-tuning adaptation, prompt adaptation preserves intrinsic bias, making pre-training fairness critical for downstream deployments. The paper has real strengths: it proposes a unified metric for intrinsic and prompted evaluations, evaluates several model families at multiple scales, averages over random seeds, randomizes answer ordering, and is transparent about the limitation to WinoBias and binary gender. The high correlations on the instruction-tuned checkpoints actually used are themselves interesting. However, the abstract and conclusion make a stronger claim about pre-trained models than the experimental setup in Section 3.1 supports, and the ambiguity in the SB definition affects the interpretation of every correlation coefficient. These issues are fixable but currently prevent the headline result from being accepted as stated.","major_comments":[{"comment":"The experiments use instruction fine-tuned versions of the models, but the abstract claims to show that 'intrinsic biases in pre-trained Mistral, Falcon and Llama models' correlate with prompted biases. Section 3.1 states: 'we investigate fairness in adaptations using the instruction fine-tuned versions of highly performant LLMs.' The intrinsic evaluations in Table 1 are next-token-generation measurements on these instruction-tuned checkpoints, so they reflect an already-adapted model, not the pre-trained checkpoint. Consequently, the reported rho values do not establish transfer from pre-training to prompt adaptation. The authors must either rerun the intrinsic and prompted evaluations on the base (non-instruction-tuned) checkpoints, or substantially revise the abstract, introduction, and conclusion to claim only that biases in instruction-tuned models predict their prompted behavior. The captions should also identify the exact checkpoints used (for example, Mistral-Instruct version, Falcon-Instruct version, and Llama 3 base vs. Instruct variants).","section":"Section 3.1, Tables 1-2, Abstract and Section 6"},{"comment":"Selection Bias is defined in Section 3.2 as 'the absolute difference in rates that an occupation is generated by a model when a male pronoun is present in a sentence vs. a female pronoun.' However, Figures 5-9 plot signed values, with the captions explaining that 'less than zero is female-biased and greater than zero is male-biased,' and Figure 2 describes biases as 'directionally aligned.' Pearson correlations between per-occupation SB values depend critically on whether the signed or absolute formulation is used; absolute values would lose direction and could inflate positive correlation through shared magnitude effects. The paper should state explicitly which definition was used for the correlations in Table 2 and Figure 3, define the signed SB formally (for example, as P(occupation | male pronoun) - P(occupation | female pronoun)), and, if advisable, report both signed and absolute correlations.","section":"Section 3.2, Figures 2 and 5-9"},{"comment":"The claims about bias transfer under fairness- and bias-inducing pre-prompts are based only on the most effective positive pre-prompt and the most effective negative pre-prompt, selected 'in an ad-hoc and iterative way' from a larger set (Appendix E). Because the selection criterion is the same SB that is later correlated, the correlations in Table 2 are conditional on this outcome-based selection and are not corrected for multiple comparisons. This makes the 'bounds of bias transfer' claim in Section 4.2 exploratory rather than confirmatory. The authors should report results for all prompts in the appendix, or pre-specify the prompt set, and show the distribution of correlations across prompts.","section":"Section 4.2 and Appendix E"},{"comment":"The claim that 'increasing number of pro-stereotypical ambiguous samples incrementally worsens SB' is not supported by the data in Table 4: the pro-stereotypical mean SB values are 23.86, 22.77, 25.29, and 23.71 for 3, 10, 20, and 30 shots, respectively, which is not a monotonic increase. If the intended claim concerns only ambiguous (Type 1) sentences, the table does not provide the Type 1/Type 2 split for this experiment. Please clarify the claim or provide the supporting per-type numbers.","section":"Section 4.3 and Table 4"}],"minor_comments":[{"comment":"The paper states that for Llama 3 8B with negative pre-prompts, 'its guardrails are triggered for nearly 4% of the dataset,' but does not describe how these cases are handled in the RPA and SB computations. Please state whether refusals are counted as incorrect answers, excluded from the denominator, or mapped to 'Unknown.'","section":"Section 4.2"},{"comment":"Pearson correlations are reported without confidence intervals, and Table 4 reports only 'p approximately zero' rather than exact p-values. Given that the correlations are computed over a fixed set of occupations, confidence intervals would help assess the stability of the rho >= 0.94 claim.","section":"Tables 1, 2, 4 and Figure 2"},{"comment":"The composition experiments are conducted only on Llama 3 8B, but the abstract states that bias transfer remains strong when few-shot length and stereotypical composition are varied without noting this model restriction. Please scope the claim explicitly or add the analogous experiments for at least one more model.","section":"Section 4.3 and Abstract"},{"comment":"The model names are inconsistent with the cited references: Table 1 and the appendix refer to 'Mistral 3 7B' and 'Llama 3 8B/70B,' but the cited Mistral paper is 'Mistral 7B' and the cited Llama paper is the original Llama. Please use exact official checkpoint names and versions throughout.","section":"Section 3.1 and Appendix A-D"},{"comment":"The legend text for the intrinsic condition says there is no standard deviation 'as there is no stochasticity involved in the next token selection,' but the intrinsic condition is still compared with seeded stochastic prompting conditions; this is helpful but should be stated once in the main text to avoid confusion about whether intrinsic results are aggregated over the same seeds.","section":"Figures 5-9"}],"recommendation":"major_revision","confidential_remarks":"The checkpoint mismatch between the abstract's 'pre-trained' language and the instruction-tuned models in Section 3.1 is the most serious issue; if the authors cannot rerun on base checkpoints, the paper should be reframed as a study of instruction-tuned models. The Selection Bias sign ambiguity is also central. The paper is otherwise well structured, and the empirical pattern is worth publishing after these corrections."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper reports that gender biases in Mistral, Falcon, and Llama models are strongly correlated (rho >= 0.94) between intrinsic next-token generation and zero-/few-shot prompted multiple-choice coreference resolution. That is a genuinely new and useful empirical result: prior bias-transfer work focused on masked LMs and fine-tuning, and this is the first to test the hypothesis for causal LLMs under prompting. The setup is careful in several ways: multiple model families, five random seeds, randomized option ordering, and a systematic variation of few-shot composition. The proposed Selection Bias metric is a reasonable attempt to unify intrinsic and prompt-adapted measurement, and the finding that correlations stay high even under explicit debiasing pre-prompts is striking.\n\nBut there is a load-bearing mismatch. The abstract and conclusion say 'pre-trained' models, yet Section 3.1 states that all evaluations use 'instruction fine-tuned versions.' Instruction-tuning is a task-specific adaptation stage; its biases are not necessarily the pre-training biases. The rho values therefore do not establish that pre-training fairness transfers to prompting. They establish that instruction-tuned model biases transfer to prompting. That is still interesting, but it is a different claim, and the current framing overstates the implication. This needs either new experiments on base checkpoints or a re-framing of the contribution.\n\nThe other soft spots are secondary but real. Selection Bias is defined in Section 3.2 as an absolute difference, while the figures plot signed values; the correlation coefficients depend on which version is used, and the paper does not clarify. The prompt selection is post hoc: only the most effective fairness- and bias-inducing prompts are reported, which can inflate apparent bounds, though the high correlations across all conditions suggest the effect is robust. There are no confidence intervals for the correlations, and no code or data are released. Guardrail refusals are mentioned for negative prompts but not described in the metric computation.\n\nIf the checkpoint issue is fixed or the claims are re-scoped, the core result is solid and worth publishing. The paper deserves a serious referee, but not in its current form. I would send it back for major revision, asking the authors to either rerun on base models or revise the abstract and conclusions to say 'instruction-tuned' consistently, and to clarify the Selection Bias sign convention. For the right audience — fairness researchers and practitioners deploying LLMs via prompts — this is a useful empirical data point, but the current version's central inference does not hold as written.","headline":"Strong prompt-adaptation bias transfer result, but the headline claim about pre-trained models is not supported by the experiments, which use instruction-tuned checkpoints.","tokens_in":742,"tokens_out":752,"would_cite":false,"duration_ms":29806,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Prompt-adapted LLMs inherit the gender bias of their pre-trained base models.","keywords":["gender bias","bias transfer hypothesis","prompt adaptation","large language models","coreference resolution","WinoBias","selection bias","model fairness"],"falsifier":"Re-run the correlation with the distractor's position in the multiple-choice options randomized blockwise; if placing the distractor first reliably lowers the Pearson coefficient from $\\rho \\ge 0.94$ to near zero on the same occupations, the transfer claim would be an answer-order artifact. A second check: measure intrinsic and prompted SB on a set of occupations not in WinoBias; if a model with high intrinsic SB shows near-zero prompted SB there, transfer would not generalize beyond the benchmark.","tokens_in":11393,"feed_emoji":"⚖️","tokens_out":7075,"duration_ms":64980,"temperature":0.7,"pith_summary":"This paper asks whether the gender bias built into a pre-trained causal language model survives when the model is adapted by zero- or few-shot prompting, the lightweight route commonly used to deploy LLMs. It answers yes: on a pronoun co-reference task built from WinoBias sentences, intrinsic biases in Mistral, Falcon, and Llama correlate with prompted biases at $\\rho \\ge 0.94$. The correlation holds when models are pre-prompted to behave fairly or biasedly ($\\rho \\ge 0.92$) and when the number and stereotypical composition of few-shot examples are varied ($\\rho \\ge 0.97$). A sympathetic reader would take this as evidence that fairness in the base model determines, to a large degree, fairness after prompt adaptation.","feed_headline":"Prompting does not erase a language model's gender bias","feed_subtitle":"Across Mistral, Falcon and Llama, intrinsic bias correlates at 0.94 or higher with zero- and few-shot prompted bias.","key_machinery":"The load-bearing object is the Selection Bias (SB) metric, designed so that intrinsic and prompted bias can be compared on the same scale. For intrinsic evaluation, the model's sum of token log-probabilities for the referent and distractor occupations decides the answer; for prompted evaluation, the next generated token picks among the referent, distractor, and 'Unknown' options in a randomized multiple-choice prompt. The transfer test is the Pearson correlation between per-occupation SB in the intrinsic setting and per-occupation SB in the prompted setting. WinoBias supplies 3,160 balanced sentences split into syntactically ambiguous (Type 1) and unambiguous (Type 2) items, and each prompting experiment is run over five random seeds with shuffled option order to reduce format noise.","core_discovery":"The central discovery is that the bias transfer hypothesis holds for prompt-adapted causal LLMs when intrinsic and prompted biases are measured with the same quantity, the Selection Bias (SB) metric. SB is the difference between the rate at which an occupation is generated when the pronoun is male and the rate when it is female; the paper computes this per occupation and correlates the intrinsic and prompted versions across occupations. Under this metric, intrinsic bias in the pre-trained model strongly predicts zero-shot and few-shot prompted bias, with Pearson coefficients $\\rho \\ge 0.94$ and negligible $p$-values. Explicit pre-prompts that instruct fairness or bias shift overall bias levels but leave the correlation essentially intact ($\\rho \\ge 0.92$), and few-shot context length and stereotypical makeup also do not break it ($\\rho \\ge 0.97$). The paper concludes that biases do transfer under prompting, making pre-trained model fairness a practical determinant of downstream prompt-adapted model fairness.","pith_inferences":["If the correlation is as stable as reported, an intrinsic audit (token log-probability scoring) could serve as a cheap screening step for likely deployed fairness before running full behavioral evaluations of prompt-adapted models.","The same experimental design could be pushed to other causal adapters—low-rank adapters, full fine-tuning, chain-of-thought prompting—to map where transfer finally breaks; earlier fine-tuning results suggest it might.","Because Section 3.2 defines SB as an absolute difference while the figures plot signed values, re-running the analysis with signed per-occupation bias would show whether the direction, not just the magnitude, of bias transfers; a flip in direction would change the fairness prescription.","Extending beyond binary gender to gender-neutral pronouns is the paper's own stated limitation; a dataset that unambiguously marks singular 'they' would test whether the strong transfer persists outside the binary."],"forward_implications":["Zero- and few-shot prompting do not dilute the gender bias already present in the pre-trained model; the base model's per-occupation bias predicts the prompted model's bias with $\\rho \\ge 0.94$.","Fairness work on the pre-trained model is therefore a practical lever for fairness in prompt-deployed systems: prompt-level instructions shift the level of bias but leave the correlation with intrinsic bias intact.","Few-shot prompt design—how many examples are used and whether they are pro-stereotypical, anti-stereotypical, or neutral—does not change the strength of bias transfer ($\\rho \\ge 0.97$).","The finding contrasts with earlier bias-transfer studies on fine-tuned masked language models, implying that the adaptation strategy (fine-tuning versus prompting) is a decisive variable in whether intrinsic bias transfers."],"supporting_citations":[{"why":"Introduces the bias transfer hypothesis and the Pearson-correlation method this paper adopts.","marker":"[Steed et al., 2022]"},{"why":"Provides the WinoBias dataset and its pro-/anti-stereotypical coreference test used throughout.","marker":"[Zhao et al., 2018]"},{"why":"The prior causal-model bias-transfer study this work contrasts with, as it found no transfer under fine-tuning.","marker":"[Cao et al., 2022]"},{"why":"Attributes conflicting bias-transfer results to metric incompatibility, which motivates the unified Selection Bias metric.","marker":"[Delobelle et al., 2022]"},{"why":"Establishes prompting as the lightweight adaptation paradigm whose fairness this paper evaluates.","marker":"[Brown et al., 2020]"},{"why":"Supplies the Mistral 7B model that is one of the three model families tested.","marker":"[Jiang et al., 2023]"},{"why":"Supplies the Falcon 40B model tested.","marker":"[Almazrouei et al., 2023]"},{"why":"Supplies the Llama 8B and 70B models tested.","marker":"[Touvron et al., 2023]"}],"fun_headline_variants":["Prompting can't wash out LLM gender bias","Bias transfer holds even with fair prompts","Pretrained bias predicts prompted bias in LLMs","Prompt adaptation fails to remove gender bias"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The result assumes that Selection Bias measured by token log-probability sums in the intrinsic setting and by next-token choice in the prompted setting capture the same underlying bias, rather than two measurements that share an artifact such as occupation-name tokenization or answer-order effects.","fun_headline_variants_meta":{"raw":{"variants":["Prompting can't wash out LLM gender bias","Bias transfer holds even with fair prompts","Pretrained bias predicts prompted bias in LLMs","Prompt adaptation fails to remove gender bias"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00052,"raw_usage":{"total_tokens":2524,"prompt_tokens":960,"completion_tokens":1564,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":576,"completion_tokens_details":{"reasoning_tokens":1507}},"tokens_in":576,"tokens_out":1564,"duration_ms":11860,"temperature":1.0,"reasoning_tokens":1507,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T22:17:07.065571+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the correlation with the distractor's position in the multiple-choice options randomized blockwise; if placing the distractor first reliably lowers the Pearson coefficient from $\\rho \\ge 0.94$ to near zero on the same occupations, the transfer claim would be an answer-order artifact. A second check: measure intrinsic and prompted SB on a set of occupations not in WinoBias; if a model with high intrinsic SB shows near-zero prompted SB there, transfer would not generalize beyond the benchmark.","supporting_citations":[],"review_version":1}