{"id":"d3d4645a-c3bb-472e-8660-a4abc2382c15","arxiv_id":"2504.14212","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A two-stage pipeline using word sense disambiguation and regard classification can audit and rebalance social bias in pretraining corpora like Common Crawl.","lead":"This paper builds an automated pipeline to find and measure social bias in web-scale text by first detecting mentions of protected groups and then classifying the tone of each mention. The authors apply it to Common Crawl and show how to trim negative mentions to reduce stereotyped word associations.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Regard labels are validated only against the Mixtral teacher, not humans; both the measured imbalance and the mitigation reductions inherit any teacher bias.","rationale":"The reader's weakest assumption identifies exactly the same load-bearing concern: the absence of human validation for the regard labels. This is the most fundamental issue because every quantitative claim in the paper—the regard distributions, the bias scores in Tables 2 and 7, the SeeGULL alignment in Table 8, and the mitigation reductions in Section 3.2—is filtered through the regard classifier. If the Mixtral teacher's polarity judgments are systematically biased, the entire empirical edifice shifts, including the central mitigation claim. The paper's own Limitations section acknowledges this risk, which is a point in its favor but does not remove the risk. The concrete test I propose is a human-annotation study on a stratified sample of the regard data; this would directly settle whether the labels are trustworthy. The paper does have independent support: the WSD stage includes human agreement checks, the SeeGULL benchmark provides an external anchor for the bias analysis, and the method is transparently described with prompts included in the appendix. These strengthen the analysis portion, but they do not validate the regard labels themselves. The mitigation claim is also weakened by the absence of a downstream training experiment, as the paper openly admits, and the reported reductions are largely arithmetic consequences of the filtering rule once the labels are taken as given. Still, the paper is careful to call the approach 'promising' rather than proven, and the limitations are disclosed. CONDITIONAL remains the appropriate verdict: the paper should be published only if the human-validation check is added or the claims are softened accordingly.","tokens_in":12143,"tokens_out":4764,"duration_ms":43803,"concrete_test":"Take a stratified random sample of sentences from the regard classification data (e.g., 50 per attribute per predicted class, roughly 1500 total) and have at least three independent human annotators apply the same three-way regard rubric used in the Mixtral prompt (Table 4). Measure Cohen's kappa or macro-F1 between the majority human label and Gloss-RoBERTa. If agreement falls below about 0.7, or if the human-labeled negative ratio for white people (or any other group) differs materially from the Mixtral-derived ratio, the imbalance and mitigation numbers are not established. As a second check, recompute Figure 2 and the post-downsampling p(w|a) values using only human labels to see whether the headline reductions survive.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's analysis and mitigation both depend on the regard labels produced by Gloss-RoBERTa, trained on Mixtral-8x7B-Instruct judgments (Section 2.2). The only reported validation is F1 = 0.91/0.82 against a test set 'double-checked with both Mixtral-8x7B and 8x22B for consistency' (Section 2.3). This measures distillation fidelity, not label validity: two Mixtral models can agree on a systematically biased polarity. No human-annotated regard test set is reported, in contrast to the WSD stage where manual annotation was performed. The Limitations explicitly concede the teacher 'may be biased in some undesirable ways.' If Mixtral systematically over-labels descriptions of white people as negative (or under-labels other groups), then Figure 2's imbalance and the reported reductions in p('supremacist'|'white') and p('terrorist'|'arab') are artifacts of the teacher's bias rather than properties of the corpus. Because the mitigation procedure removes sentences labeled negative by the same classifier, the 19–46% reductions are partly a direct consequence of the filtering rule; the scientific content of the mitigation claim therefore rests entirely on whether the regard labels correspond to human judgments.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a scalable two-stage annotation pipeline for auditing social bias in large pretraining corpora: (1) protected-attribute detection via keyword matching plus Gloss-BERT-style word sense disambiguation, and (2) regard classification into positive/neutral/negative. Both components are trained on labels distilled from Mixtral models and applied to a subset of Common Crawl. The authors introduce a frequency+regard bias score (Eq. 2) and show it aligns better with SeeGULL stereotypes than frequency-only analysis. They then demonstrate a mitigation intervention that downsamples sentences with negative regard to cap the negative ratio at 1%, reporting large reductions in p('supremacist'|'white'), p('racists'|'white'), and p('terrorist'|'arab'). The paper includes a candid Limitations section.","tokens_in":12419,"tokens_out":5150,"duration_ms":49598,"significance":"If the pipeline is reliable, it addresses a real problem: auditing massive web-crawled corpora for social bias in a scalable, interpretable way. The WSD component with hand-crafted attribute glosses is a sensible adaptation of GlossBERT, and the external validation against SeeGULL in Appendix B is encouraging evidence that the regard-aware score captures known stereotypes better than frequency alone. The paper ships a clear method, honest limitations, and illustrative examples. However, the absence of human validation for the regard labels and the self-referential nature of the mitigation evaluation substantially limit the strength of the central claims. The contribution is promising but currently under-verified.","major_comments":[{"comment":"The regard classifier's reported F1 (0.91 micro, 0.82 macro) is computed against a test set annotated by Mixtral-8x7B and Mixtral-8x22B, not by humans. This measures distillation fidelity, not the validity of the polarity labels. The Limitations correctly state that the teacher 'may be biased in some undesirable ways.' Because the regard labels are used throughout the analysis (Table 2, Figure 2, Tables 7–8) and in the mitigation procedure (§3.2), a systematic polarity bias in Mixtral would propagate to all main results. The authors should add a human-annotated regard test set—even a few hundred sentences per attribute—and report agreement, in the same way they already provide human validation for the WSD stage. Without this, the central analysis claim is not independently verified.","section":"§2.2–§2.3"},{"comment":"The mitigation evaluation is circular: the same regard classifier that selects negative-regard sentences for downsampling is used to compute the reduction in p(w|a). If 'supremacist' co-occurs with 'white' mostly in sentences that the classifier labels negative, deleting those sentences must reduce p('supremacist'|'white') regardless of whether the label is correct. The reported reductions (19%, 18%, 46%, 26%) are therefore partly a mechanical consequence of the filtering rule. The paper states in the Limitations that no full pretraining ablation was conducted. To make the mitigation claim load-bearing, the authors should evaluate the balanced corpus with an independent measure—for example, a different regard classifier, human judgments, or the SeeGULL-based alignment from §3.1—or explicitly reframe the section as a data-cleaning demonstration rather than a mitigation result.","section":"§3.2"}],"minor_comments":[{"comment":"The min operation over two ratios with different scales is a heuristic; footnote 6 acknowledges the scale mismatch but the paper does not test sensitivity to alternative combinations (e.g., product or weighted sum). A brief robustness check would increase confidence in the bias score.","section":"§3.1, Eq. (2)"},{"comment":"The 1% cap on negative regard is arbitrary, and the paper does not report how much data is removed for each attribute or how sensitive the reductions are to the cap value.","section":"§3.2"},{"comment":"The recall@k values (e.g., 2.67, 5.01) are presented as decimals without a stated unit; if they are percentages, they should be marked as such.","section":"Appendix B, Table 8"},{"comment":"There is a typo in the Limitations section: 'regard classier' should be 'regard classifier.'","section":"Limitations"},{"comment":"The paper does not release code, the attribute taxonomy, the trained models, or the annotated corpus, which would make the pipeline easier to reproduce and extend.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The authors are refreshingly transparent about limitations, but those limitations undercut the central claims as currently worded. The most fixable gap is human validation of the regard labels; this is achievable on a sample and should be within scope. The mitigation section is currently more of a data-cleaning demonstration than a validated mitigation result; the authors should either add an external evaluation or soften the claim accordingly. The paper is within the scope of the journal and the underlying pipeline has potential, but the evidence as presented is not yet sufficient for acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know before you decide how much time to spend on this. First, the paper's analysis method is a genuinely useful step: it combines frequency association with regard polarity into a single score, and it validates that score against SeeGULL, an external stereotype benchmark. The frequency+regard score consistently beats frequency-only in recall@k (e.g., for negative stereotypes, 25% to 47% at k=500). That is real empirical evidence, not just a suggestive table. Second, the mitigation claim is much weaker than the abstract suggests. The reported reductions in p('supremacist'|'white') and p('terrorist'|'arab') come from downsampling sentences that the same regard classifier labeled negative. So the selection and the measurement share the same possibly biased source. The paper admits it has not run a pretraining experiment, and the Limitations concede the Mixtral teacher may be biased.\n\nWhat's new: the score in eq. (2) — taking the min of normalized frequency bias and normalized regard bias — is a neat way to surface stereotypes rather than mere co-occurrence. The pipeline also does WSD before regard classification, which is more careful than the keyword-matching baseline and has human-annotated agreement in the 0.56–0.70 range. That's credible. The lack of human validation for regard labels is the biggest soft spot. F1 of 0.91/0.82 against a Mixtral-consistency test set measures distillation fidelity, not label accuracy. If Mixtral is systematically biased in its polarity judgments, both the imbalance in Figure 2 and the mitigation numbers inherit that bias. The stress-test note is on point here.\n\nThe citation pattern is fine; it engages with Blodgett et al. and SeeGULL, and it does not ignore the obvious related work. No code or data is released, which is a practical limitation for a pipeline paper.\n\nOverall, this is a solid methods paper for auditing pretraining corpora, not a settling of the bias problem. The analysis part deserves to be taken seriously; the mitigation part should be reframed as an illustration until it is tested with human labels or a downstream training run. I would bring it to a reading group, and I would cite the SeeGULL comparison in my own work on corpus auditing.\n\nRecommendation: send it to peer review. A serious referee will ask for human-validated regard labels and a clearer separation between the analysis and the mitigation claims, but the core idea and the external validation justify referee time.","headline":"A useful corpus-auditing method with a solid SeeGULL check, but the mitigation claim rests on LLM labels that are never validated against humans.","tokens_in":12918,"tokens_out":3747,"would_cite":true,"duration_ms":32191,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A two-step pipeline—protected attribute detection followed by regard classification—can audit and mitigate social bias in Common Crawl, the largest pretraining corpus.","keywords":["social bias","bias mitigation","protected attributes","regard classification","word sense disambiguation","Common Crawl","pretraining corpus","stereotype analysis"],"falsifier":"Have human annotators label a random sample of the same sentences that the pipeline classified as negative toward \"white\" or \"arab\" people, and compare the resulting regard distributions; if humans assign substantially fewer negative labels (or a different ordering across demographics), the reported imbalances and the 19%/18%/46% reductions would not survive. Re-running the downsampling with human labels would settle whether the mitigation is an artifact of the teacher's polarity judgments.","tokens_in":11936,"feed_emoji":"⚖️","tokens_out":7822,"duration_ms":65191,"temperature":0.7,"pith_summary":"This paper argues that a scalable two-step annotation pipeline—keyword-plus-gloss word sense disambiguation to detect protected attributes, then regard classification to label the language around each attribute as positive, neutral, or negative—can surface social biases hidden in Common Crawl, the dominant pretraining corpus. Applied to millions of sentences, the pipeline finds systematic regard imbalances, such as \"white\" appearing with negative polarity far more often than other race/ethnicity attributes. The paper then shows that downsampling negative-regard sentences until no attribute exceeds a 1% negative ratio sharply reduces offensive word associations: the conditional probability of \"supremacist\" given \"white\" falls to 19% of its original value and \"terrorist\" given \"arab\" to 46%, while neutral or positive associations are largely preserved. If correct, this gives pretraining-data curators a practical way to measure and reduce demographic stereotyping before models are trained, rather than only patching model outputs afterward.","feed_headline":"Downsampling negative regard cuts 'supremacist'–'white' odds to 19%","feed_subtitle":"Balancing the sentiment toward demographic groups in pretraining data removes most offensive word associations while keeping neutral ones.","key_machinery":"The load-bearing machinery is the two-stage annotation pipeline. Protected attribute detection first finds mentions of 97 keywords in 10 demographic classes (race/ethnicity, nationality, religion, disability, and others), using Gloss-BERT-style word sense disambiguation: a lightweight RoBERTa classifier decides, for each keyword occurrence, whether the word is used in the sense defined by a hand-crafted gloss (e.g., \"blind\" as visual impairment, not \"blind date\"). Regard classification then labels the polarity toward the detected attribute as positive, neutral, or negative, using a RoBERTa classifier distilled from an instruction-tuned Mixtral teacher. The bias score combines both stages by taking, for each word and attribute, the minimum of the word's frequency bias and its regard bias, so that a word only ranks high when it both co-occurs with the attribute and does so with a specific polarity. The mitigation intervention is the distributional consequence of that score: downsample negative-regard sentences until the negative ratio for every attribute is at most 1%.","core_discovery":"On its own terms, the paper's central claim is that regard-aware co-occurrence statistics, not raw frequency, are the right lens for social-bias auditing of pretraining text. It defines a frequency+regard bias score—the minimum of the frequency bias $p(w|a)/\\mathbb{E}_{a\\in A}p(w|a)$ and the regard bias $p(r|w,a)/\\mathbb{E}_{r\\in R}p(r|w,a)$—and shows that sorting words by this score recovers known stereotypes from a geo-cultural stereotype benchmark at higher recall than frequency alone (e.g., negative-stereotype recall at rank 200 rises from 13.97% to 33.09%). It further reports that Common Crawl's regard distributions are imbalanced across demographics, with \"white\" receiving over 20% negative regard in the race/ethnicity class, and that capping negative regard at 1% by downsampling reduces $p(\\text{supremacist}|\\text{white})$ to 19%, $p(\\text{racists}|\\text{white})$ to 18%, and $p(\\text{terrorist}|\\text{arab})$ to 46% of their original values, while neutral or positive associations stay effectively unchanged.","pith_inferences":["The paper leaves open whether the same balancing would change what a model actually learns; a direct test would pretrain a small language model on the original and the regard-balanced subsets and compare downstream stereotype measures.","Because the regard labels come from a single teacher model, the whole analysis inherits that teacher's polarity judgments; if those judgments are themselves biased, the measured imbalances and reductions describe the teacher's worldview more than the corpus.","The same pipeline could be pointed at any large web-crawled corpus or at finer-grained attributes within a demographic class, and the 1% negative-regard cap is an arbitrary threshold—the paper does not explore where the reduction saturates."],"forward_implications":["Pretraining corpora can be audited at scale for demographic regard imbalance without manual annotation, because the two distilled RoBERTa classifiers run efficiently over hundreds of thousands of sentences.","Rebalancing regard distributions is a concrete mitigation: keeping neutral and positive descriptions of a group while thinning negative ones removes the statistical signal behind offensive stereotypes rather than merely suppressing them at generation time.","Regard-aware scores identify stereotypes that pure co-occurrence misses, such as \"terrorist\"–\"arab\" and \"supremacist\"–\"white\", and align better with a stereotype benchmark.","The approach can target problematic associations while preserving benign ones—for example, retaining a sentence denying that all white people are racists while downsampling one describing a racist attack."],"supporting_citations":[{"why":"Supplies the WSD framework of binary classification of a keyword against a gloss, which the paper adapts to protected attribute detection.","marker":"Huang et al., 2019"},{"why":"Provides the Mixtral teacher model used to synthesize both the WSD training data and the regard labels.","marker":"Jiang et al., 2024"},{"why":"Provides RoBERTa, the base model for the lightweight WSD and regard classifiers that make scalable inference possible.","marker":"Liu, 2019"},{"why":"Defines regard classification and its positive/neutral/negative criteria, which the paper extends from generated text to pretraining corpora.","marker":"Sheng et al., 2019"},{"why":"Supplies the frequency-based word association score that the paper extends with regard information.","marker":"Bordia and Bowman, 2019"},{"why":"Provides the SeeGULL stereotype benchmark used to verify that frequency+regard scores recover known positive and negative stereotypes.","marker":"Jha et al., 2023"},{"why":"Supplies chain-of-thought prompting, which consistently improves the Mixtral teacher's WSD judgments.","marker":"Wei et al., 2022"},{"why":"Motivates the paper's decision to compare attributes only within the same class, avoiding conflation of incomparable demographics.","marker":"Blodgett et al., 2021"}],"fun_headline_variants":["Regard-aware bias score more than doubles stereotype recall","Downsampling negative regard cuts supremacist-white odds by 81%","Balancing demographic regard in pretraining text strips biased links","Frequency-only audit misses stereotypes; regard-aware finds them","Capping negative regard at 1% preserves neutral associations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire analysis and mitigation rest on the assumption that the regard labels produced by the teacher model are an accurate proxy for human judgments of positive, neutral, and negative language toward each demographic group; the paper verifies consistency between two teacher runs but does not compare regard labels against human annotation.","fun_headline_variants_meta":{"raw":{"variants":["Regard-aware bias score more than doubles stereotype recall","Downsampling negative regard cuts supremacist-white odds by 81%","Balancing demographic regard in pretraining text strips biased links","Frequency-only audit misses stereotypes; regard-aware finds them","Capping negative regard at 1% preserves neutral associations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001039,"raw_usage":{"total_tokens":4351,"prompt_tokens":902,"completion_tokens":3449,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":518,"completion_tokens_details":{"reasoning_tokens":3366}},"tokens_in":518,"tokens_out":3449,"duration_ms":25306,"temperature":1.0,"reasoning_tokens":3366,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:53:51.871072+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have human annotators label a random sample of the same sentences that the pipeline classified as negative toward \"white\" or \"arab\" people, and compare the resulting regard distributions; if humans assign substantially fewer negative labels (or a different ordering across demographics), the reported imbalances and the 19%/18%/46% reductions would not survive. Re-running the downsampling with human labels would settle whether the mitigation is an artifact of the teacher's polarity judgments.","supporting_citations":[],"review_version":1}