{"id":"b8a1c1f0-82c5-4d93-b533-355bde4e23cf","arxiv_id":"2505.23829","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"BiasFilter filters low-fairness segments during LLM generation using a reward model trained on a GPT-4-scored preference dataset, cutting bias on CEB and FairMT.","lead":"This paper presents BiasFilter, a method that reduces social bias in AI text generation by checking and filtering the AI's output as it writes. It works with any language model, including commercial APIs, without changing the model itself.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Most load-bearing risk: BiasFilter's reward model is trained on GPT-4 labels from the same 0-99 bias prompt used to compute the CEB Bias Score, so reported gains may reflect optimizing the judge rather than reducing bias.","rationale":"I considered the reader's Eq. 2 partial-reward concern first. It is a real secondary issue: the DPO implicit reward is defined for complete sequences, and Eq. 2's cumulative weighted log-ratio for prefixes is not validated to predict full-sequence bias; the ablations in Fig. 4-5 only test segment length and sample count, not whether partial reward correlates with final bias. However, even a perfect partial-reward model would not save a claim measured by the same judge used to train that model. The evaluation-overlap concern is the most load-bearing because it attacks the validity of every quantitative result in Tables 1-4. This is not a data-leakage claim in the usual sense: the authors say they removed HolisticBias prompts overlapping CEB. The issue is that the scoring function is shared, so the model is trained to satisfy the same function it is later evaluated by. I am not claiming fraud; this is an experimental-design confound that the paper's own text makes visible. I also verified the 'consistently outperforms' clause and found counterexamples in Table 1, which strengthens my view that the abstract overstates the results. The check is feasible because the outputs already exist and the cost is annotation/classification, not new generation. I keep the reader's CONDITIONAL verdict: the method may work, but the headline claim should not be accepted as stated without an independent judge.","tokens_in":23155,"tokens_out":9566,"duration_ms":107930,"concrete_test":"Re-score a stratified random sample of at least 200 stored outputs per condition (BiasFilter, best baseline, base) from Tables 1-4 using an independent bias judge: human annotators applying the CEB/FairMT annotation guidelines, or a separately trained stereotype classifier whose training data, prompt, and model family differ from Figure 7 (e.g., a DeBERTa-based bias model or a different LLM with a distinct rubric). Compare the BiasFilter-vs-best-baseline and BiasFilter-vs-base gaps under the original GPT-4 scoring and under the independent judge. If the BiasFilter advantage shrinks by more than 50% or reverses, the central debiasing claim is not yet established and the paper should require human-validated results (CONDITIONAL). If the advantage persists, the evaluation-overlap concern is resolved.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The most load-bearing risk is that the success metric is the same GPT-4 rubric used to create the reward model's training signal. In §3.1, GPT-4 scores HolisticBias-derived completions for stereotypical bias using the prompt in Figure 7/Appendix A.2. In §4.1, the CEB Bias Score—the paper's primary metric—is 'computed by GPT-4 based on the degree of stereotypical bias, with the prompt shown in Figure 7.' The DPO reward model in §3.2 is trained on those GPT-4 labels, and Algorithm 1 selects/filters candidates with that model (Eq. 3-5). Tables 1, 2, and 4 therefore report how well BiasFilter optimizes the exact judge used to label its training data. Improvements may reflect learned stylistic patterns (e.g., hedged, non-committal phrasing) that receive low bias scores from GPT-4, rather than removal of stereotypical content; Regard score is a partial independent check but is a sentiment/favorability measure, not a stereotype measure. Human validation during dataset construction does not resolve this, because the final evaluation still uses the same judge. This concern is empirical: it would be settled by re-judging the stored outputs with an independent instrument. A separate weakness is that even under the paper's own metric, 'consistently outperforms six competitive baselines' is contradicted by Table 1 cells such as Llama-3-8B CEB-conversation race (BiasFilter 19.1 vs. RLRF 16.5), gender (9.8 vs. ARGS 8.6), and religion (17.1 vs. SD-Ex 13.1).","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"BiasFilter is an inference-time debiasing framework. The authors construct a fairness preference dataset by sampling completions from five LLMs for HolisticBias-derived prompts, scoring them with GPT-4 for stereotypical bias, and human-validating preference pairs. They train a DPO-based reward model (Llama-3-8b-SFT2) with a token-level partial reward defined as a weighted sum of log-ratios. At decoding time, the method maintains a beam of candidate sequences, generates segments of length l, scores each partial sequence with the reward model, prunes to the top-N, and finally returns the highest-scoring complete candidate. Experiments on CEB and FairMT across open-source (Llama, Mistral, Qwen) and API models (GPT-3.5-Turbo, GPT-4o) are reported against six baselines, using GPT-4-based Bias Score/Bias Rate and Regard score, plus perplexity and distinct-2 for quality. The authors claim substantial bias reduction and consistent outperformance of baselines while preserving generation quality.","tokens_in":23536,"tokens_out":9513,"duration_ms":87493,"significance":"The paper addresses a practically important problem: reducing social bias in LLM generation without retraining the base model. The main contributions—a fairness preference dataset, a token-level reward model, and a model-agnostic inference-time filtering algorithm—are well motivated, and the experimental scope is broad (multiple model families, two benchmarks, six baselines, ablations). If the results hold under an independent evaluation instrument, BiasFilter would be a useful addition to the debiasing toolbox, particularly for API-based models. The framework ships a concrete algorithm (Algorithm 1), and the inference-time design leaves the base model untouched. However, the current evidence is weakened by a circularity between the reward-model training signal and the primary evaluation metric, by the absence of error bars or significance tests, and by results that contradict the 'consistently outperforms' claim. These issues are addressable with additional experiments and more careful reporting.","major_comments":[{"comment":"The primary evaluation metric is the same GPT-4 rubric used to create the reward model's training labels. In §3.1, GPT-4 scores responses with the prompt in Figure 7 to construct preference pairs, and the DPO reward model in §3.2 is trained on those pairs. In §4.1, the CEB Bias Score is 'computed by GPT-4 based on the degree of stereotypical bias, with the prompt shown in Figure 7.' Algorithm 1 then filters and selects outputs using that reward model. Tables 1, 2, and 4 therefore largely report how well BiasFilter optimizes the exact judge that labeled its training data. The Regard score is an independent metric, but it measures sentiment/favorability rather than stereotype content, so it does not break the loop. I request an independent evaluation of the stored outputs—for example, human annotations on a sample of final generations, or a bias classifier that does not derive from GPT-4 with the same rubric—to verify that the reported debiasing transfers outside the training loop. Without such evidence, the central claim that BiasFilter 'mitigates social bias' is not yet established.","section":"§3.1 and §4.1"},{"comment":"The claim that BiasFilter 'consistently outperforms all baselines' is contradicted by the reported numbers. For Meta-Llama-3-8b-Instruct on CEB-conversation, RLRF achieves lower Bias Scores on age (16.5 vs. 19.1) and gender (9.5 vs. 9.8); ARGS achieves lower on age (17.7 vs. 19.1) and gender (8.6 vs. 9.8); SD-Ex achieves lower on age (16.7 vs. 19.1). In Table 4, applying BiasFilter to Qwen2.5-3B-Instruct increases the FairMT FixF Bias Rate from 97.6 (base) to 100.0 (with BiasFilter). The narrative should be adjusted to 'most settings' with a detailed account of where baselines win, or significance tests should be provided to show that the remaining differences are not noise. As written, the paper's headline claim overstates the results.","section":"§4.2, Table 1 and Table 4"},{"comment":"Equation (2) defines the partial reward as a weighted sum of token-level DPO log-ratios, and Algorithm 1 relies on this score to prune candidates at every segment. The paper does not validate that this cumulative score correlates with full-sequence bias, nor that the correlation holds for base models other than Llama-3-8b-SFT2 (the reward model's own family). Because the reward model is a policy trained on full-sequence preferences, its per-token likelihood ratios on partial sequences may not track the fairness of the partial output; if early-segment scores are unreliable, the beam search can discard fair continuations or keep biased ones, and the debiasing effect would not transfer across the diverse base models in Tables 1 and 4. I request a validation experiment that compares partial-reward rankings with full-sequence bias judgments (e.g., GPT-4 or human scores) on held-out prompts for several base models. Without this, the core filtering mechanism is unsubstantiated for the generality claimed.","section":"§3.2, Eq. (2)"},{"comment":"The main hyperparameters are tuned on the evaluation benchmark. The segment length l is fixed to 128 because Fig. 4 shows that this is the best trade-off on CEB-continuation with Meta-Llama-3-8B-Instruct, and §4.1 then uses l=128 for all CEB models and tasks (and l=50 for FairMT, presumably chosen similarly). Since CEB is also the benchmark on which the main results in Table 1 are reported, this is effectively test-set tuning. Additionally, no standard deviations or significance tests are reported anywhere; single-run differences such as 19.1 vs. 17.7 may be within sampling noise. Please either use a development split for hyperparameter selection, or report variance over multiple seeds, or restrict the claim of 'significant' improvements to differences that survive a significance test.","section":"§4.1 and Fig. 4"}],"minor_comments":[{"comment":"Lines 8–10 use the loop variable `yk` both for a sampled continuation and for the prefix that produced it, and the final selected output `y*` is not defined in the algorithm's notation; rewriting with distinct variable names would improve clarity.","section":"Algorithm 1"},{"comment":"The notation `rpartial(y:K | x)` mixes `y:K` and `y:k`; I recommend defining `y_{1:K}` and `y_{1:k}` explicitly and stating that K is the number of tokens in the partial sequence.","section":"Eq. (2)"},{"comment":"The sentence \"Points closer to the top-right corner indicate a better trade-off\" appears to be a typo, because the x-axis is time and the y-axis is Bias Score, so lower-left is the favorable region.","section":"Fig. 4"},{"comment":"The appendix refers to the tuning procedure as \"FairDPO,\" but the main text calls the model a fairness reward model; please align the names to avoid confusion.","section":"Appendix B.1"},{"comment":"The Mistral-7B model is cited as (Chaplot, 2023); the correct reference is Jiang et al. (2023), and the current citation does not point to a proper Mistral technical report.","section":"References"},{"comment":"Several cells show that BiasFilter worsens the Bias Rate relative to the base model (e.g., Llama-3-8b-Base IntM: 93.3 to 96.4; Qwen2.5-3B FixF: 97.6 to 100.0), yet the text states that the method \"consistently improves\" fairness across models; these counterexamples should be acknowledged and discussed.","section":"Table 4"}],"recommendation":"major_revision","confidential_remarks":"No additional concerns beyond those raised in the major comments. The paper's scope fits the journal, and the novelty is adequately framed relative to prior work."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Core idea is solid: train an implicit reward model on fairness preference pairs, then at inference time score partial generations and beam-search over candidate continuations, keeping the most fair ones. That's a sensible engineering combination of DPO rewards and best-of-n filtering, and it's genuinely useful because it works on API models you can't fine-tune. The fairness preference dataset is a real artifact, with human validation on top of GPT-4 scoring, and the experiments cover a lot of ground: seven open-source plus two API models, two benchmarks, and reasonable quality metrics. The case studies look convincing.\n\nThe biggest issue is the evaluation loop. The reward model is trained on GPT-4 bias scores using the prompt in Figure 7, and the headline CEB metric is also GPT-4 bias scores using the same prompt. So Tables 1 and 4 partly measure how well the filter optimizes the exact judge that labeled its training data. Gains could be stylistic hedging that GPT-4 rewards rather than genuine stereotype removal. The Regard score is a partial independent check, and the FairMT prompt is different, but the primary evidence suffers. Fix: re-judge stored outputs with a different instrument (e.g., a second LLM with a different rubric, or human annotation on a sample), and report that as the main table.\n\nSecond, the claim of consistently outperforming six baselines is overstated. In Table 1, on Llama-3-8B conversation, ARGS beats BiasFilter on gender bias score (8.6 vs 9.8), and there are a few other cells where a baseline edges ahead. The overall trend is still in BiasFilter's favor, especially on continuation tasks, but 'consistently' needs softening.\n\nThird, there are no error bars or significance tests anywhere, and the segment length was selected after looking at the CEB-continuation results. That's hyperparameter tuning on the evaluation set; it inflates confidence. Also, the partial reward in Equation 2 is assumed to track full-sequence bias, but that's not validated away from the reward model's own family.\n\nNone of this sinks the paper. The approach is practical, the dataset is useful, and the direction is right. But the evaluation needs to be cleaned up before this is trustworthy as a production recommendation. I'd send it to review with a request for major revision: add an independent judge, add variance estimates, and fix the claims. It's for practitioners who want a drop-in debiasing filter for APIs, and for researchers working on decoding-time alignment. Worth a serious referee.","headline":"Sound inference-time debiasing idea with a real judge circularity problem; worth serious refereeing after evaluation is tightened.","tokens_in":24052,"tokens_out":3514,"would_cite":true,"duration_ms":34138,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"BiasFilter claims that social bias in LLM generation can be reduced at inference time by filtering partial outputs with a fairness reward.","keywords":["social bias mitigation","inference-time debiasing","fairness reward model","direct preference optimization","open-ended generation","large language models","multi-turn dialogue fairness","fairness preference dataset"],"falsifier":"On a held-out set of CEB continuation prompts, run BiasFilter with an unseen base model, log the partial-reward score at every checkpoint, and have independent judges rate the final responses for bias. If a substantial fraction of outputs that passed every checkpoint are rated biased—or if discarded prefixes would have led to fair completions—then the partial reward is not a faithful fairness signal.","tokens_in":22968,"feed_emoji":"⚖️","tokens_out":8832,"duration_ms":85281,"temperature":0.7,"pith_summary":"BiasFilter is an attempt to make social-bias mitigation a plug-in operation rather than a retraining procedure. The framework stops a base LLM every fixed number of tokens, scores each partial continuation with a fairness reward, keeps a small set of high-reward candidates, and continues only those candidates. The reward signal comes from an implicit reward model trained with direct preference optimization on a newly constructed fairness preference dataset of biased and unbiased response pairs. On the CEB and FairMT benchmarks the paper reports lower bias scores and bias rates across age, gender, race, and religion for seven open-source models and two API-based models, while perplexity and distinct-2 measures stay roughly the same or improve. If these results hold, debiasing can be added to any generator at inference time without touching its weights.","feed_headline":"Inference-time filter cuts LLM social bias without retraining","feed_subtitle":"A DPO-trained reward model scores partial outputs every few tokens and keeps only fair continuations.","key_machinery":"The load-bearing mechanism is a beam-search-style filter driven by a partial fairness reward. At each segment of length $l$, every surviving candidate spawns $N_{\\mathrm{children}}$ new continuations sampled from the base model; all candidates are scored by the partial reward $r_{\\mathrm{partial}}(y_{:K}\\mid x) = \\sum_{k=0}^{K-1} w_k \\log \\pi(y_k \\mid x, y_{:k}) / \\pi_{\\mathrm{ref}}(y_k \\mid x, y_{:k})$ with weight $w_k = 1/|y_{:k}|$, and the top $N$ survive to the next round. This partial reward is the DPO-style policy ratio of a fairness-trained model $\\pi$ versus its reference $\\pi_{\\mathrm{ref}}$, so it is the trained component that carries the bias judgment. The base generator only samples text; all debiasing work is done by pruning on this reward.","core_discovery":"The paper's central claim is that a sequence does not have to be completed before its fairness is judged. BiasFilter generates in segments, and after each segment it uses a token-level fairness reward to delete the candidates most likely to end in stereotyped text. The authors state that this procedure substantially reduces social bias on age, gender, race, and religion in both continuation and conversation/multi-turn tasks, and that it consistently beats six existing debiasing methods while preserving—or in several cases slightly improving—fluency and diversity. The base generator is never modified; the framework only decides which sampled continuations survive.","pith_inferences":["The same filtering loop could in principle carry other safety objectives—toxicity, privacy, or hallucination avoidance—if a token-level reward model for that objective is trained.","The design hides a testable premise: partial-sequence reward must match full-sequence bias judgments. A calibration study on unseen base-model families would settle it.","Because the filter optimizes the reward rather than the truth of the text, it may be vulnerable to reward hacking—continuations that game the fairness score while remaining biased in ways the judge does not catch.","The most practical long-run shape of this idea is a centralized fairness service that wraps arbitrary third-party generators; whether that works depends on how well the reward model's judgments transfer across domains and languages."],"forward_implications":["A single fairness reward model can be reused across many base generators, because BiasFilter never modifies the generator.","API-based black-box models can be debiased without any access to weights or gradients.","Users can dial latency against bias strength by changing the segment length $l$ and the candidate count $N_{\\mathrm{children}}$.","Multi-turn fairness improves alongside single-turn fairness, suggesting the filter does not disrupt dialogue structure.","Fluency and diversity survive filtering, since reported perplexity and Distinct-2 values are comparable or better after debiasing."],"supporting_citations":[{"why":"Supplies the HolisticBias descriptor terms and sentence templates used to build the prompt pool for the fairness preference dataset.","marker":"Smith et al., 2022"},{"why":"Provides the DPO loss that turns fairness preference pairs into the policy-ratio reward used in training.","marker":"Rafailov et al., 2023"},{"why":"Defines the partial-reward formulation as a cumulative weighted sum of token-level log-ratios.","marker":"Qiu et al., 2024"},{"why":"The CEB benchmark whose continuation and conversation tasks, prompts, and Bias Score protocol evaluate the debiasing results.","marker":"Wang et al., 2024"},{"why":"The FairMT benchmark providing the multi-turn dialogue tasks and Bias Rate metric.","marker":"Fan et al., 2024a"},{"why":"BiasDPO, a fine-tuning baseline BiasFilter is compared against on the main tables.","marker":"Allam, 2024"},{"why":"Dexperts, an inference-time expert/anti-expert baseline in the comparison.","marker":"Tong et al., 2024"},{"why":"Self-debiasing baselines (reprompting and explanation variants) in the comparison.","marker":"Gallegos et al., 2024b"},{"why":"RLRF, a PPO-based reflection baseline that must be outperformed.","marker":"Cheng et al., 2024"},{"why":"ARGS, a reward-guided decoding-time alignment baseline that BiasFilter compares against.","marker":"Khanov et al., 2024"}],"fun_headline_variants":["Filter LLM outputs mid-generation to kill bias","Token-level fairness reward prunes biased LLM continuations","Debias LLMs on the fly with a token-level reward","No retraining: score partial LLM outputs for fairness","Prune biased text as LLM generates, no retraining"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire filter rests on the premise that the weighted sum of token-level preference ratios for an incomplete text correctly orders continuations by how biased the finished text will be, even for base models the reward model was not trained on.","fun_headline_variants_meta":{"raw":{"variants":["Filter LLM outputs mid-generation to kill bias","Token-level fairness reward prunes biased LLM continuations","Debias LLMs on the fly with a token-level reward","No retraining: score partial LLM outputs for fairness","Prune biased text as LLM generates, no retraining"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001356,"raw_usage":{"total_tokens":5449,"prompt_tokens":837,"completion_tokens":4612,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":453,"completion_tokens_details":{"reasoning_tokens":4531}},"tokens_in":453,"tokens_out":4612,"duration_ms":30346,"temperature":1.0,"reasoning_tokens":4531,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:14:56.099454+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a held-out set of CEB continuation prompts, run BiasFilter with an unseen base model, log the partial-reward score at every checkpoint, and have independent judges rate the final responses for bias. If a substantial fraction of outputs that passed every checkpoint are rated biased—or if discarded prefixes would have led to fair completions—then the partial reward is not a faithful fairness signal.","supporting_citations":[],"review_version":1}