{"id":"328c9885-a549-4a1d-aa0b-07690abcda4b","arxiv_id":"2603.08091","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"JudgeBiasBench measures 12 LLM-judge bias types under a 4-dimension taxonomy, and bias-aware GRPO/InfoNCE training cuts BSR substantially while preserving general evaluation performance.","lead":"This paper introduces JudgeBiasBench, a benchmark that measures 12 types of judgment bias in LLM-based judges across both generative and discriminative evaluation setups. It also proposes a bias-aware training approach that reduces these biases while keeping general judging accuracy roughly intact.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"BSR's asymmetric flip-count can reward bias reversal: Table 5's Acc_inj > Acc_ori pattern suggests debiasing may be shifting, not removing, preference.","rationale":"The reader's weakest assumption concerns the reliability of Gemini-2.5-Pro/GPT-4o as verifiers for ground-truth labels. That is a legitimate concern and would affect the validity of JudgeBiasBench. However, the verifier issue is external and empirical: a verifier could turn out to be sufficiently reliable. The BSR asymmetry is an internal, formal property of the paper's central metric. It directly threatens the debiasing claim: a judge that merely swaps which side of a bias it favors will show low BSR while remaining biased. The paper's own reported numbers make this concrete, because Acc_inj exceeds Acc_ori in every bias-aware row, which is exactly the signature of wrong-to-correct flips. A symmetric flip metric or a direction probe would settle the question. Thus I partly agree with the reader's conditionality but identify a different, more load-bearing soft spot. The verdict remains CONDITIONAL pending the proposed checks.","tokens_in":19297,"tokens_out":12522,"duration_ms":145171,"concrete_test":"Recompute the full 2x2 original/injected confusion matrix per bias type and per model for Table 5, and report a symmetric flip rate: (correct->incorrect + incorrect->correct) / N. Also run a position-swap probe on JudgeBiasBench—present every pair in both orders for generative judges—and report P(prefer first) before and after bias-aware training. If the symmetric flip rate is not substantially reduced, or if the debiased judge shows a strong ordering preference, the low BSR is a metric artifact rather than genuine debiasing.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"BSR (Eq. 4) counts only one cell of the original-vs-injected confusion matrix: originally correct judgments that flip to incorrect. Flips in the opposite direction—originally incorrect becoming correct after bias injection—are ignored. A judge with a strong but opposite bias (e.g., always preferring the second response, or systematically penalizing a particular style) will have low BSR because its original errors are excluded from the denominator, even though it is still fully bias-driven. This is not a remote possibility: in Table 5, every bias-aware row shows Acc_inj higher than Acc_ori (generative Qwen2.5: 77.4 vs 72.1; discriminative Llama-3.1: 83.7 vs 77.2; discriminative Qwen2.5: 80.5 vs 75.5). Since JudgeBiasBench mostly places the chosen response first, that pattern is consistent with the debiased model having acquired a first-position heuristic or an inverted stylistic preference. The central claim that bias-aware training 'effectively reduces judgment biases' therefore rests on an asymmetric metric that can be lowered by reversing biases rather than removing them; the reported BSR reductions are not by themselves evidence of unbiased judgment.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes JudgeBiasBench, a benchmark for evaluating judgment biases in LLM-based judges, with a taxonomy of 12 bias types across 4 dimensions (superficial quality, context, presentation, diversity). Bias-augmented instances are created through counterfactual rewriting, context distraction, layout manipulation, and identity injection; a verifier model filters instances to preserve the original preference. The paper evaluates a variety of generative and discriminative judges on JudgeBiasBench using a new metric, Bias Sensitivity Rate (BSR), and proposes a bias-aware training framework: SFT plus GRPO for generative judges, and InfoNCE contrastive learning for discriminative judges. The central claim is that this training substantially reduces judgment bias while largely preserving general evaluation capability on RewardBench, JudgeBench, RMB, and RM-Bench.","tokens_in":19547,"tokens_out":6838,"duration_ms":63394,"significance":"The problem is timely and important: LLM-as-a-judge is widely deployed, and a unified taxonomic benchmark covering both generative and discriminative judges would be a useful community resource. The paper's empirical coverage is substantial (12 bias types, 18 judges), and the debiasing recipe is clearly described. However, the current evidence does not yet support the central claim. The BSR metric is asymmetric and can be lowered by replacing one bias with another; the training and test bias-injection pipelines are nearly identical; and the ground-truth verification rests on unvalidated proprietary verifiers. These issues are fixable but require additional analysis, so the paper is not acceptable in its present form.","major_comments":[{"comment":"The BSR metric is asymmetric: it counts only flips from originally correct to incorrect and ignores flips from originally incorrect to correct. A judge with a systematic opposite bias (e.g., a first-position preference) will have low BSR because its original errors are excluded from the denominator. This is not a theoretical edge case: in Table 5, every bias-aware row shows Acc_inj > Acc_ori (generative Qwen2.5: 77.4 vs 72.1; discriminative Qwen2.5: 80.5 vs 75.5; Llama-3.1: 83.7 vs 77.2). Since the benchmark places the chosen response first for most bias types (Section 3.3.2), this pattern is exactly what a first-position heuristic would produce. The reported BSR reductions are therefore not by themselves evidence of bias removal. Please report a symmetric metric (e.g., total flip rate or both directions of the confusion matrix) and show that Acc_inj is not inflated by a position heurist","section":"Eq. (4), Table 5"},{"comment":"The training data for debiasing uses the same bias-injection recipe as the test benchmark. Section 5.1 stage 2 explicitly says context and presentation biases are introduced by the manipulation methods of Section 3.3.2, and the same rewriting/injection templates are used for the other categories. The low BSR of the bias-aware models on JudgeBiasBench may therefore reflect overfitting to the specific injection templates rather than a general reduction in bias. The abstract and Section 5.4.2 claim robustness, but the evaluation is on the same distribution. Please add a held-out test set with bias types or injection styles not seen in training, or at least report per-bias-type results that separate seen and unseen manipulations.","section":"Sections 3.3.2 and 5.1"},{"comment":"The validity of JudgeBiasBench rests on the assumption that Gemini-2.5-Pro (test) and GPT-4o (training) reliably verify that the original preference is preserved after bias injection. No human validation, inter-verifier agreement, or error analysis of the verifier is reported. If the verifier shares the target bias or imposes its own preference, the filtered instances are mislabeled and every BSR number is systematically wrong. Please provide a human-evaluation sample of the filtered test set and report per-bias-type verifier agreement; a robust approach would use multiple verifiers and retain only instances with consensus.","section":"Section 3.3.3 and Section 5.1 stage 3"},{"comment":"No confidence intervals, bootstrap estimates, or significance tests are reported. Sample sizes per bias type are 291–500, and differences such as BSR 10.8 vs 20.7 in Table 5 may be within noise for some columns. The claim of a 'substantial reduction' in judgment bias should be accompanied by uncertainty quantification or a significance test. This is a standard requirement for benchmark comparisons and is needed to support the paper's main quantitative conclusion.","section":"Tables 4-6"}],"minor_comments":[{"comment":"The InfoNCE loss is labeled 'InfoNSE' in the figure; correct the typo.","section":"Figure 3"},{"comment":"The model is referred to as 'Claude-3-7-Sonnet' in Finding 1, but Section 4.1 and Table 4 use 'Claude-3.7-Sonnet'. Unify the name.","section":"Section 4.3 Finding 1"},{"comment":"The definition of BSR in Eq. (4) should explicitly state that flips from incorrect to correct are not counted. Consider adding a sentence and perhaps a full confusion-matrix table.","section":"Section 4.2"},{"comment":"The paper does not include a data/code availability statement. A clear statement that the benchmark, the bias-aware data, and code will be publicly released would strengthen reproducibility.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses an important problem and the proposed benchmark has potential, but the main quantitative evidence needs re-analysis. The BSR asymmetry is a serious issue that the authors may not have fully considered; the empirical pattern in Table 5 is consistent with bias reversal (e.g., a first-position heuristic) rather than bias removal, so the title-level claim of 'debiasing' is not yet established. The overlap between training and test bias-injection recipes further weakens the generalization claim. I recommend major revision rather than rejection because these issues are addressable with additional experiments and re-analysis."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know: JudgeBiasBench is a genuinely useful resource, but the paper's central claim—that bias-aware training reduces bias—rests on a metric that can be lowered by flipping the model's bias rather than removing it, and the training data and test set are built from the same injection recipe.\n\nWhat's actually new and good: the taxonomy is a real step up from CALM/EvalBiasBench. Four dimensions, 12 bias types, both generative and discriminative judges, and a clean separation between bias and judgment error. The evaluation across many models is broad, and the findings—reasoning helps, length/position/beauty stay sticky, fine-tuned judges are more vulnerable—are plausible and worth knowing. The paper is clearly written and the construction pipeline is transparent enough to reproduce.\n\nThe soft spots, in proportion. The BSR metric (Eq. 4) counts only flips from correct to incorrect after injection. Flips the other way—incorrect becoming correct—are ignored. A judge that simply inverts its position preference will have low BSR even though it is still fully bias-driven. Table 5 shows exactly that pattern: every bias-aware row has Acc_inj higher than Acc_ori (e.g., generative Qwen2.5: 77.4 vs 72.1; discriminative Llama-3.1: 83.7 vs 77.2). That is not what you expect if bias were removed. It is what you expect if the model learned a new heuristic that happens to align with the injection. So the headline \"effectively reduces judgment bias\" is not established by the reported BSR numbers alone.\n\nAlso real: ground truth for the test set depends on Gemini-2.5-Pro as verifier, and training data verification uses GPT-4o. If those verifiers share the target biases, the labels are wrong. That's a risk, not a nitpick. And there are no error bars or significance tests, and no code/data release. The circularity concern from the shared injection pipeline is genuine, though the paper does report Acc_ori and Acc_inj separately, which is enough to see the anomaly.\n\nNot everything is soft. The benchmark itself may still be diagnostic even if BSR is imperfect, and the preservation of general benchmark agreement is plausible. The paper is worth serious referee time because the community needs a careful discussion of bias metrics and independently constructed bias tests.\n\nWho this is for: people building reward models and LLM-based judges, and anyone designing bias benchmarks. I would send it to peer review, but I would expect major revision: fix or supplement BSR with the full confusion matrix, add an independently constructed bias test not built from the same recipe, release data and code, and report variance. Without those, the debiasing claim stays unverified.","headline":"Useful benchmark, but the debiasing result is likely inflated by an asymmetric metric and the eval/train pipeline sharing bias recipes.","tokens_in":20074,"tokens_out":2451,"would_cite":true,"duration_ms":28260,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LLM judges are systematically biased, and a new training method measurably reduces it.","keywords":["judgment bias","LLM-as-a-judge","automated evaluation","reward modeling","bias taxonomy","debiasing","GRPO","contrastive learning"],"falsifier":"Take a random sample of JudgeBiasBench's filtered pairs, have independent human annotators relabel which response is better, and check agreement with the verifier's labels. If human agreement is notably lower than the verifier's internal consistency, the benchmark is measuring verifier bias rather than judge bias — and the reported BSR reductions would need re-interpreting.","tokens_in":19153,"feed_emoji":"⚖️","tokens_out":4703,"duration_ms":47858,"temperature":0.7,"pith_summary":"This paper argues that large language models used as judges — the systems that score and rank AI-generated answers — are systematically swayed by task-irrelevant cues such as verbosity, formatting, confidence, majority opinion, and identity. To make that argument testable, it introduces JudgeBiasBench, a benchmark with a four-part taxonomy and twelve concrete bias types, plus a metric called Bias Sensitivity Rate that isolates bias from ordinary reasoning errors. Across many generative and discriminative judges, the paper shows that bias is widespread, persists in strong models, and is not predicted by normal accuracy. It then proposes bias-aware training — reinforcement learning for generative judges and contrastive learning for discriminative judges — and reports that it sharply lowers bias sensitivity while keeping general evaluation quality roughly unchanged. A sympathetic reader would care because these judges are also reward models in alignment pipelines, so unmeasured bias can become reward hacking.","feed_headline":"Twelve hidden biases skew AI judge scores; new training cuts them","feed_subtitle":"A benchmark measures bias in AI-as-judge systems, then shows how to reduce it without losing evaluation accuracy.","key_machinery":"The machinery is JudgeBiasBench: a controlled bias-injection pipeline that rewrites or recontextualizes preference pairs so that the task-relevant quality stays fixed while one task-irrelevant factor varies, then uses a strong verifier to filter out pairs whose preference would actually reverse. The quantitative instrument is the Bias Sensitivity Rate (BSR), defined as the share of originally correct judgments that turn wrong after injection. The debiasing machinery is a set of bias-augmented preference pairs — a chosen response contrasted with several rejected responses that look superficially attractive for different bias reasons — trained with GRPO (a reinforcement-learning objective) for","core_discovery":"On the paper's own terms, the discovery is that judgment bias in LLM-based judges is a measurable, general phenomenon under a clear definition: a judge is biased when changing a task-irrelevant attribute — style, context, layout, or identity — flips an otherwise correct preference. JudgeBiasBench operationalizes this with controlled injections over twelve bias types and a consistency filter that keeps only instances where the original quality ordering is preserved. Using the resulting measure, the authors find that most judges, including strong reasoning models and high-accuracy reward models, reverse correct judgments at rates that vary from roughly 10% to over 40%, and that length, positio","pith_inferences":["The benchmark's numbers depend on the verifier's own impartiality; a natural extension is to re-label the filtered pairs with human annotators or with several independent verifiers and compare BSR estimates.","The bias-aware training recipe could be tested on bias types outside its taxonomy, such as language, dialect, or formatting quirks, since the contrastive and reinforcement objectives are not tied to the twelve predefined cues.","If bias-aware judges are used inside RLHF, one would expect fewer spurious reward signals and less reward hacking; that is a testable downstream prediction the paper does not run.","The tension between bias-aware data proportion and general performance suggests an optimal-calibration curve; a practical extension would be a scheduling rule that anneals bias-aware data as training progresses."],"forward_implications":["If BSR is adopted as a reporting standard, judge evaluations will need to report robustness alongside accuracy, because the paper shows high clean accuracy does not imply low bias.","Bias-aware training can cut BSR substantially on the twelve tested bias types with only a small change in general benchmark agreement, making it a practical recipe for reward-model training.","Because length, position, and beauty biases persist even in strong models, any evaluation pipeline should treat these as residual known risks.","The taxonomy's separation of bias from judgment errors gives downstream users a cleaner diagnostic: when a judge fails, one can ask whether it is a reasoning failure or a systematic preference for a task-irrelevant cue.","Discriminative judges' elevated gender and race biases imply alignment pipelines using them need explicit fairness checks."],"fun_headline_variants":["AI judges reverse correct calls up to 40%—new training fixes bias","Twelve hidden biases skew AI evaluators; debiasing training works","JudgeBiasBench: 12 bias types, up to 40% reversals, and a remedy","Measure AI judge bias across 12 types, then train it away"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The entire benchmark rests on the claim that the verifier used to filter bias-injected pairs agrees with true quality; if that verifier shares the target biases, the retained test pairs are mislabeled and every BSR number is systematically wrong.","fun_headline_variants_meta":{"raw":{"variants":["AI judges reverse correct calls up to 40%—new training fixes bias","Twelve hidden biases skew AI evaluators; debiasing training works","JudgeBiasBench: 12 bias types, up to 40% reversals, and a remedy","Measure AI judge bias across 12 types, then train it away"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000312,"raw_usage":{"total_tokens":1603,"prompt_tokens":724,"completion_tokens":879,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":468,"completion_tokens_details":{"reasoning_tokens":793}},"tokens_in":468,"tokens_out":879,"duration_ms":8204,"temperature":1.0,"reasoning_tokens":793,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T05:49:31.558409+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a random sample of JudgeBiasBench's filtered pairs, have independent human annotators relabel which response is better, and check agreement with the verifier's labels. If human agreement is notably lower than the verifier's internal consistency, the benchmark is measuring verifier bias rather than judge bias — and the reported BSR reductions would need re-interpreting.","supporting_citations":[],"review_version":1}