REVIEW 3 major objections 5 minor 8 references
AI vs. Human Judgment of Content Moderation: LLM-as-a-Judge and Ethics-Based Response Refusals
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read LLM-as-a-Judge systems rate ethical refusals more favorably than human users do, a gap the paper calls moderation bias.
desk verdict A genuinely new LLM-as-a-judge bias finding that rests on a single-author classifier the paper doesn't fully validate. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is a two-part measurement setup: a five-way classifier that sorts responses into ethical refusals, ethical disclaimers, technical refusals, technical disclaimers, and standard responses, paired with a pairwise LLM-as-a-Judge protocol in which a judge model sees the prompt and two anonymized responses and picks a winner, a loser, or a tie. The classifier is a transformer-based text classifier trained on 3,500 hand-annotated responses and applied to all 99,876 responses; the judge protocol follows a standardized prompt, with response length and prompt-response similarity included as controls. This setup lets the paper compare win/loss/tie rates for each response type across human voters and two AI judges, and the contrast between ethical and technical refusals is what isolates the bias.
What would settle it
Re-label a random sample of the 99,876 responses by hand, re-run the user-versus-judge win/loss regressions using only the responses where the hand label matches the classifier, and check whether the ethical-refusal gap persists; if the gap shrinks to nothing on the confirmed labels, the moderation bias depends on classifier error rather than on genuine judgment differences.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that LLM-as-a-Judge evaluation carries a moderation bias: model judges consistently rate ethical refusals more favorably than human users, and this divergence disappears for technical refusals. In the descriptive data, ethical refusals win only 8% of human comparisons but 31% under GPT-4o and 27% under Llama 3 70B, while technical refusals move from 16% under humans to 27% and 24% respectively. The regression results show the user penalty for an ethical refusal is a 32 percentage point reduction in win rate, versus only 12 points under GPT-4o and 23 under Llama 3 70B; for technical refusals the penalties are statistically indistinguishable between users and GPT-4o. The authors interpret the pattern as alignment signaling: models trained to value safety and normative compliance reward ethical refusals as good behavior, whereas human users experience them as evasive or moralizing.
Load-bearing premise
The whole user-versus-judge comparison rests on the automated classifier correctly sorting every response into ethical refusal, technical refusal, disclaimer, or standard response; if its labeling errors line up with how judges rate those responses, the moderation bias could be an artifact of misclassification rather than a real difference in judgment.
Editorial extensions
If this is right
- If LLM judges are used as training signals, ethical refusals will be rewarded more than human users reward them, pushing deployed models toward more refusal-heavy moderation.
- Benchmarks that score models with LLM judges will systematically favor safety-aligned refusal responses, so high benchmark scores can coexist with low user satisfaction.
- Because the effect shows up in both a proprietary and an open judge model, it is likely a general property of alignment-trained judges rather than a quirk of one model.
- Since technical refusals do not show the divergence, the bias is about ethical alignment signaling specifically, not about declining to answer in general.
Reading between the lines
- Editorial inference: The paper measures an association, not a causal effect; a direct test would compare the same base model as a judge before and after safety fine-tuning to see whether alignment training alone creates the bias.
- Editorial inference: The user-vote baseline is itself a particular population, so the 'bias' partly reflects whose preferences count; a more diverse human rater pool could shrink or reshape the gap.
- Editorial inference: A practical extension would be to report refusal-type-specific agreement between LLM judges and human raters on evaluation cards, making moderation bias visible in routine model audits.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper asks whether LLM-as-a-Judge (LaaJ) systems evaluate ethical refusals differently than human users. Using 49,938 one-turn Chatbot Arena comparison pairs, the authors classify responses into ethical refusals, technical refusals, ethical disclaimers, technical disclaimers, and standard responses using a RoBERTa classifier taken from the author's prior work (Pasch, 2025). They then compare human win/loss/tie decisions with judgments from GPT-4o and Llama 3 70B under the standardized Zheng et al. (2023) pairwise prompt. Descriptive and OLS results show that ethical refusals are penalized much less by both LaaJ judges than by human users (win rate 31% for GPT-4o and 27% for Llama versus 8% for users), while technical refusals show a smaller or reversed divergence. The paper terms this divergence 'moderation bias' and discusses implications for transparency, alignment, and LLM-based evaluation pipelines.
Significance. If the classification and inference hold, the finding is significant: it documents a systematic divergence between automated judges and human users on a safety-relevant behavior, with direct implications for training, benchmarking, and model-selection pipelines that increasingly rely on LLM-as-a-Judge evaluations. The two-judge replication (proprietary GPT-4o and open-weight Llama 3 70B) is a strength, as are the controls for response length, prompt-response similarity, and opponent response features. The paper is also appropriately cautious in disclaiming normative conclusions about whether users or judges are 'right' about refusals. The main weakness is that the entire empirical claim rests on classifier labels that are not sufficiently validated in this manuscript, and label uncertainty is never propagated into the regression estimates; this makes the headline result less credible than it should be.
major comments (3)
- [§3.2, Table 2] The central estimate of moderation bias rests entirely on predicted labels from the RoBERTa classifier taken from Pasch (2025). The manuscript reports only 'Accuracy & F1 Score = 88%' and defers all details elsewhere; no per-class confusion matrix, class-wise precision/recall, or external validation is given. Because the contrast of interest is the 'Refusal Ethical' category (2.6% of responses), non-differential misclassification cannot explain the user-judge gap, but differential error can: if the classifier systematically labels safety-phrased disclaimers or standard responses as ethical refusals, and if LLM judges reward that phrasing more than users do, then the Refusal Ethical coefficients in Table 2 would be inflated for GPT-4o and Llama relative to the user equation. The controls for length and similarity do not address this because the confound is the category label itself. The paper should report the full confusion matrix, propagate classification uncertainty into Table 2 (e.g., by bootstrapping labels), and provide a robustness version estimated on the 3,500 hand-labeled responses. The limitation discussion in §5.4 does not currently mention this threat.
- [§4, Table 2 and text] The text reports p < 0.001 for the difference between the user and GPT-4o ethical-refusal coefficients and p > 0.05 for technical refusals, but no test statistic or procedure for comparing coefficients across the separate regressions is described anywhere in §4. Separate OLS columns do not supply a cross-equation p-value; the authors should specify the test used (e.g., a Wald test on a pooled model with evaluator interactions, seemingly unrelated estimation, or a bootstrap) and report the test statistics for the ethical versus technical contrasts. Without this, the inferential claim for H1 is not auditable.
- [§3.1, Table 2] The unit of analysis is the individual response, but responses are nested in 49,938 comparison pairs, and within a pair the win/loss outcomes are exactly complementary (and ties are shared). The reported robust standard errors are not clustered by pair or prompt, so the effective sample size for inference on the refusal coefficients is smaller than the number of response-level observations suggests. Clustering by pair (and possibly by prompt) should be reported; given the large effects this is unlikely to overturn H1, but it is needed for correct inference. The paper should also explain why the GPT-4o regression has 49,900 observations rather than 49,938.
minor comments (5)
- [§3.2] The phrase 'Accuracy & F1 Score = 88%' is ambiguous; please state whether this is macro-F1, weighted F1, or a single overall number, and report class-wise F1 for all five categories, especially 'Refusal Ethical', which is the category driving the main claim.
- [§3.3] The paper says the standardized prompt format from Zheng et al. (2023) is used, but does not provide the exact prompt, model temperature, max tokens, or API versions; an appendix with the precise prompt and inference settings would help reproducibility.
- [§3.1] Please report the date/version of the Chatbot Arena data used, the exact criterion for selecting 'one-turn' conversations, and how the three-way outcome (win/loss/tie) is coded in the regression (presumably two indicator outcomes with tie as the omitted reference).
- [Table 2] Since the opponent-side controls are included but not shown, report their coefficients or at least state clearly that all four opponent refusal categories are included as dummies; this would help readers assess whether the Refusal Ethical effect is net of opponent behavior.
- [References] The reference list contains minor inconsistencies, such as in-text 'Saha et al., 2024' versus the reference year 2025, and 'Whittlestone et al.' appearing with both 2019 and 2021 dates; please reconcile these.
Circularity Check
No significant circularity: the moderation-bias claim is a new empirical comparison and does not reduce to the paper's own definitions, fitted values, or cited prior work.
full rationale
The paper's central claim—that LLM-as-a-Judge systems rate ethical refusals more favorably than human users—is estimated from a new pairwise comparison: the same Chatbot Arena responses are judged by human users, GPT-4o, and Llama 3 70B, and the refusal categories are labels applied to the responses before either judgment is modeled. No equation in the paper defines judge ratings as a function of the refusal labels; the labels come from a RoBERTa classifier trained in Pasch (2025) on 3,500 hand-annotated responses, which is external to the present paper's regression outcomes. The classifier's error rate and the lack of a published confusion matrix are legitimate validity threats, and the paper explicitly acknowledges its descriptive, non-causal scope in Section 5.4, but these are correctness concerns rather than circularity: a misclassification would bias the estimates, not make the estimate equal to its input by construction. The self-citation to Pasch (2025) is load-bearing for the category labels, but the cited classifier was not fitted to the LaaJ-versus-human divergence it is used to measure, so the citation functions as independent empirical support rather than as a self-referential premise. The 'moderation bias' label is a new term for the observed coefficient pattern, not a renaming of the input data. Accordingly, no circular step is present.
Assumptions & free parameters
free parameters (1)
- RoBERTa classifier parameters (Pasch, 2025) =
not reported (88% reported F1)
assumptions (3)
- domain assumption The RoBERTa classifier from Pasch (2025) accurately reproduces the manual boundary-setting labels when applied to the full Chatbot Arena dataset, and its errors do not alter the human-vs-LaaJ comparison.
- domain assumption Human votes from Chatbot Arena, restricted to single-turn pairs, are a valid measure of user preference for refusal behavior.
- domain assumption GPT-4o and Llama 3 70B prompted with the Zheng et al. (2023) evaluation template are representative of LLM-as-a-Judge frameworks.
Cite this review
Pith. "Pith review of AI vs. Human Judgment of Content Moderation: LLM-as-a-Judge and Ethics-Based Response Refusals." pith.science (2026). https://pith.science/paper/SIDUSORE
@misc{pith2026250515365,
author = {Pith},
title = {Pith review of: AI vs. Human Judgment of Content Moderation: LLM-as-a-Judge and Ethics-Based Response Refusals},
year = {2026},
howpublished = {\url{https://pith.science/paper/SIDUSORE}},
note = {Machine review of arXiv:2505.15365}
}
read the original abstract
As large language models (LLMs) are increasingly deployed in high-stakes settings, their ability to refuse ethically sensitive prompts-such as those involving hate speech or illegal activities-has become central to content moderation and responsible AI practices. While refusal responses can be viewed as evidence of ethical alignment and safety-conscious behavior, recent research suggests that users may perceive them negatively. At the same time, automated assessments of model outputs are playing a growing role in both evaluation and training. In particular, LLM-as-a-Judge frameworks-in which one model is used to evaluate the output of another-are now widely adopted to guide benchmarking and fine-tuning. This paper examines whether such model-based evaluators assess refusal responses differently than human users. Drawing on data from Chatbot Arena and judgments from two AI judges (GPT-4o and Llama 3 70B), we compare how different types of refusals are rated. We distinguish ethical refusals, which explicitly cite safety or normative concerns (e.g., "I can't help with that because it may be harmful"), and technical refusals, which reflect system limitations (e.g., "I can't answer because I lack real-time data"). We find that LLM-as-a-Judge systems evaluate ethical refusals significantly more favorably than human users, a divergence not observed for technical refusals. We refer to this divergence as a moderation bias-a systematic tendency for model-based evaluators to reward refusal behaviors more than human users do. This raises broader questions about transparency, value alignment, and the normative assumptions embedded in automated evaluation systems.
Reference graph
Works this paper leans on
-
[1]
I’m sorry, I can’t assist with that request because it involves unsafe content
Introduction As large language models (LLMs) are deployed at scale across increasingly sensitive domains, concerns about their ethical alignment, safety, and trustworthiness have emerged as key priorities for developers, researchers, and policymakers (Hagendorff, 2020). A core strategy for addressing these concerns is for models to refuse to answer certai...
work page 2025
-
[2]
Theoretical Background 2.1. LLM-as-a-Judge As large language models (LLMs) become increasingly central to artificial intelligence research and deployment, their role has expanded from generation to evaluation. One widely adopted strategy is the use of LLM-as-a-Judge (LaaJ) frameworks, in which one model ranks or scores outputs from other models to enable ...
work page 2024
-
[3]
I cannot help with that request as it may cause harm
Methodology 3.1. Data We use data from Chatbot Arena, a large-scale evaluation platform for conversational AI models (Chiang et al., 2024). The dataset consists of 57,477 response pairs, where each pair includes answers from two different models to the same user-submitted prompt. Human users are asked to choose which response they prefer or to select a ti...
work page 2025
-
[4]
Ethical 1235 1.2 0.38 0.34 0.28 1.12 0.46 0.46 0.08 1.01 0.49 0.49 0.02 1.00 Discl
Results Table 1: Response Refusals and Win/Loss/Tie Rates for Users & LLM-as-a-Judge Distribution User Decision LLM-as-a-Judge (GPT-4o) LLM-as-a-Judge (Llama 3 70b) Response # Share Win Loss Tie Win/ Loss Win Loss Tie Win/ Loss Win Loss Tie Win/ Loss Normal 87054 87.1 0.36 0.33 0.31 1.09 0.46 0.44 0.09 1.05 0.50 0.47 0.03 1.09 Discl. Ethical 1235 1.2 0.38...
work page 2025
-
[5]
standard responses that do not include any explicit boundary-setting behavior. To classify these behaviors at scale, Pasch (2025) manually annotated 3,500 LLM responses from the Chatbot Arena dataset and trained a RoBERTa-based transformer classifier to replicate these labels. The model achieved high performance in distinguishing among the five categories...
work page 2025
-
[6]
Discussion 5.1. Key Findings In line with H1, our results show that LLM-as-a-Judge (LaaJ) frameworks consistently rate ethical refusals more favorably than human users. This pattern holds across both judge models we analyzed—GPT-4o and Llama 3 70B— indicating that the divergence is not model-specific but generalizes across both proprietary and open-source...
work page 2025
-
[7]
Conclusion This study examines how safety- and ethics-based refusal responses are evaluated differently by human users and LLM-as-a-Judge (LaaJ) systems. Drawing on paired human and model-based evaluations, we find consistent evidence that LaaJ frameworks—across both GPT-4o and Llama 3 70B as judge systems—rate ethical refusals more favorably than human u...
arXiv 2023
-
[24]
Saha, S., Li, X., Ghazvininejad, M., Weston, J., & Wang, T. (2025). Learning to Plan & Reason for Evaluation with Thinking-LLM-as-a-Judge. arXiv preprint arXiv:2501.18099. Welbl, J., Glaese, A., Uesato, J., Dathathri, S., Mellor, J., Hendricks, L. A., ... & Huang, P. S. (2021). Challenges in detoxifying language models. arXiv preprint arXiv:2109.07445. Wh...
arXiv 2025
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.