Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Beyond the Surface: Measuring Self-Preference in LLM Judgments

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLM judges' self-preference can be measured separately from response quality using a gold-reference DBG score, and larger models, style-matched responses, and shared post-training data all reduce it.

desk verdict The DBG score is a reasonable idea for deconfounding self-preference, but the headline size trend is likely an artifact of saturation because the metric measures boundary-crossing probability, not bias magnitude. read the letter →

arxiv 2506.02592 v1 pith:KCKIZSDP submitted 2025-06-03 cs.CL

Zhi-Yuan Chen , Hao Wang , Xinyu Zhang , Enrui Hu , Yankai Lin This is my paper
classification cs.CL
keywords self-preferencebiasLLM-as-a-judgeDBGscoregoldjudgmentsresponsequalityconfoundingmodelscalingpost-trainingdataattentionanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that existing ways of measuring self-preference bias in LLM judges, subtracting what a judge scores for other models' responses from what it scores for its own, conflate bias with actual response quality. To separate the two, the authors introduce the DBG score: the difference between the judge's win rate for its own responses and a gold judgment obtained by aggregating three strong LLM judges, which serves as a proxy for true quality. Using this measure across models of different sizes, versions, and reasoning abilities, the paper reports that self-preference exists in both pretrained and post-trained models, that larger models show less of it, and that style alignment and shared post-training data reduce it. If true, it gives evaluators a way to compare judges without mistaking quality advantages for bias.

What carries the argument

The DBG score is the central object: $\hat{w}_A = \mathbb{E}_x[\sigma(\delta+b_A) - \sigma(\delta)]$, the expected difference between a biased judge's preference probability and the gold judge's preference probability for the judge's own response. It is computed in practice through pairwise win rates, with response order swapped to control position bias, and the gold judgment is the averaged pairwise decision of three strong LLM judges treated as a proxy for ground truth. A first-order Taylor argument shows that when the quality gap and bias are weakly correlated, DBG is a linearly scaled estimate of the true bias $b_A$, which is what lets it separate bias from quality.

What would settle it

Recompute the DBG scores using human preference labels as the gold reference on the same response pairs used in the paper (not only the 100 AlpacaEval samples currently validated); if the relative ordering of models by DBG changes materially, the gold ensemble carries its own preferences and DBG does not isolate self-preference. Alternatively, construct response pairs that are matched for human-rated quality but vary only in which model generated them; a DBG that remains positive on such pairs would confirm bias, while a null result would indict the metric.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that self-preference bias can be separated from response quality by replacing the opposing model's response with an unbiased gold judgment in the subtraction. Formally, if judge A's score is $S_A(r) \approx Q(r) + b_A(r)$ with true quality $Q$ and bias $b$, then subtracting the gold judge's preference removes the quality gap $\delta = Q(r_A)-Q(r_B)$, leaving $\hat{w}_A = \mathbb{E}_x[\sigma(\delta + b_A) - \sigma(\delta)]$, a quantity that is positive only when the judge favors its own response beyond what quality justifies. The paper then uses this DBG score to chart bias across model families, sizes, and reasoning models, and to show that superficial style and shared post-training data both move the score toward zero.

Load-bearing premise

The load-bearing premise is that the three-model gold judgment (GPT-4o-mini, Gemini-1.5-Flash, DeepSeek-V3) is an unbiased measure of true response quality; if those three models share a systematic preference in style, format, or family, then the DBG score measures deviation from their taste rather than from ground truth.

Editorial extensions

If this is right

  • Pretrained base models already show self-preference; post-training does not systematically add to it, so bias is not solely an alignment artifact.
  • Across sizes from 0.5B to 72B, larger judges show smaller DBG scores, so judge selection should favor larger models when unbiased evaluation matters.
  • Rewriting both responses into a common style lowers DBG scores, meaning stylistic familiarity is a substantial component of the bias.
  • Fine-tuning two different models on the same dataset (UltraChat-200k) lowers both models' DBG scores, suggesting shared post-training data aligns judgment tendencies.
  • Large reasoning models also show positive DBG scores, sometimes larger than those of standard LLMs, so reasoning ability does not automatically remove self-preference.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If DBG is adopted, previously published self-preference numbers computed as own-versus-other win-rate gaps would need to be reinterpreted: part of what looked like bias in strong models may have been quality, and part of what looked like fairness in weak models may have been low quality.
  • The same subtract-a-gold-reference logic could be applied to reward models and RLHF pipelines, where a model judging its own generated candidates may inflate reward signals; a DBG-style audit would make the contamination visible.
  • A testable extension of the style result is to control content while varying only formatting and lexical register; if DBG still tracks the judge's own style, it would confirm the attention-based mechanism rather than merely correlate with it.
  • Because the gold ensemble itself consists of three API models, an external audit could replace them with human labels on the full test set; the stability of DBG under that replacement would bound how much of the reported effect is an artifact of gold-judge preference.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes the DBG score, defined as the expected difference between a judge model's preference probability for its own response and the corresponding probability from a 'gold' judgment, which is obtained by aggregating three strong LLMs (GPT-4o-mini, Gemini-1.5-Flash, DeepSeek-V3). The authors argue that subtracting gold judgments removes the confounding effect of response quality and thus yields a more accurate measure of self-preference bias than prior win-rate-difference methods. They apply the DBG score to models of varying versions, sizes, and reasoning abilities, and to two mitigation factors (response style and shared post-training data), concluding that larger models exhibit less self-preference bias and that style alignment and shared training data reduce it. They also report an attention-based mechanism for the bias.

Significance. If the DBG score were a valid measure of self-preference bias magnitude, this would be a valuable contribution to LLM-as-a-judge evaluation. The paper is commendable for explicitly targeting the quality-confounding problem, for its broad empirical sweep (multiple model families, sizes from 0.5B to 72B, three datasets), and for releasing code and data. The attention-level analysis is a useful exploratory addition. However, the central metric as defined and implemented is not a quality-free or cross-pair-comparable measure of bias magnitude; the theoretical derivation relies on unverified assumptions, and the hard-threshold implementation creates a strong dependence on the gold win rate. These issues directly threaten the headline conclusions about size trends and reasoning-model comparisons.

major comments (3)
  1. [Section 3.1 and Section 3.3] The DBG score as implemented is not a quality-free measure of bias magnitude. The continuous estimator \hat w_A = E[\sigma(\delta+b_A) - \sigma(\delta)] depends on the distribution of \delta even for fixed b_A; it is largest near \delta = -b_A/2 and vanishes as |\delta| grows. The hard-threshold implementation described in Appendix A.4 makes this dependence even more severe: with binary A/B decisions, DBG equals P(-b_A < \delta \le 0) up to rare reversals, i.e., only items where gold says the opponent wins and bias flips the judgment contribute. Consequently, a model whose own responses are genuinely much better has a small DBG no matter how large b_A is. This invalidates cross-pair magnitude comparisons in Figures 3 and 4 and Table 2. For example, QwQ-32B's gold opponent-loss rate is only 7.6%, so its DBG is capped at 7.6%; a small DBG for QwQ cannot be read as small bias. The paper should either estimate b_A within a model that accounts for the \delta distribution, or restrict conclusions to the sign of bias and to comparisons where gold win rates are matched.
  2. [Section 3.1 and Section 3.3] The gold panel's unbiasedness is asserted but not sufficiently validated. Aggregating three strong LLMs does not remove systematic shared biases (e.g., style, format, verbosity, or family affinity) that these models may hold in common. The human study in Section 3.3 covers only 100 AlpacaEval samples and only one model pair (Llama-3.1-70B-Instruct versus Llama-3.1-8B-Instruct), so it does not establish that gold judgments are unbiased proxies for the other model pairs or datasets used in the paper. This is load-bearing because the DBG score measures deviation from the gold ensemble, not from ground truth; if the gold panel carries systematic preferences, the reported size and version comparisons inherit that bias. A validation study spanning multiple model pairs and datasets is needed, or the paper should explicitly frame the results as relative to the chosen gold ensemble.
  3. [Section 2 and Appendix A.4] The Taylor-expansion argument that \hat w_A \approx E[\sigma'(\delta)] E[b_A] requires both b_A small and a weak correlation between \delta and b_A. These conditions are not tested on the data, and the subsequent implementation replaces \sigma by a hard threshold, which is not justified by the Taylor argument. The discrepancy between the theoretical soft-probability formulation and the empirical hard-decision win rate is acknowledged in Appendix A.4, but the claim that the approximation error is 'small when the underlying probabilities are well-separated' is not validated; in fact, the most informative items for measuring bias are precisely those with probabilities near 0.5, where the thresholding error is largest. The paper should report the distribution of judge probabilities and either verify the approximation or adjust the estimator accordingly.
minor comments (4)
  1. [Table 2] The DBG values for the reasoning models are not explicitly shown; adding a dedicated DBG column would make the quantitative comparisons clearer and easier to verify.
  2. [Figure 3 and Figure 4] The figures show win rates but not the resulting DBG differences; annotating the DBG values directly on the figures would help readers see the claimed size trend.
  3. [Section 3.3] The 74% agreement and the 63% vs 66% win-rate comparison are reported without confidence intervals or a description of inter-annotator agreement; adding these would strengthen the validation claim.
  4. [Abstract] The claim that 'larger models tend to exhibit less self-preference bias' is stated as a general trend, but Figure 4 shows a sharp discontinuity between 7B and 14B; a more nuanced statement would be appropriate.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the DBG estimator is derived from an explicit Bradley-Terry model and its validity rests on an empirically stated, rather than definitionally smuggled, assumption about gold judgments.

full rationale

The paper's central construct, the DBG score, is defined in Section 2 as \hat w_A = E[\sigma(\delta + b_A) - \sigma(\delta)], the difference between a judge model's expected preference probability and an unbiased gold judge's expected preference probability. This is an explicit estimator built from a stated generative model, not a quantity secretly defined in terms of the conclusion. The claim that DBG mitigates quality confounding follows from the model assumption S_A(r) = Q(r) + b_A(r); subtracting the gold judge removes Q(r) at the level of the linearized expectation. The subsequent Taylor approximation \hat w_A \approx E[\sigma'(\delta)] E[b_A] is a mathematical consequence of stated assumptions (small b_A, weak correlation), and the hard-decision implementation in Appendix A.4 is acknowledged as an approximation rather than presented as exactly identical to the theory. The load-bearing assumption that aggregated judgments from GPT-4o-mini, Gemini-1.5-Flash, and DeepSeek-V3 serve as unbiased gold quality proxies is an empirical premise, not a circular reduction. The gold models are external to the tested judge models, and the authors explicitly select test models of different types from the gold judges to avoid preference leakage. The Limitations section candidly states that more powerful models could provide more reliable gold-standard judgments, and Section 3.3 reports an independent human-annotation check on 100 AlpacaEval samples, providing external evidence for the gold-judgment premise. Even if that premise is arguable, the derivation does not assume the DBG conclusion; it assumes the gold judge is quality-estimating. No self-citation is load-bearing: the paper cites prior work for the existence of self-preference and for the limitation of win-rate-based metrics, but the DBG construction and its properties are derived in the present paper from the Bradley-Terry setup. No uniqueness theorem from the authors' prior work is invoked, and no ansatz is smuggled in via citation. The claimed size and reasoning-model comparisons are empirical interpretations of the computed DBG values; concerns about the metric's scale-dependence or the capped magnitude of DBG for models with very high gold win rates are statistical and interpretational issues, not instances where the prediction equals the input by construction. The paper therefore exhibits no significant circularity.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The DBG derivation rests on an additive score decomposition, an assumption that aggregated gold judgments are unbiased, a vanishing remainder term in the Taylor expansion, and a weak-correlation assumption. No numeric constants are fitted to data. No new physical or conceptual entities are introduced beyond the gold-judgment ensemble.

assumptions (5)
  • domain assumption Judge scores decompose additively as S_A(r) approximately Q(r) + b_A(r).
    Invoked in Section 2 to derive P(r_A preferred) = sigma(delta + b_A); this linearity is not derived or tested.
  • domain assumption Gold aggregation from three strong LLMs yields unbiased quality estimates.
    Section 2 and Section 3.1 select GPT-4o-mini, Gemini-1.5-Flash, and DeepSeek-V3 as gold judges; only a 100-sample human validation on AlpacaEval supports this.
  • domain assumption The aggregation remainder Delta is negligible because each gold model's bias is small or zero-mean.
    Section 2 Taylor expansion states that if the bias of each individual model is relatively small or fluctuates around zero, then Delta is approximately 0.
  • domain assumption Weak correlation between the quality gap delta and the bias term b_A.
    Section 2 uses E[sigma'(delta) b_A] approximately equal to E[sigma'(delta)] E[b_A] to claim that DBG is a linearly scaled estimator of true bias.
  • domain assumption Thresholded win rate approximates Bernoulli sampling from the soft preference probability.
    Appendix A.4 states that the empirical win rate is an approximation to sampling from a Bernoulli distribution with success probability sigma(delta + b_A), and that the error is small when probabilities are well separated, without empirical verification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Beyond the Surface: Measuring Self-Preference in LLM Judgments." pith.science (2026). https://pith.science/paper/KCKIZSDP

@misc{pith2026250602592,
  author       = {Pith},
  title        = {Pith review of: Beyond the Surface: Measuring Self-Preference in LLM Judgments},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KCKIZSDP}},
  note         = {Machine review of arXiv:2506.02592}
}
read the original abstract

Recent studies show that large language models (LLMs) exhibit self-preference bias when serving as judges, meaning they tend to favor their own responses over those generated by other models. Existing methods typically measure this bias by calculating the difference between the scores a judge model assigns to its own responses and those it assigns to responses from other models. However, this approach conflates self-preference bias with response quality, as higher-quality responses from the judge model may also lead to positive score differences, even in the absence of bias. To address this issue, we introduce gold judgments as proxies for the actual quality of responses and propose the DBG score, which measures self-preference bias as the difference between the scores assigned by the judge model to its own responses and the corresponding gold judgments. Since gold judgments reflect true response quality, the DBG score mitigates the confounding effect of response quality on bias measurement. Using the DBG score, we conduct comprehensive experiments to assess self-preference bias across LLMs of varying versions, sizes, and reasoning abilities. Additionally, we investigate two factors that influence and help alleviate self-preference bias: response text style and the post-training data of judge models. Finally, we explore potential underlying mechanisms of self-preference bias from an attention-based perspective. Our code and data are available at https://github.com/zhiyuanc2001/self-preference.

Figures

Figures reproduced from arXiv: 2506.02592 by the authors.

Figure 1
Figure 1. Current methods (Top) measure the self￾preference bias of the judge model by comparing the scores (such as win rate) that the judge model assigns to its own responses with those assigned to other models’ responses. However, these methods overlook the impact of the intrinsic quality of the responses on the scores provided by the judge model. Our approach (Bottom) introduces gold judgments as proxies for the ground tr… view at source ↗
Figure 2
Figure 2. Judgment results for model pairs of the same size on AlpacaEval. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Judgment results for model pairs of different sizes on AlpacaEval. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Judgment results for Qwen2.5-Instruct models [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Analysis of response style transfer on model self-preference. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Analysis of post-training data on model self-preference. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Attention scores of each layer in judge models. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Judgment results for proprietary models on AlpacaEval. [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Judgment results on TruthfulQA [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Judgment results on WMT19 (de-en). Attractive Original Responses Rewritten Responses Humorous [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: t-SNE visualizations of the embedding representations of responses before and after rewriting. [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Memory Reward Inflation in Self-Improving LLM Agents

    cs.AI 2026-06 conditional novelty 6.0 of 10

    Self-graded memory in LLM agents systematically overvalues wrong episodes, compounding through reuse, and can be corrected by a de-correlated, answer-free signal.

Reference graph

Works this paper leans on

55 extracted references · 7 canonical work pages · cited by 1 Pith paper

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Anthropic. 2024. Claude 3.5 sonnet, 2024. https://www.anthropic.com/news/claude-3-5-sonnet

    work page 2024

  4. [4]

    Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, and 1 others. 2022. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073

    arXiv 2022

  5. [5]

    Ralph Allan Bradley and Milton E Terry. 1952. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324--345

    1952

  6. [6]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

  7. [7]

    Guiming Hardy Chen, Shunian Chen, Ziche Liu, Feng Jiang, and Benyou Wang. 2024. Humans or llms as the judge? a study on judgement biases. arXiv preprint arXiv:2402.10669

    arXiv 2024

  8. [8]

    Wei-Lin Chen, Zhepei Wei, Xinyu Zhu, Shi Feng, and Yu Meng. 2025. Do llm evaluators prefer themselves for a reason? arXiv preprint arXiv:2504.03846

    arXiv 2025

Show all 55 references
  1. [9]

    Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael Jordan, Joseph E Gonzalez, and 1 others. 2024. Chatbot arena: An open platform for evaluating llms by human preference. In Forty-first Internation...

    2024

  2. [10]

    Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Bingxiang He, Wei Zhu, Yuan Ni, Guotong Xie, Ruobing Xie, Yankai Lin, and 1 others. 2023. Ultrafeedback: Boosting language models with scaled ai feedback. arXiv preprint arXiv:2310.01377

    2023 arXiv

  3. [11]

    Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Zhi Zheng, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. 2023. https://arxiv.org/abs/2305.14233 Enhancing chat language models by scaling high-quality instructional conversations . Preprint, arXiv:2305.14233

    2023 arXiv

  4. [12]

    Yann Dubois, Chen Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy S Liang, and Tatsunori B Hashimoto. 2023. Alpacafarm: A simulation framework for methods that learn from human feedback. Advances in Neural Information Processing System...

    2023

  5. [13]

    Wikimedia Foundation. 2019. http://www.statmt.org/wmt19/translation-task.html Acl 2019 fourth conference on machine translation (wmt19), shared task: Machine translation of news

    2019

  6. [14]

    Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Dan Zhang, Diego Rojas, Guanyu Feng, Hanlin Zhao, and 1 others. 2024. Chatglm: A family of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406.12793

    2024 arXiv

  7. [15]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  8. [16]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    2025 arXiv

  9. [17]

    Yiju Guo, Ganqu Cui, Lifan Yuan, Ning Ding, Zexu Sun, Bowen Sun, Huimin Chen, Ruobing Xie, Jie Zhou, Yankai Lin, and 1 others. 2024. Controllable preference optimization: Toward controllable multi-objective alignment. arXiv preprint arXiv:2402.19085

    2024 arXiv

  10. [18]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874

    2021 arXiv

  11. [19]

    Zhengyu Hu, Linxin Song, Jieyu Zhang, Zheyuan Xiao, Zhengyu Chen, and Hui Xiong. 2024. Explaining length bias in llm-based preference evaluations. In ICLR 2025 Workshop on Navigating and Addressing Data Problems for Foundation Models

    2024

  12. [20]

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276

    2024 arXiv

  13. [21]

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others. 2024. Openai o1 system card. arXiv preprint arXiv:2412.16720

    2024 arXiv

  14. [22]

    Ryan Koo, Minhwa Lee, Vipul Raheja, Jong Inn Park, Zae Myung Kim, and Dongyeop Kang. 2023. Benchmarking cognitive biases in large language models as evaluators. arXiv preprint arXiv:2309.17012

    2023 arXiv

  15. [23]

    Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, and 1 others. 2024. Rewardbench: Evaluating reward models for language modeling. arXiv preprint arXiv:2403.13787

    2024 arXiv

  16. [24]

    Harrison Lee, Samrat Phatale, Hassan Mansoor, Thomas Mesnard, Johan Ferret, Kellie Lu, Colton Bishop, Ethan Hall, Victor Carbune, Abhinav Rastogi, and 1 others. 2023. Rlaif vs. rlhf: Scaling reinforcement learning from human feedback with ai feedback. arXiv preprint arXiv:2309.00267

    2023 arXiv

  17. [25]

    Dawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi, Chengshuai Zhao, Zhen Tan, Amrita Bhattacharjee, Yuxuan Jiang, Canyu Chen, Tianhao Wu, and 1 others. 2024. From generation to judgment: Opportunities and challenges of llm-as-a-judge. arXiv preprint arXiv:2411.16594

    2024

  18. [26]

    Dawei Li, Renliang Sun, Yue Huang, Ming Zhong, Bohan Jiang, Jiawei Han, Xiangliang Zhang, Wei Wang, and Huan Liu. 2025. Preference leakage: A contamination problem in llm-as-a-judge. arXiv preprint arXiv:2502.01534

    2025

  19. [27]

    Hashimoto

    Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Alpacaeval: An automatic evaluator of instruction-following models. https://github.com/tatsu-lab/alpaca_eval

    2023

  20. [28]

    Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, Nouha Dziri, Melanie Sclar, Khyathi Chandu, Chandra Bhagavatula, and Yejin Choi. 2023. The unlocking spell on base llms: Rethinking alignment via in-context learning. arXiv preprint arXiv:2312.01552

    2023 arXiv

  21. [29]

    Stephanie Lin, Jacob Hilton, and Owain Evans. 2021. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958

    2021 arXiv

  22. [30]

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437

    2024 arXiv

  23. [31]

    Xiao Liu, Xuanyu Lei, Shengyuan Wang, Yue Huang, Zhuoer Feng, Bosi Wen, Jiale Cheng, Pei Ke, Yifan Xu, Weng Lam Tam, and 1 others. 2023 a . Alignbench: Benchmarking chinese alignment of large language models. arXiv preprint arXiv:2311.18743

    2023 arXiv

  24. [32]

    Yiqi Liu, Nafise Sadat Moosavi, and Chenghua Lin. 2023 b . Llms as narcissistic evaluators: When ego inflates evaluation scores. arXiv preprint arXiv:2311.09766

    2023 arXiv

  25. [33]

    Remi Mir, Bjarke Felbo, Nick Obradovich, and Iyad Rahwan. 2019. Evaluating style transfer for text. arXiv preprint arXiv:1904.02295

    2019 arXiv

  26. [34]

    Phil Ostheimer, Mayank Nagda, Marius Kloft, and Sophie Fellenz. 2023. Text style transfer evaluation using large language models. arXiv preprint arXiv:2308.13577

    2023 arXiv

  27. [35]

    Arjun Panickssery, Samuel Bowman, and Shi Feng. 2024. Llm evaluators recognize and favor their own generations. Advances in Neural Information Processing Systems, 37:68772--68802

    2024

  28. [36]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311--318

    2002

  29. [37]

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, and 1 others. 2023. Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.16789

    2023 arXiv

  30. [38]

    Lin Shi, Chiyu Ma, Wenhua Liang, Weicheng Ma, and Soroush Vosoughi. 2024. Judging the judges: A systematic investigation of position bias in pairwise comparative assessments by llms. arXiv preprint arXiv:2406.07791

    2024

  31. [39]

    Zhiqing Sun, Yikang Shen, Hongxin Zhang, Qinhong Zhou, Zhenfang Chen, David Cox, Yiming Yang, and Chuang Gan. 2023. Salmon: Self-alignment with instructable reward models. arXiv preprint arXiv:2310.05910

    2023 arXiv

  32. [40]

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, and 1 others. 2024 a . Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530

    2024 arXiv

  33. [41]

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, L \'e onard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram \'e , and 1 others. 2024 b . Gemma 2: Improving open language models at a practical size. arXiv preprint arX...

    2024 arXiv

  34. [42]

    Qwen Team. 2025. https://qwenlm.github.io/blog/qwq-32b/ Qwq-32b: Embracing the power of reinforcement learning

    2025

  35. [43]

    Laurens van der Maaten and Geoffrey Hinton. 2008. http://jmlr.org/papers/v9/vandermaaten08a.html Visualizing data using t-sne . Journal of Machine Learning Research, 9(86):2579--2605

    2008

  36. [44]

    Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. 2023 a . Large language models are not fair evaluators. arXiv preprint arXiv:2305.17926

    2023 arXiv

  37. [45]

    Yidong Wang, Zhuohao Yu, Zhengran Zeng, Linyi Yang, Cunxiang Wang, Hao Chen, Chaoya Jiang, Rui Xie, Jindong Wang, Xing Xie, and 1 others. 2023 b . Pandalm: An automatic evaluation benchmark for llm instruction tuning optimization. arXiv preprint arXiv:2306.05087

    2023 arXiv

  38. [46]

    Yuxia Wang, Haonan Li, Xudong Han, Preslav Nakov, and Timothy Baldwin. 2023 c . Do-not-answer: A dataset for evaluating safeguards in llms. arXiv preprint arXiv:2308.13387

    2023 arXiv

  39. [47]

    Koki Wataoka, Tsubasa Takahashi, and Ryokan Ri. 2024. Self-preference bias in llm-as-a-judge. arXiv preprint arXiv:2410.21819

    2024 arXiv

  40. [48]

    Tianhao Wu, Weizhe Yuan, Olga Golovneva, Jing Xu, Yuandong Tian, Jiantao Jiao, Jason Weston, and Sainbayar Sukhbaatar. 2024. Meta-rewarding language models: Self-improving alignment with llm-as-a-meta-judge. arXiv preprint arXiv:2407.19594

    2024 arXiv

  41. [49]

    Shijie Xia, Xuefeng Li, Yixin Liu, Tongshuang Wu, and Pengfei Liu. 2024. Evaluating mathematical reasoning beyond accuracy. arXiv preprint arXiv:2404.05692

    2024 arXiv

  42. [50]

    Wenda Xu, Guanglei Zhu, Xuandong Zhao, Liangming Pan, Lei Li, and William Yang Wang. 2024. Pride and prejudice: Llm amplifies self-bias in self-refinement. arXiv preprint arXiv:2402.11436

    2024 arXiv

  43. [51]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, and 1 others. 2024. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115

    2024 arXiv

  44. [52]

    Jiayi Ye, Yanbo Wang, Yue Huang, Dongping Chen, Qihui Zhang, Nuno Moniz, Tian Gao, Werner Geyer, Chao Huang, Pin-Yu Chen, and 1 others. 2024. Justice or prejudice? quantifying biases in llm-as-a-judge. arXiv preprint arXiv:2410.02736

    2024 arXiv

  45. [53]

    Xin Zhang, Yanzhao Zhang, Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang, Pengjun Xie, Fei Huang, and 1 others. 2024. mgte: Generalized long-context text representation and reranking models for multilingual text retrieval. arXiv preprint arXiv:2407.19669

    2024 arXiv

  46. [54]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, and 1 others. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595--46623

    2023

  47. [55]

    Lianghui Zhu, Xinggang Wang, and Xinlong Wang. 2023. Judgelm: Fine-tuned large language models are scalable judges. arXiv preprint arXiv:2310.17631

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.