Pith. sign in

REVIEW 4 major objections 4 minor 63 references

CompassJudger-2: Towards Generalist Judge Model via Verifiable Rewards

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 7B-parameter judge model trained on verifiable rewards reaches an average of 72.11 on four judge and reward benchmarks, matching much larger models.

desk verdict Solid empirical paper with a flawed derivation and an unvalidated benchmark; the 7B parity claim is real but fragile. read the letter →

arxiv 2507.09104 v1 pith:4YJNBQ7G submitted 2025-07-12 cs.CL cs.AI

Taolin Zhang , Maosong Cao , Alexander Lam , Songyang Zhang , Kai Chen This is my paper
classification cs.CLcs.AI
keywords LLM-as-a-judgegeneralistjudgemodelverifiablerewardrejectionsamplingmarginpolicygradientlossJudgerBenchV2Mix-of-Judgersevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that a general-purpose LLM judge—one that grades, ranks, and critiques responses across tasks—can be trained more effectively when the supervisor signal is a verifiable reward rather than human-written judgment alone. The authors build CompassJudger-2 from Qwen2.5 base models using a data mix of reconstructed public judge data, reward data filtered by rejection sampling, and synthetic knowledge and chat data. Their 7B model scores 72.11 on average across JudgerBenchV2, JudgeBench, RMB, and RewardBench, above all 7B judge baselines and roughly level with much larger models around 72 points. The margin policy-gradient loss contributes a 2.21-point average gain over the SFT-only baseline, the largest of the three loss variants tested. The paper also introduces JudgerBenchV2, a 10,000-question benchmark whose ground truth is the majority consensus of three strong judge models and whose metric combines per-sample accuracy with ranking consistency.

What carries the argument

The central mechanism is verifiable reward supervision for judgments. The model is prompted to reason through a fixed structure—user demand, strengths and weaknesses of each response, reasoning, then a final choice—so the reward is a rule-based function: 1 if the choice token matches the ground-truth label at a designated position, 0 otherwise. Rejection sampling generates eight candidate reasoning trajectories that end in the correct verdict, creating diverse prefixes for training; the policy-gradient-style loss then concentrates on the conditional log-probability of the correct token, and the margin variant enforces a margin between correct and incorrect verdicts. This machinery lets the authors train a 7B judge without large volumes of human critique labels while retaining generalization across judge, reward, and general instruction tasks.

What would settle it

Take a random sample of JudgerBenchV2's 10,000 pairwise questions, have human experts label which response is better, then recompute CompassJudger-2-7B's score against human labels; if its advantage over the larger models and baselines shrinks or reverses, the Mix-of-Judgers labels are carrying the result rather than the verifiable-reward training.

Watch

Extended reading notes

Core claim

CompassJudger-2 is presented as a generalist all-in-one judge series that outperforms existing 7B judge models and matches substantially larger general models on judge and reward benchmarks. On the four-benchmark average, the 7B Instruct model reaches 72.11, compared with 71.86 for DeepSeek-V3-0324 and 71.91 for Qwen3-235B-A22B, and it improves on the prior CompassJudger-1-7B by 16.39 percent on average. The decisive training choice is to treat the final verdict as a classification step with a deterministic ground-truth reward: candidate reasoning paths that produce the correct verdict are kept via rejection sampling, and a margin loss pushes the probability of the correct token above the wrong-token probability by a margin of 10. This margin loss adds 2.21 points on average over the SFT baseline, while DPO and temperature variants add less or nothing. The companion JudgerBenchV2 contributes a 10,000-sample, ten-scenario benchmark with Mix-of-Judgers ground truth and a metric that penalizes rank and score disagreement, not just sample-level accuracy.

Load-bearing premise

JudgerBenchV2's ground truth is the majority vote of three LLM judges with no human verification, so if those judges share a systematic bias, the benchmark's labels favor models that align with LLM preferences rather than with genuinely better responses.

Editorial extensions

If this is right

  • A 7B-parameter judge model reaches an average of 72.11 across four judge and reward benchmarks, making high-quality LLM evaluation feasible with far less compute than a 235B-level judge.
  • Supervising judges with verifiable rewards plus rejection sampling can reduce reliance on expensive human-written critiques for training judge models.
  • The margin loss is the best of the three tested mapping functions, adding 2.21 points over the SFT baseline and improving on all four benchmarks.
  • Including rejection-sampled reward data improves judge consistency and also lifts general benchmarks such as GPQA-Diamond and ArenaHard.
  • Critiques generated by CompassJudger-2 improve downstream policy models on AlignBench, AlpacaEval, and ArenaHard, while a comparison judge's critiques often lower those scores.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if the 7B result transfers to other base models, verifiable-reward judging could lower the cost of model evaluation enough for small research groups to run their own reward models rather than relying on external judge APIs.
  • Editorial inference: because JudgerBenchV2's ground truth is a vote among three LLM judges, the benchmark's labels inherit whatever shared preference bias those judges have; a human-verified subset would test how much of the reported 72.11 is genuine judging skill.
  • Editorial inference: the margin-loss idea can be applied to any classification-style LLM task with checkable answers, such as grading code outputs or math solutions, not just pairwise response judging.
  • Editorial inference: the same pipeline has a natural next test in multi-turn and multimodal evaluation, which the paper names as future work and which would exercise the claim of generality more aggressively.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper presents CompassJudger-2, a family of LLM judge models trained with a task-driven data pipeline that includes curation of public judge/reward data, synthesis of knowledge- and chat-based judgment data, rejection sampling, and a proposed 'margin policy gradient loss.' The authors also introduce JudgerBenchV2, a benchmark whose ground truth is the majority vote of three LLM judges (DeepSeek-R1, DeepSeek-V3-0324, Qwen3-235B-A22B) and whose metric combines sample-level accuracy with rank and score consistency. Empirical results report that CompassJudger-2-7B-Instruct outperforms existing 7B judge models and approaches the average score of much larger models such as DeepSeek-V3 and Qwen3-235B-A22B on a combination of JudgerBenchV2, JudgeBench, RMB, and RewardBench.

Significance. If the claims hold, this is a practically valuable contribution: a 7B judge model competitive with much larger models, with demonstrated robustness to prompt-style changes, and a new benchmark addressing judge evaluation beyond simple accuracy. The paper provides extensive evaluations across four judge benchmarks and multiple general benchmarks, plus ablations showing consistent improvements from the proposed data and loss components. The main methodological claim, however, rests on a policy-gradient derivation that is not mathematically correct as written, and the new benchmark's ground truth is not independently validated. These issues affect the interpretation of the headline results and need to be addressed before the paper can be accepted.

major comments (4)
  1. [Section 3.2, Eq. (2); Appendix A, Eq. (10)] The simplification of the policy gradient is mathematically incorrect. The reward r(x,y) = 1{y_kx = y*_kx} depends only on the answer token, but the expectation E_y~πθ[r(y) ∇θ log πθ(y|x)] expands to a sum over all token positions. While the terms for t > kx vanish in expectation by the score-function identity, the terms for t < kx do not: the probability of the answer token depends on all preceding tokens, and those tokens carry gradient information about the distribution of the answer. The loss in Eq. (2) is therefore not the gradient of the expected reward; it is a teacher-forced token-level cross-entropy on the answer position. Because the paper's central methodological framing ('policy gradient loss with verifiable reward signals') relies on this derivation, it should be corrected or the method should be explicitly presented as a heuristic token-level objective supported by the ablations in Table 4.
  2. [Section 4, 'Mixture of Judges'; Table 2] JudgerBenchV2 labels are the majority vote of DeepSeek-R1, DeepSeek-V3-0324, and Qwen3-235B-A22B, with no human validation, no inter-judge agreement analysis, and no discussion of shared bias. The headline claim in Table 2 — CompassJudger-2-7B (72.11) comparable to DeepSeek-V3 (71.86) and Qwen3-235B-A22B (71.91) — depends on the JudgerBenchV2 score of 60.52. If the three judge models share a systematic bias (e.g., preference for style, length, or a particular model family), the JudgerBenchV2 score could be inflated and the parity claim would fail. The training data are also synthesized with Qwen2.5-72B-Instruct, and no contamination analysis is reported. Please add a human-verified subset, report judge agreement, and analyze potential bias and contamination in JudgerBenchV2.
  3. [Section 3.2, Eq. (12); Appendix A] The rejection-sampling loss in Eq. (3)-(4) is not a policy-gradient estimator; it is a supervised token-level loss over accepted samples. In addition, Eq. (12) contains indexing errors: the conditioning context uses x^{(j)} where x^{(i)} is intended, and the decomposition of L_total into L_SFT and L_PG is not derived consistently. These issues obscure the relationship between the proposed loss and the claimed policy-gradient objective, and they should be fixed.
  4. [Section 7, Limitations] The Limitations section acknowledges only inference cost and synthesis hallucination. It does not mention the lack of validation for JudgerBenchV2 labels or the risk of contamination between training data and evaluation benchmarks. Since these are the main threats to the paper's central empirical claim, they should be addressed or explicitly discussed as limitations.
minor comments (4)
  1. [Table 2] In the CompassJudger-1-32B-Instruct row, the scores for JudgeBench, RMB, and RewardBench are printed without spaces ('62.2977.6386.17'), making the row difficult to read.
  2. [Section 5.3] The sentence 'delivering an 2.21% performance on average boost compared to the baseline model' contains a grammatical error; it should read 'a 2.21-point average improvement over the baseline model.'
  3. [Appendix A, Eq. (12)] The notation x^{(j)} in the conditioning context is inconsistent with the outer index i; it should be x^{(i)} throughout the equation.
  4. [Section 4, Eq. (5)] The metric formula contains a stray asterisk before the rank-difference term and appears to have unbalanced parentheses in the printed version; please format the equation cleanly so the normalization is unambiguous.

Circularity Check

0 steps flagged · score 2.0 of 10

No constructional circularity found: the training objective is supervised by rule-based verifiable rewards anchored to external labels, and the headline parity claim holds on the three external benchmarks alone; the unvalidated Mix-of-Judgers ground truth is a benchmark-validity risk rather than a circularity reduction.

full rationale

The paper's derivation chain is self-contained and does not reduce to its own inputs. The training objective (Eqs. 1-4, Table 1) uses a reward defined as 1 iff the prediction at position kx matches the ground-truth label (Section 3.2), and the training labels are anchored outside the model: outdated judge data is verified against human-labeled ground truth, knowledge-based judgments are validated against answer keys, and reward data labels come from public preference datasets. Evaluation is anchored to three external benchmarks (JudgeBench, RMB, RewardBench) plus the self-proposed JudgerBenchV2, whose Mix-of-Judgers ground truth is the majority of DeepSeek-R1, DeepSeek-V3-0324, and Qwen3-235B-A22B; none of these generated the training judgments (synthesis uses Qwen2.5-72B-Instruct), so there is no identity between training signal and evaluation target. The headline 72.11 average does not depend on JudgerBenchV2: excluding it, CompassJudger-2-7B averages 75.97 on the three external benchmarks versus DeepSeek-V3's 74.34 and Qwen3's 75.41, so the parity claim survives removal of the self-built benchmark. The only self-citation of note is the inclusion of general instruction data 'collected from CompassJudger-1' (Section 3.1), justified partly by the authors' own prior work [3]; this is not load-bearing because Figure 4 shows G-SFT data has 'minimal impact on judge ability,' and the practice is independently corroborated by external reference [23]. The margin-loss default is selected via the fully disclosed ablation (Table 4), which is selection-on-test-set reporting rather than a fitted parameter renamed as a prediction. The genuine weaknesses noted by the skeptic — JudgerBenchV2's MoJ consensus has no reported human validation, and no contamination analysis is given for the self-sourced CompassArena queries — are threats to construct validity and benchmark independence, not circularity reductions; no equation in the paper equates a predicted quantity to an input by construction.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the correctness of the ground-truth labels used as verifiable rewards, the reliability of Qwen2.5-72B for data reconstruction, and the validity of the LLM majority vote as a benchmark ground truth. The margin and sampling hyperparameters are chosen by hand without sensitivity analysis.

free parameters (6)
  • margin gamma = 10
    Set for margin loss; no sensitivity study reported.
  • beta (DPO) = 0.1
    Used in DPO ablation only.
  • tau (temperature) = 5
    Used in temperature ablation only.
  • top-k logits = 10
    Margin loss applied to top 10 logits.
  • rejection samples M = 8
    Number of candidate responses per instruction.
  • data cutoff = October 2024
    Split between outdated and up-to-date public judge data.
assumptions (4)
  • ad hoc to paper Policy gradient loss can be simplified to the answer-token log-probability because the reward depends only on that token.
    This simplification in Eq (2) and Eq (10) is not mathematically valid; the gradient terms for other token positions do not vanish.
  • domain assumption Ground truth labels in public reward datasets are correct.
    These labels serve as the verifiable reward signal in training (Section 3.2).
  • domain assumption The Mix-of-Judgers majority is a reliable ground truth for judge evaluation.
    JudgerBenchV2 uses majority of three LLMs (Section 4); no human validation is provided.
  • domain assumption Evaluation benchmarks are free of contamination from the training data.
    The training data includes public judge and reward data that may overlap with evaluation sets; no contamination check is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CompassJudger-2: Towards Generalist Judge Model via Verifiable Rewards." pith.science (2026). https://pith.science/paper/4YJNBQ7G

@misc{pith2026250709104,
  author       = {Pith},
  title        = {Pith review of: CompassJudger-2: Towards Generalist Judge Model via Verifiable Rewards},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4YJNBQ7G}},
  note         = {Machine review of arXiv:2507.09104}
}
read the original abstract

Recently, the role of LLM-as-judge in evaluating large language models has gained prominence. However, current judge models suffer from narrow specialization and limited robustness, undermining their capacity for comprehensive evaluations. In this work, we present CompassJudger-2, a novel generalist judge model that overcomes these limitations via a task-driven, multi-domain data curation strategy. Central to our approach is supervising judgment tasks with verifiable rewards, guiding intrinsic critical reasoning through rejection sampling to foster robust, generalizable judgment capabilities. We introduce a refined learning objective with margin policy gradient loss to enhance performance. Empirically, CompassJudger-2 achieves superior results across multiple judge and reward benchmarks, and our 7B model demonstrates competitive judgment accuracy with significantly larger models like DeepSeek-V3 and Qwen3-235B-A22B. Additionally, we propose JudgerBenchV2, a comprehensive benchmark evaluating cross-domain judgment accuracy and rank consistency to standardize judge model evaluation. These contributions advance robust, scalable LLM judgment and establish new performance and evaluation standards.

Figures

Figures reproduced from arXiv: 2507.09104 by the authors.

Figure 1
Figure 1. The data construction pipeline of CompassJudger-2, including data curation and synthesis. The Data Curation stage include reconstruction of public judge and reward data, while the data synthesis stage contains response generation over knowledge-based and chat based datasets. 3.1 Overall Data Pipeline Data Curation. We begin by collecting open-source judge-related datasets, including Public Judge Data and Public Rewa… view at source ↗
Figure 2
Figure 2. Illustration of the reasoning path in the judge task. The reasoning path involves critical analysis of the instruction and responses from various models. The final answer prediction can be treated as a classification task, which is further guided by a verified reward for supervision. Prompt Ground Truth Reasoning Path Answer Logit CoT Answer B CoT Answer A CoT Answer B [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Training framework of CompassJudger-2. CompassJudger-2 utilize rejection sampling to choose correct reasoning paths for SFT training and apply policy gradient loss over the answer logit to incorporate verifiable reward. Critical Thinking. The SFT training of judge models requires high-quality instruction-response data, which can be costly to obtain. To tackle this challenge, we introduce an innovative chain-of-thoug… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Data ablation results on different benchmarks [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Comparison results over style judge of CompassJudger-2 and RISE. CompassJudger-1 series, CompassJudger-2 enhances judge performance by 16.39% for the 7B model and 1.71% for the 32B model, on average. General Ability Analysis. We further highlight the improvements in ge…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

63 extracted references · 32 canonical work pages

  1. [1]

    Mt-bench-101: A fine-grained benchmark for evaluating large language models in multi-turn dialogues

    Ge Bai, Jie Liu, Xingyuan Bu, Yancheng He, Jiaheng Liu, Zhanhui Zhou, Zhuoran Lin, Wenbo Su, Tiezheng Ge, Bo Zheng, et al. Mt-bench-101: A fine-grained benchmark for evaluating large language models in multi-turn dialogues. arXiv preprint arXiv:2402.14762, 2024

    arXiv 2024

  2. [2]

    Internlm2 technical report, 2024

    Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, Xiaoyi Dong, Haodong Duan, Qi Fan, Zhaoye Fei, Yang Gao, Jiaye 9 Ge, Chenya Gu, Yuzhe Gu, Tao Gui, Aijia Guo, Qipeng Guo, Conghui He, Yingfan Hu, Ting Huang, Tao Jiang, Penglong Jiao, Zhenjiang Jin, Zhikai Lei, Jiaxing Li, Jingwen Li, Linyang Li,...

    work page 2024

  3. [3]

    Compassjudger-1: All-in-one judge model helps model evaluation and evolution.arXiv preprint arXiv:2410.16256, 2024

    Maosong Cao, Alexander Lam, Haodong Duan, Hongwei Liu, Songyang Zhang, and Kai Chen. Compassjudger-1: All-in-one judge model helps model evaluation and evolution.arXiv preprint arXiv:2410.16256, 2024

    arXiv 2024

  4. [4]

    xverify: Efficient answer verifier for reasoning model evaluations

    Ding Chen, Qingchen Yu, Pengyuan Wang, Wentao Zhang, Bo Tang, Feiyu Xiong, Xinchi Li, Minchuan Yang, and Zhiyu Li. xverify: Efficient answer verifier for reasoning model evaluations. arXiv preprint arXiv:2504.10481, 2025

    arXiv 2025

  5. [5]

    Rm-r1: Reward modeling as reasoning

    Xiusi Chen, Gaotang Li, Ziqi Wang, Bowen Jin, Cheng Qian, Yu Wang, Hongru Wang, Yu Zhang, Denghui Zhang, Tong Zhang, et al. Rm-r1: Reward modeling as reasoning. arXiv preprint arXiv:2505.02387, 2025

    arXiv 2025

  6. [6]

    Boolq: Exploring the surprising difficulty of natural yes/no questions, 2019

    Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions, 2019. URLhttps://arxiv.org/abs/1905.10044

    arXiv 2019

  7. [7]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

    arXiv

  8. [8]

    Opencompass: A universal evaluation platform for foundation models.https://github.com/open-compass/opencompass, 2023

    OpenCompass Contributors. Opencompass: A universal evaluation platform for foundation models.https://github.com/open-compass/opencompass, 2023

    2023

Show all 63 references
  1. [9]

    Enhancing chat language models by scaling high-quality instructional conversations

    Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Zhi Zheng, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. Enhancing chat language models by scaling high-quality instructional conversations. arXiv preprint arXiv:2305.14233, 2023

    2023 arXiv

  2. [10]

    Drop: A reading comprehension benchmark requiring discrete reasoning over paragraphs,

    Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gard- ner. Drop: A reading comprehension benchmark requiring discrete reasoning over paragraphs,

  3. [11]

    Length-controlled alpacaeval: A simple way to debias automatic evaluators

    Yann Dubois, Balázs Galambosi, Percy Liang, and Tatsunori B Hashimoto. Length-controlled alpacaeval: A simple way to debias automatic evaluators. arXiv preprint arXiv:2404.04475, 2024

    2024 arXiv

  4. [12]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  5. [13]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  6. [14]

    Openai o1 system card

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720, 2024

    2024 arXiv

  7. [15]

    Livecodebench: Holistic and contamination free evaluation of large language models for code

    Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Ar- mando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974, 2024. 10

    2024 arXiv

  8. [16]

    Rewardbench: Evaluating reward models for language modeling

    Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, et al. Rewardbench: Evaluating reward models for language modeling. arXiv preprint arXiv:2403.13787, 2024

    2024 arXiv

  9. [17]

    Cmmlu: Measuring massive multitask language understanding in chinese

    Haonan Li, Yixuan Zhang, Fajri Koto, Yifei Yang, Hai Zhao, Yeyun Gong, Nan Duan, and Timothy Baldwin. Cmmlu: Measuring massive multitask language understanding in chinese. arXiv preprint arXiv:2306.09212, 2023

    2023 arXiv

  10. [18]

    Generative judge for evaluating alignment

    Junlong Li, Shichao Sun, Weizhe Yuan, Run-Ze Fan, Hai Zhao, and Pengfei Liu. Generative judge for evaluating alignment. arXiv preprint arXiv:2310.05470, 2023

    2023 arXiv

  11. [19]

    From crowdsourced data to high-quality benchmarks: Arena-hard and benchbuilder pipeline

    Tianle Li, Wei-Lin Chiang, Evan Frick, Lisa Dunlap, Tianhao Wu, Banghua Zhu, Joseph E Gonzalez, and Ion Stoica. From crowdsourced data to high-quality benchmarks: Arena-hard and benchbuilder pipeline. arXiv preprint arXiv:2406.11939, 2024

    2024 arXiv

  12. [20]

    Wildbench: Benchmarking llms with challenging tasks from real users in the wild

    Bill Yuchen Lin, Yuntian Deng, Khyathi Chandu, Faeze Brahman, Abhilasha Ravichander, Valentina Pyatkin, Nouha Dziri, Ronan Le Bras, and Yejin Choi. Wildbench: Benchmarking llms with challenging tasks from real users in the wild. arXiv preprint arXiv:2406.04770, 2024

    2024 arXiv

  13. [21]

    Deepseek-v3 technical report

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024

    2024 arXiv

  14. [22]

    Alignbench: Benchmarking chinese alignment of large language models

    Xiao Liu, Xuanyu Lei, Shengyuan Wang, Yue Huang, Zhuoer Feng, Bosi Wen, Jiale Cheng, Pei Ke, Yifan Xu, Weng Lam Tam, et al. Alignbench: Benchmarking chinese alignment of large language models. arXiv preprint arXiv:2311.18743, 2023

    2023 arXiv

  15. [23]

    Inference-time scaling for generalist reward modeling

    Zijun Liu, Peiyi Wang, Runxin Xu, Shirong Ma, Chong Ruan, Peng Li, Yang Liu, and Yu Wu. Inference-time scaling for generalist reward modeling. arXiv preprint arXiv:2504.02495, 2025

    2025

  16. [24]

    Gpqa: A graduate-level google-proof q&a benchmark

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, 2024

    2024

  17. [25]

    Skywork critic model se- ries

    Tu Shiwen, Zhao Liang, Chris Yuhao Liu, Liang Zeng, and Yang Liu. Skywork critic model se- ries. https://huggingface.co/Skywork, September 2024. URL https://huggingface. co/Skywork

    2024

  18. [26]

    Judgebench: A benchmark for evaluating llm-based judges

    Sijun Tan, Siyuan Zhuang, Kyle Montgomery, William Y Tang, Alejandro Cuadron, Chenguang Wang, Raluca Ada Popa, and Ion Stoica. Judgebench: A benchmark for evaluating llm-based judges. arXiv preprint arXiv:2410.12784, 2024

    2024 arXiv

  19. [27]

    Qwen3: Think deeper, act faster

    Qwen team. Qwen3: Think deeper, act faster. https://qwenlm.github.io/blog/qwen3/, 2025

    2025

  20. [28]

    Mmlu-pro: A more robust and challenging multi-task language understanding benchmark

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. In The Thirty-eight Conference on Neural Information Pr...

    2024

  21. [29]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  22. [30]

    Beyond scalar reward model: Learning generative judge from preference data, 2024

    Ziyi Ye, Xiangsheng Li, Qiuchi Li, Qingyao Ai, Yujia Zhou, Wei Shen, Dong Yan, and Yiqun Liu. Beyond scalar reward model: Learning generative judge from preference data, 2024. URL https://arxiv.org/abs/2410.03742

    2024 arXiv

  23. [31]

    Learning llm-as-a-judge for preference alignment

    Ziyi Ye, Xiangsheng Li, Qiuchi Li, Qingyao Ai, Yujia Zhou, Wei Shen, Dong Yan, and Yiqun Liu. Learning llm-as-a-judge for preference alignment. In The Thirteenth International Conference on Learning Representations, 2025

    2025

  24. [32]

    Improve llm-as-a-judge ability as a general ability

    Jiachen Yu, Shaoning Sun, Xiaohui Hu, Jiaxu Yan, Kaidong Yu, and Xuelong Li. Improve llm-as-a-judge ability as a general ability. arXiv preprint arXiv:2502.11689, 2025. 11

    2025 arXiv

  25. [33]

    Rmb: Comprehensively benchmarking reward models in llm alignment

    Enyu Zhou, Guodong Zheng, Binghai Wang, Zhiheng Xi, Shihan Dou, Rong Bao, Wei Shen, Limao Xiong, Jessica Fan, Yurong Mou, et al. Rmb: Comprehensively benchmarking reward models in llm alignment. arXiv preprint arXiv:2410.09893, 2024

    arXiv

  26. [34]

    Instruction-following evaluation for large language models

    Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-following evaluation for large language models. arXiv preprint arXiv:2311.07911, 2023

    2023 arXiv

  27. [35]

    Judgelm: Fine-tuned large language models are scalable judges

    Lianghui Zhu, Xinggang Wang, and Xinlong Wang. Judgelm: Fine-tuned large language models are scalable judges. arXiv preprint arXiv:2310.17631, 2023. 12 Appendix A Deriving the Loss Function Judge Reward.In the judge task, given a instruction-response pair (x, y), prediction po...

    2023 arXiv

  28. [37]

    User’s Demand: - Carefully analyze the user’s request. What is the user specifically asking for? What are the key aspects of the request that need to be fulfilled? Identify any constraints (e.g., time, format, quantity) the user has provided

  29. [38]

    Consider how well it addresses the user’s demand, meets the user’s constraints, and how well it serves the intended purpose

    Strengths of Model A: - Identify the strengths of Model A’s response. Consider how well it addresses the user’s demand, meets the user’s constraints, and how well it serves the intended purpose

  30. [39]

    What aspects of the response fail to meet the user’s request or constraints? What could have been improved?

    Weaknesses of Model A: - Identify the weaknesses of Model A’s response. What aspects of the response fail to meet the user’s request or constraints? What could have been improved?

  31. [40]

    Consider how well it addresses the user’s demand, meets the user’s constraints, and how well it serves the intended purpose

    Strengths of Model B: - Identify the strengths of Model B’s response. Consider how well it addresses the user’s demand, meets the user’s constraints, and how well it serves the intended purpose

  32. [41]

    What aspects of the response fail to meet the user’s request or constraints? What could have been improved?

    Weaknesses of Model B: - Identify the weaknesses of Model B’s response. What aspects of the response fail to meet the user’s request or constraints? What could have been improved?

  33. [42]

    Discuss which model’s response is more suitable given the user’s request and con- straints

    Reasoning: - Based on your analysis of both responses, explain which model better addresses the user’s needs. Discuss which model’s response is more suitable given the user’s request and con- straints

  34. [43]

    User’s Demand

    Choice: - Conclude with a choice between Model A and Model B based on your reasoning. Indicate which model provides the more appropriate and useful response for the user’s request. Your final reply must be structured in the following format: { "User’s Demand": "[The user’s req...

  35. [49]

    hushed, waiting world

    Enhancing Emotional Depth: While the response is atmospheric, adding a bit more emotional depth or personal touch could make the poem more engaging. For example, describing the "hushed, waiting world" as "yearning for spring" could add a layer of emotional resonance. Overall, ...

  36. [50]

    The model adheres to the instruction of creating a poem where each sentence is exactly five words long

  37. [51]

    winter" or

    The poem effectively evokes the essence of winter without explicitly using the words "winter" or "snow."

  38. [52]

    Frost paints silent trees

    The language used is descriptive and creates a vivid image of the season, such as "Frost paints silent trees" and "Windows gleam with light."

  39. [53]

    Areas for Improvement:

    There is a coherent flow and structure to the poem, with each line contributing to a unified theme of winter. Areas for Improvement:

  40. [54]

    While the poem avoids the word "snow," which was part of the instruction, the absence of any snow-related imagery or metaphor might have strengthened the winter theme further

  41. [55]

    Hushed, the world awaits

    The phrase "Hushed, the world awaits" could be interpreted as a generic description of anticipation and not necessarily tied to the cold or silence of winter, which might make the winter theme slightly less explicit

  42. [56]

    Overall, the model’s response is a well-crafted poem that meets most of the criteria set by the user’s request

    To further enhance the specificity of the season, incorporating additional sensory details or specific winter activities (like ice skating or building snowmen) could make the poem more evocative and engaging. Overall, the model’s response is a well-crafted poem that meets most...

  43. [57]

    Conciseness and Adherence to Word Count: Each sentence is exactly five words long, as requested by the user

  44. [58]

    winter" or

    Evocation of Winter: The response effectively evokes the sense of winter without explicitly using the words "winter" or "snow." Phrases like "Frost paints silent trees" and "Chill lingers in the air" convey the cold and serene aspects of winter

  45. [59]

    Footsteps fade on paths

    Imagery and Atmosphere: The response uses vivid imagery to create a strong sense of atmosphere. Descriptions such as "Footsteps fade on paths" and "Windows gleam with light" paint a clear picture of a winter scene

  46. [60]

    Snow": While the user specifically asked to avoid the word

    Coherence and Flow: The sentences flow well together, creating a coherent and engaging poem. Areas for Improvement: 1.Inclusion of "Snow": While the user specifically asked to avoid the word "snow," it is a common and essential element of winter that could be subtly included t...

  47. [61]

    Introducing a bit of variation in sentence structure could add to the poetic quality, such as using a question or exclamation to create a different tone or emphasis

    Variety in Sentence Structure: The response has a consistent structure with each sentence being a simple statement. Introducing a bit of variation in sentence structure could add to the poetic quality, such as using a question or exclamation to create a different tone or emphasis

  48. [62]

    hushed, waiting world

    Enhancing Emotional Depth: While the response is atmospheric, adding a bit more emotional depth or personal touch could make the poem more engaging. For example, describing the "hushed, waiting world" as "yearning for spring" could add a layer of emotional resonance. Overall, ...

  49. [63]

    winter" or

    Adherence to Constraints: The model successfully adheres to the requirement of each sentence being exactly five words long and avoids using the words "winter" or "snow."

  50. [64]

    Frost paints silent trees

    Imagery and Atmosphere: The poem effectively evokes a sense of the season through vivid imagery and subtle hints. For example, "Frost paints silent trees" and "Chill lingers in the air" convey the cold and stillness characteristic of winter

  51. [65]

    Areas for Improvement:

    Consistency in Tone: The tone remains consistent throughout the poem, maintaining a sense of hushed and serene atmosphere that aligns with the winter season. Areas for Improvement:

  52. [66]

    For example, including different sensory details (e.g., sounds, smells) could make the poem more immersive

    Variety in Imagery: While the imagery is strong, it could benefit from a bit more variety. For example, including different sensory details (e.g., sounds, smells) could make the poem more immersive

  53. [67]

    For instance, a line that hints at nostalgia or anticipation could deepen the reader’s connection to the season

    Emotional Depth: Adding a slight emotional undertone could enhance the poem. For instance, a line that hints at nostalgia or anticipation could deepen the reader’s connection to the season

  54. [68]

    Rhythm and Flow: The rhythm is good, but slightly varying the structure or adding a rhyming scheme could make the poem more engaging and memorable. 18

  55. [2019]

    URLhttps://arxiv.org/abs/1903.00161

    1903 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.