{"id":"b5d059c3-8e65-47b3-a8c2-b89411693c67","arxiv_id":"2507.09104","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A 7B judge model trained with verifiable reward signals and a margin contrastive loss matches the judgment accuracy of models tens of times larger, and a new benchmark JudgerBenchV2 standardizes judge evaluation.","lead":"This paper trains a generalist LLM judge that scores model responses across many tasks, using verifiable correctness labels and a margin-based loss to improve accuracy. If the training recipe holds up, smaller judge models could replace much larger ones, and the new JudgerBenchV2 gives the field a standard way to measure judges.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"JudgerBenchV2's Mix-of-Judgers ground truth is unvalidated; if the three LLM judges share bias, the 7B model's ~1-point JudgerBenchV2 surplus over the parity threshold could vanish, breaking the headline 'competitive with DeepSeek-V3/Qwen3-235B-A22B' claim.","rationale":"The reader's weakest assumption identifies exactly the load-bearing vulnerability: JudgerBenchV2's ground truth is an unvalidated Mix-of-Judgers consensus, and the paper's flagship claim of 7B-level parity with 200B+ models rests on a razor-thin 0.25-point average margin. My analysis quantifies why this is decisive: the JudgerBenchV2 score is the only benchmark where CJ2-7B has less than a ~1-point cushion above the parity threshold, so any systematic bias in MoJ labels can erase the central comparison. The paper provides no human agreement study, no bias analysis, and no contamination check, and its Limitations section does not mention these gaps. I also note that the policy-gradient derivation in Appendix A (Eqs. 10–11) is mathematically incorrect: conditioning on the accepted answer changes the estimator and the implemented loss is essentially token-level SFT/margin classification rather than a proper policy gradient. However, this is a methodological correctness problem that does not by itself falsify the empirical results; the benchmark validity issue is more load-bearing because it directly threatens the headline comparison. For these reasons, the reader's CONDITIONAL verdict is appropriate, and I recommend no change to the verdict. The proposed human-validation test would settle whether the concern actually lands, and bootstrapping the margin-loss gain would add needed statistical rigor to the ablation claim.","tokens_in":15309,"tokens_out":8194,"duration_ms":95147,"concrete_test":"Human-validate a random sample of 300–500 JudgerBenchV2 items (stratified by scenario and language): have 3+ human annotators independently choose the better response in each pair, with majority as human label. Compute (i) human–MoJ agreement, and (ii) each model's JudgerBenchV2 accuracy and final score using human labels instead of MoJ (or restricting to items where human and MoJ agree). If CompassJudger-2-7B's four-benchmark average remains ≥ 71.86 and its rank vs. all 7B baselines is unchanged, the parity claim survives; if the average drops below DeepSeek-V3 or below Con-J/RISE on the human-labeled subset, the headline comparison should be revised. Also report bootstrapped confidence intervals on the 2.21-point margin-loss gain to check whether the Table 4 improvement is statistically distinguishable from noise.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In §4 (Data Construction and Mixture of Judges), JudgerBenchV2 sets ground truth as the majority vote of DeepSeek-R1, DeepSeek-V3-0324, and Qwen3-235B-A22B, and reports no human verification of these labels. The strongest claim in Table 2 — CompassJudger-2-7B (72.11) comparable to DeepSeek-V3 (71.86) and Qwen3-235B (71.91) — depends on all four benchmark numbers being meaningful. Given the 7B model's other scores (JudgeBench 63.06, RMB 73.90, RewardBench 90.96), matching DeepSeek-V3's 71.86 average requires a JudgerBenchV2 score of at least 59.52; the reported 60.52 leaves only a 1.00-point cushion. If MoJ consensus carries a shared LLM bias (e.g., preference for style, length, or a model family), the JudgerBenchV2 score can be inflated by more than this margin, and the parity claim fails. The concern is compounded by the training pipeline: judgments are synthesized with Qwen2.5-72B-Instruct and public judge/reward data are used without a reported contamination analysis, so high JudgerBenchV2 agreement may reflect alignment with LLM-judge preferences rather than general judging skill. The paper's own Limitations section acknowledges only inference cost and synthesis hallucination, not label validation or contamination; these are missing support for the benchmark's central validity claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents CompassJudger-2, a family of LLM judge models trained with a task-driven data pipeline that includes curation of public judge/reward data, synthesis of knowledge- and chat-based judgment data, rejection sampling, and a proposed 'margin policy gradient loss.' The authors also introduce JudgerBenchV2, a benchmark whose ground truth is the majority vote of three LLM judges (DeepSeek-R1, DeepSeek-V3-0324, Qwen3-235B-A22B) and whose metric combines sample-level accuracy with rank and score consistency. Empirical results report that CompassJudger-2-7B-Instruct outperforms existing 7B judge models and approaches the average score of much larger models such as DeepSeek-V3 and Qwen3-235B-A22B on a combination of JudgerBenchV2, JudgeBench, RMB, and RewardBench.","tokens_in":15715,"tokens_out":6458,"duration_ms":79309,"significance":"If the claims hold, this is a practically valuable contribution: a 7B judge model competitive with much larger models, with demonstrated robustness to prompt-style changes, and a new benchmark addressing judge evaluation beyond simple accuracy. The paper provides extensive evaluations across four judge benchmarks and multiple general benchmarks, plus ablations showing consistent improvements from the proposed data and loss components. The main methodological claim, however, rests on a policy-gradient derivation that is not mathematically correct as written, and the new benchmark's ground truth is not independently validated. These issues affect the interpretation of the headline results and need to be addressed before the paper can be accepted.","major_comments":[{"comment":"The simplification of the policy gradient is mathematically incorrect. The reward r(x,y) = 1{y_kx = y*_kx} depends only on the answer token, but the expectation E_y~πθ[r(y) ∇θ log πθ(y|x)] expands to a sum over all token positions. While the terms for t > kx vanish in expectation by the score-function identity, the terms for t < kx do not: the probability of the answer token depends on all preceding tokens, and those tokens carry gradient information about the distribution of the answer. The loss in Eq. (2) is therefore not the gradient of the expected reward; it is a teacher-forced token-level cross-entropy on the answer position. Because the paper's central methodological framing ('policy gradient loss with verifiable reward signals') relies on this derivation, it should be corrected or the method should be explicitly presented as a heuristic token-level objective supported by the ablations in Table 4.","section":"Section 3.2, Eq. (2); Appendix A, Eq. (10)"},{"comment":"JudgerBenchV2 labels are the majority vote of DeepSeek-R1, DeepSeek-V3-0324, and Qwen3-235B-A22B, with no human validation, no inter-judge agreement analysis, and no discussion of shared bias. The headline claim in Table 2 — CompassJudger-2-7B (72.11) comparable to DeepSeek-V3 (71.86) and Qwen3-235B-A22B (71.91) — depends on the JudgerBenchV2 score of 60.52. If the three judge models share a systematic bias (e.g., preference for style, length, or a particular model family), the JudgerBenchV2 score could be inflated and the parity claim would fail. The training data are also synthesized with Qwen2.5-72B-Instruct, and no contamination analysis is reported. Please add a human-verified subset, report judge agreement, and analyze potential bias and contamination in JudgerBenchV2.","section":"Section 4, 'Mixture of Judges'; Table 2"},{"comment":"The rejection-sampling loss in Eq. (3)-(4) is not a policy-gradient estimator; it is a supervised token-level loss over accepted samples. In addition, Eq. (12) contains indexing errors: the conditioning context uses x^{(j)} where x^{(i)} is intended, and the decomposition of L_total into L_SFT and L_PG is not derived consistently. These issues obscure the relationship between the proposed loss and the claimed policy-gradient objective, and they should be fixed.","section":"Section 3.2, Eq. (12); Appendix A"},{"comment":"The Limitations section acknowledges only inference cost and synthesis hallucination. It does not mention the lack of validation for JudgerBenchV2 labels or the risk of contamination between training data and evaluation benchmarks. Since these are the main threats to the paper's central empirical claim, they should be addressed or explicitly discussed as limitations.","section":"Section 7, Limitations"}],"minor_comments":[{"comment":"In the CompassJudger-1-32B-Instruct row, the scores for JudgeBench, RMB, and RewardBench are printed without spaces ('62.2977.6386.17'), making the row difficult to read.","section":"Table 2"},{"comment":"The sentence 'delivering an 2.21% performance on average boost compared to the baseline model' contains a grammatical error; it should read 'a 2.21-point average improvement over the baseline model.'","section":"Section 5.3"},{"comment":"The notation x^{(j)} in the conditioning context is inconsistent with the outer index i; it should be x^{(i)} throughout the equation.","section":"Appendix A, Eq. (12)"},{"comment":"The metric formula contains a stray asterisk before the rank-difference term and appears to have unbalanced parentheses in the printed version; please format the equation cleanly so the normalization is unambiguous.","section":"Section 4, Eq. (5)"}],"recommendation":"major_revision","confidential_remarks":"The paper is from the authors' own platform (CompassArena) and introduces a benchmark in the same paper; independent validation of JudgerBenchV2 labels is therefore important. The GitHub link is provided, but the paper does not clearly state whether the benchmark data and training data will be released. If the authors release the benchmark and code, it would substantially strengthen reproducibility and the credibility of the empirical claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know up front. The empirical recipe is real: CompassJudger-2-7B beats every 7B judge baseline on average across four benchmarks, and its average (72.11) is a hair above DeepSeek-V3 (71.86) and Qwen3-235B (71.91). The margin loss with rejection sampling is a new combination and the ablations are consistent: it adds 2.2 points over the SFT baseline. That's a genuine result.\n\nBut the paper overreaches in two places. First, the policy-gradient derivation in Eq (2)/(10) is simply wrong. The reward depends only on token k, but that does not let you drop the sum over all token positions in the log-probability gradient. The conditional expectation they write after simplifying does not follow from REINFORCE. The margin loss itself is a fine heuristic, but it is not derived from the expected-reward objective. This is fixable by presenting it as a heuristic, but as written it is an error.\n\nSecond, the new JudgerBenchV2 benchmark uses a Mix-of-Judgers majority vote (DeepSeek-R1, DeepSeek-V3-0324, Qwen3-235B) as ground truth, with no human verification. The stress-test math is correct: with the other three benchmark scores fixed, the 7B model needs a JudgerBenchV2 score above ~59.5 to match DeepSeek-V3's average; it reports 60.5. A few points of shared LLM bias would erase the parity claim. The paper doesn't report error bars or contamination checks, and the Limitations section doesn't mention label validation.\n\nThe good news is that the main empirical claim doesn't rest entirely on JudgerBenchV2. The 7B model also shows solid gains on RewardBench (90.96), JudgeBench (63.06), and RMB (73.90) compared to other 7B judges. So the model is a genuine improvement, even if the 'comparable to 235B' phrasing is fragile.\n\nBottom line: this is a solid engineering paper worth sending to serious referees. The derivation must be corrected or reframed, and the benchmark needs human labels or a bias analysis. With those revisions, it would be a useful contribution to the LLM-evaluation literature. I'd accept it for review, but I'd expect major revision.","headline":"Solid empirical paper with a flawed derivation and an unvalidated benchmark; the 7B parity claim is real but fragile.","tokens_in":16198,"tokens_out":4080,"would_cite":false,"duration_ms":44306,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 7B-parameter judge model trained on verifiable rewards reaches an average of 72.11 on four judge and reward benchmarks, matching much larger models.","keywords":["LLM-as-a-judge","generalist judge model","verifiable reward","rejection sampling","margin policy gradient loss","JudgerBenchV2","Mix-of-Judgers","reward model evaluation"],"falsifier":"Take a random sample of JudgerBenchV2's 10,000 pairwise questions, have human experts label which response is better, then recompute CompassJudger-2-7B's score against human labels; if its advantage over the larger models and baselines shrinks or reverses, the Mix-of-Judgers labels are carrying the result rather than the verifiable-reward training.","tokens_in":15144,"feed_emoji":"⚖️","tokens_out":7771,"duration_ms":84081,"temperature":0.7,"pith_summary":"The paper argues that a general-purpose LLM judge—one that grades, ranks, and critiques responses across tasks—can be trained more effectively when the supervisor signal is a verifiable reward rather than human-written judgment alone. The authors build CompassJudger-2 from Qwen2.5 base models using a data mix of reconstructed public judge data, reward data filtered by rejection sampling, and synthetic knowledge and chat data. Their 7B model scores 72.11 on average across JudgerBenchV2, JudgeBench, RMB, and RewardBench, above all 7B judge baselines and roughly level with much larger models around 72 points. The margin policy-gradient loss contributes a 2.21-point average gain over the SFT-only baseline, the largest of the three loss variants tested. The paper also introduces JudgerBenchV2, a 10,000-question benchmark whose ground truth is the majority consensus of three strong judge models and whose metric combines per-sample accuracy with ranking consistency.","feed_headline":"A 7B judge model scores 72.11, matching 235B rivals","feed_subtitle":"Verifiable rewards plus a margin loss lift the small judge above all 7B baselines on four evals.","key_machinery":"The central mechanism is verifiable reward supervision for judgments. The model is prompted to reason through a fixed structure—user demand, strengths and weaknesses of each response, reasoning, then a final choice—so the reward is a rule-based function: 1 if the choice token matches the ground-truth label at a designated position, 0 otherwise. Rejection sampling generates eight candidate reasoning trajectories that end in the correct verdict, creating diverse prefixes for training; the policy-gradient-style loss then concentrates on the conditional log-probability of the correct token, and the margin variant enforces a margin between correct and incorrect verdicts. This machinery lets the authors train a 7B judge without large volumes of human critique labels while retaining generalization across judge, reward, and general instruction tasks.","core_discovery":"CompassJudger-2 is presented as a generalist all-in-one judge series that outperforms existing 7B judge models and matches substantially larger general models on judge and reward benchmarks. On the four-benchmark average, the 7B Instruct model reaches 72.11, compared with 71.86 for DeepSeek-V3-0324 and 71.91 for Qwen3-235B-A22B, and it improves on the prior CompassJudger-1-7B by 16.39 percent on average. The decisive training choice is to treat the final verdict as a classification step with a deterministic ground-truth reward: candidate reasoning paths that produce the correct verdict are kept via rejection sampling, and a margin loss pushes the probability of the correct token above the wrong-token probability by a margin of 10. This margin loss adds 2.21 points on average over the SFT baseline, while DPO and temperature variants add less or nothing. The companion JudgerBenchV2 contributes a 10,000-sample, ten-scenario benchmark with Mix-of-Judgers ground truth and a metric that penalizes rank and score disagreement, not just sample-level accuracy.","pith_inferences":["Editorial inference: if the 7B result transfers to other base models, verifiable-reward judging could lower the cost of model evaluation enough for small research groups to run their own reward models rather than relying on external judge APIs.","Editorial inference: because JudgerBenchV2's ground truth is a vote among three LLM judges, the benchmark's labels inherit whatever shared preference bias those judges have; a human-verified subset would test how much of the reported 72.11 is genuine judging skill.","Editorial inference: the margin-loss idea can be applied to any classification-style LLM task with checkable answers, such as grading code outputs or math solutions, not just pairwise response judging.","Editorial inference: the same pipeline has a natural next test in multi-turn and multimodal evaluation, which the paper names as future work and which would exercise the claim of generality more aggressively."],"forward_implications":["A 7B-parameter judge model reaches an average of 72.11 across four judge and reward benchmarks, making high-quality LLM evaluation feasible with far less compute than a 235B-level judge.","Supervising judges with verifiable rewards plus rejection sampling can reduce reliance on expensive human-written critiques for training judge models.","The margin loss is the best of the three tested mapping functions, adding 2.21 points over the SFT baseline and improving on all four benchmarks.","Including rejection-sampled reward data improves judge consistency and also lifts general benchmarks such as GPQA-Diamond and ArenaHard.","Critiques generated by CompassJudger-2 improve downstream policy models on AlignBench, AlpacaEval, and ArenaHard, while a comparison judge's critiques often lower those scores."],"supporting_citations":[{"why":"It supplies the critical-thinking chain-of-thought template and rule-based reward idea that the training pipeline adapts.","marker":"[13]"},{"why":"It provides the Qwen2.5-Instruct base checkpoints that CompassJudger-2 is trained from.","marker":"[29]"},{"why":"It is the prior all-in-one judge whose data mix and general instruction data are reused.","marker":"[3]"},{"why":"It serves as one of the three Mix-of-Judgers voter models and as a general-model baseline.","marker":"[21]"},{"why":"It serves as one of the three Mix-of-Judgers voter models and as a general-model baseline.","marker":"[27]"},{"why":"It supplies the RewardBench evaluation used in the main results.","marker":"[16]"},{"why":"It supplies the JudgeBench evaluation used in the main results.","marker":"[26]"},{"why":"It supplies the RMB evaluation used in the main results.","marker":"[33]"},{"why":"It is the Con-J all-in-one judge baseline trained with DPO.","marker":"[31]"},{"why":"It is the RISE judge baseline used in comparisons and in the critique-improvement experiment.","marker":"[32]"}],"fun_headline_variants":["7B judge scores 72.11 to match 235B","7B judge matches 235B rivals via verifiable rewards","Verifiable rewards let a 7B judge rival 235B models","Small 7B judge ties giants with verifiable rewards","7B judge model matches 235B via margin loss"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"JudgerBenchV2's ground truth is the majority vote of three LLM judges with no human verification, so if those judges share a systematic bias, the benchmark's labels favor models that align with LLM preferences rather than with genuinely better responses.","fun_headline_variants_meta":{"raw":{"variants":["7B judge scores 72.11 to match 235B","7B judge matches 235B rivals via verifiable rewards","Verifiable rewards let a 7B judge rival 235B models","Small 7B judge ties giants with verifiable rewards","7B judge model matches 235B via margin loss"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000861,"raw_usage":{"total_tokens":3753,"prompt_tokens":980,"completion_tokens":2773,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":596,"completion_tokens_details":{"reasoning_tokens":2686}},"tokens_in":596,"tokens_out":2773,"duration_ms":20841,"temperature":1.0,"reasoning_tokens":2686,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T18:04:42.599589+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a random sample of JudgerBenchV2's 10,000 pairwise questions, have human experts label which response is better, then recompute CompassJudger-2-7B's score against human labels; if its advantage over the larger models and baselines shrinks or reverses, the Mix-of-Judgers labels are carrying the result rather than the verifiable-reward training.","supporting_citations":[{"cited_title":"Qwen3: Think deeper, act faster","cited_arxiv_id":null,"evidence_quote":"It serves as one of the three Mix-of-Judgers voter models and as a general-model baseline."},{"cited_title":"Learning llm-as-a-judge for preference alignment","cited_arxiv_id":null,"evidence_quote":"It is the Con-J all-in-one judge baseline trained with DPO."}],"review_version":1}