Calibrating the full set of LLM judges with labeled data halves calibration error versus top-5 accuracy selection on RewardBench2 and outperforms on four benchmarks.
Steps 1–2 provide the Bayesian posterior probabilities that serve as input for beta calibration; Step 4 adds distribution-free coverage guarantees
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
stat.ME 1years
2026 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
Calibrate, Don't Curate: Label-Efficient Estimation from Noisy LLM Judges
Calibrating the full set of LLM judges with labeled data halves calibration error versus top-5 accuracy selection on RewardBench2 and outperforms on four benchmarks.