Human adjudication of conflicts between original benchmark labels and LLM predictions on QAGS-C and SummEval increases triple agreement by 6-8% and LLM accuracy by 2-9%, with adjudicators often siding with models that provide explicit reasoning.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2026 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
Do Benchmarks Underestimate LLM Performance? Evaluating Hallucination Detection With LLM-First Human-Adjudicated Assessment
Human adjudication of conflicts between original benchmark labels and LLM predictions on QAGS-C and SummEval increases triple agreement by 6-8% and LLM accuracy by 2-9%, with adjudicators often siding with models that provide explicit reasoning.