A debate-based evaluation protocol on 50 MMLU-Pro questions: fine-tuning on the test set boosts standard accuracy from 50% to 82% but not debate win rates.
ARC-AGI-2 + ARC Prize 2025 is Live! Blog Post, https://arcprize.org/blog/announcing-arc-agi-2-and-arc-prize-2025, March 24 2025
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Pretraining on the Test Set Is No Longer All You Need: A Debate-Driven Approach to QA Benchmarks
A debate-based evaluation protocol on 50 MMLU-Pro questions: fine-tuning on the test set boosts standard accuracy from 50% to 82% but not debate win rates.