The paper shows how bootstrapping and Bayesian hierarchical modeling can quantify uncertainty in task-aggregated benchmark scores, using simulated VTAB data, and that a low-ranked model can dominate under certain task weightings.
Statistical comparisons of classifiers over multiple data sets
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
stat.ML 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Statistical Uncertainty Quantification for Aggregate Performance Metrics in Machine Learning Benchmarks
The paper shows how bootstrapping and Bayesian hierarchical modeling can quantify uncertainty in task-aggregated benchmark scores, using simulated VTAB data, and that a low-ranked model can dominate under certain task weightings.