For comparing two binary classifiers using a budget of noisy labels, collecting one label per sample across more samples outperforms aggregating multiple labels per sample.
Title resolution pending
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
fields
cs.LG 2verdicts
UNVERDICTED 2representative citing papers
SySRs is a hyperparameter-free best-arm identification algorithm with performance guarantees that improve with model similarity, outperforming baselines on 15 LLM benchmarks.
citing papers explorer
-
Don't Label Twice: Quantity Beats Quality when Comparing Binary Classifiers on a Budget
For comparing two binary classifiers using a budget of noisy labels, collecting one label per sample across more samples outperforms aggregating multiple labels per sample.
-
Cutting LLM Evaluation Costs with SySRs: A Bandit Algorithm that Provably Exploits Model Similarity
SySRs is a hyperparameter-free best-arm identification algorithm with performance guarantees that improve with model similarity, outperforming baselines on 15 LLM benchmarks.