A perturbation framework with Drop/Add/Flip and player-removal operations demonstrates that Bradley-Terry leaderboards are non-robust to sub-1% targeted changes that alter top ranks, Kendall tau, and confidence intervals.
Confidence diagram of nonparametric ranking for uncertainty assessment in large language models evaluation.arXiv preprint arXiv:2412.05506, 2024
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
years
2026 3representative citing papers
A hierarchical framework generates statistically valid task-level rank confidence intervals via pairwise comparisons and leaderboard-level rank prediction intervals via conformal prediction.
A statistical survey of RLHF for LLM alignment that connects preference learning and policy optimization to models like Bradley-Terry-Luce while reviewing methods, extensions, and open challenges.
citing papers explorer
-
A Unified Perturbation Framework for Analyzing Leaderboard Stability and Manipulation
A perturbation framework with Drop/Add/Flip and player-removal operations demonstrates that Bradley-Terry leaderboards are non-robust to sub-1% targeted changes that alter top ranks, Kendall tau, and confidence intervals.
-
Rank Intervals for Leaderboards: A Hierarchical Framework for Model Evaluation
A hierarchical framework generates statistically valid task-level rank confidence intervals via pairwise comparisons and leaderboard-level rank prediction intervals via conformal prediction.