SidConArena is a new multi-phase benchmark framework formalizing a partially observable stochastic game for evaluating LLM agents in open-ended positive-sum bargaining with negotiation, converter production, and sealed-bid auctions.
Benchmark agreement testing done right: A guide for LLM benchmark evaluation
4 Pith papers cite this work. Polarity classification is still indexing.
4
Pith papers citing it
citation-role summary
background 1
citation-polarity summary
verdicts
UNVERDICTED 4roles
background 1polarities
background 1representative citing papers
BenchMarker toolkit audits 12 MCQA benchmarks for contamination, shortcuts, and writing errors using LLM judges, finding widespread flaws that inflate or deflate accuracy and alter rankings.
MTR-Bench is a new automated benchmark for multi-turn reasoning in LLMs covering diverse tasks and difficulty levels with 3600 instances.
Submodular maximization under a Gaussian model selects small benchmark subsets that outperform random selection for imputing leaderboard scores, with mutual information better than entropy at small sizes.
citing papers explorer
-
SidConArena: An Environment Evaluating Agents in Open-Ended,Positive-Sum Bargaining Game
SidConArena is a new multi-phase benchmark framework formalizing a partially observable stochastic game for evaluating LLM agents in open-ended positive-sum bargaining with negotiation, converter production, and sealed-bid auctions.
-
BenchMarker: An Education-Inspired Toolkit for Highlighting Flaws in Multiple-Choice Benchmarks
BenchMarker toolkit audits 12 MCQA benchmarks for contamination, shortcuts, and writing errors using LLM judges, finding widespread flaws that inflate or deflate accuracy and alter rankings.
-
MTR-Bench: A Comprehensive Benchmark for Multi-Turn Reasoning Evaluation
MTR-Bench is a new automated benchmark for multi-turn reasoning in LLMs covering diverse tasks and difficulty levels with 3600 instances.
-
Submodular Benchmark Selection
Submodular maximization under a Gaussian model selects small benchmark subsets that outperform random selection for imputing leaderboard scores, with mutual information better than entropy at small sizes.