Pith. sign in

BRIDGE: Predicting Human Task Completion Time From Model Performance

1 Pith paper cite this work. Polarity classification is still indexing.

1 Pith paper citing it
abstract

Evaluating the real-world capabilities of AI systems requires grounding benchmark performance in human-interpretable measures of task difficulty. Existing approaches that rely on direct human task completion time annotations are costly, noisy, and difficult to scale across benchmarks. In this work, we propose BRIDGE, a unified psychometric framework that learns a latent difficulty scale from model responses and anchors it to human task completion time. Using a two-parameter logistic Item Response Theory model, we jointly estimate latent task difficulty and model capability from model performance data across multiple benchmarks. We demonstrate that latent task difficulty varies linearly with the logarithm of human completion time, allowing human task completion time to be inferred for new benchmarks from model performance alone. Leveraging this alignment, we forecast frontier model capabilities in terms of human task length and independently reproduce METR's exponential scaling results, with the 50% solvable task horizon doubling approximately every 6 months.

fields

cs.LG 1

years

2026 1

verdicts

CONDITIONAL 1

representative citing papers

Predicting Task Difficulty Without Rollouts

cs.LG · 2026-08-06 · conditional · novelty 6.0

Pre-rollout task difficulty for agentic benchmarks is predictable from token-level entropy features, with Spearman rho=0.399 in-distribution and 0.225 out-of-distribution.

citing papers explorer

Showing 1 of 1 citing paper.

  • Predicting Task Difficulty Without Rollouts cs.LG · 2026-08-06 · conditional · none · ref 15 · internal anchor

    Pre-rollout task difficulty for agentic benchmarks is predictable from token-level entropy features, with Spearman rho=0.399 in-distribution and 0.225 out-of-distribution.