Pith. sign in

REVIEW 4 minor 2 cited by

Historical model scores let you evaluate new models on fewer prompts while keeping unbiased estimates and valid confidence intervals.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 09:38 UTC pith:YHRSAPUH

load-bearing objection Clean, usable combination of matrix completion and CrossPPI that actually cuts evaluation cost on real generative-model benchmarks while keeping coverage.

arxiv 2607.05046 v1 pith:YHRSAPUH submitted 2026-07-06 cs.LG

CollabEval: Statistically Efficient Collaborative Model Evaluation via Matrix Completion

Adam Fisch , Daniel Deutsch , Joshua Maynez , Alekh Agarwal , Jonathan Berant , William Cohen , Amir Globerson , Jacob Eisenstein This is my paper
classification cs.LG
keywords model evaluationmatrix completionprediction-powered inferencecontrol variatesconfidence intervalscollaborative filteringstatistical efficiency
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Evaluating generative models is expensive because each new model still needs many prompts scored. CollabEval treats the collection of model-by-prompt scores as a partially observed matrix. Historical “anchor” models supply dense rows; new “target” models supply only a sparse fraction p of labels. A cross-fold matrix-completion step imputes the missing entries; those imputations are then used as control variates inside a prediction-powered estimator. The resulting point estimates of mean performance (and of pairwise differences) remain unbiased, and the accompanying confidence intervals are asymptotically valid, even when the completion is imperfect. Across five real benchmarks the method shrinks average interval widths by up to 30 percent relative to classical sampling at the same annotation budget, effectively converting past evaluations into extra statistical power.

Core claim

By casting multi-model evaluation as low-rank matrix completion and feeding the completed entries into a cross-prediction-powered control-variate estimator, one obtains unbiased estimates of both individual model means and pairwise differences whose asymptotic variance is never worse than the classical sample mean and is typically substantially smaller, together with confidence intervals that retain valid coverage under mild stability conditions on the completion algorithm.

What carries the argument

Cross-fold matrix completion (IterativeSVD by default) produces predictions that are independent of the held-out labels; those predictions enter a power-tuned CrossPPI control-variate estimator whose asymptotic covariance is given explicitly in terms of the true and imputed scores, guaranteeing both unbiasedness and valid normal-based confidence intervals.

Load-bearing premise

The matrix-completion algorithm must become stable: as more data arrive, its cross-fold predictions must converge to a fixed function of only the anchor-model scores on each prompt.

What would settle it

On a new benchmark where the score matrix has no low-rank structure (so that completed entries are essentially uncorrelated with true target scores), measure whether CollabEval’s confidence-interval widths remain no larger than the classical baseline and whether empirical coverage still meets the nominal rate; any systematic widening or under-coverage would falsify the claim.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

0 major / 4 minor

Summary. The paper proposes CollabEval, which reframes model evaluation as matrix completion over an M imes N score matrix of models by prompts. For a subset of target models only a fraction p of entries are observed; historical (anchor) models supply dense rows. Cross-fold matrix completion produces imputed scores that are used as control variates inside a CrossPPI estimator, yielding unbiased point estimates of individual means θ_i and pairwise differences Δ_ij together with asymptotically valid confidence intervals. Power-tuning weights are derived so that asymptotic variance is never larger than that of the classical sample mean. Empirically, on five generation benchmarks the method reduces average CI width by up to ~30 % (and MSE correspondingly) at fixed labeling budget while empirical coverage tracks the nominal 90 % level.

Significance. The work cleanly marries collaborative filtering with prediction-powered inference for a practically important problem—repeated, expensive evaluation of generative models. The theoretical package (multivariate CLT under a mild stability assumption, consistent covariance estimators, optimal power-tuning that is guaranteed never to inflate variance) is complete and carefully stated; the empirical gains are consistent across diverse tasks and sampling regimes. Because the method never requires generation or labeling of the skipped prompts, the cost reduction is essentially free once historical evaluations exist. The framework is modular: any stable completer can be substituted, and the validity guarantees survive even when the low-rank hypothesis is only approximate.

minor comments (4)
  1. Assumption A.1 (imputation stability) is stated but not proved for the concrete IterativeSVD algorithm used throughout the experiments. A short remark or reference to concentration results for regularized SVD would strengthen the theoretical narrative, even though the empirical coverage plots already support the claim.
  2. Figure 2 and the per-task appendix figures would benefit from a brief note on the number of bootstrap trials (stated as 10 k) and whether the shaded min/max bands are across datasets or across bootstrap replicates.
  3. In §4.3 the rewritten control-variate form of the estimator (Eq. 3) is convenient, but a one-sentence reminder that it is asymptotically equivalent to the classical PPI form under optimal scaling would help readers coming from the PPI literature.
  4. Appendix D.2 compares four completers; a short discussion of wall-clock cost relative to the evaluation budget itself would be useful for practitioners deciding whether to adopt the method.

Circularity Check

0 steps flagged

No circularity: CrossPPI rectifier guarantees unbiasedness independent of matrix-completion quality; low-rank structure is used only for efficiency, never for validity.

full rationale

The derivation is self-contained and non-circular. The estimator (Eq. 3) is the classical CrossPPI control-variate form: the sample mean of observed scores is corrected by a rectifier that subtracts the difference between the empirical mean of the imputed scores on the labeled set and their mean over the full set. Unbiasedness holds for any (even completely uncorrelated) imputations; the low-rank matrix-completion step is invoked solely to produce useful control variates that reduce residual variance. Power-tuning coefficients λ* are estimated from the same data but are shown (Prop. A.7–A.9) to converge to the population optimum that never increases asymptotic variance relative to the classical mean. Assumption A.1 (imputation stability) is required only for the CLT that justifies the plug-in confidence intervals; it is an external regularity condition on the completer, not a quantity fitted from the target estimands. No step defines a quantity in terms of the quantity it later claims to predict, no uniqueness theorem is imported from overlapping authors, and no fitted parameter is re-presented as an independent first-principles prediction. The empirical efficiency gains are therefore genuine statistical consequences of the control-variate construction, not tautologies.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 0 invented entities

The statistical validity rests on standard i.i.d. sampling of prompts, finite second moments, and the mild stability Assumption A.1 on the completion algorithm. Efficiency gains further rely on the empirical low-rank structure of score matrices (documented but not assumed for correctness). Free parameters are the usual algorithmic knobs of iterative SVD and cross-validation; none are fitted to produce the headline coverage claims.

free parameters (3)
  • rank schedule R = (1,2,4,8,16,16,16,16)
    Default sequence (1,2,4,8,16,…) chosen for speed/accuracy trade-off; ablation shows performance plateaus around rank 16 but the schedule itself is a free design choice.
  • number of folds K = 10
    Default K=10; ablation shows mild sensitivity but the value is selected by the authors rather than derived.
  • power-tuning coefficients λ
    Estimated from empirical covariances on the observed data; they are free in the sense that they are data-dependent scalars, though the paper proves they converge to the population optimum.
axioms (3)
  • domain assumption Imputation stability (Assumption A.1): cross-fold predictions converge in L1 to a fixed function of the observed anchor scores.
    Required for the CLT of the CrossPPI estimator; stated formally in Appendix A and used throughout the coverage proofs.
  • domain assumption Prompts are drawn i.i.d. from a fixed distribution of interest; observation mask Ω is independent of the scores.
    Standard sampling assumption that justifies treating rows as exchangeable and enables the Lindeberg-Feller argument.
  • standard math Evaluation scores have finite second moments (and are strictly bounded for the consistency of the empirical covariance estimator).
    Invoked to apply the multivariate CLT and to guarantee that empirical covariances converge.

pith-pipeline@v1.1.0-grok45 · 36209 in / 2450 out tokens · 37236 ms · 2026-07-11T09:38:15.306258+00:00 · methodology

0 comments
read the original abstract

Evaluating generative AI models is a routine, but resource-intensive, process that is conducted over and over again during the course of model development. In this work, we propose Collaborative Evaluation (CollabEval), a simple, effective, and principled method for exploiting dependencies between historical runs of different models on the same tasks to improve statistical efficiency. Specifically, our approach treats model evaluation as a matrix completion problem over an $M \times N$ matrix of evaluation scores, where $M$ is the total number of models and $N$ is the total number of evaluation prompts. We assume that a subset of these $M$ models are targeted for evaluation. For these target models only a small fraction, $p$, of prompts has been annotated with evaluation scores. Leveraging recent results in prediction-powered inference, we build a low-rank approximation of the score matrix, and use the reconstructed values as control variates in a manner that guarantees unbiased estimates of the true evaluation metric mean, in addition to statistically valid confidence intervals. Empirically, across a wide range of datasets, models, and sparsity levels $p$, we find that CollabEval substantially reduces the mean confidence interval size, and the mean squared error of the point estimate, compared to baseline methods at the same annotation budget.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. BayesAME: Bayesian Active Model Evaluation

    cs.LG 2026-07 conditional novelty 6.0

    A sequential Bayesian method automatically grows a coreset until performance estimate and uncertainty stabilize, outperforming adapted baselines and showing active selection beats random when reference signals are rich.

  2. Efficient Sequential Evaluation of Large Language Models

    stat.ML 2026-07 conditional novelty 5.0

    A confidence-sequence framework for sequentially estimating an LLM's average benchmark accuracy under adaptive question selection, with growth-oriented sampling rules that in practice often lose to uniform sampling.

Reference graph

Works this paper leans on

37 extracted references · 9 linked inside Pith · cited by 2 Pith papers

  1. [1]

    Prediction-powered inference.Science, 382(6671):669–674, 2023

    Anastasios N Angelopoulos, Stephen Bates, Clara Fannjiang, Michael I Jordan, and Tijana Zrnic. Prediction-powered inference.Science, 382(6671):669–674, 2023

    2023

  2. [2]

    PPI++: Efficient prediction- powered inference.arXiv preprint arXiv:2311.01453, 2023

    Anastasios N Angelopoulos, John C Duchi, and Tijana Zrnic. PPI++: Efficient prediction- powered inference.arXiv preprint arXiv:2311.01453, 2023

    Pith/arXiv arXiv 2023

  3. [3]

    Angelopoulos, Jacob Eisenstein, Jonathan Berant, Alekh Agarwal, and Adam Fisch

    Anastasios N. Angelopoulos, Jacob Eisenstein, Jonathan Berant, Alekh Agarwal, and Adam Fisch. Cost-optimal active ai model evaluation.arXiv preprint arXiv:2506.07949, 2025. URL https://arxiv.org/abs/2506.07949

    Pith/arXiv arXiv 2025

  4. [4]

    Pythia: A suite for analyzing large language models across training and scaling

    Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. Pythia: A suite for analyzing large language models across training and scaling. In International Conference on Machine Learning, pages 2397–2430. PMLR, 2023

    2023

  5. [5]

    Bernd Bohnet, Vinh Q. Tran, Pat Verga, Roee Aharoni, Daniel Andor, Livio Baldini Soares, Massimiliano Ciaramita, Jacob Eisenstein, Kuzman Ganchev, Jonathan Herzig, Kai Hui, Tom Kwiatkowski, Ji Ma, Jianmo Ni, Lierni Sestorain Saralegui, Tal Schuster, William W. Cohen, Michael Collins, Dipanjan Das, Donald Metzler, Slav Petrov, and Kellie Webster. Attribute...

    Pith/arXiv arXiv 2023

  6. [6]

    AutoEval done right: Using synthetic data for model evaluation.arXiv preprint arXiv:2403.07008, 2024

    Pierre Boyeau, Anastasios N Angelopoulos, Nir Yosef, Jitendra Malik, and Michael I Jordan. AutoEval done right: Using synthetic data for model evaluation.arXiv preprint arXiv:2403.07008, 2024

    Pith/arXiv arXiv 2024

  7. [7]

    Prediction- powered ranking of large language models

    Ivi Chatzi, Eleni Straitouri, Suhas Thejaswi, and Manuel Gomez Rodriguez. Prediction- powered ranking of large language models. In A. Globerson, L. Mackey, D. Bel- grave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors,Advances in Neural Infor- mation Processing Systems, volume 37, pages 113096–113133. Curran Associates, Inc.,

  8. [8]

    URL https://proceedings.neurips.cc/paper_files/paper/2024/file/ cd47cd67caa87f5b1944e00f6781598f-Paper-Conference.pdf

    2024

  9. [9]

    WMT24++: Expanding the language coverage of WMT24 to 55 languages & dialects

    Daniel Deutsch, Eleftheria Briakou, Isaac Rayburn Caswell, Mara Finkelstein, Rebecca Galor, Ju- raj Juraska, Geza Kovacs, Alison Lui, Ricardo Rei, Jason Riesa, Shruti Rijhwani, Parker Riley, Eliz- abeth Salesky, Firas Trabelsi, Stephanie Winkler, Biao Zhang, and Markus Freitag. WMT24++: Expanding the language coverage of WMT24 to 55 languages & dialects. ...

    2025

  10. [10]

    ISBN 979-8-89176-256-5

    Association for Computational Linguistics. ISBN 979-8-89176-256-5. doi: 10.18653/v1/ 2025.findings-acl.634. URLhttps://aclanthology.org/2025.findings-acl.634/

    doi:10.18653/v1/ 2025

  11. [11]

    Alex Hofer, Bhuwan Dhingra, Amir Globerson, and William W

    Adam Fisch, Joshua Maynez, R. Alex Hofer, Bhuwan Dhingra, Amir Globerson, and William W. Cohen. Stratified prediction-powered inference for effective hybrid evaluation of language models. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URLhttps://openreview.net/forum?id=8CBcdDQFDQ

    2024

  12. [12]

    Can uncon- fident llm annotations be used for confident conclusions?arXiv preprint arXiv:2408.15204, 2024

    Kristina Gligorić, Tijana Zrnic, Cinoo Lee, Emmanuel J Candès, and Dan Jurafsky. Can uncon- fident llm annotations be used for confident conclusions?arXiv preprint arXiv:2408.15204, 2024

    Pith/arXiv arXiv 2024

  13. [13]

    Olmo: Accelerating the science of language models

    DirkGroeneveld, IzBeltagy, EvanWalsh, AkshitaBhagia, RodneyKinney, OyvindTafjord, Ananya Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, et al. Olmo: Accelerating the science of language models. InProceedings of the 62nd annual meeting of the association for computational linguistics (volume 1: Long papers), pages 15789–15809, 2024

    2024

  14. [14]

    Measuring massive multitask language understanding.arXiv preprint arXiv:2009.03300, 2020

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding.arXiv preprint arXiv:2009.03300, 2020

    Pith/arXiv arXiv 2009

  15. [15]

    Valentin Hofmann, David Heineman, Ian Magnusson, Kyle Lo, Jesse Dodge, Maarten Sap, Pang Wei Koh, Chun Wang, Hannaneh Hajishirzi, and Noah A. Smith. Fluid language model benchmarking. InSecond Conference on Language Modeling, 2025

    2025

  16. [16]

    SWE-bench: Can language models resolve real-world github issues? InThe Twelfth International Conference on Learning Representations, 2024

    Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. SWE-bench: Can language models resolve real-world github issues? InThe Twelfth International Conference on Learning Representations, 2024. URLhttps://openreview.net/ forum?id=VTF8yNQM66

    2024

  17. [17]

    MetricX-24: The Google submission to the WMT 2024 metrics shared task

    Juraj Juraska, Daniel Deutsch, Mara Finkelstein, and Markus Freitag. MetricX-24: The Google submission to the WMT 2024 metrics shared task. In Barry Haddow, Tom Kocmi, Philipp Koehn, and Christof Monz, editors,Proceedings of the Ninth Conference on Machine Translation, pages 492–504, Miami, Florida, USA, November 2024. Association for Computational Lingui...

    2024

  18. [18]

    Schulze Buschoff, and Eric Schulz

    Alex Kipnis, Konstantinos Voudouris, Luca M. Schulze Buschoff, and Eric Schulz. metabench - a sparse benchmark of reasoning and knowledge in large language models. InThe Thirteenth International Conference on Learning Representations, 2025. URLhttps://openreview.net/ forum?id=4T33izzFpK

    2025

  19. [19]

    Hashimoto

    Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Alpacaeval: An automatic evaluator of instruction-following models.https://github.com/tatsu-lab/alpaca_eval, 5 2023

    2023

  20. [20]

    Llm360: Towards fully transparent open-source llms.arXiv preprint arXiv:2312.06550, 2023

    Zhengzhong Liu, Aurick Qiao, Willie Neiswanger, Hongyi Wang, Bowen Tan, Tianhua Tao, Junbo Li, Yuqi Wang, Suqi Sun, Omkar Pangarkar, et al. Llm360: Towards fully transparent open-source llms.arXiv preprint arXiv:2312.06550, 2023

    Pith/arXiv arXiv 2023

  21. [21]

    Llm360 k2: Building a 65b 360-open-source large language model from scratch.arXiv preprint arXiv:2501.07124, 2025

    Zhengzhong Liu, Bowen Tan, Hongyi Wang, Willie Neiswanger, Tianhua Tao, Haonan Li, Fajri Koto, Yuqi Wang, Suqi Sun, Omkar Pangarkar, et al. Llm360 k2: Building a 65b 360-open-source large language model from scratch.arXiv preprint arXiv:2501.07124, 2025. 12 CollabEval: Statistically Efficient Collaborative Model Evaluation via Matrix Completion

    Pith/arXiv arXiv 2025

  22. [22]

    tinyBenchmarks: evaluating LLMs with fewer examples

    Felipe Maia Polo, Lucas Weber, Leshem Choshen, Yuekai Sun, Gongjun Xu, and Mikhail Yurochkin. tinyBenchmarks: evaluating LLMs with fewer examples. In Ruslan Salakhutdi- nov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors,Proceedings of the 41st International Conference on Machine Learning, volu...

  23. [23]

    URLhttps://proceedings.mlr.press/v235/maia-polo24a.html

  24. [24]

    Rich insights from cheap signals: Efficient evaluations via tensor factorization.arXiv preprint arXiv:2603.02029, 2026

    Felipe Maia Polo, Aida Nematzadeh, Virginia Aglietti, Adam Fisch, and Isabela Albuquerque. Rich insights from cheap signals: Efficient evaluations via tensor factorization.arXiv preprint arXiv:2603.02029, 2026. URLhttps://arxiv.org/abs/2603.02029

    arXiv 2026

  25. [25]

    Item response theory in ai: Analysing machine learning classifiers at the instance level

    Fernando Martínez-Plumed, Ricardo BC Prudêncio, Adolfo Martínez-Usó, and José Hernández- Orallo. Item response theory in ai: Analysing machine learning classifiers at the instance level. Artificial intelligence, 271:18–42, 2019

    2019

  26. [26]

    Spectral regularization algorithms for learning large incomplete matrices.Journal of Machine Learning Research, 11(80):2287–2322, 2010

    Rahul Mazumder, Trevor Hastie, and Robert Tibshirani. Spectral regularization algorithms for learning large incomplete matrices.Journal of Machine Learning Research, 11(80):2287–2322, 2010

    2010

  27. [27]

    2olmo2furious.arXivpreprintarXiv:2501.00656, 2024

    Team OLMo, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bhagia, YulingGu, ShengyiHuang, MattJordan, etal. 2olmo2furious.arXivpreprintarXiv:2501.00656, 2024

    Pith/arXiv arXiv 2024

  28. [28]

    You don’t need to run every eval, 2026

    Dimitris Papailiopoulos. You don’t need to run every eval, 2026. URLhttps://github.com/ anadim/llm-benchmark-matrix

    2026

  29. [29]

    B. D. Ripley.Stochastic simulation. John Wiley & Sons, Inc., New York, NY, USA, 1987. ISBN 0-471-81884-4

    1987

  30. [30]

    Lalor, Robin Jia, and Jordan Boyd-Graber

    Pedro Rodriguez, Joe Barrow, Alexander Hoyle, John P. Lalor, Robin Jia, and Jordan Boyd-Graber. Evaluation examples are not equally informative: How should that change NLP leaderboards? In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli, editors,Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th In...

    doi:10.18653/v1/2021.acl-long.346 2021

  31. [31]

    Missing value estimation methods for dna microarrays

    Olga Troyanskaya, Michael Cantor, Gavin Sherlock, Pat Brown, Trevor Hastie, Robert Tibshirani, David Botstein, and Russ B Altman. Missing value estimation methods for dna microarrays. Bioinformatics, 17(6):520–525, 2001

    2001

  32. [32]

    Anchor points: Benchmark- ing models with much fewer examples

    Rajan Vivek, Kawin Ethayarajh, Diyi Yang, and Douwe Kiela. Anchor points: Benchmark- ing models with much fewer examples. In Yvette Graham and Matthew Purver, editors, Proceedings of the 18th Conference of the European Chapter of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pages 1576–1601, St. Julian’s, Malta, March

  33. [33]

    doi: 10.18653/v1/2024.eacl-long.95

    Association for Computational Linguistics. doi: 10.18653/v1/2024.eacl-long.95. URL https://aclanthology.org/2024.eacl-long.95/

    doi:10.18653/v1/2024.eacl-long.95 2024

  34. [34]

    How predictable are large language model capabilities? a case study on BIG-bench

    Qinyuan Ye, Harvey Fu, Xiang Ren, and Robin Jia. How predictable are large language model capabilities? a case study on BIG-bench. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 7493–7517, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/...

    doi:10.18653/v1/2023 2023

  35. [35]

    Collaborative performance prediction for large language models

    Qiyuan Zhang, Fuyuan Lyu, Xue Liu, and Chen Ma. Collaborative performance prediction for large language models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 2576–2596, Miami, Florida, USA, November 2024. Association for Computational Linguis- ti...

    doi:10.18653/v1/2024.emnlp-main.150 2024

  36. [36]

    On speeding up language model evaluation

    Jin Peng Zhou, Christian K Belardi, Ruihan Wu, Travis Zhang, Carla P Gomes, Wen Sun, and Kilian Q Weinberger. On speeding up language model evaluation. InThe Thirteenth International Conference on Learning Representations, 2025. URLhttps://openreview.net/forum?id= 3cvwO5DBZn

    2025

  37. [37]

    power tuning

    Tijana Zrnic and Emmanuel J. Candès. Cross-prediction-powered inference.Proceedings of the National Academy of Sciences, 121(15):e2322083121, 2024. doi: 10.1073/pnas.2322083121. URLhttps://www.pnas.org/doi/abs/10.1073/pnas.2322083121. 14 CollabEval: Statistically Efficient Collaborative Model Evaluation via Matrix Completion Contents A Theoretical results...

    doi:10.1073/pnas.2322083121 2024