Preliminary suggestions for rigorous GPAI model evaluations

Kevin Wei; Michael J. Byun; Patricia Paskov; Toby Webster

REVIEW 2 major objections 5 minor 6 references

Preliminary suggestions for rigorous GPAI model evaluations

T0 review · 2 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper argues that GPAI model evaluations become more internally valid, externally valid, and reproducible when evaluators follow a structured set of practices organized across design, implementation, execution, and documentation, and…

desk verdict A transparent, well-scoped checklist for GPAI evaluation rigour, with suitably hedged claims; the unenumerated literature review and untested cross-disciplinary transfer keep it from being more than a useful starting point. read the letter →

arxiv 2508.00875 v1 pith:AKYL3ORJ submitted 2025-07-22 cs.CY cs.AI

Patricia Paskov , Michael J. Byun , Kevin Wei , Toby Webster This is my paper

classification cs.CYcs.AI

keywords GPAIevaluationinternalvalidityexternalreproducibilityEUAIActhumanupliftstudiesbenchmarkevaluationslifecycle

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

General-purpose AI (GPAI) evaluations are under fire for weak statistics, unmeasured real-world risk, poor documentation, and badly defined constructs. This paper answers with a preliminary set of best practices, drawn from machine learning, statistics, psychology, economics, and biology, and organized by the four stages of an evaluation's life cycle: design, implementation, execution, and documentation. The aim is to promote internal validity (results reflect the true cause-effect relationship), external validity (results generalize beyond the test setting), and reproducibility (others can obtain consistent results). The paper is addressed to GPAI providers facing EU AI Act systemic-risk evaluation duties, third-party evaluators, and policymakers, and it argues that following these practices gives decision makers more accurate information about GPAI risks. If the paper is right, the field gets a common checklist for turning evaluations into credible evidence for compliance and preparedness.

What carries the argument

The carrying mechanism is the four-stage evaluation life cycle—design, implementation, execution, and documentation—with each suggestion tagged as promoting internal validity (I), external validity (E), or reproducibility (R). The life cycle turns a scattered list of tips into a structured protocol: design defines the construct and research question, implementation builds or selects tools, execution runs the study and analyzes results, and documentation records enough for others to audit and reproduce. The two evaluation types sit on top of this skeleton: human uplift studies measure the causal effect of GPAI access on human task performance, and benchmark evaluations grade model responses on standardized item sets. The tag system is what lets an evaluator see which rigor goal each practice serves.

What would settle it

A controlled comparison in which two independent teams evaluate the same GPAI model, one following the paper's full protocol and the other using current standard practice, and the protocol-based evaluation shows no improvement in test-retest reliability, inter-evaluator agreement, or correlation with measured real-world performance, would falsify the claim that these suggestions promote internal validity, external validity, and reproducibility.

Watch

Extended reading notes

Core claim

The central claim is that a disciplined evaluation process, not any single metric or benchmark, is what makes GPAI evaluation results trustworthy. Concretely, the paper proposes that every evaluation specify its research question and construct; validate items and grading with domain experts; run power analyses and pre-register the analysis plan; estimate statistical uncertainty; blind elicitation and grading; check for shortcut-solving and training-data contamination; and document data sources, model parameters, compute environment, and instructions. For human uplift studies it adds stratified randomization, constant user interfaces across treatment and control groups, contamination and non-compliance monitoring, and blinded grading. For benchmarks it adds private validation sets, contamination measurement, human baselines, canary strings, and reference scores. The payoff is framed in regulatory terms: providers of general-purpose AI models presenting systemic risk (GPAISR) under the EU AI Act must show high scientific and technical rigour, and these practices are offered as concrete ways to demonstrate it.

Load-bearing premise

The load-bearing premise is that the research standards of clinical trials, psychometrics, economics, and biology transfer intact to GPAI models, even though those models are updated frequently, can have benchmark items in their training data, and behave differently across deployments.

Editorial extensions

If this is right

Evaluators who follow the design-stage suggestions would define research questions, constructs, and evaluation environments before running anything, so results would be interpretable rather than ad hoc.
Human uplift studies using stratified randomization, uniform user interfaces, blinding, and power analyses would produce causal estimates of GPAI uplift less distorted by confounding and non-compliance.
Benchmark evaluations using private validation sets, contamination checks, human baselines, and documented reference scores would give policymakers more reliable evidence near capability thresholds that trigger safety decisions.
Standardized documentation of model versions, prompts, compute, and code would make evaluations reproducible by third parties, satisfying the Code of Practice's call for demonstrable rigour.
A common life-cycle vocabulary would let the field compare evaluations and accumulate methodological lessons rather than treating each benchmark as a one-off.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An unstated but testable corollary is that the suggestions can be prioritized by cost-benefit: practices like pre-registration and blinding are cheap, while private test sets and human baselines are expensive, so different evaluators may rationally adopt different subsets and still call themselves rigorous.
The paper's transfer premise could be checked directly by paired evaluations: run the same GPAI evaluation twice, once with the full protocol and once without, and compare stability, inter-evaluator agreement, and agreement with deployment outcomes.
If these suggestions become the de facto interpretation of 'high scientific and technical rigour' under the EU AI Act, they would effectively set the evidentiary bar for systemic-risk determinations, giving the compilation regulatory weight beyond its preliminary status.
The four-stage frame may need extension for agentic and long-horizon evaluations, where the execution stage involves dynamic environments and tool use; the paper notes such benchmarks exist, but its execution suggestions are largely written for static, short-horizon studies.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This is a genuinely useful synthesis, not a research result. The authors take established methodological standards from statistics, clinical trials, and economics, and organize them into a four-stage checklist (design, implementation, execution, documentation) for GPAI evaluations, with a focus on human uplift studies and benchmarks. The execution stage is a real addition over Reuel et al.'s benchmark-centric lifecycle, and the human-uplift material is the most distinctive contribution. The tagging process (three authors independently categorizing, external expert resolving disagreements) is a reasonable way to add discipline to a checklist paper. Credit where due: the claims are consistently hedged as suggestions that 'may promote' rigour, and the limitations in Appendix A are candid about the convenience-sampled interdisciplinary review and the lack of cost-benefit prioritization.

The soft spots are real but not fatal. The 64-paper GPAI review is not enumerated, so the selection is not auditable; a reader cannot tell whether important work was missed or whether the sample is biased. The bigger conceptual gap is the one the stress-test note flags: the transfer of clinical-trial and psychometric standards to GPAI settings is assumed rather than demonstrated. Power analyses assume a stable outcome distribution, and GPAI models are non-stationary; private test sets can improve internal validity but sit uneasily with public code release. The paper acknowledges some of this tension (e.g., tiered-access documentation, multiple-scenario power analyses), so the central claim does not collapse, but the reader is left with a checklist whose effectiveness is unmeasured. The authors themselves say future work should validate and refine the suggestions—that is the right framing.

Who is this for? The EU AI Act audience—GPAISR providers, third-party evaluators, policymakers—will find it a practical conversation starter. Academics already familiar with Reuel et al. and Weidinger et al. will find less new material. It deserves a serious referee: the topic is timely, the writing is clear, and the suggestions are sensible. I would recommend acceptance with revisions that (a) release the list of 64 reviewed papers, and (b) add an explicit discussion of which suggestions are least portable to GPAI settings and why. It is not a paper that changes the science, but it is an honest and competent piece of policy-facing work.

Referee Report

2 major / 5 minor

Summary. This paper, authored by researchers at RAND, presents a preliminary compilation of suggestions for improving the methodological rigour of general-purpose AI (GPAI) evaluations. The suggestions are organized around two evaluation types—human uplift studies and benchmark evaluations—and are mapped to four stages of an evaluation life cycle: design, implementation, execution, and documentation. Each suggestion is tagged as potentially promoting internal validity, external validity, or reproducibility, following definitions adapted from the EU AI Act's Code of Practice. The compilation draws on a literature review of 64 GPAI evaluation methodology papers and on convenience-sampled literature from machine learning, statistics, psychology, economics, and biology. The paper is explicitly preliminary and hedged, stating that the suggestions 'may' promote rigour, and it positions itself as a contribution to an ongoing conversation about the science of GPAI evaluations, motivated by the EU AI Act's requirements for providers of general-purpose AI models presenting systemic risk.

Significance. If taken up by evaluators, this compilation could help raise the statistical and methodological quality of GPAI evaluations, which currently display well-documented shortcomings. The paper's central contribution is a structured, policy-relevant synthesis that connects established measurement standards from adjacent fields to the specific context of GPAI evaluation. Its strengths include a transparent tagging process (three independent coders with external adjudication), a clear life-cycle framework that extends prior work, and a practical orientation toward EU AI Act compliance. The suggestions are specific and actionable, and the paper is appropriately honest about its provisional nature. However, the document does not provide empirical evidence that following the suggestions improves validity or reproducibility, and its evidence base is not fully auditable because the 64 reviewed papers are not enumerated and the interdisciplinary component is explicitly convenience-sampled. These limitations are partially acknowledged by the authors, making the paper a useful starting point rather than a definitive standard.

major comments (2)

[Section III.a, 'Conducting power analyses'; Section III.c, 'Documenting model parameters'] The compilation does not address non-stationarity of GPAI models as a validity threat, which is a load-bearing gap for the paper's central claim of promoting internal validity. In a human uplift study running over weeks, the GPAI model under test may be updated by the provider mid-evaluation, changing the intervention being studied; power analyses computed from a pilot on one model version may not transfer to a different version, and the documentation suggestion to record model version(s) is purely ex post. I recommend adding a design-stage suggestion to fix the model version(s) for the duration of an evaluation, or to explicitly model version-induced variability, so that internal validity claims are grounded in a stable subject of evaluation.
[Appendix A] The literature review underlying the compilation is not auditable. The paper states that it draws on a 'literature review of 64 articles,' but Appendix A does not enumerate these papers; the Google Scholar search and snowball procedure are described, yet the final list is absent. In addition, the interdisciplinary review uses convenience sampling, which the authors acknowledge may overrepresent their home fields. Because the suggestions are derived from this review, the lack of a complete reference list prevents readers from verifying coverage or judging whether important standards from other fields were omitted. Please provide the full list of 64 papers, for example as a supplementary table, and a brief justification for the selected interdisciplinary sources.

minor comments (5)

[Section III.a, 'Specifying the research question'] The suggestion to define the context to which evaluation results aim to generalise is valuable but could be operationalized with an example or template, which would help evaluators apply it consistently.
[Table 1 and Note 20] The definition of reproducibility follows the NASEM distinction between reproducibility and replicability, but the paper occasionally uses the terms loosely (e.g., Note 20 mixes 'reproduce' and 'improve on' results); aligning terminology throughout would improve precision.
[Section III.c, 'Using a validation or test set that is not released publicly'] This suggestion is in tension with the reproducibility goal of securely releasing evaluation code; the paper could add a cross-reference to the documentation stage noting that private test sets can be made available through access-controlled mechanisms (e.g., API-based evaluation) to allow independent verification without public disclosure.
[Appendix B] The tagging criteria are broad, and while the three-author plus external-expert process is described, reporting inter-rater agreement (e.g., Cohen's kappa) would strengthen confidence in the tags.
[Section III.a, 'Estimating the statistical uncertainty'] The phrase 'where relevant' is vague; specifying conditions under which uncertainty estimation is or is not appropriate (e.g., small samples, non-probability samples) would make the suggestion more actionable.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is a literature-based compilation whose suggestions are independently sourced; the few self-citations are supporting examples, not load-bearing inputs, so the central claim does not reduce to its own premises.

full rationale

This paper does not derive a numerical prediction from fitted parameters or from a uniqueness theorem. Its central claim is that the listed suggestions "may promote" internal validity, external validity and reproducibility; the support is a 64-paper literature review plus established standards in clinical trials, psychometrics, economics and biology, with the methodology disclosed in Appendix A. The definitions of the three rigour elements trace to the EU AI Act Code of Practice and standard sources (Patino & Ferreira 2018; National Academies 2019), not to the authors' own prior results. Self-citations appear: Paskov et al. (2024) is cited for a definition of external validity and for securely releasing execution details; Wei et al. (2025) is cited for population specification and human-baseline checklists; RAND uplift studies are cited as examples with acknowledged methodological limitations. In each case the cited item supports an already-established practice rather than serving as the sole justification for the paper's premise, and no suggestion is defined in terms of the outcome it is claimed to promote. Appendix A's admission of convenience sampling and possible overrepresentation of the authors' fields is a limitation on the compilation's auditability, not a circularity; it does not make the checklist's content equivalent to its inputs. Accordingly there are no self-definitional, fitted-input, or imported-uniqueness steps to report.

Assumptions & free parameters 0 free parameters · 3 assumptions · 1 invented entities

The document's claims rest on domain assumptions about what counts as rigorous evaluation and on the transferability of methods from other fields, not on numeric fit parameters. The only new conceptual object is the execution life-cycle stage; no particles, forces, mediators, or other empirical entities are introduced.

assumptions (3)

domain assumption Internal validity, external validity and reproducibility are the appropriate quality standards for GPAI evaluations.
Adopted from the EU AI Act Code of Practice and measurement-science literature in Section II.a and Table 1; asserted rather than derived.
domain assumption Established practices from statistics, psychology, economics and biology transfer to GPAI evaluations.
Used throughout Section III; stated in the Introduction ("Adjacent fields offer established standards from which to draw") and Appendix A, with transferability not tested.
domain assumption The EU AI Act's requirement for high scientific and technical rigour can be operationalized as internal validity, external validity and reproducibility.
Section II.a interprets European Commission (2025) Measure II.4.5 and the Code of Practice glossary in this way; this is a policy interpretation that the rest of the document depends on.

invented entities (1)

Execution stage in the GPAI evaluation life cycle
purpose: Adds a fourth stage to Reuel et al. (2024a)'s benchmark life cycle to cover running evaluations and collecting results, especially for human uplift studies.
Proposed in Section II.c as a structural taxonomy element; it is a framing device, not an empirical entity, and carries no falsifiable prediction.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Preliminary suggestions for rigorous GPAI model evaluations." pith.science (2026). https://pith.science/paper/AKYL3ORJ

@misc{pith2026250800875,
  author       = {Pith},
  title        = {Pith review of: Preliminary suggestions for rigorous GPAI model evaluations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AKYL3ORJ}},
  note         = {Machine review of arXiv:2508.00875}
}

read the original abstract

This document presents a preliminary compilation of general-purpose AI (GPAI) evaluation practices that may promote internal validity, external validity and reproducibility. It includes suggestions for human uplift studies and benchmark evaluations, as well as cross-cutting suggestions that may apply to many different evaluation types. Suggestions are organised across four stages in the evaluation life cycle: design, implementation, execution and documentation. Drawing from established practices in machine learning, statistics, psychology, economics, biology and other fields recognised to have important lessons for AI evaluation, these suggestions seek to contribute to the conversation on the nascent and evolving field of the science of GPAI evaluations. The intended audience of this document includes providers of GPAI models presenting systemic risk (GPAISR), for whom the EU AI Act lays out specific evaluation requirements; third-party evaluators; policymakers assessing the rigour of evaluations; and academic researchers developing or conducting GPAI evaluations.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

6 extracted references · 2 canonical work pages

[6]

‘Building an Early Warning System for LLM-Aided Biological Threat Creation.’ OpenAI. As of 10 April 2025: https://openai.com/index/building-an-early-warning-sys - tem-for-llm-aided-biological-threat-creation/ Peppin, Aidan, Anka Reuel, Stephen Casper, Elliot Jones, Andrew Strait, Usman Anwar, Anurag Agrawal, Sayash Kapoor, Sanmi Koyejo, Marie Pellat, Rish...

arXiv 2025
[2016]

‘What Does Research Reproducibility Mean?’ Science Translational Medicine 8(341):312–41. Gu, Jiawei, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Saizhuo Wang, Kun Zhang, Yuanzhuo Wang, Wen Gao, Lionel Ni & Jian Guo. 2024. ‘ A Survey on LLM-as- a-Judge.’ arXiv, arXiv:2411.15594 Gururangan...

arXiv 2024
[2019]

doi: 10.17226/25303 National Institute of Standards and Technology

‘Reproducibility and Replicability in Science.’ National Academies Press. doi: 10.17226/25303 National Institute of Standards and Technology. 2024. ‘ Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile: NIST Trustworthy and Responsible AI.’ U.S. Department of Commerce. doi: 10.6028/NIST.AI.600-1 OpenAI. 2024. ‘Ope...

arXiv 2024
[2021]

What Makes an Evaluation Useful? Common Pitfalls and Best Practices

‘External Validity.’ Annual Review of Political Science 24:365–93. Frontier Model Forum. 2024. ‘Issue Brief: Preliminary Taxonomy of Pre-Deployment Frontier AI Safety Evaluations.’ 20 December. As of 12 March 2025: https://www.frontiermodelforum.org/updates/issue-brief-pre - liminary-taxonomy-of-pre-deployment-frontier-ai-safety-eval - uations/ Frontier M...

work page Pith review arXiv 2024
[2024]

As of 10 October 2024: https://data.consilium.europa.eu/doc/document/ PE-24-2024-INIT/en/pdf Findley, Michael G., Kyosuke Kikuta & Michael Denly

Annex XI.1 in Regulation of the European Parliament and of the Council Laying Down Harmonised Rules on Artificial Intelligence and Amending Regulations (EC) No 300/2008, (EU) No 167/2013, (EU) No 168/2013, (EU) 2018/858, (EU) 2018/1139 and (EU) 2019/2144 and Directives 2014/90/EU, (EU) 2016/797 and (EU) 2020/1828 (Artificial Intelligence Act), 2021/0106(C...

work page 2008
[2025]

ICLR, 26 February

‘MLE-BENCH: Evaluating Machine Learning Agents on Machine Learning Engineering.’ Conference paper. ICLR, 26 February. arXiv, arXiv:2410.07095 Chen, Mark et al. 2021. ‘Evaluating Large Language Models Trained on Code.’ arXiv, arXiv:2107.03374 Chouldechova, Alexandra, Chad Atalla, Solon Barocas, A. Feder Cooper, Emily Corvi, P. Alex Dow, Jean Garcia- Gathri...

arXiv 2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [6]

‘Building an Early Warning System for LLM-Aided Biological Threat Creation.’ OpenAI. As of 10 April 2025: https://openai.com/index/building-an-early-warning-sys - tem-for-llm-aided-biological-threat-creation/ Peppin, Aidan, Anka Reuel, Stephen Casper, Elliot Jones, Andrew Strait, Usman Anwar, Anurag Agrawal, Sayash Kapoor, Sanmi Koyejo, Marie Pellat, Rish...

arXiv 2025

[2] [2016]

‘What Does Research Reproducibility Mean?’ Science Translational Medicine 8(341):312–41. Gu, Jiawei, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Saizhuo Wang, Kun Zhang, Yuanzhuo Wang, Wen Gao, Lionel Ni & Jian Guo. 2024. ‘ A Survey on LLM-as- a-Judge.’ arXiv, arXiv:2411.15594 Gururangan...

arXiv 2024

[3] [2019]

doi: 10.17226/25303 National Institute of Standards and Technology

‘Reproducibility and Replicability in Science.’ National Academies Press. doi: 10.17226/25303 National Institute of Standards and Technology. 2024. ‘ Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile: NIST Trustworthy and Responsible AI.’ U.S. Department of Commerce. doi: 10.6028/NIST.AI.600-1 OpenAI. 2024. ‘Ope...

arXiv 2024

[4] [2021]

What Makes an Evaluation Useful? Common Pitfalls and Best Practices

‘External Validity.’ Annual Review of Political Science 24:365–93. Frontier Model Forum. 2024. ‘Issue Brief: Preliminary Taxonomy of Pre-Deployment Frontier AI Safety Evaluations.’ 20 December. As of 12 March 2025: https://www.frontiermodelforum.org/updates/issue-brief-pre - liminary-taxonomy-of-pre-deployment-frontier-ai-safety-eval - uations/ Frontier M...

work page Pith review arXiv 2024

[5] [2024]

As of 10 October 2024: https://data.consilium.europa.eu/doc/document/ PE-24-2024-INIT/en/pdf Findley, Michael G., Kyosuke Kikuta & Michael Denly

Annex XI.1 in Regulation of the European Parliament and of the Council Laying Down Harmonised Rules on Artificial Intelligence and Amending Regulations (EC) No 300/2008, (EU) No 167/2013, (EU) No 168/2013, (EU) 2018/858, (EU) 2018/1139 and (EU) 2019/2144 and Directives 2014/90/EU, (EU) 2016/797 and (EU) 2020/1828 (Artificial Intelligence Act), 2021/0106(C...

work page 2008

[6] [2025]

ICLR, 26 February

‘MLE-BENCH: Evaluating Machine Learning Agents on Machine Learning Engineering.’ Conference paper. ICLR, 26 February. arXiv, arXiv:2410.07095 Chen, Mark et al. 2021. ‘Evaluating Large Language Models Trained on Code.’ arXiv, arXiv:2107.03374 Chouldechova, Alexandra, Chad Atalla, Solon Barocas, A. Feder Cooper, Emily Corvi, P. Alex Dow, Jean Garcia- Gathri...

arXiv 2021