Pith. sign in

REVIEW 4 major objections 5 minor 48 references

Using predictive multiplicity to measure individual performance within the AI Act

T0 review · 4 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that predictive multiplicity—many equally accurate models that disagree on individuals—is exactly the tool the EU AI Act needs for its individual-accuracy reporting duty.

desk verdict A useful bridge between predictive multiplicity and the AI Act, but the legal centerpiece rests on an incomplete quote and the empirical support is thinner than the recommendations. read the letter →

arxiv 2602.11944 v2 pith:MT2XERPT submitted 2026-02-12 cs.LG cs.CY

Karolin Frohnapfel , Mara Seyfert , Sebastian Bordt , Ulrike von Luxburg , Kristof Meding This is my paper
classification cs.LGcs.CY
keywords predictivemultiplicityEUAIActconflictratioδ-ambiguityRashomonsetindividualperformancehigh-risksystemsmodeldocumentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the EU AI Act requires providers of high-risk AI systems to report accuracy not only on entire datasets but also for specific individuals, and that predictive multiplicity provides a workable way to do so. It introduces two concrete measures—an individual conflict ratio and a dataset-level δ-ambiguity—that quantify how much equally accurate models disagree on a given case. The paper further shows that a small set of models trained with bootstrapped data and varied hyperparameters approximates the full set of comparably accurate models well enough for these measures to be useful. If accepted, providers would deliberately search for equally accurate alternatives and disclose their disagreements to deployers, so that a person whose case is contested by many good models is not silently decided by one arbitrary pick.

What carries the argument

The Rashomon set is the set of all models whose statistical accuracy lies within a small threshold ε of the best achievable accuracy. The conflict ratio for a data point is the fraction of models in that set that predict the opposite label from the majority, ranging from 0 (all agree) to 0.5 (perfect split). The δ-ambiguity is the fraction of data points whose conflict ratio exceeds a chosen threshold δ. Because exhaustive enumeration of the Rashomon set is infeasible for general model classes, the paper uses an ad-hoc approximation: train many models with bootstrapped data and varied hyperparameters, then keep those within the ε threshold. Experiments show that including dataset multiplicit

What would settle it

Compute conflict ratios and δ-ambiguity from an exhaustively enumerated Rashomon set of sparse decision trees on a moderate-size binary dataset, then compute the same quantities from the ad-hoc approach; if the δ-ambiguity for, say, δ=0.1 differs by more than a few percentage points, the claim that the ad-hoc set faithfully approximates the true set would be falsified.

Watch

Extended reading notes

Core claim

The central claim is that the AI Act's accuracy provisions (Article 15, read with Articles 11 and 13 and Annex IV) require providers of high-risk AI systems to assess and report performance for specific individuals, not just at dataset level, and that predictive multiplicity—measured by conflict ratios across an approximate set of comparably accurate models—is a technically suitable way to do so. When many models of similar statistical accuracy give conflicting predictions for a particular case, the paper argues that no meaningful decision can be made for that individual, and this unreliability should be disclosed to deployers so they can route such cases to human oversight.

Load-bearing premise

The load-bearing premise is that a small, ad-hoc set of models trained with bootstrapped data and varied hyperparameters faithfully represents the full set of comparably accurate models; the paper only validates this on one binarized recidivism dataset, where the ad-hoc approach shows a tendency to underestimate δ-ambiguity for δ>0.

Editorial extensions

If this is right

  • Providers of high-risk AI systems could satisfy the AI Act's individual-performance documentation duty by including conflict ratios and δ-ambiguity in the technical documentation required by Article 11 and Annex IV.
  • Deployers could use conflict ratios to flag individual cases where the system's output is not reliable enough for automated decision-making, and route those cases to human oversight as required by Article 14.
  • A relatively small Rashomon set—hundreds of models rather than millions—suffices to approximate δ-ambiguity, provided it incorporates both dataset and parametric multiplicity.
  • Because the ad-hoc approach works for any model class, these measures are usable in practice without requiring specialized exhaustive-enumeration algorithms.
  • Standard ambiguity is monotone in the size of the Rashomon set, but δ-ambiguity for δ>0 is not, so reporting should state the δ threshold and the construction of the Rashomon set.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If regulators adopt this framework, the compliance burden shifts toward evidence that a provider actually searched for equally accurate alternatives, strengthening arguments that providers should seek less discriminatory or less arbitrary models—a move parallel to existing less-discriminatory-algorithm duties in other jurisdictions.
  • The paper's validation is limited to a single binarized recidivism dataset, where the ad-hoc approach tends to underestimate δ-ambiguity for δ>0; a natural extension is a 'calibration step' in which providers compare ad-hoc estimates against exhaustive enumeration on a small proxy dataset before deployment.
  • Because conflict ratios require only stored models, deployers could compute them on new, unlabeled cases during deployment without involving the provider, turning human oversight from a static design feature into a live, per-case mechanism.
  • The AI Act's phrase 'performance regarding specific persons' could plausibly be read as demanding exactly the kind of per-instance disagreement measure proposed here; if harmonized standards adopt δ-ambiguity, model documentation formats would need to evolve to include the Rashomon-set construction and threshold.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper connects predictive multiplicity—the existence of many equally accurate classifiers that differ on individual predictions—to the accuracy and transparency provisions of the EU AI Act (Regulation (EU) 2024/1689). It argues that Art. 15 in combination with Art. 11(1), Art. 13(3), and Annex IV(3) requires providers of high-risk systems to evaluate and report performance not only at dataset level but also for specific individuals. To operationalize this, the paper defines a per-data-point conflict ratio and a dataset-level δ-ambiguity, and constructs an ad-hoc Rashomon set using bootstrap resampling and hyperparameter variation. Experiments on synthetic data, ACS Employment, and COMPAS (benchmarked against TreeFarms) are used to argue that including dataset multiplicity and using only a few hundred models is sufficient. The paper concludes with recommendations for providers to search for conflicting models and for deployers to use conflict ratios to identify cases for human oversight.

Significance. If the legal interpretation holds, the paper gives a concrete, actionable way to give the AI Act's 'specific persons' language operational content. The formal definitions are simple and clear; the synthetic experiment with known ground truth and the comparison to TreeFarms on COMPAS are genuinely useful validation steps, and the ad-hoc approach is model-agnostic. The main uncertainty is legal: the cited provisions actually read 'specific persons or groups of persons', so the inference from the statutory text to a duty to assess every affected individual is not established. With a corrected legal framing and stronger uncertainty quantification in the experiments, the paper would be a useful contribution at the machine-learning/law interface.

major comments (4)
  1. [Section 3.2 / Art. 11(1) & Art. 13(3); Fig. 1] The statutory quotations in the abstract, Section 3.2, and Figure 1 omit the phrase 'or groups of persons' from Art. 11(1)/Annex IV(3) and Art. 13(3)(b)(v). The official text requires information on 'degrees of accuracy for specific persons or groups of persons on which the system is intended to be used,' not necessarily on each affected individual. The paper itself concedes (Section 3.2) that the Act gives no criteria for selecting specific individuals and that provider-side assessments may not transfer to deployment. This is load-bearing: the claim that the AI Act mandates individual-level accuracy reporting for all affected individuals is not established by the cited provisions. Please either quote the full statutory text and argue why the stronger reading follows, or reframe the contribution as a normative/policy proposal rather than a compliance requirement.
  2. [Section 4.4 / Fig. 4] The validation against TreeFarms is a valuable check, but Fig. 4 shows that the ad-hoc approximation tends to underestimate δ-ambiguity for δ > 0, with no error bars or repeated seeds. Since the paper's central recommendation is that deployers use conflict ratios to detect unreliable individual predictions, a systematic underestimation at higher conflict thresholds would provide false assurance and undermine the proposed compliance tool. Please quantify the deviations, report repeated runs, and discuss the direction of bias. The claim that the deviations are 'rather small' is not supported by the figure alone.
  3. [Sections 4.3 and 4.5 / Figs. 2-3] The practical conclusions—that bootstrapped dataset multiplicity dominates and that a few hundred models suffice—rest on single-run point estimates on one synthetic dataset and one state-year of ACS Employment. Figs. 2 and 3 contain no error bars or multiple seeds, and the threshold ε = 0.1 is fixed. These claims are load-bearing for the 'easy-to-implement rules' in Section 5. Please add repeated seeds, report variance, and test at least one additional dataset or model class before asserting generalizability.
  4. [Section 4.1 / Def. 4.2] Definition 4.2 treats the Rashomon set as a uniform collection of models, but the ad-hoc Rashomon set is generated by a convenience selection of bootstrap samples and hyperparameter values. The conflict ratio and δ-ambiguity therefore depend on the number of models, the chosen grid, and the random seed; no protocol or uncertainty quantification is specified. For a compliance-oriented metric, providers need a clearly defined construction procedure; otherwise the reported numbers are not comparable across providers and can be influenced by arbitrary protocol choices. Section 4.2 should specify a reproducible protocol for forming the ad-hoc Rashomon set.
minor comments (5)
  1. [Section 4.4 / Appendix B] The comparison uses the penalized objective Obj(g) = acc(g) - λH(g), so the 'Rashomon set' in Fig. 4 is not the same as in Definition 4.1. Make this distinction explicit in the main text.
  2. [Section 4.3] The synthetic data description lists N((5,5),(1,1)) twice in the same sentence; clarify that this is an intentional overlap of the two label distributions.
  3. [Throughout] 'ad-hoc' is sometimes capitalized as 'AdHoc' (e.g., Section 4.4); please use one consistent spelling.
  4. [Reproducibility] The paper does not state whether code and training seeds are available. For reproducibility of the empirical claims, please provide code, seeds, and package/version information.
  5. [Section 5] The language moves between 'can serve compliance' and 'mandates providers'; please align the conclusions with the revised legal framing suggested in the major comments.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the legal argument and technical metrics are not self-referential, and the ad-hoc Rashomon set is checked against external benchmarks.

full rationale

The paper's derivation chain has three independent parts. (1) Legal: Articles 15, 11/Annex IV, and 13 AIA are quoted and interpreted to require individual-level performance reporting. This is an interpretive legal argument; its main weakness is that the quotations' ellipses omit the qualifier 'or groups of persons', which is a correctness/interpretation concern, not circularity, because the paper argues for the reading 'all affected individuals' rather than defining the statutory phrase in those terms by construction. (2) Formalization: the conflict ratio and δ-ambiguity are explicitly defined as functions of a Rashomon set, and no fitted parameter is later reported as a prediction. The Rashomon-set threshold ε is a modeling parameter with stated values and is not tuned to match the reported conflict ratios. (3) Empirical support: the ad-hoc Rashomon-set approximation is validated against a synthetic ground-truth setup and against TreeFarms' exhaustive enumeration on COMPAS, an external benchmark. The paper reports a tendency to underestimate δ-ambiguity for δ>0, which is inconsistent with post-hoc fitting. There are no load-bearing self-citations: the cited prior work on Rashomon sets, arbitrariness, and ad-hoc approximations comes from independent authors and is used as background or as a benchmark, not to force the paper's conclusion. Every load-bearing computational claim is either definitional in the transparent sense or externally checked. Thus no circular step is exhibited.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical entities. Its main conceptual imports are the Rashomon set (existing) and the two metrics (variants of existing measures). The key free parameters are the thresholds ε and δ, plus experimental design choices; these are not justified by a principled method.

free parameters (3)
  • ε (Rashomon set threshold) = 0.1 (synthetic/ACS), 0.05 (COMPAS), 0.02 (XGBoost/MLP)
    The accuracy gap defining the Rashomon set is chosen by the authors per experiment; no principled rule is given, and all conflict-ratio estimates depend on it.
  • δ (ambiguity threshold) = 0.4 (used in ACS experiments)
    The threshold defining 'high conflict' is set ad hoc; the paper does not provide guidance on selecting δ for real deployments.
  • Bootstrap sample sizes and feature-noise levels = 800, 2800, 4200, 0.5*|D_train|; σ=0.05, p=0.001
    These are chosen heuristically; feature noise levels are explicitly 'chosen in a way to maximize statistical accuracy' (Section 4.3), i.e., fitted to the data.
assumptions (3)
  • domain assumption The ad-hoc Rashomon set approximates the true set of comparably good models.
    Central to computing conflict ratios; validated only against TreeFarms on COMPAS for sparse decision trees, and against synthetic ground truth for two Gaussians.
  • domain assumption Articles 11(1), 13(3), and Annex IV(3) of the AI Act require performance evaluation for all affected individuals.
    The paper's legal interpretation of 'degrees of accuracy for specific persons' is the load-bearing legal premise; the Act itself does not define 'specific persons', and the authors acknowledge this gap.
  • standard math Standard ambiguity is monotone in Rashomon set size.
    Used in Section 4.4 to explain why ad-hoc sets have lower δ=0 ambiguity than the full TreeFarms set.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Using predictive multiplicity to measure individual performance within the AI Act." pith.science (2026). https://pith.science/paper/MT2XERPT

@misc{pith2026260211944,
  author       = {Pith},
  title        = {Pith review of: Using predictive multiplicity to measure individual performance within the AI Act},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MT2XERPT}},
  note         = {Machine review of arXiv:2602.11944}
}
abstract

When building AI systems for decision support, one often encounters the phenomenon of predictive multiplicity: a single best model does not exist; instead, one can construct many models with similar overall accuracy that differ in their predictions for individual cases. Especially when decisions have a direct impact on humans, this can be highly unsatisfactory. For a person subject to high disagreement between models, one could as well have chosen a different model of similar overall accuracy that would have decided the person's case differently. We argue that this arbitrariness conflicts with the EU AI Act, which requires providers of high-risk AI systems to report performance not only at the dataset level but also for specific persons. The goal of this paper is to put predictive multiplicity in context with the EU AI Act's provisions on accuracy and to subsequently derive concrete suggestions on how to evaluate and report predictive multiplicity in practice. Specifically: (1) We introduce the AI Act's accuracy provisions and argue that incorporating information about predictive multiplicity could serve compliance with specific provisions for providers. (2) Based on this legally rigorous analysis, we suggest individual conflict ratios and $\delta$-ambiguity as tools to quantify the disagreement between models on individual cases and to help detect individuals subject to conflicting predictions. (3) Based on computational insights, we derive easy-to-implement rules on how model providers could evaluate predictive multiplicity in practice. (4) Ultimately, we suggest that information about predictive multiplicity should be made available to deployers under the AI Act, enabling them to judge whether system outputs for specific individuals are reliable enough for their use case.

Figures

Figures reproduced from arXiv: 2602.11944 by the authors.

Figure 1
Figure 1. Our work in a nutshell. Predictive multiplicity means that classifiers with the same statistical accuracy may decide a large number of individual cases differently (left side). We argue that this challenges the AI Act requirements on accuracy and transparency (right side). To resolve this issue, we recommend that providers and deployers use methods to identify conflicting cases and involve human oversight in the dec… view at source ↗
Figure 2
Figure 2. The importance of dataset multiplicity motivated on two-dimensional synthetic data. For fixed parametric multiplicity, we compare different ways to include dataset multiplicity into the ad-hoc approach from (a) no dataset multiplicity to (d) full dataset multiplicity. The ability to discover conflicting data points in the overlap region N ((5, 5),(1, 1)) increases with stronger integration of dataset multiplicity (s… view at source ↗
Figure 3
Figure 3. The importance of dataset multiplicity on ACSEmployment data. For fixed parametric multiplicity, we compare different ways to include dataset multiplicity into the ad-hoc approach from (a) no dataset multiplicity to (f) full dataset multiplicity. The integration of dataset multiplicity increases the ability to discover conflicting data points (left plot) and highly conflicting data points (middle plot). The effectiv… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Comparison of the ad-hoc approach and TreeFarms [47] on COMPAS data. With the ad-hoc approach, we train Rashomon sets of different sizes and track the δ￾ambiguities. Results are compared to the ground truth Rashomon set found by TreeFarms. The dataset has 12 features, …
Figure 5
Figure 5. Figure 5: The best accuracy is displayed depending on the size of the subset of the data. [PITH_FULL_IMAGE:figures/full_fig_p020_5.png]
Figure 6
Figure 6. Figure 6: The importance of dataset multiplicity on ACSEmployment data for XGBoost. For fixed parametric multiplicity, we compare different ways to include dataset multiplicity into the ad-hoc approach from (a) no dataset multiplicity to (f) full dataset multiplicity. Distances …
Figure 7
Figure 7. Figure 7: The importance of dataset multiplicity on ACSEmployment data for MLPs. For fixed parametric multiplicity, we compare different ways to include dataset multiplicity into the ad-hoc approach from (a) no dataset multiplicity to (f) full dataset multiplicity. Distances bet…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 3 linked inside Pith

  1. [1]

    Allen, R

    E. Allen, R. Taylor, A. Gimson, and D. Thorburn. Transplant benefit-based offering of deceased donor livers in the United Kingdom.Journal of Hepatology, 81(3):471–478, 2024

    2024

  2. [2]

    Almada and N

    M. Almada and N. Petit. The EU AI Act: Between the rock of product safety and the hard place of fundamental rights.Common Market Law Review, 62(1), 2025

    2025

  3. [3]

    Angwin, J

    J. Angwin, J. Larson, S. Mattu, and L. Kirchner. Machine bias: There’s software used across the country to predict future criminals. And it’s biased against blacks.https://www.propublica.org/article/ machine-bias-risk-assessments-in-criminal-sentencing, 2016

    2016

  4. [4]

    Black, M

    E. Black, M. Raghavan, and S. Barocas. Model multiplicity: Opportunities, concerns, and solutions. InProceedings of the ACM Conference on Fairness, Accountability, and Transparency (F AccT), 2022

    2022

  5. [5]

    Black, L

    E. Black, L. Koepke, P. Kim, S. Barocas, and M. Hsu. The legal duty to search for less discriminatory algorithms.arXiv preprint arXiv:2406.06817, 2024

    arXiv 2024

  6. [6]

    Boner, H

    Z. Boner, H. Chen, L. Semenova, R. Parr, and C. Rudin. Using noise to infer aspects of simplicity without learning. InAdvances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  7. [7]

    L. Breiman. Statistical modeling: The two cultures (with comments and a rejoinder by the author).Statistical Science, 16(3):199–231, 2001

    2001

  8. [8]

    B. Buchner. KI-VO Art. 15 Genauigkeit, Robustheit und Cybersicherheit. In J. Schefzig and R. Kilian, editors,BeckOK KI-Recht. C.H. Beck, M¨ unchen, 4 edi- tion, 2025

    2025

Show all 48 references
  1. [9]

    Buscemi, T

    A. Buscemi, T. Deckenbrunnen, F. Kabir, N. Mowla, and K. Mishchenko. As- sessing high-risk systems: An EU AI Act verification framework.arXiv preprint arXiv:2512.13907, 2025

    2025 arXiv

  2. [10]

    CEN/CLC/JTC 21 Work Programme.https: //standards.cencenelec.eu/ords/fp=205:22:::::FSP_ORG_ID,FSP_LANG_ID: 2916257,25&cs=114251C6C0B684FBBC069923513BF6348, 2025

    CEN-CENELEC. CEN/CLC/JTC 21 Work Programme.https: //standards.cencenelec.eu/ords/fp=205:22:::::FSP_ORG_ID,FSP_LANG_ID: 2916257,25&cs=114251C6C0B684FBBC069923513BF6348, 2025. 15

    2025

  3. [11]

    A. F. Cooper, J. Frankle, and C. De Sa. Non-determinism and the lawlessness of machine learning code. InProceedings of the 2022 Symposium on Computer Science and Law, pages 1–8, 2022

    2022

  4. [12]

    A. F. Cooper, K. Lee, M. Z. Choksi, S. Barocas, C. De Sa, J. Grimmelmann, J. Klein- berg, S. Sen, and B. Zhang. Arbitrariness and social prediction: The confounding role of variance in fair classification. InProceedings of the AAAI Conference on Artificial Intelligence, 2024

    2024

  5. [13]

    D’Amour, K

    A. D’Amour, K. Heller, D. Moldovan, B. Adlam, B. Alipanahi, A. Beutel, C. Chen, J. Deaton, J. Eisenstein, and M. D. et al Hoffman. Underspecification presents chal- lenges for credibility in modern machine learning.Journal of Machine Learning Re- search (JMLR), 23(226):1–61, 2022

    2022

  6. [14]

    F. Ding, M. Hardt, J. Miller, and L. Schmidt. Retiring adult: New datasets for fair machine learning. InAdvances in Neural Information Processing Systems (NeurIPS), 2021

    2021

  7. [15]

    White paper on artificial intelligence – A European ap- proach to excellence and trust.https://eur-lex.europa.eu/legal-content/EN/ TXT/?uri=celex:52020DC0065, 2020

    European Commission. White paper on artificial intelligence – A European ap- proach to excellence and trust.https://eur-lex.europa.eu/legal-content/EN/ TXT/?uri=celex:52020DC0065, 2020

    2020

  8. [16]

    European Commission. Proposal for a regulation of the European parliament and of the council laying down harmonised rules on artificial intelligence (Artificial Intelli- gence Act) and amending certain union legislative acts.https://eur-lex.europa. eu/legal-content/EN/TXT/?uri...

    2021

  9. [17]

    The ‘Blue Guide’ on the implementation of EU prod- uct rules 2022.https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=OJ:C: 2022:247:TOC, 2022

    European Commission. The ‘Blue Guide’ on the implementation of EU prod- uct rules 2022.https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=OJ:C: 2022:247:TOC, 2022

    2022

  10. [18]

    C(2023)3215 – Standardisation request M/593.https://ec

    European Commission. C(2023)3215 – Standardisation request M/593.https://ec. europa.eu/growth/tools-databases/enorm/mandate/593_en, 2025

    2023

  11. [19]

    C(2025)3871 – Standardisation request M/613.https://ec

    European Commission. C(2025)3871 – Standardisation request M/613.https://ec. europa.eu/growth/tools-databases/enorm/mandate/613_en, 2025

    2025

  12. [20]

    New legislative framework.https: //single-market-economy.ec.europa.eu/single-market/goods/ new-legislative-framework_en, 2025

    European Commission. New legislative framework.https: //single-market-economy.ec.europa.eu/single-market/goods/ new-legislative-framework_en, 2025

    2025

  13. [21]

    Fisher, C

    A. Fisher, C. Rudin, and F. Dominici. All models are wrong, but many are useful: Learning a variable’s importance by studying an entire class of prediction models simultaneously.Journal of Machine Learning Research (JMLR), 20(177):1–81, 2019

    2019

  14. [22]

    Ganesh, A

    P. Ganesh, A. Taik, and G. Farnadi. Systemizing multiplicity: The curious case of arbitrariness in machine learning. InProceedings of the AAAI/ACM Conference on AI, Ethics, and Society (AIES), 2025

    2025

  15. [23]

    George, D

    E. George, D. Needell, and B. Ustun. Observational multiplicity.arXiv preprint arXiv:2507.23136, 2025

    2025 arXiv

  16. [24]

    Gerdemann and M

    S. Gerdemann and M. K. W¨ obbeking. Art. 40. In M. Martini and C. Wendehorst, editors,KI-VO: Verordnung ¨ uber K¨ unstliche Intelligenz: Kommentar. C.H. Beck, M¨ unchen, 2024

    2024

  17. [25]

    Gorywoda

    L. Gorywoda. The new European legislative framework for the marketing of goods. Columbia Journal of European Law, 16(1):161–170, 2009. 16

    2009

  18. [26]

    Gur-Arieh and C

    S. Gur-Arieh and C. Lee. Consistently arbitrary or arbitrarily consistent: Navigating the tensions between homogenization and multiplicity in algorithmic decision-making. InProceedings of the ACM Conference on Fairness, Accountability, and Transparency (F AccT), 2025

    2025

  19. [27]

    Ethics guidelines for trustworthy AI.https://digital-strategy.ec.europa.eu/en/library/ ethics-guidelines-trustworthy-ai, 2019

    High-Level Expert Group on Artificial Intelligence. Ethics guidelines for trustworthy AI.https://digital-strategy.ec.europa.eu/en/library/ ethics-guidelines-trustworthy-ai, 2019

    2019

  20. [28]

    Hsu and F

    H. Hsu and F. Calmon. Rashomon capacity: A metric for predictive multiplicity in classification. InAdvances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  21. [29]

    H. Hsu, G. Li, S. Hu, and C.-F. R. Chen. Dropout-based Rashomon set exploration for efficient predictive multiplicity estimation. InProceedings of the International Conference on Learning Representations (ICLR), 2024

    2024

  22. [30]

    X. Hu, C. Rudin, and M. Seltzer. Optimal sparse decision trees. InAdvances in Neural Information Processing Systems (NeurIPS), 2019

    2019

  23. [31]

    M. E. Kaminski and A. D. Selbst. An American’s guide to the EU AI Act.Available at SSRN 5373345, 2025

    2025

  24. [32]

    Kilian, L

    R. Kilian, L. J¨ ack, and D. Ebel. European AI standards – Technical standardisation and implementation challenges under the EU AI Act.European Journal of Risk Regulation, 16(3):1038–1062, 2025

    2025

  25. [33]

    Kirschke-Biller and A

    J. Kirschke-Biller and A. L. F¨ ullsack. KI-VO Art. 3 Begriffsbestimmungen. In J. Schefzig and R. Kilian, editors,BeckOK KI-Recht. C.H. Beck, M¨ unchen, 4 edi- tion, 2025

    2025

  26. [34]

    Kulynych, H

    B. Kulynych, H. Hsu, C. Troncoso, and F. P. Calmon. Arbitrary decisions are a hidden cost of differentially private training. InACM Conference on Fairness, Accountability, and Transparency (F AccT), 2023

    2023

  27. [35]

    Laufer, M

    B. Laufer, M. Raghavan, and S. Barocas. What constitutes a less discriminatory algorithm? InProceedings of the 2025 Symposium on Computer Science and Law, 2025

    2025

  28. [36]

    M. Martini. Art. 15. In M. Martini and C. Wendehorst, editors,KI-VO: Verordnung ¨ uber K¨ unstliche Intelligenz: Kommentar. C.H. Beck, M¨ unchen, 2024

    2024

  29. [37]

    C. Marx, F. Calmon, and B. Ustun. Predictive multiplicity in classification. In Proceedings of the International Conference on Machine Learning (ICML), 2020

    2020

  30. [38]

    A. P. Meyer, A. Albarghouthi, and L. D’Antoni. The dataset multiplicity problem: How unreliable data impacts predictions. InProceedings of the ACM Conference on Fairness, Accountability, and Transparency (F AccT), 2023

    2023

  31. [39]

    Nolte, M

    H. Nolte, M. Rateike, and M. Finck. Robustness and cybersecurity in the EU Artificial Intelligence Act. InProceedings of the ACM Conference on Fairness, Accountability, and Transparency (F AccT), 2025

    2025

  32. [40]

    Pouget, K

    H. Pouget, K. Holtman, and T. Emborg. Standard setting.https:// artificialintelligenceact.eu/standard-setting-overview/, 2022

    2022

  33. [41]

    Rudin, C

    C. Rudin, C. Zhong, L. Semenova, M. Seltzer, R. Parr, J. Liu, S. Katta, J. Donnelly, H. Chen, and Z. Boner. Position: Amazing things come from having many good models. InProceedings of the International Conference on Machine Learning (ICML), 2024. 17

    2024

  34. [42]

    Semenova, C

    L. Semenova, C. Rudin, and R. Parr. On the existence of simpler machine learning models. InProceedings of the ACM Conference on Fairness, Accountability, and Transparency (F AccT), 2022

    2022

  35. [43]

    Shalev-Shwartz and S

    S. Shalev-Shwartz and S. Ben-David.Understanding machine learning: From theory to algorithms. Cambridge University Press, 2014

    2014

  36. [44]

    Suresh and J

    H. Suresh and J. Guttag. A framework for understanding sources of harm throughout the machine learning life cycle. InProceedings of the ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO), 2021

    2021

  37. [45]

    Review into bias in al- gorithmic decision-making.https://www.gov.uk/government/publications/ cdei-publishes-review-into-bias-in-algorithmic-decision-making, 2020

    UK Centre for Data Ethics and Innovation. Review into bias in al- gorithmic decision-making.https://www.gov.uk/government/publications/ cdei-publishes-review-into-bias-in-algorithmic-decision-making, 2020

    2020

  38. [46]

    Watson-Daniels, D

    J. Watson-Daniels, D. C. Parkes, and B. Ustun. Predictive multiplicity in probabilistic classification. InProceedings of the AAAI Conference on Artificial Intelligence, 2023

    2023

  39. [47]

    R. Xin, C. Zhong, Z. Chen, T. Takagi, M. Seltzer, and C. Rudin. Exploring the whole Rashomon set of sparse decision trees. InAdvances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  40. [48]

    2018" horizon=

    C. Zhong, Z. Chen, J. Liu, M. Seltzer, and C. Rudin. Exploring and interacting with the set of good sparse generalized additive models. InAdvances in Neural Information Processing Systems (NeurIPS), 2023. 18 Supplemental Material A Additional information on the experiments in ...

    2023

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.