REVIEW 3 major objections 5 minor 1 cited by
Observational Multiplicity
T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read A closed-form estimate shows which probability predictions are arbitrary, because the observed labels themselves are a random draw.
desk verdict Introduces a useful regret measure for probabilistic classification with a rigorous logistic-regression variance bound, but the estimator answers model-implied uncertainty rather than the true-label regret of Definition 2.1 unless the model is well-specified. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the regret identity $Q_i = p_i^2(1-p_i)^2 x_i^\top H^{-1} x_i$, with $H = \sum_j p_j(1-p_j)x_j x_j^\top$ the Hessian of the logistic loss at the fitted parameters. It expresses per-point instability as the product of label noise, $p_i^2(1-p_i)^2$, and leverage in the inverse-Hessian metric, $x_i^\top H^{-1} x_i$, so regret is large near $p_i=1/2$ and in directions where the data barely constrain the model. The proof machinery is a quantitative self-concordance bound on one Newton step for logistic regression, which controls the difference between the retrained parameters and a single linearized step, plus Bernstein concentration on the resampled-label gradient. Algorithm 1 supplies the estimator: resample labels from the fitted model, retrain, record the point's predicted probability, repeat, and take the variance.
What would settle it
Take a dataset whose labels are generated by a known nonlinear probability model, fit a logistic regression to it, and compute both true regret (resampling labels from the known generator, retraining, and taking variances) and estimated regret from Algorithm 1. If the two diverge systematically, and in particular if estimated regret underestimates true regret for points whose fitted probabilities are extreme, then the well-specification premise fails and Algorithm 1 measures the model's internal plausibility, not the label randomness the definition targets.
Extended reading notes
Core claim
The central discovery is that arbitrariness in probabilistic classification can be defined as a per-point variance and, for logistic regression, estimated without retraining on fresh data. Regret is the variance of $f_{A(D')}(x_i)$ when $D'$ is formed by resampling each label from the true conditional probability $p_i^*$; since those probabilities are unknown, Algorithm 1 resamples from the fitted model and averages the resulting predictions. Theorem 3.1 gives that when the resampling model is the initial logistic model, the resulting variance obeys $|\operatorname{Var}(\hat p_i) - Q_i| \le \epsilon Q_i$ with $Q_i = p_i^2(1-p_i)^2 x_i^\top H^{-1} x_i$ and $\epsilon$ a small constant depending on dimension, feature norms, and the Hessian's smallest eigenvalue, provided $\epsilon<1$. The paper shows this matches true regret in semi-synthetic experiments and uses it to flag high-regret points for abstention and targeted data collection, including on a large mortgage-loan dataset where estimated regret follows a heavy-tailed distribution.
Load-bearing premise
The load-bearing premise is that the fitted model's probabilities are the true conditional label probabilities, so that resampling labels from the model mimics resampling labels from nature; when the model is misspecified, the estimated regret describes arbitrariness implied by the model, not by the data-generating process, and the paper reports exactly this kind of underestimation for high-regret points in the bank dataset.
Editorial extensions
If this is right
- For logistic regression, per-point regret can be computed or bounded from a single dataset using $Q_i$, so no fresh label collection is needed to know which predictions are arbitrary.
- Regret concentrates: points with predicted probability near $1/2$ and points lying in low-curvature directions of the Hessian carry most of the instability.
- Estimated regret supports selective abstention: withholding predictions for high-regret points reduces average KL divergence to the true probabilities, close to the best possible in the bank dataset.
- Regret-guided data collection outperforms uniform sampling in two of the three semi-synthetic datasets, reaching full-dataset accuracy after a few additional samples.
- On a real mortgage-loan dataset, 98% of points have negligible regret while a small tail follows an approximate power law, so arbitrariness is a minority phenomenon that is identifiable.
Reading between the lines
- Inference: the same $Q_i$ formula suggests a cheap diagnostic for feature engineering: adding features that raise the minimum eigenvalue of $H$ in a cluster's subspace should shrink regret there, which is testable.
- Inference: regret as defined is a property of the pairing of a model class and a dataset, so comparing regret across model classes on the same data would separate label noise from model flexibility; the paper's gradient-boosting experiment on loan data is a step in that direction.
- Inference: because the estimator resamples from the deployed model, a model that is confidently wrong will report low regret; calibration audits should run alongside regret estimates before using abstention in high-stakes decisions.
- Inference: the power-law tail found on the loan dataset, if it persists, implies that fairness audits should report regret distributions per group rather than aggregate accuracy, since group-level means can hide a small set of arbitrarily predicted individuals.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. Across probabilistic classification, the paper argues that a single observed label vector is one draw from a stochastic process, so different draws produce different plausible training sets and different models. It defines observational multiplicity as the resulting arbitrariness and formalizes a per-example regret (Definition 2.1) as the variance of the prediction f_A(D')(x_i) when D' is created by resampling labels from the true conditional probabilities. Algorithm 1 estimates this variance by resampling labels from a fitted model, and Theorem 3.1 gives a finite-sample bound for logistic regression: if epsilon < 1, the variance of the resampled prediction is within a factor (1 +/- epsilon) of Q_i = p_i^2(1-p_i)^2 x_i^T H^{-1} x_i. The paper validates the estimator on three semi-synthetic datasets, applies it to 13 million Fannie Mae loans with a GBDT, and demonstrates abstention and active-learning uses.
Significance. The paper tackles a timely and important question: when multiple equally performant models disagree on an individual, how much of that disagreement is irreducible randomness in the observation process? The proof of Theorem 3.1 is a genuine technical contribution, applying Bach's self-concordance bounds in a nontrivial way and producing an explicit, actionable formula. The semi-synthetic study is careful, and the Fannie Mae experiment shows the framework scales. The significance is conditional, however, on the identification of the model-resampled variance with the true-probability variance of Definition 2.1; the paper itself acknowledges the proxy, and the experimental evidence does not close the gap under misspecification. With a redefinition or a transfer theorem, the work would be a solid contribution to model multiplicity and uncertainty quantification.
major comments (3)
- [2.3, Algorithm 1, Definition 2.1] Definition 2.1 defines regret as variance under resampling from the true conditional probabilities of the labels, but Algorithm 1 resamples from the fitted model's own probabilities and Theorem 3.1 analyzes exactly that model-resampled process (labels ~ Bern(p_i) from an initial logistic fit). Section 2.3 concedes that the proxy is 'exactly the model we were attempting to learn in the first place.' These are different stochastic mechanisms unless the fitted model coincides with the true conditional probability function; under misspecification, Algorithm 1 estimates a property of the deployed model rather than the quantity in Definition 2.1. The underestimation of high-regret points in the bank data is consistent with this divergence. The paper should either prove a bound that transfers from model-resampled variance to true regret under well-specification, or reframe regret and the safety applications as explicitly model-implied rather than as estimates of the true observational multiplicity.
- [Theorem 3.1 and Section 4] The central theorem applies only when epsilon < 1, but the paper never reports whether this condition holds in any of its experiments, and the condition depends on d, Xmax/Xmin, ||theta*||_2, and the smallest Hessian eigenvalue lambda_min, so it is most likely to fail in exactly the ill-conditioned or high-dimensional settings where regret matters. In the loan experiment the model is a GBDT, which is outside the theorem's scope entirely. Please state how a practitioner can verify epsilon < 1 and provide at least one experiment, synthetic or real, where the condition and the bound are actually checked; otherwise the empirical support for the theorem's regime is missing.
- [4.1 and 4.2] The semi-synthetic validation in Section 4.1 generates ground-truth labels from a logistic model whose parameters are themselves fit to the original data, so the experiments are confined to the well-specified logistic family. The Fannie Mae experiment is fully real and therefore lacks any ground-truth regret to compare against, and the GBDT estimator has no supporting theory. There is no experiment that quantifies the gap between true regret (Definition 2.1) and Algorithm 1's output under a deliberately misspecified model (e.g., nonlinear label-generating function with a logistic estimator). Such an experiment would directly address the main weakness identified in Major Comment 1 and would determine whether the reported bank underestimation is an isolated finite-sample artifact or a systematic feature.
minor comments (5)
- [Abstract] The abstract contains several grammatical errors, including 'can can undermine', 'as a result of different training labels change', and 'estimating regret promote safety'; these should be corrected.
- [5.3] Lemma 5.5 states Pr(B_c) <= 2d exp(...), but the proof bounds the complement B_c^c; the event/complement notation is also inconsistent in the lower-bound part of Section 5.4, which uses 'B_c' for both the event and its complement.
- [Algorithm 1] Algorithm 1 does not list k (the number of resamples) as an input; the experiments use k = 300, but the pseudocode should state k explicitly.
- [4.2] The claim that the tail of the regret distribution 'appears to follow a power law' is supported only by an eyeballed line y = 3.5e-10 x^{-3}; a fitted exponent and goodness-of-fit measure would make the claim precise.
- [Figure 1] The caption 'The five points at the top and five points at the bottom all have an 80% probability of lying in the positive and negative class' is ambiguous about which points are positive and which are negative; please clarify.
Circularity Check
No significant circularity: the finite-sample bound is a genuine derivation, and the model-based resampling is an acknowledged approximation, not a hidden identity.
full rationale
The paper's derivation chain is not circular. Definition 2.1 defines regret as prediction variance under labels resampled from the true conditional probabilities. Algorithm 1 instead resamples labels from the fitted model's probabilities, and Section 2.3 explicitly flags this: "This approximation, of course, is exactly the model we were attempting to learn in the first place." The paper then reframes the output as uncertainty "directly implied by the model we are seeking to deploy." This is an honest, disclosed assumption rather than a disguised reduction. Theorem 3.1 is a substantive finite-sample bound for the model-resampled variance, proved from Bach's Proposition 2 and concentration arguments, not from the conclusion; it is not the variance formula written as an input. The semi-synthetic experiments validate the estimator against known ground-truth labels, providing independent evidence, and the bank dataset's reported underestimation of high-regret points is a misspecification symptom consistent with the acknowledged proxy gap. The only self-citations (e.g., references [10], [13], and [19]) appear in related-work or motivation and are not load-bearing. Therefore no claim reduces by construction to its own inputs; the central result has independent mathematical content.
Assumptions & free parameters
free parameters (4)
- initial logistic regression parameters theta* =
not reported
- number of label resamples k =
300
- ground-truth logistic parameters for semi-synthetic data =
fit by regularized logistic regression
- GBDT hyperparameters for Fannie Mae experiment =
not specified
assumptions (5)
- domain assumption Labels are independent Bernoulli draws from true conditional probabilities p_i^*
- ad hoc to paper Fitted logistic model is well specified, so resampling from f_theta*(x_i) approximates resampling from true probabilities
- standard math Bach's Proposition 2 (self-concordance bound) is correct and applicable
- domain assumption Feature matrix is full rank and not linearly separable, so the Hessian is positive definite
- domain assumption Compact support and label probabilities bounded away from 0 and 1
Cite this review
Pith. "Pith review of Observational Multiplicity." pith.science (2026). https://pith.science/paper/7OQ7GVAN
@misc{pith2026250723136,
author = {Pith},
title = {Pith review of: Observational Multiplicity},
year = {2026},
howpublished = {\url{https://pith.science/paper/7OQ7GVAN}},
note = {Machine review of arXiv:2507.23136}
}
abstract
Many prediction tasks can admit multiple models that can perform almost equally well. This phenomenon can can undermine interpretability and safety when competing models assign conflicting predictions to individuals. In this work, we study how arbitrariness can arise in probabilistic classification tasks as a result of an effect that we call \emph{observational multiplicity}. We discuss how this effect arises in a broad class of practical applications where we learn a classifier to predict probabilities $p_i \in [0,1]$ but are given a dataset of observations $y_i \in \{0,1\}$. We propose to evaluate the arbitrariness of individual probability predictions through the lens of \emph{regret}. We introduce a measure of regret for probabilistic classification tasks, which measures how the predictions of a model could change as a result of different training labels change. We present a general-purpose method to estimate the regret in a probabilistic classification task. We use our measure to show that regret is higher for certain groups in the dataset and discuss potential applications of regret. We demonstrate how estimating regret promote safety in real-world applications by abstention and data collection.
Figures
Figures from the paper (5 more)
Forward citations
Cited by 1 Pith paper
-
Using predictive multiplicity to measure individual performance within the AI Act
Providers of high-risk AI should report how often equally accurate models disagree on a person's case, using the proposed conflict ratio and δ-ambiguity metrics.
Reference graph
Works this paper leans on
-
[1]
Self-concordant analysis for logistic regression.Electronic Journal of Statistics, 4: 384–414, 2010
Francis Bach. Self-concordant analysis for logistic regression.Electronic Journal of Statistics, 4: 384–414, 2010
work page 2010
-
[2]
Model multiplicity: Opportunities, concerns, and solutions
Emily Black, Manish Raghavan, and Solon Barocas. Model multiplicity: Opportunities, concerns, and solutions. InProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’22, pages 850–863, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450393522. doi: 10.1145/3531146.3533149. URL https://doi.org/ 1...
arXiv 2022
-
[3]
Olivier Bousquet and André Elisseeff. Stability and generalization. Journal of Machine Learning Research, 2(Mar):499–526, 2002. URL https://www.jmlr.org/papers/v2/ bousquet02a.html
work page 2002
-
[4]
Statistical modeling: The two cultures.Statistical Science, 16(3):199–231, 2001
Leo Breiman. Statistical modeling: The two cultures.Statistical Science, 16(3):199–231, 2001. doi: 10.1214/ss/1009213726. URL https://doi.org/10.1214/ss/1009213726
arXiv 2001
-
[5]
Alexander D’Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew D. Hoffman, Farhad Hormozdiari, Neil Houlsby, Shaobo Hou, Ghassen Jerfel, Alan Karthikesalingam, Mario Lucic, Yian Ma, Cory McLean, Diana Mincu, Akinori Mitani, Andrea Montanari, Zachary Nado, Vivek Nataraj...
work page 2022
-
[6]
Fannie Mae single-family loan performance data, 2025
Fannie Mae. Fannie Mae single-family loan performance data, 2025. URL https: //capitalmarkets.fanniemae.com/credit-risk-transfer/single-family- credit-risk-transfer/fannie-mae-single-family-loan-performance-data . [Online; accessed: 2025-1-16]
work page 2025
-
[7]
Rashomon capacity: A metric for predictive multiplicity in classification
Hsiang Hsu and Flavio Calmon. Rashomon capacity: A metric for predictive multiplicity in classification. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 28988–29000. Curran Associates, Inc., 2022. URLhttps://proceedings.neurips.cc/paper_files/paper/ 2022/fil...
work page 2022
-
[8]
Stability and generalization analy- sis of gradient methods for shallow neural networks
Yunwen Lei, Rong Jin, and Yiming Ying. Stability and generalization analy- sis of gradient methods for shallow neural networks. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Infor- mation Processing Systems , volume 35, pages 38557–38570. Curran Associates, Inc.,
Show all 22 references
-
[9]
Trans- formers as algorithms: Generalization and stability in in-context learning
Yingcong Li, Muhammed Emrullah Ildiz, Dimitris Papailiopoulos, and Samet Oymak. Trans- formers as algorithms: Generalization and stability in in-context learning. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors...
2023
-
[10]
Predictive multiplicity in classification
Charles Marx, Flavio Calmon, and Berk Ustun. Predictive multiplicity in classification. In Hal Daumé, III and Aarti Singh, editors,Proceedings of the 37th International Conference on Machine Learning, volume 119 ofProceedings of Machine Learning Research, pages 6765–6774. Proc...
2020
-
[12]
A data-driven approach to predict the success of bank telemarketing
Sérgio Moro, Paulo Cortez, and Paulo Rita. A data-driven approach to predict the success of bank telemarketing. Decision Support Systems, 62:22–31, 2014. doi: 10.1016/j.dss.2014.03.001. URL https://www.sciencedirect.com/science/article/pii/S016792361400061X
2014 doi
-
[13]
Regretful decisions under label noise
Sujay Nagaraj, Yang Liu, Flavio P Calmon, and Berk Ustun. Regretful decisions under label noise. 13th International Conference on Learning Representations, 2025. URL https: //openreview.net/forum?id=7B9FCDoUzB
2025
-
[14]
Martin Pawelczyk, Klaus Broelemann, and Gjergji. Kasneci. On counterfactual explanations under predictive multiplicity. In Jonas Peters and David Sontag, editors,Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI), volume 124 ofProceedings of Mac...
2020
-
[15]
Brandon Westover, Cynthia Rudin, and Matt T
Berk Ustun, M. Brandon Westover, Cynthia Rudin, and Matt T. Bianchi. Clinical prediction models for sleep apnea: The importance of medical history over symptoms.Journal of Clinical Sleep Medicine, 12(02):161–168, 2016. doi: 10.5664/jcsm.5476. URLhttps://jcsm.aasm.org/ doi/abs/...
2016 doi
-
[16]
Stability and generalization of graph convolutional neural networks
Saurabh Verma and Zhi-Li Zhang. Stability and generalization of graph convolutional neural networks. InProceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’19, pages 1539–1548, New York, NY, USA, 2019. Association for Computing...
2019
-
[17]
Cambridge Series in Statistical and Probabilistic Mathematics
Roman Vershynin.High-Dimensional Probability: An Introduction with Applications in Data Sci- ence. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press,
-
[18]
Hofman, and Alexandra Chouldechova
Jamelle Watson-Daniels, Solon Barocas, Jake M. Hofman, and Alexandra Chouldechova. Multi- target multiplicity: Flexibility and fairness in target specification under resource constraints. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, ...
2023
-
[19]
Parkes, and Berk Ustun
Jamelle Watson-Daniels, David C. Parkes, and Berk Ustun. Predictive multiplicity in probabilistic classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 10306–10314, 2023. doi: 10 .1609/aaai.v37i9.26227. URL https://ojs.aaai.org/ ind...
2023
-
[20]
William Wolberg, Olvi Mangasarian, Nick Street, and W. Street. Breast cancer Wisconsin (diagnostic). UCI Machine Learning Repository, 1993. 26
1993
-
[21]
On model selection consistency of lasso.Journal of Machine Learning Research, 7(90):2541–2563, 2006
Peng Zhao and Bin Yu. On model selection consistency of lasso.Journal of Machine Learning Research, 7(90):2541–2563, 2006. URLhttp://jmlr.org/papers/v7/zhao06a.html. 27
2006
-
[2018]
URL https://doi.org/10.1017/9781108231596
doi: 10.1017/9781108231596. URL https://doi.org/10.1017/9781108231596
-
[2022]
URL https://proceedings.neurips.cc/paper_files/paper/2022/file/ fb8fe6b79288f3d83696a5d276f4fc9d-Paper-Conference .pdf
2022
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.