Pith. sign in

REVIEW 3 major objections 5 minor 63 references

"Who experiences large model decay and why?" A Hierarchical Framework for Diagnosing Heterogeneous Performance Drift

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SHIFT is a two-stage hypothesis testing framework that locates the subgroups where a deployed model loses unacceptable accuracy and then names the variable subsets responsible, with controlled false alarms.

desk verdict A credible and useful framework for diagnosing heterogeneous performance drift, with a real gap between the proven theory (binned outcome-shift estimand) and the null the analyst actually wants. read the letter →

arxiv 2506.00756 v1 pith:EONNYV5G submitted 2025-05-31 cs.LG cs.AIstat.ML

Harvineet Singh , Fan Xia , Alexej Gossmann , Andrew Chuang , Julian C. Hong , Jean Feng This is my paper
classification cs.LGcs.AIstat.ML MSC 62G1062G2068T05
keywords distributionshiftsubgroupscanningperformancedecaycovariateoutcomehypothesistestingdebiasedmachinelearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

When a machine-learning model is moved to a new setting—another hospital, another state, another customer base—its accuracy usually does not fall evenly: some subgroups lose a lot while others are barely touched. This paper introduces SHIFT, a two-stage hypothesis testing framework: it first tests whether any subgroup of at least a chosen size has an accuracy loss above a chosen tolerance $\tau$, attributable to a shift in the input features or in how outcomes are generated, and if so it tests which subset of variables explains the loss. The paper claims both stages give valid statistical inference without a known causal graph or parametric model: false alarms are controlled and genuinely shifted variables are flagged with probability at least $1-\alpha$ as the sample grows. That matters because a model developer could then fix only the affected subgroup and the specific variables responsible, instead of retraining the whole model and risking new harm elsewhere.

What carries the argument

The load-bearing object is the Maximum conditional Expectation of the Exceedence (McEE): the supremum over subgroups $A$ of the conditional expected amount by which the performance drop exceeds tolerance $\tau$. Each of the four hypotheses is rewritten as a test that this supremum is non-positive, and the search over all subgroups is made tractable by sample splitting—one split learns a small set of candidate subgroup detectors (for covariate shifts, found by solving a dual problem over scaled density ratios) and the held-out split evaluates a one-step debiased estimator of the McEE for those detectors, giving asymptotically normal statistics under the null. The detailed outcome-shift test requires an extra device: its candidate shift is defined through the source outcome probability $\mu_0(x)$, whose sharp dependence blocks pathwise differentiability, so the paper replaces $\mu_0$ with a binned version and proves the binned McEE is pathwise differentiable and hence admits an asymptotically linear estimator. P-values are computed with a Gaussian multiplier bootstrap.

What would settle it

Simulate a shift with known ground truth where $\mu_0(x)$ is continuous and steep, apply the detailed outcome-shift test at the paper's own setting ($B = 40$ bins, $\tau$ and $\epsilon$ chosen), and check whether the flagged variable subsets match the truth and stay stable as $B$ grows; if the flags change with bin count or disagree with an unbinned oracle analysis, the binning assumption is doing real work at the recommended setting, and the guarantee on the binned estimand does not extend to the hypothesis the analyst cares about.

Watch

Extended reading notes

Core claim

SHIFT's central claim is that heterogeneous performance decay can be diagnosed through valid, nonparametric hypothesis tests rather than point estimates. The aggregate stage tests whether any subgroup with prevalence at least $\epsilon$ has an expected-loss increase above tolerance $\tau$ due to covariate shift or outcome shift; the detailed stage tests whether a candidate variable-subset-specific shift reproduces the aggregate decay across all subgroups, and flags the subsets whose nulls are not rejected as potential explanations. The paper proves that the debiased estimators of the Maximum conditional Expectation of the Exceedence (McEE) are asymptotically linear, so the tests control Type I error and have asymptotic power one; consequently, if the variable subset corresponding to the true shift is among the candidates, the probability that SHIFT fails to flag it is at most $\alpha$. In simulations with known ground truth, SHIFT outperforms kernel-based and parametric baselines, and in two real deployments it flags interpretable subgroups and suggests targeted updates that improve the affected subgroup without degrading others.

Load-bearing premise

The detailed outcome-shift test does not test the shift the analyst actually defined: it replaces the source outcome probability $\mu_0(x)$ with a binned version and assumes the binned and unbinned quantities are similar for a large enough number of bins, so the proven assertions about error control attach to the binned test rather than the original hypothesis.

Editorial extensions

If this is right

  • Deployed models can be audited subgroup by subgroup with controlled error rates, so a hospital, agency, or platform can tell whether a reported accuracy drop is real rather than noise.
  • Corrective action becomes targeted: in the insurance case study, updating the model only for the flagged subgroup and demographic variables beat full retraining and avoided the accuracy losses that retraining caused in other subgroups.
  • Because the tests are omnibus, they work on modest sample sizes and can serve as screens that trigger deeper analysis only when a problem exists.
  • The same machinery applies to related questions: the authors note the tests parallel testing heterogeneous direct and indirect effects in mediation analysis, so they may transfer to subgroup-specific causal questions.
  • Domain experts can set the thresholds (minimum subgroup size $\epsilon$ and minimum decay $\tau$) so that negligible shifts do not raise alarms, addressing alarm fatigue in clinical monitoring.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If SHIFT is right, average-accuracy monitoring can be actively misleading: in the paper's readmission case a subgroup covering 41.8% of target patients lost 15.4% accuracy while the reported overall drop was 6.1%, so aggregate dashboards may certify models that are failing a specific population.
  • The collider-bias preprocessing—dropping features uncorrelated with the loss before covariate-shift testing—may, in other settings, discard a genuine mediator; a sensitivity check comparing flagged subsets with and without the filter would reveal how often the fix removes real explanations.
  • The binning assumption suggests a cheap robustness check that the paper does not report: rerun the detailed outcome-shift stage at several bin counts $B$ and confirm the flagged subsets are stable, since only the binned estimand is covered by the theorems.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces SHIFT, a two-stage nonparametric hypothesis-testing framework for diagnosing heterogeneous performance drift under distribution shift. The first stage ('Where?') tests whether any sufficiently large subgroup experiences performance decay exceeding a tolerance τ due to aggregate covariate or outcome shifts; the second stage ('How?') tests whether variable-subset-specific candidate shifts explain the decay. The inference procedure uses sample splitting, ML-based nuisance estimation, one-step debiased estimators of a maximum conditional expectation of exceedence (McEE), and a multiplier bootstrap. The central theoretical claim is that the estimators are asymptotically linear, so the tests control Type I error and have asymptotic power one; simulations and two real-world case studies (insurance coverage, hospital readmission) plus a text-data application support the practical value.

Significance. If the stated guarantees hold, SHIFT is a meaningful advance: it provides valid inference for subgroup-level performance decay without parametric models or known causal graphs, and it produces actionable variable-subset explanations. The paper's strengths include a clear problem formulation, the connection to MMD and mediation analysis, the use of debiased ML for boundary nulls, and reproducible code. The real-world case studies, especially the readmission example where correcting the extraction of 'num ED encounters' removes the covariate-shift signal, demonstrate genuine practical utility. However, the formal guarantees are narrower than the central claims: the detailed outcome-shift theorem applies to a binned estimand, and the power claims presuppose that a singleton learned detector suffices for the supremum null. These gaps need to be addressed before the stated claims can be accepted.

major comments (3)
  1. [Section 4, Appendix C.3, Theorem D.2] The detailed outcome-shift test is proven for the binned estimand McEE_bin(s), not for the original Hypothesis 3.4 estimand. The paper asserts in Appendix C.3 that 'We expect the binned and original version of MEE to be similar for a large enough number of bins,' but no theorem quantifies this approximation and no sensitivity analysis over B (fixed at B=40 in Appendix H) is reported. Because the 'How' stage flags variable subsets by failing to reject the binned null, a material binning error would make the flagged explanations pertain to a different outcome shift than the one the analyst specified. Please either prove a bound on |McEE_bin(s) − McEE(s)| that is uniform over A∈Aε (under explicit regularity conditions on the loss and p1(y|x,·)), or report a sensitivity analysis over B and explicitly state that the reported p-values are for the binned estimand.
  2. [Section 4 and Appendix D] The asymptotic linearity proofs (Lemma D.3, Theorems D.2 and D.5) concern a fixed singleton detector h_A, whereas the procedure estimates the detector on the training split and the null hypotheses in Section 3 are suprema over A∈Aε. The manuscript acknowledges conservativeness but does not state conditions under which the data-dependent detector yields a test with power against the original sup-null; the claim that s*,shift will be flagged with probability at least 1−α requires that the plug-in detector recovers the worst-case subgroup sufficiently well. This is load-bearing for both stages. Please either provide detector-estimation conditions and a formal power statement for the restricted test, or revise the claims to refer explicitly to the singleton-detector null.
  3. [Appendix B / Remark 3.1] The modified covariate test first screens features to X_corr = {Xi : Xi not independent of loss} on the training split and then defines the null hypotheses 3.1 and 3.3 with respect to X := X_corr. This changes the target of inference relative to the stated null for the full covariate vector, and the screening threshold is an additional free hyperparameter. The manuscript does not discuss how finite-sample screening errors affect the validity or interpretation of the covariate-shift tests, nor what population quantity the test is consistent for. Please state the conditional-on-selection null explicitly and discuss the screening step's impact on the test's interpretation.
minor comments (5)
  1. [Section 5.1, Table 2] Table 2 reports power from only 25 random draws, whereas Figure 3 and Appendix I use 50; the wide confidence intervals (e.g., SHIFT Setup 1a: 0.56 with CI 0.42–0.70) make it difficult to compare methods reliably. Please report the number of repetitions consistently and discuss the Monte Carlo error.
  2. [Section 5.1 vs Appendix G.1] The subgroup definition is inconsistent between the main text and the appendix: Section 5.1 uses A={x|x1∉[−3.5,3.5]} for both setups, while Appendix G.1 defines Setup 1b with A={x∈R^10|x1∉[−4,4]}. Please clarify the exact subgroup definitions for Setups 1a and 1b.
  3. [Appendix C.3, equations (14)–(16)] The notation in equations (14)–(16) is dense and the role of πV is not clearly explained; a short intuitive description of the V-statistic correction would improve accessibility.
  4. [Appendix D.2] The covariate detailed test proof is omitted with a reference to Lemma G.2 in Singh et al. (2024); since this is a key component of the 'How' stage, please state the result or provide a proof sketch.
  5. [Table 1] In Table 1, the row for 'Detect loss shift' lists 'Detailed explanations for outcome/covariate shifts: Outcome only' for Podkopaev & Ramdas (2022); the cited work does not provide variable-subset explanations, so this entry should be 'No' to match the text.

Circularity Check

0 steps flagged · score 1.0 of 10

No circularity: pre-specified nulls and held-out testing keep the derivation independent; binning approximation and delegated covariate-shift proof are gaps, not circular steps.

full rationale

The derivation chain is not circular. The null hypotheses (3.1)-(3.4) are fixed before any data are used, and the testing procedure splits the sample into a training part for detectors/nuisances and a held-out part for the one-step corrected McEE estimators (Section 4, Appendix C), so the reported p-values are not fit on the same data that defines the test decision. The theoretical results are proven for the binned detailed-outcome estimand: Theorem D.2 establishes asymptotic linearity of the debiased McEEY(s) estimator under Condition D.1, and the 'true shift will be flagged' statement is a consequence of power against the binned null rather than a construction that forces the flag. Two passages warrant attention but are not circular. First, Appendix C.3 states 'We expect the binned and original version of MEE to be similar for a large enough number of bins'; this is an unproved approximation bridging Hypothesis 3.4 to the estimand in Theorem D.2, a correctness/robustness gap rather than an equivalence-by-construction. Second, Appendix D.2 omits the derivation of asymptotic linearity for the detailed covariate estimator and points to 'Lemma G.2 in Singh et al. (2024)', a prior paper with overlapping authors; this is an omitted proof and a self-citation, but it is a delegation to an external published lemma, not a reduction of the present test statistic to its own fitted inputs. The real-world case studies use held-out target data and compare SHIFT's flagged variables to external fixes, so the empirical claims are not baked into the method. Overall, the central derivation has independent content; the noted gaps lower confidence in specific 'How' explanations under coarse binning but do not make the derivation circular.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central inference procedure rests on standard semiparametric assumptions (sample splitting, consistency and product-of-error rates for nuisance models), on domain assumptions about support, and on two ad hoc choices: the binning of mu0 and the feature-screening step in the modified covariate test. Performance thresholds tau, epsilon, alpha and the bin count B are user-chosen; they define the null hypotheses but are not fitted to data.

free parameters (4)
  • tau (tolerance) = 0, 0.05, or 0.02 depending on experiment
    User-specified minimum shift magnitude; defines the null hypotheses. Not fitted, but its choice affects what SHIFT flags.
  • epsilon (minimum subgroup prevalence) = 0.05 in experiments
    User-specified minimum subgroup size; only subgroups with prevalence above epsilon are considered.
  • B (number of bins for binned outcome shift) = 40
    Chosen to discretize mu0 for pathwise differentiability; the theoretical guarantee is for the binned estimand, not the original outcome shift.
  • Feature screening threshold for modified covariate test = Not specified in main text
    Features are screened for correlation with loss before covariate tests; the criterion is not formally stated and is not part of the theoretical analysis.
assumptions (5)
  • standard math i.i.d. samples from source and target domains; sample splitting into training and evaluation partitions
    Assumed in Section 4 and Appendix C; standard for semiparametric inference.
  • domain assumption Nuisance estimators are consistent and the product of estimation errors converges at o_p(n^{-1/2}) (Condition D.1 for outcome, D.4 for covariate)
    Appendix D; allows debiased estimators to be asymptotically linear. These are nontrivial assumptions about the ML models used.
  • domain assumption Support of source covariates contains target support (contiguity), p0(x)>0 whenever p1(x)>0
    Condition D.4; needed for density ratios to be well-defined.
  • ad hoc to paper Binned mu_bin does not fall on bin edges almost surely and binned MEE approximates the original MEE
    Appendix C.3 and Section 4; the binned outcome shift is substituted for the original to ensure pathwise differentiability. The approximation claim is not proven.
  • domain assumption Candidate shift definitions: ps(y|x)=p1(y|xs,mu0(x)) and ps(x)=p1(xs)p0(x-s|xs) are the explanations considered
    Section 3.2; the 'How' stage is limited to these candidate shifts, so explanations are relative to this restricted class.

how reviews work

0 comments
Cite this review

Pith. "Pith review of "Who experiences large model decay and why?" A Hierarchical Framework for Diagnosing Heterogeneous Performance Drift." pith.science (2026). https://pith.science/paper/EONNYV5G

@misc{pith2026250600756,
  author       = {Pith},
  title        = {Pith review of: "Who experiences large model decay and why?" A Hierarchical Framework for Diagnosing Heterogeneous Performance Drift},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EONNYV5G}},
  note         = {Machine review of arXiv:2506.00756}
}
read the original abstract

Machine learning (ML) models frequently experience performance degradation when deployed in new contexts. Such degradation is rarely uniform: some subgroups may suffer large performance decay while others may not. Understanding where and how large differences in performance arise is critical for designing targeted corrective actions that mitigate decay for the most affected subgroups while minimizing any unintended effects. Current approaches do not provide such detailed insight, as they either (i) explain how average performance shifts arise or (ii) identify adversely affected subgroups without insight into how this occurred. To this end, we introduce a Subgroup-scanning Hierarchical Inference Framework for performance drifT (SHIFT). SHIFT first asks "Is there any subgroup with unacceptably large performance decay due to covariate/outcome shifts?" (Where?) and, if so, dives deeper to ask "Can we explain this using more detailed variable(subset)-specific shifts?" (How?). In real-world experiments, we find that SHIFT identifies interpretable subgroups affected by performance decay, and suggests targeted actions that effectively mitigate the decay.

Figures

Figures reproduced from arXiv: 2506.00756 by the authors.

Figure 1
Figure 1. Subgroup-scanning Hierarchical Inference Framework for performance drifT (SHIFT) is a two-stage hypothesis testing procedure that first checks if there is a subgroup with unacceptably large performance decay due to aggregate covariate and outcome shifts with respect to all X variables. If so, it checks if this can be explained by detailed variable(subset)-specific shifts. Red indicates the shift was flagged for furt… view at source ↗
Figure 2
Figure 2. Overview of testing procedure First, estimating a supremum over the infinite number of binary detectors hA is computationally intractable. Never￾theless, our goal is simply hypothesis testing, not estimation. We can accomplish this by sample splitting, where one por￾tion of the data is for learning one (or a few) good candidate detector (hˆA) and the remaining data is for testing the ex￾pected exceedence for hˆA. Th… view at source ↗
Figure 3
Figure 3. Hypothesis testing results for variable(subset)- specific shifts. SHIFT shown in outlined boxes; baselines for covariate and outcome shifts shown on the bottom left and right, respectively. Null hypotheses either state that a shift should be flagged (†), in which case we flag it in red if the p-value > 0.05 and show the p-value in the colored box, or that a shift should not be flagged (‡), in which case we flag it i… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Graphical description of the hypothesis tests. Differences in distributions of X and Y (and hence average performance of a model) across the two domains D = 0, 1 is represented by effect of variable D on X, Y . (left) Performance can vary due to changes in conditional …
Figure 5
Figure 5. Figure 5: Modified covariate shift test. Graph provides an example data generating mechanism where X1 is marginally independent of Y . The feature X2 influences Y but does not shift. Conditioning on A introduces a collider on the path from D to Y (and hence ℓ). Therefore, the lo…
Figure 6
Figure 6. Figure 6: Testing VIs at boundary of parameter space. Existing methods for the related problem of explaining treatment effect heterogeneity (Hines et al., 2023; Quinzan et al., 2023; Williamson et al., 2021) do not give valid tests for importance of a variable. We observe that t…
Figure 7
Figure 7. Figure 7: Covariate shift setup. Covariate test, tolerance τ = 0.0 and prevalence ϵ = 0.05. 29 [PITH_FULL_IMAGE:figures/full_fig_p029_7.png]
Figure 8
Figure 8. Figure 8: Outcome shift setup. Outcome test, tolerance τ = 0.05 and prevalence ϵ = 0.05. 30 [PITH_FULL_IMAGE:figures/full_fig_p030_8.png]
Figure 9
Figure 9. Figure 9: Detailed tests for variable subsets in simulation setups. [PITH_FULL_IMAGE:figures/full_fig_p031_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

63 extracted references · 44 canonical work pages

  1. [1]

    Ali, A., Cauchois, M., and Duchi, J. C. The lifecycle of a statistical model: Model failure detection, identification, and refitting, 2022. URL https://arxiv.org/abs/2202.04166

    arXiv 2022

  2. [2]

    Generalized random forests

    Athey, S., Tibshirani, J., and Wager, S. Generalized random forests . The Annals of Statistics, 47 0 (2): 0 1148 -- 1178, 2019. doi:10.1214/18-AOS1709. URL https://doi.org/10.1214/18-AOS1709

    doi:10.1214/18-aos1709 2019

  3. [3]

    Baron, R. M. and Kenny, D. A. The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. Journal of personality and social psychology, 51 6: 0 1173--82, 1986. URL https://api.semanticscholar.org/CorpusID:1925599

    work page 1986

  4. [4]

    and Chernozhukov, V

    Belloni, A. and Chernozhukov, V. _1 -penalized quantile regression in high-dimensional sparse models . The Annals of Statistics, 39 0 (1): 0 82 -- 130, 2011. doi:10.1214/10-AOS827. URL https://doi.org/10.1214/10-AOS827

    doi:10.1214/10-aos827 2011

  5. [5]

    Why did the distribution change? In Banerjee, A

    Budhathoki, K., Janzing, D., Bloebaum, P., and Ng, H. Why did the distribution change? In Banerjee, A. and Fukumizu, K. (eds.), Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, volume 130 of Proceedings of Machine Learning Research, pp.\ 1666--1674. PMLR, 13--15 Apr 2021. URL https://proceedings.mlr.press/v130/bu...

    work page 2021

  6. [6]

    T., Namkoong, H., and Yadlowsky, S

    Cai, T. T., Namkoong, H., and Yadlowsky, S. Diagnosing model performance under distribution shift. March 2023. URL http://arxiv.org/abs/2303.02011

    arXiv 2023

  7. [7]

    C., Walker, I., and Glocker, B

    Castro, D. C., Walker, I., and Glocker, B. Causality matters in medical imaging. Nat. Commun., 11 0 (1): 0 3673, July 2020

    work page 2020

  8. [8]

    Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors

    Chernozhukov, V., Chetverikov, D., and Kato, K. Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors . The Annals of Statistics, 41 0 (6): 0 2786 -- 2819, 2013. doi:10.1214/13-AOS1161. URL https://doi.org/10.1214/13-AOS1161

    doi:10.1214/13-aos1161 2013

Show all 63 references
  1. [9]

    Double/debiased machine learning for treatment and structural parameters

    Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., and Robins, J. Double/debiased machine learning for treatment and structural parameters. Econom. J., 21 0 (1): 0 C1--C68, February 2018

    2018

  2. [10]

    Monitor alarm fatigue: an integrative review

    Cvach, M. Monitor alarm fatigue: an integrative review. Biomed. Instrum. Technol., 46 0 (4): 0 268--277, 2012

    2012

  3. [11]

    R., and Leyton-Brown, K

    d'Eon, G., d'Eon, J., Wright, J. R., and Leyton-Brown, K. The spotlight: A general method for discovering systematic errors in deep learning models. In 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT '22, pp.\ 1962–1981, New York, NY, USA, 2022. Associ...

    2022

  4. [12]

    Retiring adult: New datasets for fair machine learning

    Ding, F., Hardt, M., Miller, J., and Schmidt, L. Retiring adult: New datasets for fair machine learning. Advances in Neural Information Processing Systems, 34, 2021

    2021

  5. [13]

    Efficiently mitigating the impact of data drift on machine learning pipelines

    Dong, S., Wang, Q., Sahri, S., Palpanas, T., and Srivastava, D. Efficiently mitigating the impact of data drift on machine learning pipelines. Proc. VLDB Endow., 17 0 (11): 0 3072–3081, August 2024. ISSN 2150-8097. doi:10.14778/3681954.3681984. URL https://doi.org/10.14778/368...

    2024

  6. [14]

    K., Delbrouck, J.-B., Lee-Messer, C., Dunnmon, J., Zou, J., and Re, C

    Eyuboglu, S., Varma, M., Saab, K. K., Delbrouck, J.-B., Lee-Messer, C., Dunnmon, J., Zou, J., and Re, C. Domino: Discovering systematic errors with cross-modal embeddings. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=FPCMqjI0jXN

    2022

  7. [15]

    Is this model reliable for everyone? testing for strong calibration

    Feng, J., Gossmann, A., Pirracchio, R., Petrick, N., A Pennello, G., and Sahiner, B. Is this model reliable for everyone? testing for strong calibration. In Dasgupta, S., Mandt, S., and Li, Y. (eds.), Proceedings of The 27th International Conference on Artificial Intelligence ...

    2024

  8. [16]

    Feng, J., Kothari, A., Zier, L., Singh, C., and Tan, Y. S. Bayesian concept bottleneck models with LLM priors. NeurIPS Workshop on Statistical Frontiers in LLMs and Foundation Models, October 2024 b

    2024

  9. [17]

    Not all clinical AI monitoring systems are created equal: Review and recommendations

    Feng, J., Xia, F., Singh, K., and Pirracchio, R. Not all clinical AI monitoring systems are created equal: Review and recommendations. NEJM AI, 2 0 (2), January 2025

    2025

  10. [18]

    G., Subbaswamy, A., Singh, K., Bowers, J., Kupke, A., Zittrain, J., Kohane, I

    Finlayson, S. G., Subbaswamy, A., Singh, K., Bowers, J., Kupke, A., Zittrain, J., Kohane, I. S., and Saria, S. The clinician and dataset shift in artificial intelligence. New England Journal of Medicine, 385 0 (3): 0 283--286, 2021. doi:10.1056/NEJMc2104626. URL https://www.ne...

    2021 doi

  11. [19]

    Ghosh, B., Malioutov, D., and Meel, K. S. Efficient learning of interpretable classification rules. Journal of Artificial Intelligence Research, 74: 0 1823--1863, 2022

    2022

  12. [20]

    An algorithmic framework for bias bounties

    Globus-Harris, I., Kearns, M., and Roth, A. An algorithmic framework for bias bounties. In 2022 ACM Conference on Fairness, Accountability, and Transparency, New York, NY, USA, June 2022. ACM

    2022

  13. [21]

    M., Rasch, M

    Gretton, A., Borgwardt, K. M., Rasch, M. J., Sch \"o lkopf, B., and Smola, A. A kernel two-sample test. Journal of Machine Learning Research, 13 0 (25): 0 723--773, 2012 a . URL http://jmlr.org/papers/v13/gretton12a.html

    2012

  14. [22]

    M., Rasch, M

    Gretton, A., Borgwardt, K. M., Rasch, M. J., Schölkopf, B., and Smola, A. A kernel two-sample test. J. Mach. Learn. Res., 13 0 (25): 0 723--773, 2012 b

    2012

  15. [23]

    Multicalibration: Calibration for the ( C omputationally-identifiable) masses

    Hebert-Johnson, U., Kim, M., Reingold, O., and Rothblum, G. Multicalibration: Calibration for the ( C omputationally-identifiable) masses. In Dy, J. and Krause, A. (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine...

    1939

  16. [24]

    Diagnostic runtime monitoring with martingales, 2024

    Hindy, A., Luo, R., Banerjee, S., Kuck, J., Schmerling, E., and Pavone, M. Diagnostic runtime monitoring with martingales, 2024. URL https://arxiv.org/abs/2407.21748

    2024 arXiv

  17. [25]

    Variable importance measures for heterogeneous causal effects, 2023

    Hines, O., Diaz-Ordaz, K., and Vansteelandt, S. Variable importance measures for heterogeneous causal effects, 2023

    2023

  18. [26]

    Consistent tests for conditional treatment effects

    Hsu, Y. Consistent tests for conditional treatment effects. The Econometrics Journal, 20 0 (1): 0 1--22, 03 2017. ISSN 1368-4221. doi:10.1111/ectj.12077. URL https://doi.org/10.1111/ectj.12077

    2017 doi

  19. [27]

    Nonparametric inference on non-negative dissimilarity measures at the boundary of the parameter space, 2023

    Hudson, A. Nonparametric inference on non-negative dissimilarity measures at the boundary of the parameter space, 2023. URL https://arxiv.org/abs/2306.07492

    2023 arXiv

  20. [28]

    Kearns, M., Neel, S., Roth, A., and Wu, Z. S. Preventing fairness gerrymandering: Auditing and learning for subgroup fairness. In Dy, J. and Krause, A. (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Re...

    2018

  21. [29]

    Kennedy, E. H. Semiparametric doubly robust targeted double machine learning: A review. In Handbook of Statistical Methods for Precision Medicine, pp.\ 207--236. Chapman and Hall/CRC, Boca Raton, 1st edition edition, October 2024

    2024

  22. [30]

    P., Ghorbani, A., and Zou, J

    Kim, M. P., Ghorbani, A., and Zou, J. Multiaccuracy: Black-box post-processing for fairness in classification. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, AIES '19, pp.\ 247–254, New York, NY, USA, 2019. Association for Computing Machinery. ISBN ...

    2019

  23. [31]

    W., Nguyen, T., Tang, Y

    Koh, P. W., Nguyen, T., Tang, Y. S., Mussmann, S., Pierson, E., Kim, B., and Liang, P. Concept bottleneck models. In III, H. D. and Singh, A. (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, p...

    2020

  24. [32]

    W., Sagawa, S., Marklund, H., Xie, S

    Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L., Gao, I., Lee, T., David, E., Stavness, I., Guo, W., Earnshaw, B., Haque, I., Beery, S. M., Leskovec, J., Kundaje, A., Pierson, E., Levine, S., Finn, C., and Li...

    2021

  25. [33]

    Kulinski, S., Bagchi, S., and Inouye, D. I. Feature shift detection: Localizing which features have shifted via conditional distribution tests. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Processing Systems, volum...

    2020

  26. [34]

    H., and Leskovec, J

    Lakkaraju, H., Bach, S. H., and Leskovec, J. Interpretable decision sets: A joint framework for description and prediction. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16, pp.\ 1675–1684, New York, NY, USA, 2016. ...

    2016

  27. [35]

    On the need for a language describing distribution shifts: Illustrations on tabular datasets

    Liu, J., Wang, T., Cui, P., and Namkoong, H. On the need for a language describing distribution shifts: Illustrations on tabular datasets. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023. URL https://openreview.net/foru...

    2023

  28. [36]

    W., Moerkerke, B., Loeys, T., and Vansteelandt, S

    Loh, W. W., Moerkerke, B., Loeys, T., and Vansteelandt, S. Heterogeneous indirect effects for multiple mediators using interventional effect models. Epidemiol. Method., 9 0 (1), January 2020

    2020

  29. [37]

    Luedtke, A., Carone, M., and van der Laan, M. J. An Omnibus Non-Parametric Test of Equality in Distribution for Unknown Functions . Journal of the Royal Statistical Society Series B: Statistical Methodology, 81 0 (1): 0 75--99, 11 2018. ISSN 1369-7412. doi:10.1111/rssb.12299. ...

    2018 doi

  30. [38]

    Algorithmic fairness: Choices, assumptions, and definitions

    Mitchell, S., Potash, E., Barocas, S., D'Amour, A., and Lum, K. Algorithmic fairness: Choices, assumptions, and definitions. Annu. Rev. Stat. Appl., 8 0 (1): 0 141--163, March 2021

    2021

  31. [39]

    P., Perelman, A., Ramesh, A., Clark, A., and et al

    OpenAI, :, Hurst, A., Lerer, A., Goucher, A. P., Perelman, A., Ramesh, A., Clark, A., and et al. Gpt-4o system card, 2024. URL https://arxiv.org/abs/2410.21276. Accessed on March 28, 2025

    2024 arXiv

  32. [40]

    S., Balasubramanian, V

    Panda, P., Kancheti, S. S., Balasubramanian, V. N., and Sinha, G. Interpretable model drift detection. In Proceedings of the 7th Joint International Conference on Data Science & Management of Data (11th ACM IKDD CODS and 29th COMAD), CODS-COMAD '24, pp.\ 1–9, New York, NY, USA...

    2024

  33. [41]

    Scikit-learn: Machine learning in P ython

    Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. Scikit-learn: Machine learning in P ython. Journal of ...

    2011

  34. [42]

    and Ramdas, A

    Podkopaev, A. and Ramdas, A. Tracking the risk of a deployed model and detecting harmful distribution shifts. In International Conference on Learning Representations, 2022

    2022

  35. [43]

    Quinonero-Candela, J., Sugiyama, M., Schwaighofer, A., and Lawrence, N. D. Dataset Shift in Machine Learning. The MIT Press, 2009

    2009

  36. [44]

    T., Santiago, E., Mu, J., and Heckerman, D

    Quintas-Martinez, V., Bahadori, M. T., Santiago, E., Mu, J., and Heckerman, D. Multiply-robust causal change attribution. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F. (eds.), Proceedings of the 41st International Confer...

    2024

  37. [45]

    R., and Bauer, S

    Quinzan, F., Soleymani, A., Jaillet, P., Rojas, C. R., and Bauer, S. DRCFS : Doubly robust causal feature selection. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), Proceedings of the 40th International Conference on Machine Learning...

    2023

  38. [46]

    Failing loudly: An empirical study of methods for detecting dataset shift

    Rabanser, S., G\" u nnemann, S., and Lipton, Z. Failing loudly: An empirical study of methods for detecting dataset shift. In Wallach, H., Larochelle, H., Beygelzimer, A., d Alch\' e -Buc, F., Fox, E., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, ...

    2019

  39. [47]

    R., and van der Schaar, M

    Rauba, P., Seedat, N., Luyten, M. R., and van der Schaar, M. Context-aware testing: A new paradigm for model testing with large language models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=d75qCZb7TX

    2024

  40. [48]

    Rubinstein, M., Branson, Z., and Kennedy, E. H. Heterogeneous interventional effects with multiple mediators: Semiparametric and nonparametric approaches. J. Causal Inference, 11 0 (1), July 2023

    2023

  41. [49]

    Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter

    Sanh, V., Debut, L., Chaumond, J., and Wolf, T. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. ArXiv, abs/1910.01108, 2019

    1910 arXiv

  42. [50]

    A hierarchical decomposition for explaining ML performance discrepancies

    Singh, H., Xia, F., Subbaswamy, A., Gossmann, A., and Feng, J. A hierarchical decomposition for explaining ML performance discrepancies. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, September 2024

    2024

  43. [51]

    An empirical analysis of backward compatibility in machine learning systems

    Srivastava, M., Nushi, B., Kamar, E., Shah, S., and Horvitz, E. An empirical analysis of backward compatibility in machine learning systems. In KDD, August 2020

    2020

  44. [52]

    Steyerberg, E. W. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. Springer, New York, NY, 2009

    2009

  45. [53]

    C., and Saria, S

    Subbaswamy, A., Sahiner, B., Petrick, N., Pai, V., Adams, R., Diamond, M. C., and Saria, S. A data-driven framework for identifying patient subgroups on which an AI /machine learning model may underperform. NPJ Digit. Med., 7 0 (1): 0 334, November 2024

    2024

  46. [54]

    Direct importance estimation with model selection and its application to covariate shift adaptation

    Sugiyama, M., Nakajima, S., Kashima, H., Buenau, P., and Kawanabe, M. Direct importance estimation with model selection and its application to covariate shift adaptation. Adv. Neural Inf. Process. Syst., 2007

    2007

  47. [55]

    M., Ghassemi, M., and Ustun, B

    Suriyakumar, V. M., Ghassemi, M., and Ustun, B. When personalization harms performance: Reconsidering the use of group attributes in prediction. Proc. Int. Conf. Mach. Learn., 2023

    2023

  48. [56]

    van der Vaart, A. W. Asymptotic Statistics. Cambridge University Press, October 1998

    1998

  49. [57]

    and Walther, G

    Wager, S. and Walther, G. Adaptive concentration of regression trees, with application to random forests. arXiv preprint arXiv:1503.06388, 2015

    2015 arXiv

  50. [58]

    D., Gilbert, P

    Williamson, B. D., Gilbert, P. B., Carone, M., and Simon, N. Nonparametric variable importance assessment using machine learning techniques. Biometrics, 77 0 (1): 0 9--22, 2021. doi:https://doi.org/10.1111/biom.13392. URL https://onlinelibrary.wiley.com/doi/abs/10.1111/biom.13392

    2021 doi

  51. [59]

    Explaining medical AI performance disparities across sites with confounder shapley value analysis

    Wu, E., Wu, K., and Zou, J. Explaining medical AI performance disparities across sites with confounder shapley value analysis. November 2021. URL http://arxiv.org/abs/2111.08168

    2021 arXiv

  52. [60]

    Change is hard: A closer look at subpopulation shift

    Yang, Y., Zhang, H., Katabi, D., and Ghassemi, M. Change is hard: A closer look at subpopulation shift. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 ...

    2023

  53. [61]

    v., Hinz, O., and Feuerriegel, S

    Zahn, M. v., Hinz, O., and Feuerriegel, S. Locating disparities in machine learning. In 2023 IEEE International Conference on Big Data (BigData), pp.\ 1883--1894, 2023. doi:10.1109/BigData59044.2023.10386485

    2023

  54. [62]

    `` W hy did the model fail?'': Attributing model performance changes to distribution shifts

    Zhang, H., Singh, H., Ghassemi, M., and Joshi, S. `` W hy did the model fail?'': Attributing model performance changes to distribution shifts. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), Proceedings of the 40th International Conf...

    2023

  55. [63]

    Kernel-based conditional independence test and application in causal discovery

    Zhang, K., Peters, J., Janzing, D., and Sch\" o lkopf, B. Kernel-based conditional independence test and application in causal discovery. In Proceedings of the Twenty-Seventh Conference on Uncertainty in Artificial Intelligence, UAI'11, pp.\ 804–813, Arlington, Virginia, USA, ...

    2011

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.