Label-free estimation of clinically relevant performance metrics under distribution shifts

Alceu Bissoto; Lisa M. Koch; Tim Fl\"uhmann; Trung-Dung Hoang

REVIEW 3 major objections 4 minor 23 references

Label-free estimation of clinically relevant performance metrics under distribution shifts

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proposes label-free estimators that recover the full confusion matrix on unlabelled chest X-ray data, and shows they track clinically relevant metrics under real-world and simulated covariate shifts, while prevalence shifts…

desk verdict A useful empirical extension of ATC/DoC to confusion-matrix metrics, honestly benchmarked but with an abstract that oversells reliability. read the letter →

arxiv 2507.22776 v1 pith:344HISZR submitted 2025-07-30 cs.LG

Tim Flühmann , Alceu Bissoto , Trung-Dung Hoang , Lisa M. Koch This is my paper · ORCID

classification cs.LG

keywords performanceestimationlabel-freeconfusionmatrixdistributionshiftchestX-rayprevalencecovariatepostmarketsurveillance

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to show that a deployed medical image classifier's clinically relevant performance, including precision, recall, F1, balanced accuracy, specificity, and AUC, can be estimated on unlabelled target data by extending two confidence-based accuracy estimators to estimate the full confusion matrix. It introduces CM-ATC and CM-DoC, which separately calibrate positive and negative confidence scores and use the calibration to estimate positive and negative predictive values. On chest X-ray datasets with real-world and simulated covariate shifts, the proposed estimators track realized metrics with lower error than the existing confidence-based CBPE baseline. The same experiments expose a hard limit: under prevalence shifts, all tested estimators degrade, especially for prevalence-dependent metrics. If correct, the methods offer a low-cost postmarket surveillance tool, provided shift monitoring accompanies them.

What carries the argument

The machinery is the class-specific confidence-score calibration carried over from validation to test. For CM-ATC the carrying objects are two thresholds, $t^+_{ATC}$ and $t^-_{ATC}$, where the fraction of validation scores above each threshold reproduces the validation PPV and NPV, and the same fractions computed on test scores are taken as the test PPV and NPV. For CM-DoC the carrying objects are the offsets between mean confidence on positive validation predictions and positive test predictions, and the analogous offset for negative predictions, which are applied to the validation PPV and NPV. The reconstructed confusion matrix then turns any counting metric into a label-free estimate.

What would settle it

Take a fixed classifier and a fixed feature distribution, resample its test set to shift only the positive-class prevalence away from the validation level while keeping class-conditional features fixed, and then compare estimated PPV from CM-ATC or CM-DoC with realized PPV; the paper's Figure 4 already shows such divergence, and a reader could reproduce it by reporting estimated minus realized PPV across prevalence levels.

Watch

Extended reading notes

Core claim

The central claim is that two established label-free accuracy predictors, Average Threshold Confidence and Difference of Confidences, can be revised from accuracy-only methods into confusion-matrix estimators. CM-ATC learns class-specific thresholds on validation such that the fraction of positive-class scores above a threshold matches validation PPV, and analogously for NPV, then applies these thresholds to test scores. CM-DoC offsets test mean confidences by the gap between validation and test mean confidences within each predicted class, producing PPV and NPV estimates. From these values the paper reconstructs true-positive, false-positive, true-negative, and false-negative counts, and hence all counting metrics plus a multi-threshold AUC. In the chest X-ray benchmark, CM-ATC gives the lowest estimation error on real-world shifts, and the methods capture the decline in performance under simulated covariate shift; however, prevalence shift breaks the transfer assumptions and all methods fail on prevalence-dependent metrics.

Load-bearing premise

The estimates inherit the assumption that the relationship between confidence scores and true positive and negative rates measured on validation still holds on test data after a simple threshold or offset correction, and this fails when the prevalence of the positive class changes on its own.

Editorial extensions

If this is right

Deployed chest X-ray classification models can be monitored for precision, recall, F1, and AUC without ground-truth labels, using only confidence scores and validation statistics.
Because the estimators only require computing thresholds or averages on validation and test scores, they add negligible computational overhead at inference time.
Performance monitoring should be paired with distribution-shift detection and prevalence monitoring, since label shifts cause all evaluated estimators to misreport.
Under covariate shifts that lower confidence on minority subgroups, the estimators still capture the direction and rough magnitude of performance decline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension is to compose these estimators with a label-shift correction step; if prevalence is estimated first, the failure mode under prevalence shift might shrink substantially.
The paper's comparison suggests that the benefit of class-specific treatment grows when a shift affects positive and negative predictions asymmetrically; one could construct shifts affecting only one class and compare CM-ATC against the naive ATC baseline.
Because CM-ATC depends on a validation-learned threshold, the stability of that threshold across distributions could itself serve as a distribution-shift detector, connecting performance estimation to shift detection.
The confusion-matrix formulation extends naturally to multi-class and multi-label settings by estimating per-class positive predictive values, though the paper only demonstrates binary classification.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read this if you work on label-free model monitoring. The core contribution is real: they extend ATC and DoC from accuracy to PPV/NPV, giving the full confusion matrix, and benchmark on chest X-ray data with real-world shifts plus controlled covariate and prevalence shifts. Code is up, i.d. results are strong, and CM-ATC does beat the CBPE baseline on o.o.d. accuracy, PPV, and F1 by a wide margin. The paper is also unusually honest: Section 6 and Figure 4 explicitly show all methods failing under prevalence shifts. That alone makes it worth a look.

The soft spots are mostly in the packaging. The abstract says the methods 'reliably predicted' clinically relevant counting metrics under distribution shifts. The paper's own numbers don't support 'reliably': o.o.d. MAE for PPV is about 0.15 and F1 about 0.14; under prevalence shifts the estimates systematically diverge. That's not reliable in any practical monitoring sense. The extension in Eqs (4) and (5) is heuristic, with no stated condition for when the validation-to-test transfer of score statistics holds. The authors don't pretend otherwise, but the abstract overreaches.

Also, the 'CM-ATC performing best' claim is based on averaged MAE without significance testing. In several in-distribution rows plain ATC and DoC have lower MAE. So the ranking is suggestive, not conclusive.

None of this kills the contribution. The benchmark is careful, the methods are simple to implement, and the failure-mode analysis is valuable. The right fix is to scope the claims: say the methods work on moderate covariate shifts and fail on prevalence shifts, and give practical diagnostics (e.g., calibration error, prevalence estimate) for when to trust them.

Who should read it: people building postmarket surveillance for medical AI, and anyone comparing confidence-based estimators. It deserves a serious referee; a good reviewer will ask for the assertion fixes and maybe a significance analysis, but the empirical foundation is there.

Referee Report

3 major / 4 minor

Summary. The paper proposes two new confidence-based estimators, CM-ATC and CM-DoC, that generalize ATC and DoC to directly estimate the confusion matrix on unlabeled test data, and therefore any counting metric such as PPV, recall, and F1. The methods are evaluated on chest X-ray datasets (CheXpertPlus, PadChest, NIH) under real-world distribution shifts and under simulated covariate and prevalence shifts, with CBPE and naive ATC/DoC as baselines. The results indicate that the CM estimators, especially CM-ATC, are the most accurate in real-world shifts and covariate shift, but all methods degrade substantially under prevalence shift.

Significance. The benchmark is valuable and the code is publicly available, which enables reproducibility. The paper addresses an important clinical monitoring problem and provides a systematic comparison that moves beyond accuracy prediction. The main weakness is the mismatch between the abstract's claim of reliable prediction and the observed errors and failure modes, particularly under prevalence shift; the empirical contributions are nevertheless useful for postmarket surveillance research.

major comments (3)

[Abstract; §5.1; Table S2] The abstract's statement that the proposed methods 'reliably predicted clinically relevant counting metrics ... under distribution shifts' is not supported by the reported results. In Table S2, the best o.o.d. MAEs for PPV and F1 are 0.148 and 0.140, and Fig. 4 shows that under pure prevalence shift CM-ATC and CM-DoC diverge systematically from realized values. The claim should be qualified to the demonstrated scope (real-world shifts and covariate shift), and the prevalence-shift failure should be reflected in the abstract. The discussion already acknowledges this limitation; the abstract and Section 5.1 should be aligned accordingly.
[§3, Eqs. (4)-(5)] The proposed estimators are introduced without stating the statistical assumption that justifies transferring validation-set thresholds/offsets to the test set. For CM-ATC, the fraction of scores above t+_ATC on I+_test is taken as test PPV; for CM-DoC, the difference in mean confidence is used as a shift estimate. No condition is given under which these quantities are invariant or how they should transform under a specified shift model. The paper would be substantially stronger if a formal assumption were stated (e.g., a location-shift or quantile-coupling condition) and the prevalence-shift failure in Fig. 4 was explained with reference to it. Alternatively, the methods should be explicitly characterized as empirical heuristics and the claims adjusted accordingly.
[§5.1; Table S2] The conclusion that 'CM-ATC performing best' is based on MAE values averaged over metrics and datasets without significance testing. In the in-distribution rows of Table S2, plain ATC and DoC have lower MAE than CM-ATC for recall, AUC, accuracy, PPV, and F1 (e.g., recall: ATC 0.0252 vs. CM-ATC 0.0400; PPV: ATC 0.0084 vs. CM-ATC 0.0088). To support the claim, the authors should report per-metric comparisons with confidence intervals or paired significance tests, or soften the wording.

minor comments (4)

[Abstract] There is a typo: 'covariateand' should be 'covariate and'.
[Fig. 2 caption] The phrase 'On average, CM methods outperformed' should refer to a specific average (e.g., mean MAE across datasets and metrics) and ideally to a statistical test.
[§4.2] The phrase 'keep the class distribution consistent between the validation and test sets' is ambiguous because the majority fraction of the artefact is varied; clarify that the positive-class prevalence is held fixed.
[Table S2] The column header uses 'Precision' instead of 'PPV'; unify terminology with the main text.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: the estimators transfer validation-set statistics to unlabeled test confidence scores, and the self-citations are not load-bearing.

full rationale

The derivation chain is self-contained rather than circular. CBPE in Eq. (1) follows directly from the calibration identity P(Y=1|S=s)=s, and Eq. (2) is only confusion-matrix bookkeeping. CM-ATC in Eq. (4) learns a validation threshold t+_ATC so that the score fraction above it equals validation PPV and then evaluates the same fraction on test scores; this is a transfer assumption about threshold stability, not an identity that feeds test labels back into the estimator. CM-DoC in Eq. (5) shifts validation PPV by the change in mean confidence between validation and test positive predictions; again, the target PPV is not used in constructing the estimate. The paper's controlled prevalence-shift experiments (Fig. 4) show that these transfer assumptions break under label shift, which is consistent with an honest domain assumption rather than circularity. The two self-citations involving the last author ([19], used for the spurious-correlation covariate-shift construction, and [21], cited for the general recommendation to combine monitoring with shift detection) support experimental design and discussion points, not the estimators themselves; the core extension builds on external prior work [4,7,8] and is benchmarked against external chest X-ray datasets. No equation in the paper reduces by construction to its own input, and no fitted test-label quantity is renamed as a prediction.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The core methods rest on two domain assumptions about confidence calibration and how distribution shifts affect the confidence-output relationship. No free parameters are fit to test labels; thresholds and offsets come from validation. No new entities are introduced.

assumptions (2)

domain assumption Model confidence scores are calibrated on the validation set, so that mean confidence approximates the true positive rate or PPV for positive predictions.
CBPE (Eq. 1) and the proposed CM methods (Eqs. 4-5) all rely on confidence scores being meaningful probability estimates. The paper notes calibration errors as a source of failure in Section 5.1.
domain assumption The relationship between confidence scores and true outcomes changes only through a simple shift (DoC) or through a threshold-preserving transformation (ATC) between validation and test.
CM-DoC (Eq. 5) assumes the gap between PPV and mean confidence is identical on val and test, while CM-ATC (Eq. 4) assumes the threshold learned on val remains valid. This assumption is violated under prevalence shift, as shown in Section 5.3.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Label-free estimation of clinically relevant performance metrics under distribution shifts." pith.science (2026). https://pith.science/paper/344HISZR

@misc{pith2026250722776,
  author       = {Pith},
  title        = {Pith review of: Label-free estimation of clinically relevant performance metrics under distribution shifts},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/344HISZR}},
  note         = {Machine review of arXiv:2507.22776}
}

read the original abstract

Performance monitoring is essential for safe clinical deployment of image classification models. However, because ground-truth labels are typically unavailable in the target dataset, direct assessment of real-world model performance is infeasible. State-of-the-art performance estimation methods address this by leveraging confidence scores to estimate the target accuracy. Despite being a promising direction, the established methods mainly estimate the model's accuracy and are rarely evaluated in a clinical domain, where strong class imbalances and dataset shifts are common. Our contributions are twofold: First, we introduce generalisations of existing performance prediction methods that directly estimate the full confusion matrix. Then, we benchmark their performance on chest x-ray data in real-world distribution shifts as well as simulated covariate and prevalence shifts. The proposed confusion matrix estimation methods reliably predicted clinically relevant counting metrics on medical images under distribution shifts. However, our simulated shift scenarios exposed important failure modes of current performance estimation techniques, calling for a better understanding of real-world deployment contexts when implementing these performance monitoring techniques for postmarket surveillance of medical AI models.

Figures

Figures reproduced from arXiv: 2507.22776 by the authors.

**Figure 2.** (a-g) MAE evaluated on i.d. and o.o.d. data represented by solid and lined bars, respectively. (h) shows the mean RBS and ACE over all models. On average, CM methods outperformed the other estimators. Exact numerical values are reported in the Supplementary Table S2. encouraging the model to rely on the spurious feature. Test sets are constructed by sampling 1000 images with varying proportions of majority samples f… view at source ↗

**Figure 3.** (a-g) The estimated performance metrics capture the decline in performance under the simulated covariate shift, yet show overconfident performance towards minority groups in the test set. (h) Calibration error rises significantly for the minority group. To estimate metrics beyond accuracy, only CBPE has been proposed before. While estimation of balanced accuracy, specificity, and AUC performed comparably to accura… view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: (a–d) Prevalence-independent metrics remain constant under label shift; naive estimators perform best, whereas confusion-based ones remain prevalencesensitive. (e–g) Prevalence-dependent metrics vary under label shifts; confusionmatrix-based methods capture these cha…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 21 canonical work pages

[1]

The Lancet Digital Health 1(6), e271–e297 (Oct 2019)

Liu, X., Faes, L., Kale, A.U., Wagner, S.K., Fu, D.J., Bruynseels, A., Mahendiran, T., Moraes, G., Shamdas, M., Kern, C., Ledsam, J.R., Schmid, M.K., Balaskas, K., Topol, E.J., Bachmann, L.M., Keane, P.A., Denniston, A.K.: A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic r...

work page 2019
[2]

npj Digital Medicine5(1), 1–8 (Apr 2022)

Varoquaux, G., Cheplygina, V.: Machine learning for medical imaging: method- ological failures and recommendations for the future. npj Digital Medicine5(1), 1–8 (Apr 2022)

work page 2022
[3]

In: ACM conference on health, inference, and learning (CHIL)

Oakden-Rayner, L., Dunnmon, J., Carneiro, G., Ré, C.: Hidden stratification causes clinically meaningful failures in machine learning for medical imaging. In: ACM conference on health, inference, and learning (CHIL). pp. 151–159 (2020)

work page 2020
[4]

In: International Conference on Computer Vi- sion (ICCV)

Guillory, D., Shankar, V., Ebrahimi, S., Darrell, T., Schmidt, L.: Predicting with confidence on unseen distributions. In: International Conference on Computer Vi- sion (ICCV). pp. 1134–1144 (2021)

work page 2021
[5]

In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Fan, W., Davidson, I.: Reverse testing: an efficient framework to select amongst classifiers under sample selection bias. In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. p. 147–156 (2006)

work page 2006
[6]

arXiv preprint arXiv:2401.08348 (2024)

Białek, J., Kuberski, W., Perrakis, N., Bifet, A.: Estimating model performance under covariate shift without labels. arXiv preprint arXiv:2401.08348 (2024)

work page arXiv 2024
[7]

In: NeurIPS Workshop on Distribution Shifts: Connecting Methods and Applications (2021)

Garg, S., Balakrishnan, S., Lipton, Z.C., Neyshabur, B., Sedghi, H.: Leveraging unlabeled data to predict out-of-distribution performance. In: NeurIPS Workshop on Distribution Shifts: Connecting Methods and Applications (2021)

work page 2021
[8]

In: Interna- tional Conference on Medical Image Computing and Computer-Assisted Interven- tion (MICCAI)

Li, Z., Kamnitsas, K., Islam, M., Chen, C., Glocker, B.: Estimating model per- formance under domain shifts with class-specific confidence scores. In: Interna- tional Conference on Medical Image Computing and Computer-Assisted Interven- tion (MICCAI). pp. 693–703. Springer (2022)

work page 2022

Show all 23 references

[9]

Journal of Artificial Intelli- gence Research82, 209–240 (2025)

Kivimäki, J., Nurminen, J.K., Białek, J., Kuberski, W.: Confidence-based estima- tors for predictive performance in model monitoring. Journal of Artificial Intelli- gence Research82, 209–240 (2025)

2025
[10]

arXiv preprint arXiv:2505.05295 (2025) 10 Flühmann et al

Kivimäki, J., Białek, J., Kuberski, W., Nurminen, J.K.: Performance estimation in binary classification using calibrated confidence. arXiv preprint arXiv:2505.05295 (2025) 10 Flühmann et al

2025
[11]

Elsahar, H., Gallé, M.: To annotate or not? predicting performance drop under domainshift.In:ConferenceonEmpiricalMethodsinNaturalLanguageProcessing (EMNLP). pp. 2163–2173 (2019)

2019
[12]

Advances in Neural Information Processing Systems (NeurIPS)35, 19274–19289 (2022)

Baek, C., Jiang, Y., Raghunathan, A., Kolter, J.Z.: Agreement-on-the-line: Pre- dicting the performance of neural networks under distribution shift. Advances in Neural Information Processing Systems (NeurIPS)35, 19274–19289 (2022)

2022
[13]

Nature methods21(2), 195–212 (2024)

Maier-Hein, L., Reinke, A., Godau, P., Tizabi, M.D., Buettner, F., Christodoulou, E., Glocker, B., Isensee, F., Kleesiek, J., Kozubek, M., et al.: Metrics reloaded: recommendations for image analysis validation. Nature methods21(2), 195–212 (2024)

2024
[14]

In: International Conference on Machine Learning (ICML)

Guo, C., Pleiss, G., Sun, Y., Weinberger, K.Q.: On calibration of modern neural networks. In: International Conference on Machine Learning (ICML). pp. 1321– 1330 (2017)

2017
[15]

In: International Conference on Learning Representations (ICLR) (2017)

Hendrycks, D., Gimpel, K.: A baseline for detecting misclassified and out-of- distribution examples in neural networks. In: International Conference on Learning Representations (ICLR) (2017)

2017
[16]

arXiv preprint arXiv:2405.19538 (2024)

Chambon, P., Delbrouck, J.B., Sounack, T., Huang, S.C., Chen, Z., Varma, M., Truong, S.Q., Chuong, C.T., Langlotz, C.P.: Chexpert plus: Augmenting a large chest x-ray dataset with text radiology reports, patient demographics and addi- tional image formats. arXiv preprint arXiv...

2024 arXiv
[17]

Medical Image Anal- ysis 66, 101797 (2020)

Bustos, A., Pertusa, A., Salinas, J.M., De La Iglesia-Vaya, M.: Padchest: A large chest x-ray image dataset with multi-label annotated reports. Medical Image Anal- ysis 66, 101797 (2020)

2020
[18]

In: Conference on Computer Vision and Pattern Recognition (CVPR)

Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classi- fication and localization of common thorax diseases. In: Conference on Computer Vision and Pattern Recognition (CVPR). p...

2017
[19]

Sun, S., Koch, L.M., Baumgartner, C.F.: Right for the wrong reason: Can inter- pretable ml techniques detect spurious correlations? In: Medical Image Computing and Computer Assisted Interventions (MICCAI) (2023)

2023
[20]

In: Medical Imaging with Deep Learning (MIDL)

Cohen, J.P., Hashir, M., Brooks, R., Bertrand, H.: On the limits of cross-domain generalization in automated x-ray prediction. In: Medical Imaging with Deep Learning (MIDL). pp. 136–155 (2020)

2020
[21]

npj Digital Medicine (2024)

Koch, L.M., Baumgartner, C.F., Berens, P.: Distribution shift detection for the postmarket surveillance of medical ai algorithms: A retrospective simulation study. npj Digital Medicine (2024)

2024
[22]

arXiv preprint arXiv:2411.07940 (2024)

Roschewitz, M., Mehta, R., Jones, C., Glocker, B.: Automatic dataset shift identification to support safe deployment of medical imaging ai. arXiv preprint arXiv:2411.07940 (2024)

2024 arXiv
[23]

In: International Conference on Machine Learning (ICML)

Alexandari, A., Kundaje, A., Shrikumar, A.: Maximum likelihood with bias- corrected calibration is hard-to-beat at label shift adaptation. In: International Conference on Machine Learning (ICML). pp. 222–232 (2020) Supplementary Material A Calibration 0.0 0.2 0.4 MAE a bal. ac...

2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

The Lancet Digital Health 1(6), e271–e297 (Oct 2019)

Liu, X., Faes, L., Kale, A.U., Wagner, S.K., Fu, D.J., Bruynseels, A., Mahendiran, T., Moraes, G., Shamdas, M., Kern, C., Ledsam, J.R., Schmid, M.K., Balaskas, K., Topol, E.J., Bachmann, L.M., Keane, P.A., Denniston, A.K.: A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic r...

work page 2019

[2] [2]

npj Digital Medicine5(1), 1–8 (Apr 2022)

Varoquaux, G., Cheplygina, V.: Machine learning for medical imaging: method- ological failures and recommendations for the future. npj Digital Medicine5(1), 1–8 (Apr 2022)

work page 2022

[3] [3]

In: ACM conference on health, inference, and learning (CHIL)

Oakden-Rayner, L., Dunnmon, J., Carneiro, G., Ré, C.: Hidden stratification causes clinically meaningful failures in machine learning for medical imaging. In: ACM conference on health, inference, and learning (CHIL). pp. 151–159 (2020)

work page 2020

[4] [4]

In: International Conference on Computer Vi- sion (ICCV)

Guillory, D., Shankar, V., Ebrahimi, S., Darrell, T., Schmidt, L.: Predicting with confidence on unseen distributions. In: International Conference on Computer Vi- sion (ICCV). pp. 1134–1144 (2021)

work page 2021

[5] [5]

In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Fan, W., Davidson, I.: Reverse testing: an efficient framework to select amongst classifiers under sample selection bias. In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. p. 147–156 (2006)

work page 2006

[6] [6]

arXiv preprint arXiv:2401.08348 (2024)

Białek, J., Kuberski, W., Perrakis, N., Bifet, A.: Estimating model performance under covariate shift without labels. arXiv preprint arXiv:2401.08348 (2024)

work page arXiv 2024

[7] [7]

In: NeurIPS Workshop on Distribution Shifts: Connecting Methods and Applications (2021)

Garg, S., Balakrishnan, S., Lipton, Z.C., Neyshabur, B., Sedghi, H.: Leveraging unlabeled data to predict out-of-distribution performance. In: NeurIPS Workshop on Distribution Shifts: Connecting Methods and Applications (2021)

work page 2021

[8] [8]

In: Interna- tional Conference on Medical Image Computing and Computer-Assisted Interven- tion (MICCAI)

Li, Z., Kamnitsas, K., Islam, M., Chen, C., Glocker, B.: Estimating model per- formance under domain shifts with class-specific confidence scores. In: Interna- tional Conference on Medical Image Computing and Computer-Assisted Interven- tion (MICCAI). pp. 693–703. Springer (2022)

work page 2022