Robustifying and Selecting Cohort-Appropriate Prognostic Models under Distributional Shifts

Angelos G. Koulouras; Carol Gao; Dimitris Bertsimas; Georgios Antonios Margonis

arxiv: 2604.16537 · v1 · submitted 2026-04-16 · 📊 stat.ME · cs.AI· stat.AP

Robustifying and Selecting Cohort-Appropriate Prognostic Models under Distributional Shifts

Dimitris Bertsimas , Carol Gao , Angelos G. Koulouras , Georgios Antonios Margonis This is my paper

Pith reviewed 2026-05-10 09:49 UTC · model grok-4.3

classification 📊 stat.ME cs.AIstat.AP

keywords prognostic modelsexternal validationdistributional shiftsKL divergencecalibrationmeta-analysis weightingcohort selectiontransportability

0 comments

The pith

Distributional mismatches between cohorts degrade the calibration of prognostic models, but meta-analysis weighting and similarity-based selection can improve transportability.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that external calibration success does not ensure a prognostic model will generalize because calibration deteriorates as the covariate and outcome distributions of training and validation cohorts diverge. It demonstrates this link by showing that higher Kullback-Leibler divergence between cohorts correlates with higher Integrated Calibration Index values across six real-world surgical groups. To address the problem, the authors develop a developer strategy that weights models toward a meta-analysis-derived distribution to achieve better average performance and a user strategy that measures outcome similarity to select the most suitable published model for a given target cohort.

Core claim

External calibration worsens as distributional mismatch increases, with higher KL divergence associated with higher ICI in both surgery-alone and surgery-plus-chemotherapy cohorts. Training the best-on-average model by tuning toward a meta-analysis-derived covariate and outcome distribution improves calibration in most settings without materially affecting discrimination, with clearest benefit on the aggregated external population. Models developed in more similar cohorts achieve lower ICI and greater clinical utility on decision curve analysis.

What carries the argument

Kullback-Leibler divergence quantifies mismatch in covariates and outcomes between cohorts and is linked to the Integrated Calibration Index to measure calibration degradation, while meta-analysis-informed weighting adjusts model parameters toward a broader target distribution and a similarity measure ranks existing models for a new cohort.

If this is right

Calibration performance in external validation is directly tied to the degree of distributional similarity between cohorts.
Meta-analysis-informed model weighting enhances calibration across diverse external settings without harming discrimination.
Cohort similarity measures allow selection of models that deliver lower ICI and higher decision curve analysis utility for specific target populations.
The benefit of meta-weighting appears most clearly when models are assessed on aggregated external data rather than individual cohorts.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Clinicians could adopt routine checks of outcome distribution similarity before applying any published prognostic model to their local patients.
Model developers might shift priority from optimizing on narrow single-center data toward constructing representative meta-distributions from the outset.
The same mismatch-quantification and selection logic may extend to other clinical prediction tasks that face cohort shifts, such as risk models in cardiology or oncology follow-up.

Load-bearing premise

The meta-analysis-derived covariate and outcome distribution serves as a reasonable approximation of the broader target population for developing the best-on-average model.

What would settle it

A new set of external cohorts in which higher KL divergence fails to correspond to higher ICI, or in which meta-analysis weighting produces no calibration improvement on the aggregated population, would falsify the central dependence on distributional mismatch.

read the original abstract

External validation is widely regarded as the gold standard for prognostic model evaluation. In this study, we challenge the assumption that successful external calibration guarantees model generalizability and propose two complementary strategies to improve transportability of prognostic models across cohorts. Using six real-world surgical cohorts from tertiary academic centers, we tested whether successful external calibration depends largely on similarity in covariates and outcomes between training and validation cohorts, quantified using Kullback-Leibler (KL) divergence, with calibration assessed by the Integrated Calibration Index (ICI). From the model-developer's perspective, we trained the "best-on-average" prognostic model by tuning toward a meta-analysis-derived covariate and outcome distribution as an approximation of the broader target population. From the end-user perspective, we proposed a simple measure for cohort outcome similarity to identify, among published models, the one most suitable for a given target cohort in terms of both calibration and clinical utility. External calibration worsened as distributional mismatch increased. Higher KL divergence was associated with higher ICI in both surgery-alone (Spearman $\rho=0.614$, $p=0.004$) and surgery + adjuvant chemotherapy cohorts (Spearman $\rho=0.738$, $p<0.001$). Meta-analysis-informed weighting improved calibration in most settings without materially affecting discrimination, with the clearest benefit when evaluated on the aggregated external population ($p=0.037$). Models developed in more similar cohorts achieved lower ICI in surgery-alone (Spearman $\rho=0.803$, $p<0.001$) and surgery + adjuvant chemotherapy cohorts (Spearman $\rho=0.737$, $p<0.001$), and provided greater clinical utility on DCA.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

This paper shows that KL divergence tracks calibration failure across surgical cohorts and tests two simple fixes—meta-weighted training and similarity-based selection—with real-data correlations and modest gains.

read the letter

The main thing to know is that higher KL divergence between cohorts lines up with worse integrated calibration index in both surgery-only and chemo-augmented settings, and the authors report two workable responses: reweighting a model toward a meta-analysis distribution and picking published models by outcome similarity. Both steps are demonstrated on the same six tertiary-center cohorts, with the weighting step showing a calibration lift on the pooled external set (p=0.037) while leaving discrimination largely unchanged. The selection rule also tracks lower ICI and better decision-curve utility when cohorts are closer in distribution. That is the concrete, usable part. The correlations themselves (ρ=0.614 and 0.738) are new empirical observations for this clinical domain, even if the underlying divergence and calibration metrics are standard. The work is straightforward and addresses a routine problem in prognostic modeling without overclaiming. The soft spot is the meta-analysis distribution itself. It is treated as a proxy for the broader target population, yet the paper gives no external check on whether that meta-sample is materially different from the six study sites or whether publication bias has already narrowed it. If the meta is drawn mostly from similar academic centers, the reported gains are closer to internal averaging than to genuine transportability. A second minor issue is that the abstract leaves model-training details, exact cohort definitions, and any multiple-comparison adjustment implicit, which makes the p-values harder to interpret at face value. This is the kind of applied methods paper that medical statisticians and surgical-outcomes researchers will actually read and try. It is not paradigm-shifting, but the empirical grounding is real enough that a serious editor should send it to referees rather than desk-reject. I would recommend peer review with requests for clearer justification of the meta proxy and fuller methods reporting.

Referee Report

2 major / 2 minor

Summary. The paper claims that external calibration of prognostic models worsens with greater distributional mismatch between training and validation cohorts, quantified via KL divergence and assessed by the Integrated Calibration Index (ICI). Using six real-world surgical cohorts, it reports positive Spearman correlations between KL and ICI in both surgery-alone and surgery-plus-chemotherapy settings. It further proposes a developer-side strategy of reweighting models toward a meta-analysis-derived joint covariate-outcome distribution to produce a 'best-on-average' model, and an end-user strategy of selecting published models based on outcome similarity to the target cohort; both are shown to improve calibration (with p=0.037 on aggregated external data for weighting) while preserving discrimination and enhancing decision-curve utility.

Significance. If the empirical associations and weighting gains hold after addressing the noted gaps, the work is significant for prognostic modeling in clinical statistics and medical decision-making. It supplies concrete, reproducible evidence that transportability failures are tied to measurable distributional shifts and offers two practical, complementary remedies (reweighting at development time and similarity-based selection at deployment) that could be adopted in oncology and surgical risk modeling without requiring new data collection.

major comments (2)

[Abstract (developer-side strategy) and Results (aggregated external evaluation)] The central claim that meta-analysis-informed weighting yields a model that better approximates the broader target population (and thereby improves calibration, p=0.037 on aggregated external data) rests on the untested assumption that the meta-analysis distribution is not itself dominated by cohorts similar to the six tertiary-center study sites. No sensitivity analysis, external validation of the meta-analysis representativeness, or comparison against a truly held-out population is provided; if the meta-analysis largely overlaps with the observed sample, the reported gains may reflect interpolation within internal heterogeneity rather than genuine transportability.
[Abstract and Results (correlation analyses)] Multiple Spearman rank correlations are reported with associated p-values (ρ=0.614, p=0.004; ρ=0.738, p<0.001; ρ=0.803, p<0.001; ρ=0.737, p<0.001) linking KL divergence, ICI, and model similarity, yet no adjustment for multiple comparisons or pre-specified analysis plan is described. This raises the possibility that the reported significance levels are inflated, directly affecting the strength of the evidence for the KL-ICI relationship that underpins both proposed strategies.

minor comments (2)

[Methods] The manuscript provides insufficient detail on model training procedures (e.g., hyperparameter tuning, feature selection, handling of missing data), exact definitions and inclusion criteria for the six cohorts, and the precise construction of the meta-analysis-derived weighting distribution. These omissions limit reproducibility and independent assessment of whether the reported associations are robust to reasonable analytic choices.
[Abstract and Results] The abstract and results refer to 'surgery-alone' and 'surgery + adjuvant chemotherapy cohorts' without reporting cohort sizes, event rates, or baseline characteristics; adding a table summarizing these quantities would improve interpretability of the KL and ICI values.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and insightful comments, which highlight important considerations for strengthening the rigor of our claims regarding transportability and the proposed strategies. We address each major comment point-by-point below and outline revisions to the manuscript.

read point-by-point responses

Referee: [Abstract (developer-side strategy) and Results (aggregated external evaluation)] The central claim that meta-analysis-informed weighting yields a model that better approximates the broader target population (and thereby improves calibration, p=0.037 on aggregated external data) rests on the untested assumption that the meta-analysis distribution is not itself dominated by cohorts similar to the six tertiary-center study sites. No sensitivity analysis, external validation of the meta-analysis representativeness, or comparison against a truly held-out population is provided; if the meta-analysis largely overlaps with the observed sample, the reported gains may reflect interpolation within internal heterogeneity rather than genuine transportability.

Authors: We acknowledge the validity of this concern: the published studies contributing to the meta-analysis may share similarities with our tertiary-center cohorts, potentially limiting the extent to which the reweighted model demonstrates transportability beyond internal heterogeneity. Our approach pools covariate-outcome distributions from the literature to approximate a broader population, but we did not perform sensitivity analyses excluding comparable studies or validate against a fully independent held-out population. In the revised manuscript, we will add a sensitivity analysis re-deriving the meta-distribution after excluding studies from similar academic settings and re-evaluate calibration on the aggregated external data. We will also expand the Discussion to explicitly note this as a limitation and discuss the challenges of obtaining truly representative external data from published sources. revision: yes
Referee: [Abstract and Results (correlation analyses)] Multiple Spearman rank correlations are reported with associated p-values (ρ=0.614, p=0.004; ρ=0.738, p<0.001; ρ=0.803, p<0.001; ρ=0.737, p<0.001) linking KL divergence, ICI, and model similarity, yet no adjustment for multiple comparisons or pre-specified analysis plan is described. This raises the possibility that the reported significance levels are inflated, directly affecting the strength of the evidence for the KL-ICI relationship that underpins both proposed strategies.

Authors: We agree that the four reported Spearman correlations constitute multiple testing without adjustment, which could inflate Type I error rates, and that the manuscript does not describe a pre-specified analysis plan for these associations. These correlations were intended to quantify the relationship between distributional shift and calibration as a foundation for the proposed strategies. In the revision, we will apply Bonferroni correction to the p-values (dividing the significance threshold by 4) and report both original and adjusted results. We will also update the Methods to state that these correlation analyses were pre-specified to evaluate the KL-ICI link, while noting any exploratory aspects. If adjusted p-values alter significance, we will revise the Results and Discussion language to reflect the adjusted strength of evidence. revision: yes

Circularity Check

0 steps flagged

No significant circularity; empirical results rest on direct external comparisons

full rationale

The paper reports Spearman correlations between KL divergence and ICI on real-world cohorts, plus calibration improvements from meta-analysis-informed reweighting evaluated on aggregated external data. These are standard statistical associations and hold-out evaluations with no self-definitional loops, no fitted parameters renamed as predictions, and no load-bearing self-citations that reduce the central claims to unverified inputs. The meta-analysis distribution is treated as an external proxy rather than derived from the study data itself, and all performance metrics (ICI, DCA) are computed independently on validation cohorts.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claims rest on standard statistical assumptions about divergence measures and meta-analysis representativeness rather than new free parameters or invented entities.

axioms (2)

domain assumption Kullback-Leibler divergence between covariate and outcome distributions captures the mismatches that drive calibration failure in prognostic models.
Invoked to interpret the observed Spearman correlations with ICI.
domain assumption A meta-analysis-derived distribution approximates the broader target population well enough to serve as a tuning target for robust models.
Basis for the model-developer strategy described in the abstract.

pith-pipeline@v0.9.0 · 5625 in / 1331 out tokens · 23190 ms · 2026-05-10T09:49:24.362773+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

19 extracted references · 19 canonical work pages

[1]

Jama318(14), 1377–1384 (2017)

Alba, A.C., Agoritsas, T., Walsh, M., Hanna, S., Iorio, A., Devereaux, P., McGinn, T., Guyatt, G.: Discrimination and calibration of clinical prediction models: users’ guides to the medical literature. Jama318(14), 1377–1384 (2017)

work page 2017
[2]

In: 2019 Global Conference for Advancement in Technology (GCAT), pp

Nair, N.G., Satpathy, P., Christopher, J.,et al.: Covariate shift: A review and analysis on classifiers. In: 2019 Global Conference for Advancement in Technology (GCAT), pp. 1–6 (2019). IEEE

work page 2019
[3]

Ishwaran, H., Kogalur, U.B., Blackstone, E.H., Lauer, M.S.: Random survival forests (2008)

work page 2008
[4]

Machine learning111(8), 2951–3023 (2022)

Bertsimas, D., Dunn, J., Gibson, E., Orfanoudaki, A.: Optimal survival trees. Machine learning111(8), 2951–3023 (2022)

work page 2022
[5]

Wiley encyclopedia of clinical trials, 1–3 (2007)

Woolson, R.F.: Wilcoxon signed-rank test. Wiley encyclopedia of clinical trials, 1–3 (2007)

work page 2007
[6]

European Journal of Surgical Oncology47(12), 3113–3122 (2021)

Bosma, N.A., Keehn, A.R., Lee-Ying, R., Karim, S., MacLean, A.R., Brenner, D.R.: Efficacy of perioperative chemotherapy in resected colorectal liver metas- tasis: A systematic review and meta-analysis. European Journal of Surgical Oncology47(12), 3113–3122 (2021)

work page 2021
[7]

Journal of Clinical Oncology24(31), 4976–4982 (2006)

Portier, G., Elias, D., Bouche, O., Rougier, P., Bosset, J.-F., Saric, J., Belghiti, J., Piedbois, P., Guimbaud, R., Nordlinger, B.,et al.: Multicenter randomized trial of adjuvant fluorouracil and folinic acid compared with surgery alone after resection of colorectal liver metastases: Ffcd achbth aurc 9002 trial. Journal of Clinical Oncology24(31), 4976–...

work page 2006
[8]

The lancet oncology14(12), 1208–1215 (2013)

Nordlinger, B., Sorbye, H., Glimelius, B., Poston, G.J., Schlag, P.M., Rougier, P., Bechstein, W.O., Primrose, J.N., Walpole, E.T., Finch-Jones, M.,et al.: Peri- operative folfox4 chemotherapy and surgery versus surgery alone for resectable liver metastases from colorectal cancer (eortc 40983): long-term results of a randomised, controlled, phase 3 trial....

work page 2013
[9]

PloS one11(9), 0162400 (2016)

Hasegawa, K., Saiura, A., Takayama, T., Miyagawa, S., Yamamoto, J., Ijichi, M., Teruya, M., Yoshimi, F., Kawasaki, S., Koyama, H.,et al.: Adjuvant oral uracil-tegafur with leucovorin for colorectal cancer liver metastases: a randomized controlled trial. PloS one11(9), 0162400 (2016)

work page 2016
[10]

Journal of Clinical Oncology39(34), 3789–3799 (2021) 24

Kanemitsu, Y., Shimizu, Y., Mizusawa, J., Inaba, Y., Hamaguchi, T., Shida, D., Ohue, M., Komori, K., Shiomi, A., Shiozawa, M.,et al.: Hepatectomy followed by mfolfox6 versus hepatectomy alone for liver-only metastatic colorectal can- cer (jcog0603): a phase ii or iii randomized controlled trial. Journal of Clinical Oncology39(34), 3789–3799 (2021) 24

work page 2021
[11]

Urology76(6), 1298–1301 (2010)

Vickers, A.J., Cronin, A.M.: Everything you always wanted to know about eval- uating prediction models (but were too afraid to ask). Urology76(6), 1298–1301 (2010)

work page 2010
[12]

Cancers16(9), 1645 (2024)

Kokkinakis, S., Ziogas, I.A., Llaque Salazar, J.D., Moris, D.P., Tsoulfas, G.: Clinical prediction models for prognosis of colorectal liver metastases: A compre- hensive review of regression-based and machine learning models. Cancers16(9), 1645 (2024)

work page 2024
[13]

Journal of statistical planning and inference90(2), 227–244 (2000)

Shimodaira, H.: Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of statistical planning and inference90(2), 227–244 (2000)

work page 2000
[14]

Journal of Machine Learning Research10(9) (2009)

Bickel, S., Br¨ uckner, M., Scheffer, T.: Discriminative learning under covariate shift. Journal of Machine Learning Research10(9) (2009)

work page 2009
[15]

Advances in neural information processing systems19(2006)

Huang, J., Gretton, A., Borgwardt, K., Sch¨ olkopf, B., Smola, A.: Correcting sam- ple selection bias by unlabeled data. Advances in neural information processing systems19(2006)

work page 2006
[16]

Dataset shift in machine learning 3(4), 5 (2009)

Gretton, A., Smola, A., Huang, J., Schmittfull, M., Borgwardt, K., Sch¨ olkopf, B., et al.: Covariate shift by kernel mean matching. Dataset shift in machine learning 3(4), 5 (2009)

work page 2009
[17]

Journal of Machine Learning Research8(5) (2007)

Sugiyama, M., Krauledat, M., M¨ uller, K.-R.: Covariate shift adaptation by impor- tance weighted cross validation. Journal of Machine Learning Research8(5) (2007)

work page 2007
[18]

specialization under concept shift

Nguyen, A., Schwab, D.J., Ngampruetikorn, V.: Generalization vs. specialization under concept shift. arXiv preprint arXiv:2409.15582 (2024)

work page arXiv 2024
[19]

Tian, J., Hsu, Y.-C., Shen, Y., Jin, H., Kira, Z.: Exploring covariate and concept shift for out-of-distribution detection. In: NeurIPS 2021 Workshop on Distribution Shifts: Connecting Methods and Applications (2021) 25 Supplementary Appendix 0.5 0.6 0.7 0.8 0.9 1.0 0.0 0.2 0.4 0.6 0.8 1.0 Net Benefit OST RSF All None 1:1 3:2 5:2 4:1 10:1 100:1 Threshold ...

work page 2021

[1] [1]

Jama318(14), 1377–1384 (2017)

Alba, A.C., Agoritsas, T., Walsh, M., Hanna, S., Iorio, A., Devereaux, P., McGinn, T., Guyatt, G.: Discrimination and calibration of clinical prediction models: users’ guides to the medical literature. Jama318(14), 1377–1384 (2017)

work page 2017

[2] [2]

In: 2019 Global Conference for Advancement in Technology (GCAT), pp

Nair, N.G., Satpathy, P., Christopher, J.,et al.: Covariate shift: A review and analysis on classifiers. In: 2019 Global Conference for Advancement in Technology (GCAT), pp. 1–6 (2019). IEEE

work page 2019

[3] [3]

Ishwaran, H., Kogalur, U.B., Blackstone, E.H., Lauer, M.S.: Random survival forests (2008)

work page 2008

[4] [4]

Machine learning111(8), 2951–3023 (2022)

Bertsimas, D., Dunn, J., Gibson, E., Orfanoudaki, A.: Optimal survival trees. Machine learning111(8), 2951–3023 (2022)

work page 2022

[5] [5]

Wiley encyclopedia of clinical trials, 1–3 (2007)

Woolson, R.F.: Wilcoxon signed-rank test. Wiley encyclopedia of clinical trials, 1–3 (2007)

work page 2007

[6] [6]

European Journal of Surgical Oncology47(12), 3113–3122 (2021)

Bosma, N.A., Keehn, A.R., Lee-Ying, R., Karim, S., MacLean, A.R., Brenner, D.R.: Efficacy of perioperative chemotherapy in resected colorectal liver metas- tasis: A systematic review and meta-analysis. European Journal of Surgical Oncology47(12), 3113–3122 (2021)

work page 2021

[7] [7]

Journal of Clinical Oncology24(31), 4976–4982 (2006)

Portier, G., Elias, D., Bouche, O., Rougier, P., Bosset, J.-F., Saric, J., Belghiti, J., Piedbois, P., Guimbaud, R., Nordlinger, B.,et al.: Multicenter randomized trial of adjuvant fluorouracil and folinic acid compared with surgery alone after resection of colorectal liver metastases: Ffcd achbth aurc 9002 trial. Journal of Clinical Oncology24(31), 4976–...

work page 2006

[8] [8]

The lancet oncology14(12), 1208–1215 (2013)

Nordlinger, B., Sorbye, H., Glimelius, B., Poston, G.J., Schlag, P.M., Rougier, P., Bechstein, W.O., Primrose, J.N., Walpole, E.T., Finch-Jones, M.,et al.: Peri- operative folfox4 chemotherapy and surgery versus surgery alone for resectable liver metastases from colorectal cancer (eortc 40983): long-term results of a randomised, controlled, phase 3 trial....

work page 2013

[9] [9]

PloS one11(9), 0162400 (2016)

Hasegawa, K., Saiura, A., Takayama, T., Miyagawa, S., Yamamoto, J., Ijichi, M., Teruya, M., Yoshimi, F., Kawasaki, S., Koyama, H.,et al.: Adjuvant oral uracil-tegafur with leucovorin for colorectal cancer liver metastases: a randomized controlled trial. PloS one11(9), 0162400 (2016)

work page 2016

[10] [10]

Journal of Clinical Oncology39(34), 3789–3799 (2021) 24

Kanemitsu, Y., Shimizu, Y., Mizusawa, J., Inaba, Y., Hamaguchi, T., Shida, D., Ohue, M., Komori, K., Shiomi, A., Shiozawa, M.,et al.: Hepatectomy followed by mfolfox6 versus hepatectomy alone for liver-only metastatic colorectal can- cer (jcog0603): a phase ii or iii randomized controlled trial. Journal of Clinical Oncology39(34), 3789–3799 (2021) 24

work page 2021

[11] [11]

Urology76(6), 1298–1301 (2010)

Vickers, A.J., Cronin, A.M.: Everything you always wanted to know about eval- uating prediction models (but were too afraid to ask). Urology76(6), 1298–1301 (2010)

work page 2010

[12] [12]

Cancers16(9), 1645 (2024)

Kokkinakis, S., Ziogas, I.A., Llaque Salazar, J.D., Moris, D.P., Tsoulfas, G.: Clinical prediction models for prognosis of colorectal liver metastases: A compre- hensive review of regression-based and machine learning models. Cancers16(9), 1645 (2024)

work page 2024

[13] [13]

Journal of statistical planning and inference90(2), 227–244 (2000)

Shimodaira, H.: Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of statistical planning and inference90(2), 227–244 (2000)

work page 2000

[14] [14]

Journal of Machine Learning Research10(9) (2009)

Bickel, S., Br¨ uckner, M., Scheffer, T.: Discriminative learning under covariate shift. Journal of Machine Learning Research10(9) (2009)

work page 2009

[15] [15]

Advances in neural information processing systems19(2006)

Huang, J., Gretton, A., Borgwardt, K., Sch¨ olkopf, B., Smola, A.: Correcting sam- ple selection bias by unlabeled data. Advances in neural information processing systems19(2006)

work page 2006

[16] [16]

Dataset shift in machine learning 3(4), 5 (2009)

Gretton, A., Smola, A., Huang, J., Schmittfull, M., Borgwardt, K., Sch¨ olkopf, B., et al.: Covariate shift by kernel mean matching. Dataset shift in machine learning 3(4), 5 (2009)

work page 2009

[17] [17]

Journal of Machine Learning Research8(5) (2007)

Sugiyama, M., Krauledat, M., M¨ uller, K.-R.: Covariate shift adaptation by impor- tance weighted cross validation. Journal of Machine Learning Research8(5) (2007)

work page 2007

[18] [18]

specialization under concept shift

Nguyen, A., Schwab, D.J., Ngampruetikorn, V.: Generalization vs. specialization under concept shift. arXiv preprint arXiv:2409.15582 (2024)

work page arXiv 2024

[19] [19]

Tian, J., Hsu, Y.-C., Shen, Y., Jin, H., Kira, Z.: Exploring covariate and concept shift for out-of-distribution detection. In: NeurIPS 2021 Workshop on Distribution Shifts: Connecting Methods and Applications (2021) 25 Supplementary Appendix 0.5 0.6 0.7 0.8 0.9 1.0 0.0 0.2 0.4 0.6 0.8 1.0 Net Benefit OST RSF All None 1:1 3:2 5:2 4:1 10:1 100:1 Threshold ...

work page 2021