Pith. sign in

REVIEW 4 major objections 5 minor 48 references

Tree-based methods for estimating heterogeneous model performance and model combining

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read New tree and ensemble methods identify subgroups with differential model performance and use them to combine prediction models.

desk verdict Genuine methodological contribution with a real gap in Theorem 3.1; the central idea survives, but the consistency theorem needs an added partition-regularity condition and the alpha' sensitivity needs a principled fix. read the letter →

arxiv 2506.01905 v1 pith:J525IGJC submitted 2025-06-02 stat.ME

Ruotao Zhang , Constantine Gatsonis , Jon Steingrimsson This is my paper
classification stat.ME
keywords modelperformancemethodssubgroupsensemblesheterogeneityoveralltree-based
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most prediction models, including medical risk scores, report a single accuracy number averaged over everyone. That number can hide places where the model is much weaker, for example for younger patients or a specific ethnic group. The paper treats finding those places as a statistical search problem. It modifies the classic CART tree algorithm in two ways. The first, CART-TO, uses each person's prediction error as the outcome and grows a tree that puts people with similar error sizes in the same branch. The second, PASD, chooses splits that maximize the standardized difference in model performance between the two child groups. PASD-2 also works for pair-based measures such as the area under the ROC curve. Single trees are then replaced by random forests and gradient boosting ensembles that estimate performance at every covariate pattern. To combine several prediction models, the authors use majority voting: in each bootstrap tree, whichever model has the best out-of-bag performance at a point gets a vote, and the final weight is the vote share. They also give an EM-based weighting scheme. In simulations, the ensembles always beat a single tree, and on a simulated binary outcome the combined model raised AUC from about 0.68 to 0.74. On the NLST lung cancer dataset, the most common PASD tree was a single node, meaning no strong heterogeneity in AUC was found, and the combined model gave small Brier-score improvements.
Extended reading notes

Core claim

Main load-bearing assertion: 'If Y and h(X) are bounded and the number of terminal nodes is of order o(n/log n) when n → ∞, then all three tree algorithms are L2-consistent estimators for conditional model performance at X' (Theorem 3.1, Section 3.3). The paper also claims that the PASD ensembles can estimate covariate-dependent weights for model combination that improve over individual models. If correct, the trees recover true performance subgroups and the ensembles yield consistent performance estimates.

Load-bearing premise

The most fragile premise is that the pruning schemes, especially PASD-2 with a fixed penalty alpha', produce trees with o(n/log n) terminal nodes and that the split statistic s_{j,c} in Eq. (2) is a calibrated measure of performance heterogeneity. Theorem 3.1 assumes the terminal-node rate rather than proving that cost-complexity pruning or the fixed-alpha' selection delivers it, and the paper's own simulations show the correct-tree rate in setting 2 varies from 8.6% (alpha'=2) to 0.4% (alpha'=log n). If alpha' cannot be chosen reliably or the split statistic is not calibrated, the consistency result and the practical utility of PASD-2 collapse.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper develops tree-based methods for discovering subgroups with differential predictive model performance. Three single-tree algorithms are proposed: CART with a transformed performance outcome (CART-TO), and two versions of a Prediction Accuracy Subgroup Discovery algorithm (PASD-1 and PASD-2) based on a standardized split statistic and split-complexity pruning. The methods are extended to random forest and gradient boosting ensembles for estimating conditional model performance, and the ensembles are used for model combination through majority voting and an EM-based mixture approach. The paper evaluates the methods in four simulation settings, compares single trees with ensembles, applies the methods to lung cancer screening (NLST) data, and reports an additional COMPAS analysis.

Significance. If the theoretical claims are correct, the paper makes a useful contribution by generalizing subgroup discovery for model performance to measures such as AUC that are not available at the individual level, and by providing ensemble-based model combination with covariate-dependent weights. The simulations are carefully designed with independent test data and out-of-bag terminal-node estimates, and they consistently show that the proposed ensembles improve over single trees. The NLST and COMPAS applications illustrate practical value. The main weakness is that the central consistency theorem is stated with insufficient conditions and an incomplete proof, and the PASD-2 results are highly sensitive to the user-chosen penalty alpha'.

major comments (4)
  1. [Section 3.3, Appendix C] Theorem 3.1 is false as stated. The condition that the number of terminal nodes is o(n/log n) bounds only the noise term in the proof; it does not control the bias term. A root-only tree satisfies the terminal-node rate, but it estimates a constant and cannot be L2-consistent when the conditional performance function mu(x) is nonconstant. The proof's bias term is dismissed by 'following the same argument as in the proof of Theorem 1 in [27]', but Nobel's Theorem 1 requires a partition-regularity condition, such as cell probabilities shrinking to zero, which is neither stated in Theorem 3.1 nor established for CART or PASD trees. This is a load-bearing gap because the theorem is the paper's main theoretical justification.
  2. [Section 3.3, Appendix C, Eq. (7)] Theorem 3.1 claims L2-consistency for all three algorithms for conditional model performance, but the proof only covers individual-level loss measures through the transformed outcome Y* = l(Y, h(X)). The terminal-node estimator in Eq. (7) is a sample mean of Y*, and the proof uses this representation throughout. For group-level measures such as AUC, the terminal estimator is a two-sample U-statistic (Appendix D.2) and does not have the form of Eq. (7). Thus the proof does not establish the theorem for AUC, which is the performance measure used in the NLST application in Section 7.1.
  3. [Section 3.2.1, Eq. (2)] The claim that the splitting statistic s_{j,c} is asymptotically chi-squared_1 distributed under equal performance is stated without proof or regularity conditions. The statistic is maximized over many candidate splits and covariates, and the same data are reused in nested nodes, so standard chi-squared asymptotics do not apply automatically. This matters because the split-complexity pruning in Eq. (3) treats s(m) as a calibrated additive score; if the distributional claim is only heuristic, the pruning scale and the meaning of alpha are not well founded.
  4. [Section 3.2.3, Tables 4-6] PASD-2's final tree selection depends strongly on the fixed penalty alpha'. In simulation setting 2, the correct-tree rate falls from 8.6% with alpha'=2 to 0.4% with alpha'=log n, and the underfitting rate at alpha'=log n is 97.5%. This sensitivity means the algorithm's practical subgroup recovery is not reliable without a principled way to choose alpha'. The paper acknowledges that increasing alpha' reduces overfitting but increases underfitting, yet it does not provide a data-adaptive selection rule or quantify the resulting uncertainty; this undermines the claim that PASD-2 identifies true performance subgroups in practice.
minor comments (5)
  1. [Section 7.1] The title 'Discovery of sbugroups with differential model performance' contains a typo: 'sbugroups' should be 'subgroups'.
  2. [Section 5.2, Appendix H.3] The scaled chi-squared assumption for the estimated performance bmu_k(x) in Eq. (6) needs justification. bmu_k(x) is a tree ensemble estimate, not a simple sample mean, so the exact tau^2 * chi-squared_1 distribution is not immediate and should be stated as an assumption rather than derived.
  3. [Section 2 and Section 6] The notation for the data used for tree building is inconsistent: the text uses D and Dtree, and the simulation section refers to Dtree and Dtest without a single clear definition; please standardize.
  4. [Tables 4-7] The table columns would be much clearer if the taxonomy from Appendix G.3 (e.g., '(SF, nF)' categories) were included as column headers or explained in a footnote; some entries such as '0.0' are formatted inconsistently.
  5. [Section 3.1] The sentence beginning 'For a given measure of model performance, We use mu_i...' has a capitalization inconsistency ('We' after a comma); please fix.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the methods are evaluated on independent test data and the theoretical claim is a conditional consistency theorem, not a re-statement of its inputs.

full rationale

The paper's central claim (Theorem 3.1) is a conditional consistency result: under boundedness plus an assumed terminal-node rate o(n/log n), the tree estimators are L2-consistent. The rate is stated as an assumption rather than derived from the target, and the proof decomposes the error into bias and noise terms, importing Nobel's histogram-regression bounds for the noise term. Whether Nobel's regularity conditions are fully met is a mathematical-correctness concern, not a circularity concern. The splitting criteria (Eq. 1 and Eq. 2) are defined directly in terms of model-performance differences, not in terms of the later consistency theorem. Simulations evaluate fitted trees and ensembles on independent test sets, random-forest terminal-node estimates use out-of-bag samples, and the model-combining weights are estimated on training data and assessed on held-out test data (e.g., the NLST training/test split). The self-citations that appear ([6], [15], [31]) are contextual or invoke standard sample-splitting arguments; none of them supplies the paper's central premise or forbids alternative choices. No equation is defined in terms of the result it is supposed to predict, and no fitted parameter is renamed as a prediction. The derivation chain is therefore self-contained with respect to the identified circularity patterns.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

Central claim rests on standard CART/Nobel partition-consistency machinery plus a number of hand-chosen tuning parameters. The main free parameter is alpha' in PASD-2 final tree selection, which the paper's own simulations show to be highly influential. The EM model combination adds distributional assumptions (Gaussian mixture, scaled chi-squared estimated MSE) that are introduced to make updates analytic and have no independent justification. No invented physical entities are needed; the latent variable Z in the EM is a standard mixture device.

free parameters (3)
  • alpha' pruning penalty in PASD-2 final tree selection = default 4; variants 2, 3, log(n)
    Chosen by hand in Section 3.2.3. Tables 4-7 show correct-tree rates in setting 2 range from 8.6% at alpha'=2 to 0.4% at alpha'=log n, so the central method's output depends heavily on this arbitrary value.
  • Gradient boosting ensemble size M and learning rate lambda = not reported
    Section 4 says M and lambda should be tuned together to prevent overfitting, but no default values or tuning procedure are given, so the reported ensemble results depend on undisclosed choices.
  • Random forest parameters B and mtry = not reported
    Algorithm 5 specifies bootstrap samples and random covariate subsets per split but gives no values; the simulation and application results depend on these choices.
assumptions (6)
  • domain assumption Observations are i.i.d. from the joint distribution of (X,Y).
    Section 2 states this at the outset; the consistency proof and all simulation inferences rely on it.
  • domain assumption Y and h(X) are bounded and the number of terminal nodes is o(n/log n).
    Assumed in Theorem 3.1 and used in the Appendix C proof; the paper does not show that any of its pruning schemes guarantees the terminal-node rate.
  • ad hoc to paper The split statistic s_{j,c} in Eq. (2) is asymptotically chi-squared_1 under equal performance.
    Asserted in Section 3.2.1 without proof or regularity conditions; used to motivate the split criterion and the pruning statistic.
  • standard math The conditions of Nobel (1996) hold for the partition rules generated by CART-TO, PASD-1, and PASD-2.
    Appendix C invokes Proposition 2, Lemma 3, and equations from [27] without verifying that the specific algorithms output partitions obeying these conditions.
  • domain assumption For EM model combination, the conditional density of Y given X is a Gaussian mixture with components N(h_k(X), sigma_k^2).
    Section 5.2 and Appendix H.1 define the likelihood this way; the EM derivation and MLE interpretation are only valid under this model.
  • ad hoc to paper The estimated MSE for each model follows a scaled chi-squared distribution tau^2 * chi-squared_1.
    Algorithm 8, Eq. (58) imposes this density to obtain analytic M-step updates; no justification is given, and it only applies to squared-error performance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Tree-based methods for estimating heterogeneous model performance and model combining." pith.science (2026). https://pith.science/paper/J525IGJC

@misc{pith2026250601905,
  author       = {Pith},
  title        = {Pith review of: Tree-based methods for estimating heterogeneous model performance and model combining},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/J525IGJC}},
  note         = {Machine review of arXiv:2506.01905}
}
read the original abstract

Model performance is frequently reported only for the overall population under consideration. However, due to heterogeneity, overall performance measures often do not accurately represent model performance within specific subgroups. We develop tree-based methods for the data-driven identification of subgroups with differential model performance, where splitting decisions are made to maximize heterogeneity in performance between subgroups. We extend these methods to tree ensembles, including both random forests and gradient boosting. Lastly, we illustrate how these ensembles can be used for model combination. We evaluate the methods through simulations and apply them to lung cancer screening data.

Figures

Figures reproduced from arXiv: 2506.01905 by the authors.

Figure 1
Figure 1. Figure 1a shows the distributions of the MSE of the single PASD tree, random PASD forest and gradient boosting PASD trees in estimating the squared errors of the linear regression model. The single PASD tree had higher MSEs compared to both PASD ensembles, with gradient boosting PASD trees generally having the lowest MSE. Figure 1b looks at the MSE difference by subtracting the MSE of the single PASD tree. That is, … view at source ↗
Figure 1
Figure 1. Mean squared error (MSE) of single PASD tree, random PASD forest (RF), and gradient [PITH_FULL_IMAGE:figures/full_fig_p016_1.png] view at source ↗
Figure 2
Figure 2. Difference in test set AUC between the combined model and the four individual models. [PITH_FULL_IMAGE:figures/full_fig_p018_2.png] view at source ↗
Figures from the paper (13 more)
Figure 3
Figure 3. Figure 3: Comparison between combined model and individual risk models in terms of Brier Score [PITH_FULL_IMAGE:figures/full_fig_p020_3.png]
Figure 4
Figure 4. Figure 4: True underlying tree structure for simulation setting 1. [PITH_FULL_IMAGE:figures/full_fig_p034_4.png]
Figure 5
Figure 5. Figure 5: True underlying tree structure for simulation setting 2. [PITH_FULL_IMAGE:figures/full_fig_p034_5.png]
Figure 6
Figure 6. Figure 6: True underlying tree structure for simulation setting 4. [PITH_FULL_IMAGE:figures/full_fig_p035_6.png]
Figure 7
Figure 7. Figure 7: Visualization for the proposed model combination method in a one-dimensional classifi [PITH_FULL_IMAGE:figures/full_fig_p045_7.png]
Figure 8
Figure 8. Figure 8: Visualization for the proposed model combination method in a one-dimensional classifi [PITH_FULL_IMAGE:figures/full_fig_p045_8.png]
Figure 9
Figure 9. Figure 9: A second visualization for the proposed model combination method in a one-dimensional [PITH_FULL_IMAGE:figures/full_fig_p046_9.png]
Figure 10
Figure 10. Figure 10: The second visualization for the proposed model combination method in a one [PITH_FULL_IMAGE:figures/full_fig_p047_10.png]
Figure 11
Figure 11. Figure 11: Comparison between combined model and individual models for the simulation results [PITH_FULL_IMAGE:figures/full_fig_p048_11.png]
Figure 12
Figure 12. Figure 12: In all four cases and for all splits, the tree ensembles had smaller MSE than the single [PITH_FULL_IMAGE:figures/full_fig_p049_12.png]
Figure 12
Figure 12. Figure 12: Improvements in MSE of random PASD forest and gradient boosting PASD trees [PITH_FULL_IMAGE:figures/full_fig_p050_12.png]
Figure 13
Figure 13. Figure 13: Most frequently selected tree by the PASD algorithm applied to the COMPAS data [PITH_FULL_IMAGE:figures/full_fig_p051_13.png]
Figure 14
Figure 14. Figure 14: Most frequently selected tree by the PASD algorithm applied to the COMPAS data [PITH_FULL_IMAGE:figures/full_fig_p052_14.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 46 canonical work pages

  1. [27]

    Histogram Regression Estimation Using Data-Dependent Partitions,

    A. Nobel, “Histogram Regression Estimation Using Data-Dependent Partitions,” The Annals of Statistics , vol. 24, pp. 1084–1105, 9 1996

    work page 1996

  2. [1]

    Hidden Stratification Causes Clini- cally Meaningful Failures in Machine Learning for Medical Imaging.,

    L. Oakden-Rayner, J. Dunnmon, G. Carneiro, and C. R´ e, “Hidden Stratification Causes Clini- cally Meaningful Failures in Machine Learning for Medical Imaging.,” Proceedings of the ACM Conference on Health, Inference, and Learning , vol. 2020, pp. 151–159, 4 2020

    work page 2020

  3. [2]

    Addressing fairness in artificial intelligence for medical imaging,

    M. A. Ricci Lara, R. Echeveste, and E. Ferrante, “Addressing fairness in artificial intelligence for medical imaging,” nature communications, vol. 13, no. 1, p. 4581, 2022

    2022

  4. [3]

    Algorithmic fairness in artificial intelligence for medicine and healthcare,

    R. J. Chen, J. J. Wang, D. F. Williamson, T. Y. Chen, J. Lipkova, M. Y. Lu, S. Sahai, and F. Mahmood, “Algorithmic fairness in artificial intelligence for medicine and healthcare,” Nature biomedical engineering, vol. 7, no. 6, pp. 719–742, 2023

    work page 2023

  5. [4]

    Interaction trees with censored survival data,

    X. Su, T. Zhou, X. Yan, J. Fan, and S. Yang, “Interaction trees with censored survival data,” The international journal of biostatistics , vol. 4, p. 2, 1 2008

    work page 2008

  6. [5]

    Subgroup identification from randomized clinical trial data.,

    J. C. Foster, J. M. G. Taylor, and S. J. Ruberg, “Subgroup identification from randomized clinical trial data.,” Statistics in medicine , vol. 30, pp. 2867–2880, 10 2011

    work page 2011

  7. [6]

    Subgroup identification using covariate-adjusted interaction trees,

    J. A. Steingrimsson and J. Yang, “Subgroup identification using covariate-adjusted interaction trees,” Statistics in Medicine , vol. 38, pp. 3974–3984, 9 2019

    work page 2019

  8. [7]

    Identifying subgroups of enhanced predictive accuracy from longitudinal biomarker data using tree-based approaches: applications to fetal 21 growth,

    J. C. Foster, D. Liu, P. S. Albert, and A. Liu, “Identifying subgroups of enhanced predictive accuracy from longitudinal biomarker data using tree-based approaches: applications to fetal 21 growth,” Journal of the Royal Statistical Society. Series A, (Statistics in Society) , vol. 180, pp. 247–261, 1 2017

    work page 2017

Show all 48 references
  1. [8]

    Empirical comparison of Dynamic Classifier Selection methods based on diversity and accuracy for building ensembles,

    M. C. P. de Souto, R. G. F. Soares, A. Santana, and A. M. P. Canuto, “Empirical comparison of Dynamic Classifier Selection methods based on diversity and accuracy for building ensembles,” in 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Co...

    2008

  2. [9]

    Contribution of data complexity features on dynamic classifier selection,

    A. L. Brun, A. S. Britto, L. S. Oliveira, F. Enembreck, and R. Sabourin, “Contribution of data complexity features on dynamic classifier selection,” in 2016 International Joint Conference on Neural Networks (IJCNN) , pp. 4396–4403, 2016

    2016

  3. [10]

    Dynamic classifier selection: Recent advances and perspectives,

    R. M. O. Cruz, R. Sabourin, and G. D. C. Cavalcanti, “Dynamic classifier selection: Recent advances and perspectives,” Information Fusion, vol. 41, pp. 195–216, 2018

    2018

  4. [11]

    Super learner.,

    M. J. van der Laan, E. C. Polley, and A. E. Hubbard, “Super learner.,” Statistical applications in genetics and molecular biology , vol. 6, p. Article25, 2007

    2007

  5. [12]

    Adaptive Mixtures of Local Experts,

    R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinton, “Adaptive Mixtures of Local Experts,” Neural Computation, vol. 3, no. 1, pp. 79–87, 1991

    1991

  6. [13]

    Breiman, J

    L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone, Classification and Regression Trees. Monterey, CA: Wadsworth and Brooks, 1984

    1984

  7. [14]

    Survival Trees by Goodness of Split,

    M. Leblanc and J. Crowley, “Survival Trees by Goodness of Split,” Journal of the American Statistical Association, vol. 88, pp. 457–467, 6 1993

    1993

  8. [15]

    Causal interaction trees: Finding subgroups with heterogeneous treatment effects in observational data,

    J. Yang, I. J. Dahabreh, and J. A. Steingrimsson, “Causal interaction trees: Finding subgroups with heterogeneous treatment effects in observational data,”Biometrics, vol. 78, no. 2, pp. 624– 635, 2022

    2022

  9. [16]

    Random Forests,

    L. Breiman, “Random Forests,” Machine Learning, vol. 45, no. 1, pp. 5–32, 2001

    2001

  10. [17]

    Greedy Function Approximation : A Gradient Boosting Machine,

    J. Friedman, “Greedy Function Approximation : A Gradient Boosting Machine,” The Annals of Statistics , vol. 29, no. 5, pp. 1189–1232, 2001. 22

    2001

  11. [18]

    Using Bayesian model averag- ing to calibrate forecast ensembles,

    A. E. Raftery, T. Gneiting, F. Balabdaoui, and M. Polakowski, “Using Bayesian model averag- ing to calibrate forecast ensembles,” Monthly Weather Review, vol. 133, no. 5, pp. 1155–1174, 2005

    2005

  12. [19]

    Maximum Likelihood from Incomplete Data Via the EM Algorithm,

    A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum Likelihood from Incomplete Data Via the EM Algorithm,” Journal of the Royal Statistical Society: Series B (Methodological) , vol. 39, no. 1, pp. 1–22, 1977

    1977

  13. [20]

    An Alternative Model for Mixtures of Experts,

    L. Xu, M. Jordan, and G. E. Hinton, “An Alternative Model for Mixtures of Experts,” in Advances in Neural Information Processing Systems (G. Tesauro, D. Touretzky, and T. Leen, eds.), vol. 7, MIT Press, 1994

    1994

  14. [21]

    Multivariate Adaptive Regression Splines,

    J. H. Friedman, “Multivariate Adaptive Regression Splines,” The Annals of Statistics , vol. 19, no. 1, pp. 1–67, 1991

    1991

  15. [22]

    Reduced lung-cancer mortality with low-dose computed tomographic screening,

    N. L. S. T. R. Team, “Reduced lung-cancer mortality with low-dose computed tomographic screening,” New England Journal of Medicine , vol. 365, no. 5, pp. 395–409, 2011

    2011

  16. [23]

    Selection Criteria for Lung-Cancer Screening,

    M. C. Tammem¨ agi, H. A. Katki, W. G. Hocking, T. R. Church, N. Caporaso, P. A. Kvale, A. K. Chaturvedi, G. A. Silvestri, T. L. Riley, J. Commins, and C. D. Berg, “Selection Criteria for Lung-Cancer Screening,” New England Journal of Medicine , vol. 368, pp. 728–736, 2 2013

    2013

  17. [24]

    Variations in lung cancer risk among smokers.,

    P. B. Bach, M. W. Kattan, M. D. Thornquist, M. G. Kris, R. C. Tate, M. J. Barnett, L. J. Hsieh, and C. B. Begg, “Variations in lung cancer risk among smokers.,” Journal of the Na- tional Cancer Institute , vol. 95, pp. 470–478, 3 2003

    2003

  18. [25]

    Risk Prediction Model Versus United States Preventive Services Task Force Lung Cancer Screening Eligibility Criteria: Reducing Race Disparities.,

    M. M. Pasquinelli, M. C. Tammem¨ agi, K. L. Kovitz, M. L. Durham, Z. Deliu, K. Rygalski, L. Liu, M. Koshy, P. Finn, and L. E. Feldman, “Risk Prediction Model Versus United States Preventive Services Task Force Lung Cancer Screening Eligibility Criteria: Reducing Race Dispariti...

    2020

  19. [26]

    Risk prediction models for selection of lung 23 cancer screening candidates: A retrospective validation study,

    K. ten Haaf, J. Jeon, M. C. Tammem¨ agi, S. S. Han, C. Y. Kong, S. K. Plevritis, E. J. Feuer, H. J. de Koning, E. W. Steyerberg, and R. Meza, “Risk prediction models for selection of lung 23 cancer screening candidates: A retrospective validation study,” PLOS Medicine, vol. 14...

    2017

  20. [28]

    On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other,

    H. B. Mann and D. R. Whitney, “On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other,” Ann. Math. Statist. , vol. 18, no. 1, pp. 50–60, 1947

    1947

  21. [29]

    Estimate of Variance of Wilcoxon-Mann-Whitney Statistic,

    S. Shirahata, “Estimate of Variance of Wilcoxon-Mann-Whitney Statistic,” Journal of the Japanese Society of Computational Statistics , vol. 6, no. 2, pp. 1–10, 1993

    1993

  22. [30]

    Making Sense of a Forest of Trees,

    H. A. Chipman, E. I. George, and R. E. Mcculloch, “Making Sense of a Forest of Trees,” in Proceedings of the 30th Symposium on the Interface , pp. 84–92, 2007

    2007

  23. [31]

    Transporting a prediction model for use in a new target population,

    J. A. Steingrimsson, C. Gatsonis, and I. J. Dahabreh, “Transporting a prediction model for use in a new target population,” American Journal of Epidemiology (In Press) , 2021

    2021

  24. [32]

    Scikit-learn: Machine Learning in Python,

    F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay, “Scikit-learn: Machine Learning in Python,”Journal of Machine Learnin...

    2011

  25. [33]

    A computer program used for bail and sentencing decisions was labeled biased against blacks. Its actually not that clear.,

    S. Corbett-Davies, E. Pierson, A. Feller, and S. Goel, “A computer program used for bail and sentencing decisions was labeled biased against blacks. Its actually not that clear.,” 10 2016

    2016

  26. [34]

    Machine Bias,

    J. Angwin, J. Larson, S. Mattu, and L. Kirchner, “Machine Bias,” 5 2016. 24 Appendix A Details of the CART-TO algorithm More specifically, for a tree T let IT denote the set of internal nodes. |IT | is the size of IT and thus |IT | + 1 is the number of terminal nodes. For each...

    2016

  27. [35]

    argued that dVar[bµ(w)] is the UMVU estimator for Var[ bµ(w)]. Appendix E Random forest and gradient boosting algorithm Algorithm 5 gives the random forest algorithm with PASD trees and Algorithm 6 presents the gradient boosting algorithm with PASD trees. Algorithm 5:Random PA...

  28. [36]

    compute residuals ri = µ(xi) − Tm−1(xi)

  29. [37]

    fit a PASD tree eTm to data {(xi, ri) : i = 1, . . . , n}

  30. [38]

    set Tm(x) = Tm−1(x) + λ · eTm(x) 3 Output TM (x); Appendix F Algorithms for model combination Algorithm 7 gives the algorithm for combining prediction models using majority voting. 33 Algorithm 7:Majority voting guided model combination for b = 1 to B do Generate a bootstrap s...

  31. [39]

    Running each observation down the fitted tree gives a predic- tion bµi of model performance for each observation µi

    Mean squared error (MSE) . Running each observation down the fitted tree gives a predic- tion bµi of model performance for each observation µi. The mean squared error is defined as 1 N PN i=1 (µi − bµi)2, where µi is the observed model performance for observation i. This mea- ...

  32. [40]

    Let IT (i, j) and IF (i, j) be the indicators of whether observations i and j are placed in the same subgroup (terminal node) in the true tree and the fitted tree, respectively

    Pairwise prediction similarity (PPS) between the true tree and the fitted tree [30]. Let IT (i, j) and IF (i, j) be the indicators of whether observations i and j are placed in the same subgroup (terminal node) in the true tree and the fitted tree, respectively. PPS is defined...

  33. [41]

    In simulation setting 1 there are no prediction error modifiers

    Proportion of fitted trees that only split on the covariates that affect prediction performance (e.g., prediction error modifiers [31]). In simulation setting 1 there are no prediction error modifiers. For simulation settings 2 and 3, X (3) and X (5) are prediction error modif...

  34. [42]

    For setting 1, there should be zero splits (one terminal node)

    Proportion of fitted trees (out of 1,000) that only split on the correct covariates and have an equal number of subgroups compared to the true tree. For setting 1, there should be zero splits (one terminal node). We expect three splits for settings 2 and 3, and one split for s...

  35. [43]

    KY k=1 ϕ(yi; hk(xi), σ2 k)I(zi=k) # ·

    the number of subgroups (terminal nodes) in the fitted tree to determine what type of error the tree has. Let SF and ST denote the set of covariates used in the fitted tree and the set of correct (expected) covariates in the true tree, respectively. Let nF and nT denote the nu...

  36. [45]

    KY k=1 ϕ(yi; hk(xi), σ2 k)I(zi=k) # ·

    · πj(xi; ν′) . (39) Taking the derivative of Q(γ | γ′) w.r.t. σ2 j gives ∂Q ∂σ 2 j = nX i=1 λij · ∂ ∂σ 2 j ϕ(yi; hj(xi), σ2 j ) ϕ(yi; hj(xi), σ2 j ) = nX i=1 λij[(hj(xi) − yi)2 − σ2 j ] 2σ4 j . (40) Thus, bσ2 j = argmax σ2 j Q(γ | γ′) = Pn i=1 λij[hj(xi) − yi]2 Pn i=1 λij , j ...

  37. [46]

    · πk(xi; ν′)PK j=1 ϕ(yi; hj(xi), σ′ j

  38. [47]

    (62) Taking the derivative of Q(γ | γ′) w.r.t

    · πj(xi; ν′) . (62) Taking the derivative of Q(γ | γ′) w.r.t. σ2 j gives ∂Q ∂σ 2 j = nX i=1 λij · ∂ ∂σ 2 j ϕ(yi; hj(xi), σ2 j ) ϕ(yi; µj(xi), σ2 j ) = nX i=1 λij[(hj(xi) − yi)2 − σ2 j ] 2σ4 j . (63) Thus, bσ2 j = argmax σ2 j Q(γ | γ′) = Pn i=1 λij[hj(xi) − yi]2 Pn i=1 λij , j ...

  39. [48]

    , n, k= 1,

    E-step: calculate λik = ϕ yi; hk(xi), σ2 k (t−1) · πk(xi; ν(t−1)) PK j=1 ϕ yi; hj(xi), σ2 j (t−1) · πj(xi; ν(t−1)) , i = 1, . . . , n, k= 1, . . . , K

  40. [49]

    , K β(t) j = 1 n nX i=1 λij, j = 1,

    M-step: σ2 j (t) = Pn i=1 λij[hj(xi) − yi]2 Pn i=1 λij , j = 1, . . . , K β(t) j = 1 n nX i=1 λij, j = 1, . . . , K τ 2(t) = 1 n nX i=1 KX k=1 λik · bµk(xi) Appendix I Illustration of model combination using PASD ensem- bles with two classification examples For illustrations i...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.