{"id":"669ebeda-e311-404e-8542-531dd72f24a7","arxiv_id":"1908.03385","paper_version":5,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"GBST, a gradient-boosted survival tree with time-varying leaf weights, improves credit default prediction on two large datasets.","lead":"This paper introduces GBST, a survival tree model trained by gradient boosting to predict when a borrower will default. On two large credit datasets it reports better discrimination than seven existing models, though the statistical evidence is partly overstated.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"LOOCV t-tests on overlapping training folds make the significance claims anti-conservative; small Lending Club margins (e.g., month-5 AUC 0.7267 vs 0.7252) depend on them.","rationale":"The reader's conditional verdict is appropriate. The invalid dependence structure of the LOOCV t-tests is the most load-bearing concern because it directly underwrites the word 'significantly' in the empirical conclusions, and the margins on the public Lending Club dataset are small. The risk-set AUC/KS issue is real but less decisive: the same loans define the risk set for every model, so relative rankings are less likely to be reversed by censoring than by the faulty p-values; still, absolute metric values and some pairwise comparisons could shift with IPCW. I also note a second dataset-specific flaw not in the reader's analysis: in §3.2.2, the 50 features for the 360 dataset are selected with XGB on the January training set before the LOOCV folds of Table 7 are formed, so folds 1–7 use test labels in feature selection. That contaminates the 360 LOOCV p-values, though the March-only folds 8–10 appear clean. A bootstrap on the fixed test set would settle whether the Lending Club advantage is real. Because the model derivation is coherent and the code is public, the appropriate disposition remains conditional: accept the method, but require corrected inference before relying on the superiority claim.","tokens_in":23714,"tokens_out":12754,"duration_ms":141462,"concrete_test":"Recompute the Lending Club comparison on the fixed May–Dec 2013 test set of §3.1.1, retraining all models on the Jan 2007–Apr 2013 training set, and bootstrap the test loans (10,000 resamples) to get 95% CIs for paired differences in C-index and in month-5/month-7 AUC between GBST and XGB/CoxBoost. If a CI includes zero, the LOOCV t-test was the only statistical support for superiority; if all CIs exclude zero, the mean advantage is robust and the invalid p-values are a reporting flaw rather than a load-bearing one.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that GBST 'significantly outperforms' all baselines (abstract; §3.1.5 and §3.2.3) is supported by paired t-tests on 10 leave-one-month-out folds. The folds' test sets are disjoint, but each training set contains 9 of the 10 month blocks, so the 10 model fits—and the fold-wise scores—are strongly correlated. A paired t-test assumes independence across folds; with positive correlation arising from shared training data, the standard error of the mean difference is understated and p-values are anti-conservative. This is consequential: several reported Lending Club advantages are tiny (month-5 AUC 0.7267 vs XGB 0.7252 with p<0.001; C-index 0.6867 vs XGB 0.6809), so the headline superiority can hinge on invalid significance tests. The risk-set construction for per-month AUC/KS is a secondary issue because the risk set at each month is identical across models, but the non-independence of folds undermines the statistical comparison itself.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GBST, a gradient boosting algorithm for discrete-time survival trees. The hazard is modeled via a logistic link f(t;x), represented by an additive ensemble of survival trees whose leaf weights are functions of time. The negative log-likelihood (Eq. 10) is optimized by gradient boosting; for a fixed tree structure, the optimal leaf weights have a closed form (Eq. 16) and a corresponding split gain (Eq. 18). Experiments on the Lending Club dataset and a private 360 Finance dataset compare GBST against Cox, RSF, XGBoost, CoxBoost, GBMCI, CIF, and DeepHit, using C-index, per-month AUC and KS, and survival-group default-rate plots. The paper claims that GBST significantly outperforms all existing models on both datasets.","tokens_in":23937,"tokens_out":9000,"duration_ms":88387,"significance":"If the empirical claims hold, GBST is a practically useful nonparametric model for discrete-time credit default risk. Its likelihood-based derivation is coherent, the leaf-weight update is closed-form and convex, and the code is publicly available. The paper addresses a real gap by testing survival models on large industrial credit datasets. However, the statistical evidence for the headline superiority claim rests on a flawed cross-validation significance test, so the strength of the empirical conclusions needs to be revised before the contribution can be fully assessed.","major_comments":[{"comment":"The paired t-tests used to declare GBST significantly better are invalid because the 10 leave-one-month-out folds have heavily overlapping training sets. Each training fold contains 9 of the 10 month blocks, so the 10 model fits and their fold-wise metrics are strongly positively correlated; the standard error of the mean difference is understated, making the p-values anti-conservative. Because several reported advantages are very small (e.g., month-5 AUC of 0.7267 vs 0.7252 for XGB; C-index 0.6867 vs 0.6809), the conclusion that GBST 'significantly outperforms' the baselines can hinge on an invalid test. Please redo the significance analysis with a method that accounts for training-set overlap, such as a block bootstrap resampling months, a corrected variance estimator for K-fold CV, or a mixed-effects model with month as a random effect, and report effect sizes with confidence intervals that respect the dependence structure.","section":"Section 3.1.5 (Tables 3 and 4; also Section 3.2.3, Tables 8 and 9)"},{"comment":"The text states that 'the difference between GBST and the other models are statistically significant (p-value < 0.05) with a few exceptions,' but Table 4 itself shows many non-significant entries, e.g., month-11 AUC (GBST 0.7161 vs XGB 0.7146, no marker), month-15 AUC (GBST 0.7081 vs XGB 0.7048, no marker), and month-19 AUC (GBST 0.6966 vs XGB 0.6930, no marker). This overstatement is independent of the test-validity issue and should be corrected so that the narrative matches the reported p-values.","section":"Section 3.1.5, Table 4 and Section 3.2.3, Table 9"}],"minor_comments":[{"comment":"The handling of censoring times that fall in the interior of an interval is implicit: for a censored observation with J(t)=j, the likelihood contribution is taken as S(τ_j), i.e., the survival probability at the interval endpoint. Please state this convention explicitly.","section":"Section 2.1, Eq. (6)"},{"comment":"The per-month AUC and KS are computed on the risk set at each month; this is a conditional risk-set metric, not a marginal time-dependent AUC with inverse-probability-of-censoring weighting. The text should describe the metric as conditional on survival to the start of the month.","section":"Section 3.1.5 (and Section 3.2.3)"},{"comment":"The claim that 'there exists no gradient boosting algorithm that is applied to survival trees' is too strong; existing implementations such as gradient boosting survival analysis in scikit-survival use tree-based learners. The distinction between those methods and the proposed survival-tree representation should be clarified.","section":"Introduction, Section 1"},{"comment":"The convergence experiment reports only the training loss; the statement that '30 iterations are sufficient' would be stronger if supported by a validation-loss or out-of-sample performance metric.","section":"Section 3.1.3"}],"recommendation":"major_revision","confidential_remarks":"The core methodology is sound, but the experimental significance testing is flawed and the significance narrative is overstated. The authors should be required to fix the statistical analysis and align the text with the reported p-values before publication. The novelty claim should also be sharpened against existing tree-based gradient boosting survival methods."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The model is real and the math is clean; the paper overclaims significance because the paired t-tests are run on non-independent folds.\n\nWhat's new and good: GBST is a gradient-boosted tree ensemble where each leaf returns a function of the observation time, trained by minimizing the discrete-time survival likelihood. That exact combination is absent from the cited literature, and the derivation in Equations (16)-(18) is internally consistent: the leaf weights and split gains follow from a convex second-order approximation, and the logit hazard transformation is standard. The authors also provide a public implementation and test on two large real credit datasets, which is a step up from the small clinical datasets common in survival analysis.\n\nThe soft spots: The abstract's 'significantly outperforms' claim rests almost entirely on 10 LOOCV experiments where the folds are month blocks. Each training set contains 9 of 10 blocks, so the ten model fits are strongly correlated. A paired t-test across those folds assumes independent differences; with positive correlation the standard error is understated and the p-values are anti-conservative. Several margins are tiny (month-5 AUC 0.7267 vs. 0.7252; C-index 0.6867 vs. 0.6809), so the flags really do depend on the invalid test. A repeated holdout with independent training folds or a clustered bootstrap would be the right fix. The per-month AUC/KS issue is secondary: because the risk set at each month is constructed the same way for every model, any censoring-induced bias in the metric applies to all models equally. The citation errors (Blakely & Cox, PyCox) should be corrected but are not load-bearing.\n\nBottom line: The method is worth using and the empirical evidence is directionally positive, but the strength of the conclusion exceeds the statistical support as written. A serious referee should ask for a corrected comparison design.\n\nMy recommendation: send to peer review. The model is novel enough, the derivation is sound, and the application is relevant. The paper needs revision rather than rejection.","headline":"The GBST model is a clean extension of XGBoost to survival trees, but the paper's significance claims lean on invalid LOOCV t-tests and need a corrected comparison.","tokens_in":24466,"tokens_out":5352,"would_cite":true,"duration_ms":57703,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper introduces gradient boosting survival trees, a nonparametric ensemble that models the chance of default in each repayment period, and reports that it outperforms seven established survival and credit-scoring models on two large…","keywords":["gradient boosting","survival trees","credit scoring","discrete-time hazard","concordance index","consumer finance","ensemble learning","survival analysis"],"falsifier":"Re-run the comparison with repeated train/test splits that share no training data and recompute per-month AUC and KS on all loans, weighting by the inverse probability of remaining uncensored; if GBST no longer leads on the concordance index, AUC, and KS, the paper's central claim is not supported.","tokens_in":23467,"feed_emoji":"💳","tokens_out":9975,"duration_ms":90739,"temperature":0.7,"pith_summary":"The paper proposes gradient boosting survival trees (GBST), a nonparametric ensemble method for predicting when a borrower will default. Instead of treating default as a simple yes/no label, GBST models the probability of default in each repayment period through a discrete-time hazard function, and it learns an additive ensemble of survival trees by minimizing the negative log-likelihood. The authors apply GBST to two large consumer-finance datasets and report that it achieves higher concordance, higher per-period AUC, and higher KS statistics than seven comparison models, including proportional-hazards, random-survival-forest, tree-boosting, and deep-survival methods. If these results hold, GBST gives lenders a practical tool for ranking risk and pricing loans by maturity, rather than only by lifetime default probability.","feed_headline":"Boosted survival tree beats seven rivals at loan-default timing","feed_subtitle":"It predicts the month a borrower will default and beat seven established survival and credit models on two datasets.","key_machinery":"The load-bearing object is the survival tree with time-dependent leaf weights. A tree maps a borrower's features to a leaf, and each leaf $l$ carries weights $w_l(\\tau_j)$ for every observation period $\\tau_j$, entering the hazard through the logit link $h(\\tau_j;x)=1/(1+\\exp(-\\hat f(\\tau_j;x)))$. The ensemble $\\hat f=\\sum_k f_k$ is grown additively: at each round the algorithm takes a second-order Taylor expansion of the negative log-likelihood, computes per-period gradients and Hessians of the logistic loss, and selects splits by the resulting loss reduction. The optimal leaf weight at period $j$ is the negative ratio of the summed gradient to the summed Hessian plus the $l_2$ penalty, which makes the per-period updates parallel and closed-form. Because the objective is minimized simultaneously across all periods, the model fits the whole survival curve rather than fitting each horizon independently.","core_discovery":"The paper's central claim is that a survival tree ensemble trained by gradient boosting on the discrete-time negative log-likelihood can outperform existing survival models on real credit data. Each leaf of a GBST tree carries a function of time rather than a single constant, so the ensemble produces an entire survival curve $\\hat S(t;x)$ for each borrower. The loss decomposes into a sum over observation periods, and each boosting round uses the first and second derivatives of the logistic loss with respect to the logit of the hazard, yielding closed-form leaf weights and an analytic split-gain criterion that updates all time periods simultaneously. On a public US loan dataset and a proprietary Chinese consumer-finance dataset, the authors report that GBST achieves the highest concordance index in their leave-one-month-out experiments (0.6867 and 0.7997, respectively) and the highest per-month AUC and KS values, with differences they report as statistically significant. The paper concludes that GBST is a practically useful nonparametric model for discrete-time credit default risk.","pith_inferences":["The same second-order likelihood boosting scheme could be adapted to competing risks, where default and prepayment compete, by giving each event its own hazard logit while sharing the tree structure.","Because the per-month metrics are computed only on loans still at risk, the late-horizon advantages may shift under a censoring-adjusted evaluation; a reader should recheck those numbers before relying on them.","The consistent margin over a strong tree-boosting classifier suggests that modeling the time of default, rather than collapsing it into a binary label, carries additional information for credit scoring.","The method is general: any discretely observed time-to-event problem in which some cases never experience the event during the observation window could use the same objective."],"forward_implications":["Lenders get a predicted survival curve for every borrower, so they can set different cutoffs for short- and long-term default risk instead of relying on one lifetime score.","The closed-form split gains and leaf weights let the method handle the high-dimensional, noisy application data typical of internet consumer finance after preprocessing.","Survival probabilities at each repayment month make GBST directly comparable with classification-based scores through AUC and KS at every horizon.","The reported convergence behavior means roughly 30 boosting rounds suffice, so the model can be retrained routinely in production.","If the reported gains generalize, GBST gives a nonparametric alternative to proportional-hazards and random-survival-forest models for discrete-time default prediction."],"supporting_citations":[{"why":"Supplies the gradient boosting framework that GBST adapts to survival trees, including the additive expansion and second-order optimization.","marker":"Friedman, 2001"},{"why":"Provides the XGBoost algorithm and the weighted-quantile approximation that GBST's fast split search extends, and supplies the XGB baseline.","marker":"Chen and Guestrin, 2016"},{"why":"Defines the concordance index (C-index) used as the primary global performance metric.","marker":"Harrell et al., 1982"},{"why":"Nonparametric estimator used to initialize the hazard probabilities for all individuals.","marker":"Kaplan and Meier, 1958"},{"why":"Random survival forests, the main tree-based survival baseline that GBST is compared against.","marker":"Ishwaran et al., 2008"},{"why":"CoxBoost, a boosting approach with proportional-hazards base models, serves as a comparison baseline.","marker":"Binder and Schumacher, 2008"},{"why":"GBMCI, a gradient boosting method that directly optimizes concordance, serves as a comparison baseline.","marker":"Chen et al., 2013"},{"why":"Conditional inference survival forests (CIF) are a comparison baseline for the empirical studies.","marker":"Wright et al., 2017"},{"why":"DeepHit, the deep survival model, serves as a comparison baseline.","marker":"Lee et al., 2018"}],"fun_headline_variants":["Gradient-boosted survival trees sharpen credit default timing","Boosted survival trees beat rivals in credit scoring","New boosting method predicts default timing with higher accuracy","Survival trees boosted to top credit-risk forecasts","Gradient boosting improves survival trees for credit risk"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole conclusion depends on the fairness of the comparison: if the statistical tests or the month-by-month accuracy measures are biased, GBST's claimed edge may disappear.","fun_headline_variants_meta":{"raw":{"variants":["Gradient-boosted survival trees sharpen credit default timing","Boosted survival trees beat rivals in credit scoring","New boosting method predicts default timing with higher accuracy","Survival trees boosted to top credit-risk forecasts","Gradient boosting improves survival trees for credit risk"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000615,"raw_usage":{"total_tokens":2846,"prompt_tokens":920,"completion_tokens":1926,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":536,"completion_tokens_details":{"reasoning_tokens":1853}},"tokens_in":536,"tokens_out":1926,"duration_ms":12565,"temperature":1.0,"reasoning_tokens":1853,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:15:39.197327+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the comparison with repeated train/test splits that share no training data and recompute per-month AUC and KS on all loans, weighting by the inverse probability of remaining uncensored; if GBST no longer leads on the concordance index, AUC, and KS, the paper's central claim is not supported.","supporting_citations":[{"cited_title":"and Guestrin, C","cited_arxiv_id":null,"evidence_quote":"Provides the XGBoost algorithm and the weighted-quantile approximation that GBST's fast split search extends, and supplies the XGB baseline."},{"cited_title":"E., Califf, R","cited_arxiv_id":null,"evidence_quote":"Defines the concordance index (C-index) used as the primary global performance metric."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Random survival forests, the main tree-based survival baseline that GBST is compared against."},{"cited_title":"and Schumacher, M","cited_arxiv_id":null,"evidence_quote":"CoxBoost, a boosting approach with proportional-hazards base models, serves as a comparison baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"GBMCI, a gradient boosting method that directly optimizes concordance, serves as a comparison baseline."},{"cited_title":"N., Dankowski, T., and Ziegler, A","cited_arxiv_id":null,"evidence_quote":"Conditional inference survival forests (CIF) are a comparison baseline for the empirical studies."},{"cited_title":"R., Yoon, J., and van der Schaar, M","cited_arxiv_id":null,"evidence_quote":"DeepHit, the deep survival model, serves as a comparison baseline."}],"review_version":1}