{"id":"087078bd-e16d-4982-9449-027a60ab64eb","arxiv_id":"2607.07247","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":5,"one_line_summary":"In a cardiac prediction model simulation, k-nearest neighbors imputation provided the most stable and computationally efficient performance across complex missingness scenarios, outperforming multiple imputation at high missingness.","lead":"This study simulated missing data on a real cardiac dataset and compared five imputation methods for building clinical prediction models. It found k-nearest neighbors imputation was fastest and most stable, while standard multiple imputation struggled with non-linear variables at extreme missingness.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"kNN's apparent stability advantage at extreme missingness may reflect variance compression and deterministic-imputation bias rather than superior data recovery, confounding the core comparison.","rationale":"The reader's concern about MAR generalizability is valid but standard. The more fundamental issue is internal to the comparison itself: the MAPE-based 'prediction stability' metric structurally favors deterministic methods (kNN, missForest) over stochastic ones (MICE) by design. The paper's own supplementary data (Tables S3-S4) reveals that kNN produces badly biased, low-variance imputations at 90% missingness—median eGFR of 89.2 vs. true ~73, SD of 14.8 vs. true ~25—yet this is reframed as 'stability' rather than recognized as a failure mode that happens to produce stable (but potentially misleading) predictions. The external validation advantage of kNN at 90% missingness is real in relative terms (AUC 0.68 vs. 0.55-0.58 for MICE/missForest), but all methods perform poorly in absolute terms, and kNN's edge may stem from variance compression neutralizing the missing variable rather than from better imputation. The paper would be more convincing if it explicitly tested whether kNN's advantage survives variance preservation, or if it framed the recommendation as 'kNN is less bad when all methods fail' rather than 'kNN is superior.' The simulation design is otherwise thorough—18 scenarios, bootstrap internal validation, external validation, computational timing—and the findings are genuinely informative. The verdict should remain CONDITIONAL: the results are interesting and well-executed, but the interpretation of kNN's stability advantage needs the proposed check before the recommendation can be considered robust. The reader's CONDITIONAL verdict with MODERATE confidence is appropriate.","tokens_in":43492,"tokens_out":5687,"duration_ms":338554,"concrete_test":"Re-run the 90% NLinCon and All Conditions scenarios with a modified kNN that preserves target variance (e.g., kNN with stochastic residual addition calibrated to observed SD, or kNN with much larger k to reduce donor bias). If the variance-preserving kNN maintains similar external validation AUC and calibration, the stability advantage is genuine. If performance deteriorates toward MICE levels, the advantage is an artifact of variance compression.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that kNN provides 'superior prediction stability' in complex scenarios at 90% missingness is confounded by a mechanical property: kNN is deterministic single imputation, so it will necessarily show lower bootstrap-to-bootstrap variability than MICE's stochastic multiple imputation. More critically, Supplementary Table S3 shows that at 90% missingness in non-linear scenarios (NLinCon), kNN dramatically shrinks the variance of imputed eGFR (SD=14.8 vs. CCA reference 25.1) and substantially biases the median (89.2 vs. true 72.9, per Supplementary Table S4). MICE methods preserve variance much better (SD ~24-26) and produce medians closer to truth (~65-70). kNN's 'stability' thus appears to result from effectively neutralizing the missing variable's signal through low-variance, biased imputations, rather than from better data recovery. The paper acknowledges variance shrinkage and median overestimation but frames the resulting prediction stability as an advantage. The external validation results (AUC ~0.68 for kNN vs. reference 0.71) confirm that kNN is still substantially degraded at 90% missingness—it is merely less degraded than alternatives. Whether 'stable but biased' imputation is genuinely preferable to 'unbiased but variable' imputation for clinical prediction remains unaddressed.","agreement_with_reader":"partial"},"referee_report":{"model":"glm-5.2","summary":"This simulation study compares five missing-data strategies (complete case analysis, MICE-FCS, MICE-PMM, missForest, and kNN) for clinical prediction model development using a real-world cardiac cohort of 8,245 patients. Missingness was artificially induced under a missing-at-random (MAR) mechanism across 18 scenarios varying by variable type, predictor-outcome relationship, and missingness proportion (30%, 60%, 90%). The study evaluates imputed-value integrity, model-structure stability, optimism-corrected performance, prediction stability (MAPE), external validation performance, and computation time. The central finding is that while all methods perform comparably in simple scenarios at moderate missingness, kNN demonstrates the most consistent prediction stability and computational efficiency in complex non-linear scenarios at extreme (90%) missingness, where MICE methods show increased instability and missForest overfits externally. The simulation design is thorough in its scenario coverage and evaluation metrics, and the use of both internal bootstrap validation and external validation is commendable.","tokens_in":44323,"tokens_out":1509,"duration_ms":251699,"significance":"The study addresses a practically important question for clinical prediction model development: how imputation method choice interacts with predictor-outcome complexity and missingness severity to affect model stability and transportability. The multi-dimensional evaluation framework—spanning imputed-value fidelity, model-structure stability, individual-level prediction stability, and external validation—is a genuine strength that goes beyond standard discrimination/calibration reporting. The inclusion of computation time is practically relevant for deployment considerations. The finding that method performance depends on variable characteristics and missingness complexity is useful for practitioners. However, the significance of the central claim regarding kNN's superiority is tempered by a fundamental confound between deterministic single imputation and stochastic multiple imputation in the measurement of prediction stability, which must be addressed before the conclusions can be considered well-supported.","major_comments":[{"comment":"The central claim that kNN provides 'superior prediction stability' in complex scenarios at 90% missingness is confounded by a mechanical property: kNN is a deterministic single-imputation method, so it will necessarily show lower bootstrap-to-bootstrap variability than MICE's stochastic multiple imputation. The MAPE metric, as defined in §Model evaluation, measures the absolute difference between bootstrap-derived and original-development-model predictions; a deterministic imputation method will mechanically produce smaller such differences. More critically, Supplementary Table S3 shows that at 90% missingness in the NLinCon scenario, kNN dramatically shrinks the variance of imputed eGFR (SD=14.8 vs. CCA reference 25.1) and substantially biases the median (89.2 vs. true 72.9, per Supplementary Table S4). MICE methods preserve variance much better (SD ~24-26) and produce medians closer (","section":null},{"comment":"The external validation results in Supplementary Table S7 at 90% missingness for NLinCon show that kNN achieves external AUC of 0.68 (vs. reference 0.71), while missForest achieves 0.56 and MICE-PMM achieves 0.58. While kNN is indeed the least degraded, an AUC of 0.68 represents a meaningful loss from the 0.71 reference. The manuscript's framing of kNN as providing 'better external validation results' (Abstract) and 'superior prediction stability' (Conclusion) without adequately emphasizing that all methods are substantially degraded at 90% missingness risks overstating the practical implications. The authors should explicitly state the magnitude of performance degradation relative to the complete-data reference and discuss whether 'stable but biased' imputation is genuinely preferable for clinical practice, particularly given that the biased imputed values (e.g., eGFR median overestimat","section":null}],"minor_comments":[{"comment":"The Declaration of Generative AI states that ChatGPT-5 and Gemini were used for language editing. ChatGPT-5 does not appear to be a released model as of the manuscript date; the authors should verify and correct this reference.","section":null},{"comment":"In the Methods (§Handling of missing data), the outcome variable was excluded from imputation to prevent data leakage. While this is a defensible choice, it is at odds with standard recommendations for multiple imputation in prediction modeling (e.g., references 10, 17), where including the outcome in the imputation model is generally advised to preserve predictor-outcome associations. The rationale for this choice should be briefly discussed.","section":null},{"comment":"Supplementary Table S2 describes amputation weights but the text in the Missing Data Simulation Mechanics appendix uses slightly different variable lists (e.g., 'dbp' appears in some descriptions but not others). The mapping between the prose description and the table should be made consistent.","section":null},{"comment":"Figure 4 caption references 'Red dashed vertical line indicates optimal MAPE % threshold at 20%' but the text in §Model evaluation does not explicitly define a 20% MAPE threshold. This threshold should be defined in the Methods or the caption should be made more self-contained.","section":null},{"comment":"The sample size justification (§Sample size calculation) cites a minimum of 1,647 participants based on 18 predictors and 13% prevalence, then uses 4,123 as approximately 2.5-fold. The rationale for choosing 2.5-fold specifically (rather than 2-fold or 3-fold) is not provided and should be briefly discussed.","section":null},{"comment":"Reference 21 (Albu et al., 2025) is cited for missForest implementation but appears to describe a different package ('missForestPredict'). The original missForest reference (Stekhoven & Bühlmann 2012, ref 15) is also cited; clarification on which implementation was used would be helpful.","section":null},{"comment":"In the Results §3, the text states 'the mean and SD of the linear predictor (LP) were similar across most scenarios; however, at 90% missingness in the LinMix, NLinCon, NLinMix, and All conditions, MICE, particularly MICE-FCS, produced a notably high variance of mean and SD of the LP.' Supplementary Table S5 should be referenced here, and the specific values illustrating this high variance should be cited.","section":null},{"comment":"The term 'prediction stability' is used throughout but is defined somewhat indirectly via the MAPE metric. A brief, explicit definition in the Introduction or Methods (e.g., 'the degree to which a CPM produces consistent risk estimates for the same individual when developed on different samples from the same target population') would improve clarity for readers unfamiliar with the concept.","section":null}],"recommendation":"major_revision","confidential_remarks":"The skeptic's concern about the deterministic-imputation confound is well-founded and is the primary reason for the major revision recommendation. The authors have done substantial work and the simulation infrastructure is impressive, but the framing of kNN's 'stability advantage' needs to be substantially recontextualized. The paper would benefit from either (a) reframing the stability comparison to acknowledge the deterministic-vs-stochastic confound explicitly, or (b) adding a sensitivity analysis using a stochastic variant of kNN (e.g., random neighbor selection) to disentangle the mechanism. Option (a) is likely sufficient for revision. The single-dataset limitation is acknowledged by the authors and is acceptable for an exploratory simulation study, though it should be more prominently discussed in the limitations."},"author_rebuttal":null,"desk_editor":{"model":"glm-5.2","letter":"This paper compares five missing-data strategies (CCA, MICE-FCS, MICE-PMM, missForest, kNN) across 18 simulated missingness scenarios on a real cardiac cohort of 8,245 patients. The headline finding: at 30–60% missingness with linear or categorical variables, all methods perform comparably (AUC ~0.75). At 90% missingness or with non-linear variables, performance degrades across the board, but kNN degrades least and is fastest to run. The simulation design is genuinely thorough — 18 scenarios, bootstrap internal validation with optimism correction, external validation, and assessment of imputed-value integrity alongside predictive performance. That multi-dimensional evaluation is the paper's real contribution. Most imputation comparisons stop at AUC; this one tracks variance of imputed values, model-structure stability, and computation time too. That said, the central claim about kNN's superiority at extreme missingness has a soft spot the paper doesn't fully grapple with. kNN is deterministic single imputation, so it will mechanically show lower bootstrap-to-bootstrap variability than MICE's stochastic approach. More importantly, the supplementary tables show kNN severely compresses variance and biases medians at 90% missingness for non-linear variables — eGFR SD of 14.8 versus the reference 25.1, median of 89.2 versus true 72.9. The paper acknowledges this shrinkage but frames the resulting prediction stability as an advantage. Whether 'stable but biased' imputation is genuinely preferable for clinical prediction is a real question that goes unaddressed. The external validation confirms kNN is still substantially degraded at 90% (AUC ~0.68 vs. reference 0.71) — just less degraded than alternatives. The single-dataset limitation is real but acknowledged honestly, as is the restriction to MAR mechanisms. The paper positions itself as exploratory, which is appropriate. This deserves a serious referee. The simulation work is careful and the multi-dimensional evaluation framework is worth publishing. The interpretation of kNN's stability advantage needs sharpening — a good reviewer should push the authors to explicitly discuss the deterministic-imputation confound and whether their recommendation holds when the goal is unbiased data recovery rather than prediction stability alone.","headline":"Solid simulation study on imputation methods for clinical prediction models; the kNN advantage at extreme missingness is real but partly mechanical and needs careful interpretation.","tokens_in":44209,"tokens_out":1534,"would_cite":false,"duration_ms":112069,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"Simple kNN beats MICE for missing clinical data at high rates","keywords":[],"falsifier":"If a different real-world clinical dataset with naturally occurring missingness (rather than artificially induced) showed that MICE methods maintained superior prediction stability and external validation performance compared to kNN in non-linear or extreme missingness scenarios, the paper's central claim would be undermined.","tokens_in":43838,"feed_emoji":"🩺","tokens_out":830,"duration_ms":294772,"temperature":0.7,"pith_summary":"This paper asks a practical question: when developing clinical prediction models from real patient data with missing values, which imputation method should you use? The authors take a fully observed cardiac cohort of 8,245 patients, artificially introduce missingness under a missing-at-random (MAR) mechanism across 18 scenarios (varying variable type, predictor-outcome relationship, and missingness proportion from 30% to 90%), and compare five strategies: complete case analysis, MICE-FCS, MICE-PMM, missForest, and kNN. The core finding is that when missingness involves non-linear variables or reaches extreme levels (90%), the standard multiple imputation approaches (MICE) become unstable and overfit, while the simplest method tested, k-nearest neighbors imputation, maintains stable predictions, performs better on external validation, and runs 2-5 times faster than MICE. The paper argues that for sufficiently large development samples, kNN offers a computationally efficient and robust alternative to multiple imputation, particularly in complex missingness scenarios.","feed_headline":"Simple kNN beats MICE for missing clinical data at high rates","feed_subtitle":"When clinical prediction models face complex or extreme missing data, a simple k-nearest neighbors approach outperforms standard multiple im","key_machinery":"The key machinery is the simulation pipeline itself: a fully observed real-world cardiac dataset is split into development and validation cohorts, missingness is artificially induced under MAR using multivariate amputation across 18 scenarios, five imputation methods are applied, logistic regression models with backward stepwise elimination are developed on bootstrap samples, and performance is evaluated using optimism-corrected AUC, calibration slope, mean absolute prediction error (MAPE), external validation, and computation time. The comparison of these metrics across methods and scenarios is the mechanism that carries the argument.","core_discovery":"The central discovery is that the relative performance of imputation methods depends critically on the structure of the missing data. When missingness is isolated to linear or categorical variables at moderate levels (30-60%), all methods perform comparably. However, when missingness involves non-linear variables or reaches 90%, MICE methods show increased prediction instability and overfitting, missForest overfits substantially in external validation, while kNN maintains the most consistent performance with the shortest computation time. This suggests that the complexity of the imputation method is not always beneficial, and a simpler deterministic approach may be preferable in complex, non","pith_inferences":[],"forward_implications":["Clinical prediction model developers may consider kNN imputation as a default strategy when dealing with large datasets and complex or extreme missingness, rather than defaulting to MICE.","The finding that method performance depends on variable characteristics (linear vs. non-linear) suggests that imputation strategy selection should be guided by exploratory data analysis of the missing variables' relationships.","The computational efficiency of kNN (2-5x faster than MICE) could enable more rapid model development and validation cycles in clinical research settings with large datasets.","The observed overfitting of missForest in external validation under non-linear scenarios suggests caution when using machine learning imputation methods that do not account for imputation uncertainty."],"fun_headline_variants":["Complex imputation overfits clinical models at high missingness; kNN stays stable","kNN outperforms MICE and missForest for extreme clinical data missingness","For complex clinical missing data, kNN imputation beats standard MICE methods","kNN provides fastest and most stable imputation for clinical prediction models","Standard MICE overfits clinical models at high missingness; kNN remains stable"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The study assumes that the artificial missingness generation process, which creates missing data under a Missing at Random (MAR) mechanism with specific weight configurations, accurately reflects real-world clinical missingness patterns. If real-world missingness follows different mechanisms, such as Missing Not at Random (MNAR), the relative performance of the imputation methods may not generalize.","fun_headline_variants_meta":{"raw":{"variants":["Complex imputation overfits clinical models at high missingness; kNN stays stable","kNN outperforms MICE and missForest for extreme clinical data missingness","For complex clinical missing data, kNN imputation beats standard MICE methods","kNN provides fastest and most stable imputation for clinical prediction models","Standard MICE overfits clinical models at high missingness; kNN remains stable"]},"model":"glm-5.2","effort":"low","cost_usd":0.0,"raw_usage":{"total_tokens":1679,"prompt_tokens":607,"completion_tokens":1072,"prompt_tokens_details":null},"tokens_in":607,"tokens_out":1072,"duration_ms":61280,"temperature":1.0,"reasoning_tokens":968,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-09T16:08:25.930450+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"If a different real-world clinical dataset with naturally occurring missingness (rather than artificially induced) showed that MICE methods maintained superior prediction stability and external validation performance compared to kNN in non-linear or extreme missingness scenarios, the paper's central claim would be undermined.","supporting_citations":[],"review_version":1}