{"id":"efd4b5d8-10ad-4a0f-9555-9c6e4a50211a","arxiv_id":"1908.02334","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A k-NN classifier on multiparametric MRI voxels produces tissue composition estimates that associate with glioma grade and survival, but only in the same cohort used to train the classifier.","lead":"This paper tests a voxel-by-voxel MRI analysis that classifies glioma tissue into disease states and then uses the resulting tissue composition to predict survival, grade, and endothelial proliferation. The method is promising in principle, but the validation leaks information between training and testing, so the predictive claims are not established.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Voxel-level train/test split without patient separation invalidates predictive claims","rationale":"The reader's weakest assumption matches the most load-bearing concern: the lack of patient-level separation in train/test splits. The k-NN classifier is evaluated on voxels from the same patients used for training, so the high Dice scores reflect spatial correlation and patient-specific intensity patterns rather than generalization to new individuals. The subsequent PDC-outcome regressions and CDA are computed on the same 17 patients, meaning the p-values are not independent of the classifier's training data. The authors' own Discussion acknowledges this overfitting risk. A leave-one-patient-out cross-validation is the minimal test that would reveal whether the method truly predicts outcomes for unseen patients. Without such a test, the central claims cannot be substantiated, and the REJECT verdict is appropriate.","tokens_in":147,"tokens_out":1794,"duration_ms":25483,"concrete_test":"Re-run the entire pipeline with leave-one-patient-out cross-validation. For each of the 17 patients, train the k-NN classifier on all voxels from the other 16 patients, predict the held-out patient's voxel classes, compute PDC for that patient, then use these out-of-fold PDC values (plus age) in linear regression and CDA. If the resulting OS p-value, grade accuracy, and EP accuracy fall to non-significance or near-chance levels, the original claims are not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that PDC from multiparametric MRI can predict OS, grade, and endothelial proliferation depends on the k-NN classifier generalizing to unseen patients. The paper trains on 458,948 voxels randomly drawn from 14 patients and tests on the remaining 152,982 voxels from the same patients, with no patient-level holdout (Section 2.5). Because neighboring voxels share spatial autocorrelation and the same patient-specific intensity distributions, test accuracy (DSC 94.34%) largely reflects interpolation within the same scans, not classification of new patients. The PDC-outcome analyses (Section 3.2, Tables 4-5) then use these PDC values from the same 17 patients in linear regression and CDA, so the reported p-values (OS p=0.008, grade p=0.014, EP p=0.003) are not independent evidence of generalization. The authors explicitly acknowledge this risk in the Discussion: 'a voxel neighboring a train voxel will be assigned to the test feature vector and could lead to over fitting.' Unless patient-level separation is introduced, the predictive accuracy and p-values are inflated and do not support the conclusions.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a voxel-wise radiomics pipeline for glioma characterization: multiparametric MRI (T1, T1-GD, T2, FLAIR, ADC) is registered to T1-GD, expert annotations define nine tissue classes, a k-NN classifier is trained on 611,930 annotated voxels from 14 patients, and the classifier's predictions on 11 slices per patient yield Predicted Disease Compositions (PDC). These PDC are then used in linear regression and canonical discriminant analysis to predict overall survival, tumor grade, and endothelial proliferation in 17 patients. The paper reports a Dice similarity coefficient of 94.34% and significant p-values for OS (0.008), grade (0.014), and EP (0.003).","tokens_in":10236,"tokens_out":4621,"duration_ms":49921,"significance":"If the predictive claims were valid, the work would offer a non-invasive, voxel-resolved MRI method for assessing clinically important glioma features, with potential decision-support value. Strengths include the use of publicly available TCGA/TCIA data, a clinically grounded annotation scheme, and a clearly described machine-learning pipeline. However, the significance is currently contingent on a validation strategy that does not establish generalization to unseen patients; the voxel-level train/test split and in-sample outcome analyses substantially weaken the evidentiary value of the reported accuracies and p-values.","major_comments":[{"comment":"The k-NN classifier is trained and tested on voxels randomly split within the same 14 patients, with no patient-level separation (Section 2.5). Because neighboring voxels are spatially correlated and share patient-specific intensity distributions, the reported DSC of 94.34% largely reflects interpolation within the same scans, not classification performance on unseen patients. The authors acknowledge this risk in the Discussion ('a voxel neighboring a train voxel will be assigned to the test feature vector and could lead to over fitting'), but the abstract and conclusions still assert predictive accuracy. This is load-bearing: the PDC used in the outcome regressions (Section 3.2) and CDA are derived from the same patients, so the reported p-values (OS p=0.008, grade p=0.014, EP p=0.003) are not independent evidence of predictive utility. Patient-level cross-validation or a held-out patient cohort is required to support the central claim.","section":"Sections 2.5, 3.1, and 4"},{"comment":"The abstract states that linear combinations of PDCs and diagnostic age predicted OS (p=0.008), but Section 3.2 (Linear Regression) reports only a univariate regression of %Tumor on OS (F(1,15)=7.186, p=0.017). No multivariate model including age is presented. Please clarify which analysis yields p=0.008, report the full model, and reconcile the discrepancy.","section":"Abstract vs. Section 3.2"},{"comment":"The regression and canonical discriminant analyses are performed on the same 17 patients whose PDC were generated by the classifier, with no held-out validation or cross-validation for the outcome models. With only 10 LGG and 7 GBM patients, the reported classification accuracies (76.5%, 88.2%) and p-values are in-sample estimates and are likely optimistic. Please provide a validation scheme for the PDC-outcome associations (e.g., leave-one-patient-out for the full pipeline) or clearly label these results as exploratory.","section":"Section 3.2 and Table 5"},{"comment":"The authors state that linear regression is appropriate for OS because 'all cohort subjects experienced the same events.' This implies no censoring, but the paper does not explicitly confirm that all 17 patients had a recorded death event during follow-up. If any patient was censored, Cox proportional hazards regression or another survival analysis should be used. Please clarify the censoring status of all patients.","section":"Section 2.6"}],"minor_comments":[{"comment":"There are two subsections numbered '3.2' (Linear Regression and Canonical Discriminant Analysis). The second should be renumbered (e.g., 3.3).","section":"Section 3"},{"comment":"The row 'Parameters constant across sequences: Field strength 2.90 ±0.39 [1.50, 3.00]' lists a mean of 2.90 T, which is not a standard MRI field strength (typical values are 1.5 or 3.0 T). This appears to be a typo or a miscalculation; please correct.","section":"Table 2"},{"comment":"The caption states the model was tested using a '25% hold-out method.' Please specify that the hold-out was at the voxel level, not the patient level, to avoid ambiguity.","section":"Figure 2"},{"comment":"The k-NN model parameters in Table 3 include prior probabilities that appear to sum to approximately 0.9996; please verify that rounding is intentional and that the priors are based on the training set class frequencies.","section":"Section 2.5"}],"recommendation":"major_revision","confidential_remarks":"The manuscript addresses an important problem and the pipeline is clearly described, but the current validation design cannot support the predictive claims. The voxel-level train/test split and the use of in-sample PDC in outcome models are load-bearing issues. I recommend requiring patient-level validation (e.g., leave-one-patient-out for the classifier) and a clear reconciliation of the OS p-value discrepancy. With n=17, the revised results may be more modest, but the paper could then be positioned appropriately as a feasibility study."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know about this paper is that its main results—the p-values for OS, grade, and endothelial proliferation—come from an analysis where the classifier was trained and tested on voxels from the same patients. There is no patient-level holdout. The authors are upfront about this in the Discussion: “a voxel neighboring a train voxel will be assigned to the test feature vector and could lead to over fitting.” That is not a minor caveat; it is the load-bearing flaw. The 94% Dice score mostly tells you that the k-NN can interpolate within the same scans, not that it can classify a new patient. And the regression and discriminant analyses use those same in-sample PDC values, so the reported p-values are not evidence of generalization.\n\nThat said, the paper has real merits. The PDC concept—summarizing a tumor as percentages of disease components from a voxel-wise classifier—is a reasonable and potentially useful way to capture heterogeneity. The writing is clear, the methods are described in enough detail to reproduce (they even list exact k-NN parameters), and the authors use public TCGA/TCIA data. They also cite the relevant prior work (BraTS, Inano, Tian) and position their contribution honestly: an extension, not a brand-new framework. Credit where due.\n\nThe soft spots beyond the leakage: the %Tumor–OS association is partly a grade effect (GBM has more tumor and worse survival), and the sample is 17 patients. The authors acknowledge the small sample and protocol variability. None of this sinks the paper by itself; the leakage does.\n\nWho is this for? Someone working on radiomics or glioma imaging who wants to see the PDC framing, and anyone who wants a clean teaching example of why voxel-level cross-validation misleads. As a submission, I would reject it in current form, not because the idea is bad, but because the evidence does not support the claims as stated. Send it back with a request for patient-level cross-validation or an external cohort, and the paper could become a useful contribution. It deserves serious referee time, not a desk reject, because the direction is legitimate and the authors are transparent about the limitation.","headline":"The PDC idea is worth a look, but the voxel-level train/test split makes the predictive claims unsubstantiated; the authors admit the leakage themselves.","tokens_in":10810,"tokens_out":1303,"would_cite":false,"duration_ms":15839,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Voxel-wise MRI disease compositions predict glioma grade, endothelial proliferation, and overall survival.","keywords":["glioma","radiomics","multiparametric MRI","k-nearest neighbors","predicted disease composition","overall survival","endothelial proliferation"],"falsifier":"Retrain the k-NN model with a strict patient-level split (train on some patients, test on the remaining patients) and recompute the Dice score and the overall-survival, grade, and endothelial-proliferation statistics; if the p-values rise above 0.05 or the Dice score drops substantially below the reported 94%, the reported predictions come from voxel leakage rather than a generalizable imaging biomarker.","tokens_in":9828,"feed_emoji":"🧠","tokens_out":14082,"duration_ms":135583,"temperature":0.7,"pith_summary":"This paper sets out to show that routine multiparametric MRI, read voxel by voxel, can be converted into maps of glioma disease states and that the composition of those maps predicts outcomes. The authors train a k-nearest-neighbor classifier on 611,930 expert-annotated voxels across five MRI sequences, then apply it to about 13 million voxels in seventeen patients to produce per-slice percentages of suspicious tissue, edema, tumor, cyst, and necrosis—the predicted disease composition (PDC). They report that linear combinations of PDC components and diagnostic age predict overall survival, tumor grade, and endothelial proliferation (p = 0.008, 0.014, and 0.003 in the abstract), and that gene mutations for TP53BP1 and IDH1 were not significantly predicted. If correct, this would give clinicians a non-invasive way to estimate prognosis and tumor aggressiveness from imaging alone.","feed_headline":"MRI voxel maps predict glioma survival and grade","feed_subtitle":"A five-sequence MRI model turns tumor composition into survival and grade estimates without a biopsy.","key_machinery":"Predicted disease composition (PDC) is the central object: the per-slice percentage of voxels assigned by the k-NN classifier to each of five disease categories—suspicious, edema, tumor, cyst, and necrosis. The k-NN classifier is the mechanism that generates it: ten neighbors, Euclidean distance, standardized features, trained on a random three-quarters of 611,930 expert-annotated voxels and tested on the remaining quarter, then applied to 13,018,171 voxels from seventeen patients. These composition percentages, combined with diagnostic age, serve as the independent variables in linear regression and canonical discriminant analysis, so the whole argument reduces to whether the PDC percentages carry outcome information.","core_discovery":"The paper's central claim is that a voxel-wise k-nearest-neighbor model, trained on expert annotations of five MRI contrasts (T1, T1-GD, T2, FLAIR, ADC), can label individual voxels as one of five disease classes or four normal-tissue classes, and that the resulting slice-level percentages—the predicted disease composition—are clinically informative. In the authors' cohort, %Tumor alone explained 27.9% of the variability in overall survival through the equation Predicted OS = 7.67 − 0.037(%Tumor), and discriminant models using PDC plus age separated lower-grade glioma from glioblastoma (76.5% accuracy, p = 0.014) and endothelial proliferation status (88.2% accuracy, p = 0.003). The model's voxel labels agreed with expert annotations at a Dice similarity coefficient of 94.35% ± 2.98. The authors conclude that PDC derived from multiparametric MRI can act as a non-invasive imaging biomarker for glioma grade, endothelial proliferation, and overall survival.","pith_inferences":["A stricter test of the central claim would be patient-level cross-validation: training the k-NN on some patients and testing on the rest, which would reveal how much of the reported accuracy comes from voxel leakage between neighboring train and test voxels.","If PDC reflects biologically distinct tissue compartments, then serial MRI scans could track changes in %Tumor or %Necrosis over treatment; the paper's cross-sectional design does not test this.","The near-significant mutation results (TP53BP1 p = 0.097, IDH1 p = 0.054) suggest that adding texture, shape, or perfusion features to the PDC vector could push molecular prediction over the significance threshold.","Scanner harmonization of ADC maps may matter more than the other sequences, because DWI acquisition parameters varied most across patients and ADC was the only computed, rather than native, sequence."],"forward_implications":["Clinicians could estimate overall survival from the equation Predicted OS = 7.67 − 0.037(%Tumor) using only MRI, before any tissue is obtained.","Non-invasive discrimination of lower-grade glioma from glioblastoma at 76.5% accuracy could guide decisions about whether to biopsy, resect, or treat empirically.","The endothelial-proliferation model's 100% specificity suggests MRI-computed disease composition may identify highly angiogenic tumor zones that are the most informative biopsy targets.","Since IDH1 and TP53BP1 mutation status were not significantly predicted, PDC is best interpreted as a macroscopic tissue-composition biomarker, not a molecular surrogate.","A direct extension is to test whether PDC maps remain predictive across scanner manufacturers, field strengths, and multi-b-value DWI protocols, since the authors report their data varied on all of these."],"supporting_citations":[{"why":"Supplies the lower-grade versus glioblastoma classification and the patient-level outcome and genetic annotations used as dependent variables.","marker":"[3]"},{"why":"Provides the earlier voxel-wise MRI clustering method for glioma grading that this paper extends to multiparametric data and survival prediction.","marker":"[18]"},{"why":"Supplies the prior multiparametric MRI texture-feature approach to glioma grading that motivates combining multiple MRI sequences.","marker":"[19]"},{"why":"Provides the diffusion-weighted MRI formulation used to compute apparent diffusion coefficient maps, one of the five classifier features.","marker":"[20]"},{"why":"Gives the previous radiomics grading result whose specificity the authors use as a benchmark for their model's performance.","marker":"[24]"}],"fun_headline_variants":["No-biopsy MRI predicts glioma survival and grade","MRI voxel composition forecasts glioma outcomes","Radiomics maps glioma without biopsy","Voxel-wise MRI model estimates glioma survival"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a classifier trained on randomly selected voxels from the same patients—rather than on separate patients—produces disease-composition estimates that generalize to unseen patients, since neighboring voxels from the same tumor share nearly identical feature values.","fun_headline_variants_meta":{"raw":{"variants":["No-biopsy MRI predicts glioma survival and grade","MRI voxel composition forecasts glioma outcomes","Radiomics maps glioma without biopsy","Voxel-wise MRI model estimates glioma survival"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000668,"raw_usage":{"total_tokens":3141,"prompt_tokens":1134,"completion_tokens":2007,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":750,"completion_tokens_details":{"reasoning_tokens":1961}},"tokens_in":750,"tokens_out":2007,"duration_ms":14846,"temperature":1.0,"reasoning_tokens":1961,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:47:07.096178+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the k-NN model with a strict patient-level split (train on some patients, test on the remaining patients) and recompute the Dice score and the overall-survival, grade, and endothelial-proliferation statistics; if the p-values rise above 0.05 or the Dice score drops substantially below the reported 94%, the reported predictions come from voxel leakage rather than a generalizable imaging biomarker.","supporting_citations":[{"cited_title":"Comprehensive, integrative genomic analysis of diffuse lower-grade gliomas","cited_arxiv_id":null,"evidence_quote":"Supplies the lower-grade versus glioblastoma classification and the patient-level outcome and genetic annotations used as dependent variables."},{"cited_title":"Voxel -based clustered imaging by multiparameter diffusion tensor images for glioma grading","cited_arxiv_id":null,"evidence_quote":"Provides the earlier voxel-wise MRI clustering method for glioma grading that this paper extends to multiparametric data and survival prediction."},{"cited_title":"Diffusion weighted magnetic resonance imaging and its recent trend—a survey","cited_arxiv_id":null,"evidence_quote":"Provides the diffusion-weighted MRI formulation used to compute apparent diffusion coefficient maps, one of the five classifier features."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the previous radiomics grading result whose specificity the authors use as a benchmark for their model's performance."}],"review_version":1}