{"id":"1ef26b98-caae-4ba2-9828-29be6efd418b","arxiv_id":"2608.09721","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Pairwise multi-label learning improved PET/CT mutation prediction for KRAS and TP53 but not for EGFR/TP53 in a single-centre UK cohort of 263 NSCLC patients.","lead":"This study tested whether predicting two gene mutations at once improves PET/CT-based mutation detection in lung cancer compared with predicting each gene separately. On a small UK patient cohort, joint prediction helped only for one gene pair, suggesting that the benefit depends heavily on which mutations are modelled.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"AUC differences between single-gene and multi-label models are within seed-to-seed variability; no significance testing is reported, so the pair-dependent benefit claim is not established.","rationale":"The reader's weakest assumption was patient-level leakage from repeated PET/CT examinations split across training and test sets. This is a legitimate concern, but it may not be the most load-bearing issue. Since single-gene and multi-label models use the same split within each pair, leakage would tend to inflate both arms similarly and may not systematically distort the single-vs-multi-label comparison, although it could still affect absolute AUCs and might interact with class imbalance. The more direct threat to the central claim is the absence of any statistical significance testing: the reported AUC differences are comparable to or smaller than the standard deviations across five seeds. Without paired tests or confidence intervals, the observed improvements in KRAS/TP53 and the lack of improvement in EGFR/TP53 cannot be distinguished from random variation. This is an internal correctness concern, not a matter of external consensus, and it is concrete and checkable from the reported table. The paper remains a valuable hypothesis-generating study on a novel UK cohort, so conditional acceptance with a request for significance testing is appropriate. My assessment therefore leaves the reader's CONDITIONAL verdict unchanged.","tokens_in":7001,"tokens_out":2904,"duration_ms":26917,"concrete_test":"Perform a paired statistical comparison of single-gene vs multi-label test AUC for each gene within each pair using the five random-seed results. Because single-gene and multi-label models share the same stratified train/validation/test split for each pair, a paired Wilcoxon signed-rank test (or paired t-test on logit-transformed AUC) with n=5 can be applied. Report exact p-values and 95% bootstrap confidence intervals for the AUC difference. If the KRAS and TP53 improvements yield p>0.05, the headline claim of pair-dependent benefit is not established; if all comparisons are non-significant, the central conclusion should be revised to a hypothesis-generating observation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that multi-label learning improves KRAS/TP53 but not EGFR/TP53, supporting mutation-specific modelling strategies. However, Table 4 shows all reported AUC differences are within one standard deviation of the five-seed means: KRAS 0.575±0.068 vs 0.641±0.050, TP53 0.689±0.100 vs 0.712±0.038, EGFR/KRAS EGFR 0.718±0.057 vs 0.746±0.093, and EGFR/TP53 TP53 0.514±0.107 vs 0.491±0.048. No paired significance test, confidence interval, or effect-size measure is reported anywhere in Section 3 or the tables. The observed 'improvements' may therefore be seed noise, and the 'no improvement' findings may be underpowered rather than evidence of absent benefit. The abstract's numeric claims (0.58 to 0.64, 0.69 to 0.71) are thus not statistically supported, and the recommendation to prefer mutation-specific modelling strategies rests on an unverified difference. The repeated-scan leakage flagged by the reader is also a concern, but even if leakage were fully eliminated, without significance testing the central conclusion would remain unsupported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies PET/CT-based radiogenomic prediction of EGFR, KRAS, and TP53 mutation status in non-small cell lung cancer using a ResNet-18 backbone, comparing single-gene classification with pairwise multi-label learning. It introduces a novel UK cohort (MIRAI, 263 patients / 293 PET/CT examinations) and reports that joint learning improves AUC for the KRAS/TP53 pair, improves only EGFR for the EGFR/KRAS pair, and provides no improvement for the EGFR/TP53 pair. The authors conclude that multi-label learning should not be applied uniformly and that mutation-specific modelling strategies may be preferable.","tokens_in":7237,"tokens_out":2328,"duration_ms":19642,"significance":"If the central comparison is statistically sound, the paper makes a useful contribution to PET/CT radiogenomics: it is among the first systematic evaluations of pairwise multi-label learning for mutation prediction in NSCLC, and it provides a new real-world UK cohort (MIRAI) and an openly described preprocessing pipeline, including an nnU-Net segmentation module validated on held-out manual segmentations (Table 3, Dice 0.78±0.12). The finding that multi-label benefit is gene-pair-dependent would also inform practical choices for radiogenomic modelling. However, the headline conclusion currently rests on AUC differences that fall within seed-to-seed variability, and the evaluation protocol admits patient-level leakage, so the significance of the empirical claim is not yet established.","major_comments":[{"comment":"The central claim that multi-label learning improves KRAS/TP53 but not EGFR/TP53 is not supported by any statistical testing. All reported AUC differences are within one standard deviation of the five-seed means, e.g., KRAS 0.575±0.068 vs 0.641±0.050, TP53 0.689±0.100 vs 0.712±0.038, EGFR 0.718±0.057 vs 0.746±0.093, and TP53 0.514±0.107 vs 0.491±0.048. No paired significance test, confidence interval, or effect-size measure is reported. Since the same splits and seeds are used for single-gene and multi-label models, a paired statistical comparison (e.g., Wilcoxon signed-rank or bootstrap over patients) is feasible and should be provided; without it, the observed differences may be seed noise, and the 'no improvement' findings may be underpowered rather than evidence of absent benefit.","section":"§3, Table 4"},{"comment":"Repeated PET/CT examinations from the same patient are treated as independent samples and randomly assigned to training, validation, and test sets. If two scans from one patient appear in both training and test, the model can memorise patient-specific appearance, inflating AUC and distorting the single-gene versus multi-label comparison. The analysis should be repeated with a patient-level split (all scans from a given patient in the same fold), and the number of patients versus PET/CT examinations should be reported for each gene-pair cohort. This is a required robustness check for the quantitative conclusions.","section":"§2.4, §3"},{"comment":"The exclusion of label combinations occurring fewer than six times removes all EGFR+/TP53+ cases (footnote a in Table 2), so the EGFR/TP53 multi-label model never sees the joint-positive class. The conclusion that 'no improvement was observed for the EGFR/TP53 pair' is therefore limited to the sub-cohort without dual-positive cases and should be stated as such. The analysis also changes the target distribution by exclusion; a sensitivity analysis reporting results on the full cohort, or an explicit discussion of why such exclusion is unavoidable, is needed before the pair-dependent benefit claim is accepted.","section":"§2.4, Table 2"},{"comment":"The evaluation reports mean±SD over five random seeds, which is a good start, but the spread in some cells is very large (e.g., EGFR AUC 0.783±0.357 in the joint EGFR/TP53 model). The paper should report the full distribution of results across seeds, not only the mean, and should indicate whether the qualitative pattern (improvement for KRAS/TP53, no improvement for EGFR/TP53) is stable across individual seeds. This is essential because the claim is about a consistent effect, not a single seed outcome.","section":"§2.4, Table 4"}],"minor_comments":[{"comment":"The running title contains a typo, 'Raiodgenomic' instead of 'Radiogenomic'; the same typo also appears in the header on page 5.","section":"Header, page 3"},{"comment":"The authors state that label combinations occurring fewer than six times were excluded, but they do not specify how many patients/scans were removed for each gene pair beyond the aggregate N reported in Table 2. Please state the number of excluded examinations and the number of affected patients for each pair.","section":"§2.4"},{"comment":"The stratification is described as being based on target labels, but for multi-gene cohorts the target is a pair label while single-gene models use individual labels. It should be clarified whether the 'same patient split' means exactly the same assignment of examinations to folds for single-gene and multi-label models, and whether the stratification was recomputed for single-gene labels.","section":"§2.4, Table 2"},{"comment":"The bold formatting indicating the better-performing model per gene is visually clear, but it would be helpful to add a note that the values are means over five seeds and that the bold does not imply statistical significance.","section":"§3, Table 4"},{"comment":"The discussion of negative transfer for KRAS in the EGFR/KRAS pair and the explanation based on class-imbalance weighting would be more convincing if accompanied by per-seed results or a simple ablation varying the positive weight; as written, the explanation is plausible but not directly supported by an experiment.","section":"§3"},{"comment":"The MIRAI dataset reference [2] is cited as 'In press'; if this is a companion paper, it would be helpful to state explicitly that the current work is based on a subset of MIRAI (108 of 130 segmented volumes with mutation labels) and to clarify the overlap with the dataset description paper.","section":"§2.1"}],"recommendation":"major_revision","confidential_remarks":"The core idea—systematically comparing single-gene and pairwise multi-label PET/CT radiogenomic prediction on a novel UK cohort—is appropriate for the journal, and the MIRAI cohort is a genuine contribution. However, the manuscript's central claim is currently vulnerable on two fronts that are fully fixable within the scope of a revision: the absence of paired significance testing and the risk of patient-level leakage from random scan-level splitting. If the authors can add paired statistical tests and rerun with patient-level splits, the conclusions may well hold; if the improvements disappear under those conditions, the paper's conclusion would need to be substantially softened. I would also request the authors to explicitly address the EGFR+/TP53+ exclusion in the interpretation, because the current wording overgeneralizes."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper does something genuinely new: it compares single-gene and pairwise multi-label deep learning for PET/CT-based mutation prediction in NSCLC on a fresh UK cohort, using the same patient splits for each comparison. The segmentation pipeline, with manual masks and radiologist review, is careful, and the authors are honest about limitations.\n\nThe main claim—that multi-label benefit depends on the gene pair—is not actually supported by the numbers. In Table 4, every AUC difference between joint and single models is smaller than the five-seed standard deviations: KRAS 0.575±0.068 to 0.641±0.050, TP53 0.689±0.100 to 0.712±0.038, EGFR 0.718±0.057 to 0.746±0.093, and the EGFR/TP53 pair shows the same pattern. No paired significance test, confidence interval, or effect size is reported. The abstract's \"0.58 to 0.64, 0.69 to 0.71\" sound precise but are within seed-to-seed noise. Without significance testing, the results are suggestive at best.\n\nThe data split is a second real problem. Repeated scans from the same patient are treated as independent and randomly assigned across train/validation/test. With 293 scans from 263 patients, roughly 30 patients have two scans that can land in both training and test, letting the model memorize patient-specific appearance and inflate AUC. Patient-level splitting is needed. This could affect absolute performance and the relative comparison, though it is hard to know the direction.\n\nThe third soft spot: the EGFR/TP53 pair excludes all EGFR+/TP53+ cases due to the rare-label threshold. So the \"no improvement\" finding is really about two genes that do not co-occur in the analyzed cohort; it is not a strong test of multi-label benefit for co-mutation prediction. The authors acknowledge this, but it weakens their central conclusion.\n\nWhat is good: the cohort is new and clinically relevant, the methods are standard and described clearly, and the paper honestly lists its limitations. It is a hypothesis-generating study, not a definitive one. The recommendation to prefer mutation-specific modelling strategies goes beyond what the data currently demonstrate.\n\nI would send this to peer review, but with a firm request for statistical analysis (paired tests or confidence intervals) and patient-level splitting. Even then, the findings should be framed as preliminary. It is worth reading for anyone working on radiogenomics or multi-task learning evaluation, but not as a settled result.","headline":"Useful cohort study, but the central pair-dependent benefit claim is not statistically established and the repeated-scan splitting is a patient-leakage risk.","tokens_in":7766,"tokens_out":2812,"would_cite":false,"duration_ms":24100,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Pairwise multi-label learning improves PET/CT mutation prediction only for certain NSCLC gene pairs.","keywords":["radiogenomics","non-small cell lung cancer","PET/CT","multi-label learning","mutation prediction","EGFR","KRAS","TP53"],"falsifier":"Recompute the KRAS/TP53 comparison with a strict patient-level split, placing all scans of each patient in the same set; if the joint-model AUC gains vanish or reverse, the conclusion that joint learning helps this pair is an artefact of scan-level leakage rather than shared imaging signal.","tokens_in":6812,"feed_emoji":"🧬","tokens_out":4081,"duration_ms":25625,"temperature":0.7,"pith_summary":"Radiogenomics tries to read tumour gene mutations from medical images so patients can avoid invasive biopsies. This paper asks whether a deep-learning model trained to predict two gene mutations at once from PET/CT images outperforms separate single-gene models in non-small cell lung cancer. On a UK cohort of 263 patients, it finds the answer is not uniform: joint training raises AUC for the KRAS and TP53 pair, helps only EGFR in the EGFR/KRAS pair, and helps neither gene in the EGFR/TP53 pair. The paper concludes that multi-label learning should be applied selectively rather than as a default, and that mutation-specific modelling strategies may be preferable.","feed_headline":"Joint training boosts only some PET/CT mutation predictions","feed_subtitle":"KRAS and TP53 improve together, EGFR and TP53 do not; gene-pair-specific models may win.","key_machinery":"The load-bearing mechanism is pairwise multi-label learning: one ResNet-18 backbone with two sigmoid outputs trained jointly, so the two mutation tasks share a learned image representation. The paper tests whether this shared representation produces positive transfer, negative transfer, or neither for each gene pair. The comparison is controlled by training single-gene models on the identical patient split and by weighting the binary cross-entropy loss per gene to counter class imbalance.","core_discovery":"On its own terms, the paper's discovery is that pairwise multi-label learning transfers positively only between certain gene-mutation tasks. With a ResNet-18 taking the largest axial PET slice, the corresponding CT slice, and the tumour mask as input, joint KRAS/TP53 prediction increased test AUC from 0.575 to 0.641 for KRAS and from 0.689 to 0.712 for TP53 relative to single-gene baselines. For EGFR/KRAS, joint learning improved EGFR from 0.718 to 0.746 while KRAS fell from 0.632 to 0.557; the two mutations were mutually exclusive in this cohort. For EGFR/TP53, joint learning did not improve either task, with AUC dropping from 0.808 to 0.783 for EGFR and from 0.514 to 0.491 for TP53. The authors' conclusion is that benefit depends on the gene pair, so uniform adoption of multi-label learning is not warranted.","pith_inferences":["The paper does not split by patient, so its AUC comparison may be optimistic; a patient-level split could shrink or reverse some pair gains, especially for patients with two scans.","One testable extension is to measure gradient conflict or representation similarity between tasks; if KRAS/TP53 gradients align while EGFR/TP53 conflict, that would mechanistically explain the observed transfer pattern.","Because only 2D single slices were used, 3D tumour-volume models might change which pairs benefit; the pair ordering found here should not be assumed to carry over to volumetric inputs.","The conclusion that mutation-specific strategies are preferable would be strengthened or weakened by external multi-centre validation, which the paper lists as future work."],"forward_implications":["If KRAS and TP53 genuinely share imaging correlates, joint training is a cheap way to improve both predictions without new data.","If the EGFR/KRAS asymmetry holds, a hybrid pipeline that predicts EGFR jointly with KRAS but reports KRAS from a single-gene model would dominate either uniform strategy.","If EGFR and TP53 do not benefit from shared learning, future radiogenomic systems should treat task selection as an empirical per-pair question.","If mutation-specific modelling is adopted, clinical workflows can match the model to the gene of interest rather than assuming one multi-label network fits all."],"supporting_citations":[{"why":"Supplies the novel MIRAI PET/CT radiogenomics cohort used for all experiments.","marker":"[2]"},{"why":"Documents the lack of PET/CT and multi-label radiogenomic mutation studies that this work addresses.","marker":"[3]"},{"why":"Prior PET/CT radiomics prediction of EGFR/TP53 co-mutation that this study extends to deep multi-label learning.","marker":"[12]"},{"why":"Provides the shared-representation and transfer rationale for expecting joint learning to help.","marker":"[14]"},{"why":"Frames joint mutation prediction as multi-label learning and motivates the pairwise comparison.","marker":"[17]"},{"why":"Shows a multi-task CT-based radiogenomic approach, the closest existing comparison for simultaneous prediction.","marker":"[20]"}],"fun_headline_variants":["Multi-label learning helps only select PET/CT mutation pairs","PET/CT mutation prediction: multi-label gain depends on gene pair","Joint learning helps some gene pairs in PET/CT mutation scans","Multi-label radiogenomics: only certain mutation pairs benefit","PET/CT mutation prediction: joint training not universally beneficial"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Repeated PET/CT scans from the same patient were treated as independent samples and randomly split across training, validation, and test, so the model could memorise patient-specific appearance and inflate the reported AUC differences.","fun_headline_variants_meta":{"raw":{"variants":["Multi-label learning helps only select PET/CT mutation pairs","PET/CT mutation prediction: multi-label gain depends on gene pair","Joint learning helps some gene pairs in PET/CT mutation scans","Multi-label radiogenomics: only certain mutation pairs benefit","PET/CT mutation prediction: joint training not universally beneficial"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000647,"raw_usage":{"total_tokens":2993,"prompt_tokens":990,"completion_tokens":2003,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":606,"completion_tokens_details":{"reasoning_tokens":1929}},"tokens_in":606,"tokens_out":2003,"duration_ms":10434,"temperature":1.0,"reasoning_tokens":1929,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T12:00:48.109934+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the KRAS/TP53 comparison with a strict patient-level split, placing all scans of each patient in the same set; if the joint-model AUC gains vanish or reverse, the conclusion that joint learning helps this pair is an artefact of scan-level leakage rather than shared imaging signal.","supporting_citations":[{"cited_title":"BMJ Digital Health & AI (In press)","cited_arxiv_id":null,"evidence_quote":"Supplies the novel MIRAI PET/CT radiogenomics cohort used for all experiments."},{"cited_title":"European Radiology36, 2157–2185 (2026)","cited_arxiv_id":null,"evidence_quote":"Documents the lack of PET/CT and multi-label radiogenomic mutation studies that this work addresses."},{"cited_title":"Clinical and Transla- tional Oncology27, 1506–1515 (2025)","cited_arxiv_id":null,"evidence_quote":"Prior PET/CT radiomics prediction of EGFR/TP53 co-mutation that this study extends to deep multi-label learning."},{"cited_title":"DeepChest: Dynamic Gradient-Free Task Weighting for Effective Multi-Task Learning in Chest X-ray Classification","cited_arxiv_id":"2505.23595","evidence_quote":"Provides the shared-representation and transfer rationale for expecting joint learning to help."},{"cited_title":"Applied Intelligence53, 25135–25153 (2023)","cited_arxiv_id":null,"evidence_quote":"Frames joint mutation prediction as multi-label learning and motivates the pairwise comparison."},{"cited_title":"Cancers14(2022)","cited_arxiv_id":null,"evidence_quote":"Shows a multi-task CT-based radiogenomic approach, the closest existing comparison for simultaneous prediction."}],"review_version":1}