{"id":"ce61cd9f-322f-4f21-9b28-a373d8c78792","arxiv_id":"2607.13826","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A multimodal Swin-UNETR with auxiliary segmentation classifies PDAC resectability at AUC 0.86, but the clinical stream includes post-diagnosis treatment variables that can leak the label.","lead":"Researchers trained a neural network to read CT scans and clinical records and sort pancreatic-cancer patients into three surgery-readiness groups. On a small internal set it reached 0.86 AUC, but one of the clinical inputs—whether the patient had chemotherapy—may already encode part of the answer.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Ground-truth NCCN labels are not validated: reported metrics may reflect agreement with a single tumor board, not true resectability.","rationale":"After reading the paper carefully, I find the reader's weakest assumption is exactly the most load-bearing concern. The paper claims a robust, reproducible tool for resectability assessment, but the validity of that claim hinges on the quality of the ground-truth NCCN labels. The authors themselves cite literature showing inter-observer agreement below 70%, and they explicitly list the absence of a multi-reader study as a limitation (Section 5). This is not an external attack; it is a gap the authors disclose. The internal validation and external validation numbers are all computed against these labels, so any label noise propagates directly into the reported metrics. The external cohort's label generation is not described, so it does not mitigate the concern. I agree with the reader's verdict of CONDITIONAL: the paper is well-executed in many respects (nested CV, external validation, ablations, public code), but this ground-truth issue is a genuine soft spot that a reader study would resolve. I do not think the concern is fatal enough to reject, because the model's CT-only performance and geometry analysis provide some independent support, and the authors are transparent about the limitation. Thus the verdict should remain unchanged.","tokens_in":11539,"tokens_out":6462,"duration_ms":52968,"concrete_test":"Select a random subset of 40–60 cases from the Basel and KSA Aarau cohorts (e.g., 20–30 from each). Have 3–5 independent, board-certified radiologists/surgeons who are blinded to the original labels classify each case into the three NCCN categories using the same CT and clinical information (excluding the model's output). Compute Fleiss' kappa among readers, per-reader agreement with the original MDT label, and the model's agreement with each reader and with the original consensus. If inter-observer kappa is below 0.70 (or agreement <70%), or if the model's agreement with the MDT consensus falls within the human inter-observer agreement range, then the reported AUC/F1 numbers cannot be interpreted as measuring true resectability. This single check would directly quantify label reliability and resolve whether the central claim holds.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—that the multimodal model is a robust, reproducible tool for NCCN-based resectability assessment—depends on the reference labels being reliable. The paper itself reports that inter-observer agreement for resectability is often below 70% even with standardized criteria (Section 1, citing Giannone et al., 2021; Katz et al., 2013). Labels in the training/validation cohort were defined by MDT consensus at two Basel centers and independently reviewed by one additional physician (Section 4.1), but no pairwise agreement, kappa, or adjudication statistics are reported. For the external KSA Aarau cohort, label generation is not described at all (Section 4.1). Since the model is trained and evaluated against these labels, the reported AUC of 0.86, macro-F1 of 0.79–0.81, and accuracy of 0.85–0.87 measure agreement with one institution's MDT decisions, not with an objective 'true resectability.' The authors acknowledge this gap in Section 5: 'we did not perform a dedicated multi-reader study... important next step.' As a result, the external validation cannot establish that the model generalizes to other centers with different consensus practices; it only shows the model transfers to a second cohort whose labeling process is unspecified. This is load-bearing because if label noise is substantial, the model may be fitting idiosyncratic MDT decisions, and the headline performance overstates clinical utility.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a fully automated multimodal deep learning framework for classifying pancreatic ductal adenocarcinoma into three NCCN resectability categories (upfront resectable, borderline resectable, locally advanced) from 3D contrast-enhanced CT and 17 structured clinical variables. A Swin-UNETR encoder–decoder provides auxiliary multi-organ segmentation supervision during training, while a fused imaging–clinical representation is used for classification; an adaptive multitask loss shifts weight from segmentation to classification as tumor Dice improves. The authors report internal stratified nested 5-fold cross-validation on 159 patients (AUC 0.86, macro-F1 0.79, accuracy 0.85) and external validation on 52 patients from another institution (AUC 0.86, macro-F1 0.81, accuracy 0.87), with ablations showing improved performance over unimodal and adapted baseline methods. The central claim is that integrating anatomical supervision with clinical context yields a robust, reproducible tool for NCCN-based resectability assessment.","tokens_in":11917,"tokens_out":2129,"duration_ms":20994,"significance":"If the central claim holds, the framework is a potentially useful step toward automated, mask-free resectability assessment, and the methodological ingredients are of interest: auxiliary segmentation supervision without inference-time masks, adaptive multitask weighting, and public code release. The internal nested CV is well designed, and the ablation study is thorough. However, the validity of the reported metrics depends on two load-bearing issues: the inclusion of post-treatment clinical variables that are downstream of the label being predicted, and the reliability and consistency of the NCCN reference labels. These issues must be addressed before the headline results can be interpreted as evidence of a robust clinical tool.","major_comments":[{"comment":"The clinical feature vector includes 'neoadjuvant (radio-)chemotherapy status' and 'CA 19-9 serum level after neoadjuvant therapy'. These variables are determined after diagnosis and are strongly influenced by the initial NCCN category, since borderline/locally advanced patients are typically treated neoadjuvantly while upfront resectable patients proceed to surgery. Including them as predictors therefore introduces target leakage: the model can partially infer the label from treatment history. This is especially relevant because Table 3 shows tabular-only models achieve non-trivial performance (AUC 0.74–0.75), and the imaging-only model is weaker. The authors should retrain and re-evaluate the model with all post-diagnosis treatment variables removed (or restricted to baseline CA19-9), and report whether the multimodal advantage and the reported AUC/F1 are preserved. Without this, the c","section":"§3.1, Clinical Data"},{"comment":"The paper's own introduction cites inter-observer agreement for resectability assessment often below 70% even under standardized criteria (Giannone et al., 2021; Katz et al., 2013). Yet the training labels are MDT consensus from two Basel centers, with a single additional physician review and no reported agreement statistics, and the external KSA Aarau cohort's label generation process is not described. The reported AUC/F1 therefore measure agreement with one (or two) institutional tumor boards' decisions rather than an objective ground truth; if label noise is substantial or institutional practices differ, the model may be fitting idiosyncratic decisions. The authors acknowledge the lack of a multi-reader study in Section 5, but this is not merely a future-work item: it is load-bearing for the interpretation of every performance number. At minimum, the paper should report inter-observer","section":"§4.1 and §5, Reference standard / label noise"},{"comment":"The external validation cohort contains only 52 patients, including 10 locally advanced cases, and results are reported as point estimates with no confidence intervals. Given the small sample, an AUC of 0.86 with no uncertainty bound cannot establish 'cross-institution generalization' in a statistically meaningful way; the 95% CI on a three-class AUC from 52 patients is likely wide (of the order of ±0.1 or more). The authors should provide bootstrap or exact CIs, per-class counts, and ideally a formal comparison with the internal performance. This is a required addition to support the paper's cross-institution claim, which is a central part of the abstract and conclusion.","section":"§4.5, Table 2, External validation"}],"minor_comments":[{"comment":"Section 4.5 states that 'segmentation outputs were used only for evaluation', but Section 4.4 explicitly uses model-predicted segmentations to compute geometric contact angles for the NCCN geometry analysis. The wording should be clarified to 'not used for classification at inference'.","section":"§4.5/§4.4"},{"comment":"The coefficients α, β, ω in the segmentation loss are not given numeric values in the main text; the values are said to be grid-searched in the inner loop. Reporting the selected values (or range) would improve reproducibility.","section":"Eq. (2)"},{"comment":"The adaptive weighting ablation reports 'Mean Tumor Dice' with standard deviations, but the main text does not explain how tumor Dice is averaged across folds and whether it is computed on the validation or test set. Please clarify.","section":"Table 5"},{"comment":"The Viviers-based baseline is adapted from DPCG to NCCN criteria, but the conversion is not described. Since this baseline underperforms the proposed method, a brief explanation of the adaptation would help readers assess fairness.","section":"Table 1 and Table 2"},{"comment":"Minor typographical issues: 'Jo¨el' and 'W olleb' contain mis-encoded characters; these should be fixed in the camera-ready version.","section":"Author list"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper is a competent applied contribution, not a method breakthrough. It combines a Swin-UNETR backbone with auxiliary 16-class segmentation supervision, a 17-variable clinical MLP, and a Dice-adaptive multitask loss for NCCN resectability. The mask-free inference design is genuinely useful, since prior work needed segmentations at test time. The nested cross-validation is carefully done, the external cohort is small but real, and the ablations are informative. The code is public. That part is solid.\n\nThe soft spots are real, and one is more than minor. The clinical vector includes neoadjuvant chemotherapy status and post-neoadjuvant CA19-9. Since borderline and locally advanced patients usually receive neoadjuvant therapy, those features are partly consequences of the label the model is predicting. That can inflate the multimodal results. The paper doesn't address this, and the ablation doesn't retrain without those variables. That should be the first thing a revision does.\n\nThe other issue is the reference standard. Labels come from one tumor board's MDT consensus, with a single extra physician review. The paper itself cites inter-observer agreement below 70% for NCCN resectability. So the reported AUC is measuring agreement with one institution's decisions, not with an objective ground truth. The authors acknowledge the lack of a multi-reader study in the conclusion, which is honest, but they don't quantify label noise, and the external cohort's labeling process is not described. That limits the generalizability claim. The stress-test note gets this right.\n\nSmaller things: the external validation has no confidence intervals, and the 'outperforming' language is stronger than the overlapping standard deviations support. The cohort is small, and the locally advanced class has only 27 cases, which shows in the per-class F1.\n\nOverall, the central argument holds up as an applied contribution: the model does what it claims to do under nested CV and one external set. The circular features and label reliability are the main reasons to be conditional rather than enthusiastic. This deserves a serious referee, and I'd accept it for review. It's the kind of paper that could become useful with modest fixes. I'd cite it if I worked on this topic.","headline":"A solid applied multimodal pipeline for PDAC resectability, with known-component novelty; two upstream-information leaks and an unvalidated reference standard keep it from being more than that.","tokens_in":12467,"tokens_out":2659,"would_cite":true,"duration_ms":26218,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A multimodal deep learning model can classify pancreatic ductal adenocarcinoma resectability into the three NCCN categories from contrast-enhanced CT plus 17 clinical variables, reporting accuracy 0.85–0.87 on internal and external cohorts","keywords":["pancreatic cancer","PDAC","resectability","NCCN classification","multimodal deep learning","CT imaging","auxiliary segmentation","multitask learning"],"falsifier":"Run a prospective multi-reader study in which several blinded radiologists independently grade the same CT volumes and compare their agreement rates with the model's agreement to the MDT label; if the model agrees with the board no more often than a typical clinician does, the claim of adding reproducible objective information collapses. Additionally, checking whether the external cohort's labels were produced by an independent MDT would show whether the generalization result is merely shared-label transfer.","tokens_in":11439,"feed_emoji":"🩻","tokens_out":5177,"duration_ms":47694,"temperature":0.7,"pith_summary":"This paper aims to show that pancreatic ductal adenocarcinoma resectability—whether surgery should be attempted up front, only after neoadjuvant therapy, or not at all under NCCN rules—can be predicted automatically from a single contrast-enhanced CT scan plus routine clinical variables. The authors train a multimodal network whose imaging branch is supervised to segment the pancreas, tumor, and key peripancreatic vessels, then fuse those learned anatomical features with a compact embedding of 17 clinical variables. They report consistent performance on nested cross-validation and on an independent 52-patient cohort, with AUC around 0.86 and accuracy 0.85–0.87. They argue that because segmentation is needed only during training, the resulting system gives vessel-aware, interpretable predictions without manual masks at test time. If true, this would give clinicians a reproducible second opinion on tumor-board decisions that today vary substantially between experts.","feed_headline":"Deep learning sorts pancreatic tumors by resectability from one CT","feed_subtitle":"A CT-plus-chart model hits 0.86 AUC internally and on an outside cohort, no tumor masks needed.","key_machinery":"The load-bearing object is the fused representation: a 256-dimensional imaging vector from a Swin-UNETR bottleneck concatenated with a 32-dimensional clinical embedding from a three-layer MLP, forming a 288-dimensional multimodal vector for the classification head. Training is controlled by a dynamic multitask schedule in which the segmentation weight w_seg drops from 3.0 to 1.0 (and w_cls = 1/w_seg rises) as the tumor Dice score improves, so the shared encoder first learns anatomy and then is tuned for discrimination. This design transfers vessel-aware structure into the classifier without requiring segmentation masks at inference.","core_discovery":"The paper's central claim is that integrating anatomical supervision with clinical context yields a robust and reproducible tool for NCCN-based operability assessment in pancreatic cancer. The authors show that a Swin-UNETR encoder trained jointly on CT segmentation and classification, fused with an MLP embedding of 17 clinical variables, can predict upfront-resectable, borderline-resectable, and locally-advanced disease. They report AUC 0.86, macro-F1 0.79, and accuracy 0.85 in nested cross-validation, and AUC 0.86, macro-F1 0.81, and accuracy 0.87 on an independent 52-patient cohort. They argue that because segmentation is auxiliary to training, the system delivers vessel-aware representat","pith_inferences":["Because the reference labels are MDT consensus without quantified inter-reader variability, the reported AUC/F1 values may chiefly measure agreement with one tumor board's operational decisions; a multi-reader study is needed to show the model adds signal beyond a typical clinician.","The geometry analysis is a testable opening: if angle-based contact surrogates were made explicit outputs, the model could generate human-readable explanations that clinicians can audit, turning a black box into a decision-support tool.","The adaptive segmentation-to-classification schedule could be reused in any task with scarce labels and a strong auxiliary dense task, such as tumor staging in other organs, but that transfer remains to be demonstrated."],"forward_implications":["If the reported metrics hold in prospective testing, an automated second opinion becomes feasible for a decision whose inter-observer agreement is often below 70%.","The system's mask-free inference removes the need for a separate, fragile segmentation step in deployment, making the workflow closer to a click-and-report tool.","Ablations show CT-only (AUC 0.82) and tabular-only (AUC 0.74) each underperform the fused model (AUC 0.86), implying both modalities contribute complementary signal rather than one being redundant.","The external cohort results suggest transfer across institutions and scanner settings without retraining, although the paper treats this as preliminary evidence needing further validation."],"fun_headline_variants":["AI grades pancreatic cancer operability from CT plus chart data","CT and clinical data feed AI that predicts pancreatic tumor resectability","Automated AI scores pancreatic cancer resectability from CT and chart without tumor masks","Pancreatic cancer operability predicted by AI from CT plus clinical data, no masks needed","AI hits 0.86 AUC for pancreatic cancer resectability on CT plus chart"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The evaluation assumes the tumor-board-assigned NCCN labels are true resectability; if those labels are noisy or reflect local institutional treatment habits—the paper itself notes inter-observer agreement on NCCN categories is often below 70%—then the reported accuracy measures agreement with one board's decisions rather than true resectability.","fun_headline_variants_meta":{"raw":{"variants":["AI grades pancreatic cancer operability from CT plus chart data","CT and clinical data feed AI that predicts pancreatic tumor resectability","Automated AI scores pancreatic cancer resectability from CT and chart without tumor masks","Pancreatic cancer operability predicted by AI from CT plus clinical data, no masks needed","AI hits 0.86 AUC for pancreatic cancer resectability on CT plus chart"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000463,"raw_usage":{"total_tokens":2130,"prompt_tokens":704,"completion_tokens":1426,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":448,"completion_tokens_details":{"reasoning_tokens":1325}},"tokens_in":448,"tokens_out":1426,"duration_ms":11785,"temperature":1.0,"reasoning_tokens":1325,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T03:34:49.156218+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a prospective multi-reader study in which several blinded radiologists independently grade the same CT volumes and compare their agreement rates with the model's agreement to the MDT label; if the model agrees with the board no more often than a typical clinician does, the claim of adding reproducible objective information collapses. Additionally, checking whether the external cohort's labels were produced by an independent MDT would show whether the generalization result is merely shared-label transfer.","supporting_citations":[],"review_version":1}