{"id":"73d304ee-411e-482c-8b3e-b990a166725d","arxiv_id":"2507.15193","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Among eleven annotation strategies, Tumor + Kidney + Aorta (TKA) achieved the best pheochromocytoma segmentation (DSC 0.860, NSD 0.814, F1 0.857), beating the previous Tumor + Body baseline.","lead":"A study of 105 abdominal CT scans found that training a deep learning model to also recognize nearby kidney and aorta, alongside the pheochromocytoma tumor, improved segmentation accuracy compared with the prior approach of using the whole body region as context. If confirmed, automated tumor boundary detection and tumor burden measurement for this rare adrenal tumor could become more reliable.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Post-hoc selection of TKA on the same 31-scan test set used for significance testing invalidates the reported p-values; the descriptive ranking may hold, but the claim of significant outperformance is not supported as stated.","rationale":"The reader's weakest assumption is exactly the load-bearing concern: the 31-scan test set was used both to select TKA among eleven strategies and to compute the significance tests against TB. This invalidates the reported p-values as confirmatory evidence. I agree with that identification. I add one corroborating detail: Section 2.4's description of the TKA-versus-TB comparison as five-fold cross-validation on the full dataset conflicts with the abstract and Section 3.2's attribution of the p-values to the 70-30 split. This ambiguity strengthens the concern, because whichever reading is correct, the reported p-values do not support the stated conclusion. The descriptive ranking, large effect size, and consistency of TKA's advantage in the five-fold F1 curves are real evidence, and the use of nnU-Net with standard preprocessing is a strength. However, the central claim is explicitly inferential ('significantly outperforming'), and that claim cannot be accepted on the current statistical protocol. Since the reader already recommended conditional acceptance with a request for nested validation or corrected p-values, my stress test does not change the verdict: conditional acceptance remains appropriate. The concrete nested-validation test would settle whether the selection effect is material, and if it is, whether TKA still exceeds TB after proper accounting.","tokens_in":11149,"tokens_out":5008,"duration_ms":52328,"concrete_test":"Run a nested validation: split the 105 scans once into 74 training and 31 test. Within the 74 training scans, use 5-fold cross-validation to select the best annotation strategy among the eleven. Then evaluate only that selected model on the held-out 31 scans and report DSC, NSD, and F1. Compute both uncorrected p-values and p-values corrected for the eleven-way selection (e.g., Holm or a permutation test over the selection process). If TKA remains best with p<0.05 after this protocol, the central claim stands; otherwise it should be downgraded to descriptive.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central inferential claim depends on the Wilcoxon p-values reported for DSC (p=0.0097) and NSD (p=0.0110) in the TKA-versus-TB comparison. Section 2.4 states that 'the best-performing model from the eleven approaches was first determined' before TKA was compared with TB. Because that selection was made on the same 30% test set used for Table 1 and Figure 5, the subsequent significance test is post hoc: under the null that all eleven strategies perform equally, the chance that the best-performing strategy shows a nominally significant difference from TB is far greater than 0.05. No multiple-comparison correction is reported. The F1 improvement (25.84%) has no p-value at all, since the methods section says p-values were computed only for DSC and NSD. There is also an internal inconsistency: Section 2.4 says the TKA-vs-TB comparison was performed using five-fold cross-validation on the full dataset, while the abstract and Figure 5 attribute the p-values to the 70-30 test split. If the p-values come from the selection test set, they are invalid due to selection; if they come from cross-validation, the test-set p-values are unreported and the CV comparison lacks significance testing. Either way, the phrase 'significantly outperforming' is not currently supported. The descriptive finding that TKA ranks first may well be true, and the large effect size is encouraging, but the load-bearing inferential claim requires a valid estimation protocol.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper evaluates eleven multi-class annotation strategies for 3D nnU-Net segmentation of pheochromocytoma (PCC) in contrast-enhanced abdominal CT, using anatomical priors such as kidney, aorta, adrenal gland, liver, spleen, and body region. On a 70-30 patient-level split of 105 scans, the Tumor+Kidney+Aorta (TKA) strategy ranks first in DSC, NSD, and instance-wise F1, and the paper claims it significantly outperforms the previously used Tumor+Body (TB) strategy. The authors also report tumor-burden correlation (R^2=0.968 for TKA) and a five-fold cross-validation comparison across IoU thresholds. The descriptive ranking is well supported by the reported metrics, but the inferential claim of significant superiority is weakened by the fact that TKA was selected post hoc on the same test set used for the significance tests, and by an internal inconsistency between the Methods and the Abstract regarding where the p-values come from.","tokens_in":11407,"tokens_out":5075,"duration_ms":60032,"significance":"If the results hold, the study makes a useful clinical contribution: it systematically maps the space of anatomical priors for PCC segmentation, identifies kidney and aorta as effective auxiliary labels, and provides detailed training configurations that facilitate replication. The five-fold cross-validation and subtype-stratified results strengthen the descriptive finding. However, the central claim that TKA 'significantly outperforms' TB is currently not established at the stated confidence level because the significance tests are performed after selecting the winning strategy on the same test data, and no multiple-comparison adjustment is reported. The descriptive ranking and large effect size are encouraging, but the inferential language in the abstract and discussion needs to be supported by a valid estimation protocol or explicitly softened.","major_comments":[{"comment":"There is a direct inconsistency about which protocol produced the p-values. Section 2.4 states that 'the best-performing model from the eleven approaches was first determined' and that 'For this comparison, both models were trained on the entire dataset (105 scans) using five-fold cross-validation,' while the Abstract and Figure 5 attribute the DSC (p=0.0097) and NSD (p=0.0110) values to the 70-30 test split. The authors must state unambiguously whether the p-values come from the 31-scan test set or from cross-validation. If they come from the test set, the Methods paragraph is wrong; if they come from cross-validation, the test-set p-values are unreported and the cross-validation comparison still lacks a paired significance test across folds.","section":"Section 2.4 vs. Abstract/Figure 5"},{"comment":"The reported Wilcoxon signed-rank p-values are not valid confirmatory significance tests because TKA was selected as the best of eleven strategies on the same 31-scan test set used for the comparison. Under the null hypothesis that all eleven strategies perform equally, the maximum over eleven strategies is far more likely to yield a nominally significant p-value than a pre-specified comparison would. The authors should either report results on a confirmatory held-out split that was not used for model selection, apply a multiple-comparison correction across the eleven strategies, or present the p-values explicitly as exploratory rather than confirmatory.","section":"Section 2.4 and Section 2.5"},{"comment":"The claim that TKA 'significantly outperforms' TB on F1 score (25.84% improvement at IoU=0.5) is not supported by any significance test. Section 2.5 states that p-values were computed only for DSC and NSD, and no confidence interval or paired test is reported for F1. The wording 'significantly outperforming' should be removed for F1 unless an appropriate statistical test is provided, or the sentence should be rephrased to describe the F1 improvement as a descriptive difference.","section":"Abstract and Section 4"},{"comment":"The reference standard for approximately half of the dataset was generated by an iterative framework in which an nnU-Net model's predictions were used to seed annotations that residents then refined. Because the same nnU-Net architecture is later evaluated, the test labels may be partially biased toward model predictions. The manuscript should report annotation-refinement statistics (e.g., the extent of edits made by residents per volume) or re-annotate an independent subset by a senior radiologist to demonstrate that the reference standard is not systematically biased.","section":"Section 2.2"},{"comment":"The five-fold cross-validation comparison is presented as evidence that 'TKA consistently outperforms TB,' but Figure 4 shows only mean F1 curves without per-fold distributions, error bars, or any statistical test across folds. The descriptive consistency is useful, but the robustness claim would be stronger with fold-wise paired tests or clear visualization of the per-fold spread.","section":"Section 3.1 and Figure 4"}],"minor_comments":[{"comment":"The genetic-subtype results are based on very small test subgroups (e.g., n=5 for VHL/EPAS1 and n=6 for SDHx), so the subtype-level differences should be labeled as descriptive rather than as evidence of subtype-specific superiority.","section":"Table 3"},{"comment":"The dataset division is described as random at the patient level with 105 scans from 91 patients, but the manuscript does not state how patients with multiple scans were handled when assigning scans to train and test splits. Please clarify whether all scans from a given patient were kept in the same split and how this was ensured.","section":"Section 2.4"},{"comment":"The composite loss is standard, but the definition of L_Dice as a class-averaged Dice across C classes should state explicitly whether the average is taken over all classes including background or only foreground classes, since this affects the relative weighting of auxiliary anatomical labels.","section":"Equations (1)-(3)"},{"comment":"The caption notes that the same organ may appear in different colors across subfigures because colors are assigned by class index. To make visual comparison easier, consider using a fixed color map for each anatomical structure across all annotation strategies.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"This is a well-executed empirical study with a clear clinical motivation and detailed experimental reporting. The main obstacle is statistical: the significance claim is undermined by post-hoc model selection on the test set and by the inconsistency between Sections 2.4 and the Abstract. If the authors can validate on a truly untouched confirmatory split or apply a proper multiple-comparison correction, the paper would be suitable for publication. I did not identify concerns about data fabrication or circular reasoning beyond the acknowledged self-seeding in the annotation process."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper's real contribution is a clean head-to-head comparison of eleven organ-specific anatomical priors for pheochromocytoma segmentation, where prior work only used a body-region prior. The descriptive finding—Tumor+Kidney+Aorta (TKA) ranks first on DSC, NSD, and F1, with a large margin over Tumor+Body—is plausible and consistently reproduced across test split and five-fold CV. That is worth having.\n\nWhat the paper does well: the comparison is systematic, the metrics are standard, the genetic-subtype breakdown is useful, and the tumor-burden R² is a nice clinical tie-in. The iterative annotation framework is a reasonable way to scale labels, though it does introduce mild self-dependence in the reference standard.\n\nThe soft spot is the statistical claim. TKA was chosen as the best of eleven strategies on the same 31-scan test set used for the p-values. Under that selection, the Wilcoxon p-values (0.0097, 0.0110) are not valid significance tests—they do not account for the fact that the best of eleven will look better by chance. No multiple-comparison correction is reported. There is also an internal inconsistency: the Methods say the TKA-vs-TB comparison used five-fold CV on all 105 scans, while the Abstract and Figure 5 attribute the p-values to the 70-30 test split. Either way, 'significantly outperforming' is not supported as stated. The descriptive ranking may well hold—the effect size is large and consistent—but the inferential framing needs a fix: nested validation, a corrected threshold, or at minimum reporting the comparison as exploratory.\n\nNo code or data is released, so independent verification is limited. That is a practical weakness, not a fatal one, given the dataset is private and the task is niche.\n\nBottom line: this is a useful empirical paper for anyone working on adrenal tumor segmentation or anatomical priors in nnU-Net. It should go to peer review, but the authors need to redo or reframe the significance testing before I would trust the 'significant' language. As a reviewer, I would ask for a corrected protocol or a clear statement that the ranking is descriptive.","headline":"Useful systematic ranking of anatomical priors for PCC segmentation, but the 'significant' claim is undercut by post-hoc selection on the same test set; send to review with required statistical corrections.","tokens_in":12021,"tokens_out":2714,"would_cite":true,"duration_ms":27822,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding kidney and aorta as auxiliary training labels improves deep-learning segmentation of pheochromocytoma in abdominal CT.","keywords":["pheochromocytoma segmentation","anatomical priors","multi-class annotation","abdominal CT","nnU-Net","tumor burden quantification","deep learning"],"falsifier":"Run a confirmatory study with a pre-specified primary comparison between the TKA and TB annotations on a held-out multi-center CT cohort, assigning the split before any model selection; if TKA does not beat TB on DSC, NSD, or F1 at IoU = 0.5, the paper's central claim fails. A simpler internal check: apply a Bonferroni correction for the eleven strategy comparisons to the reported p-values and see whether TKA-vs-TB remains significant.","tokens_in":10950,"feed_emoji":"🩻","tokens_out":5099,"duration_ms":51674,"temperature":0.7,"pith_summary":"This paper asks whether the anatomical context fed to a deep-learning segmenter can be chosen to make pheochromocytoma (PCC) segmentation from contrast-enhanced abdominal CT more accurate. It trains 3D nnU-Net models under eleven different annotation schemes and claims that labeling the tumor together with the adjacent kidney and aorta (TKA) gives the best results, outperforming the previously used tumor-plus-body (TB) scheme on Dice similarity, surface distance, and instance-level F1. The improvement is large enough that the authors report $p = 0.0097$ for DSC, $p = 0.0110$ for NSD, a 25.84% F1 gain at IoU $= 0.5$, and better tumor-burden quantification ($R^2 = 0.968$). The practical stake is that a reliable automated PCC segmentation tool could support tumor burden monitoring and reduce reliance on expensive genetic testing by enabling cluster inference from imaging.","feed_headline":"Kidney plus aorta labels sharpen tumor segmentation on CT","feed_subtitle":"A new annotation recipe beats the body-region prior for pheochromocytoma, raising Dice and tumor-burden accuracy.","key_machinery":"The central object is the multi-class annotation strategy used to train nnU-Net: tumor voxels plus auxiliary voxel labels for nearby organs, with only the tumor class extracted at inference time. The auxiliary labels are generated by merging expert tumor annotations with organ masks from an automated whole-body segmentation tool, creating eleven training-label combinations. The network is a 3D full-resolution nnU-Net trained with a composite cross-entropy and soft Dice loss; the load-bearing step is that the auxiliary labels act as anatomical priors that guide the network's localization of the tumor.","core_discovery":"The paper's central discovery is that organ-specific anatomical priors, not a broad body-region prior, should accompany the tumor label when training a 3D segmentation network for pheochromocytoma. Among eleven schemes, Tumor + Kidney + Aorta (TKA) produced the highest DSC ($0.8599 \\pm 0.093$), NSD ($0.8139 \\pm 0.148$), and instance-wise F1 ($0.857$ at IoU $= 0.5$), and these gains held for every genetic subtype and across five-fold cross-validation. The authors interpret this as evidence that the kidney and aorta provide relevant spatial context that helps localize a small, heterogeneous adrenal tumor, while larger organs such as liver and spleen hurt performance, likely through class imbalance.","pith_inferences":["The reported p-values should be read as descriptive rather than confirmatory, because the test set was used both to pick TKA as the best of eleven strategies and then to test it against TB; a pre-registered split or multiple-comparison correction would be needed to make the significance claim airtight.","One plausible mechanism the paper does not isolate is that kidney and aorta act as stable anatomical landmarks near the adrenal bed, and the benefit may be mostly spatial context rather than tissue-class information; an ablation with randomly displaced organ masks could test this.","The recipe may transfer to other small retroperitoneal or adrenal lesions, such as adrenal adenomas or paragangliomas, where nearby-organ priors could supply similar localization cues.","A testable extension is whether the TKA gain grows with smaller training sets: if the prior mainly reduces the search space, its value should increase when data are scarce."],"forward_implications":["A TKA-trained model can be used to quantify PCC tumor burden from routine portal venous CT, with predicted volumes correlating strongly with ground truth ($R^2 = 0.968$).","The TKA advantage persists across SDHx, VHL/EPAS1, kinase, and sporadic genetic subtypes, so the method does not appear to overfit one tumor phenotype.","Adding kidney and aorta labels is a cheap intervention: the auxiliary masks come from an automated tool, and only the tumor label is needed at inference.","The same evaluation shows that piling on more anatomical classes (liver, spleen, body) degrades accuracy, so prior design should favor a small set of nearby organs."],"supporting_citations":[{"why":"Supplies the nnU-Net self-configuring 3D segmentation architecture used for all experiments.","marker":"[23]"},{"why":"Defines the prior Tumor + Body annotation strategy that TKA is compared against as the previous baseline.","marker":"[24]"},{"why":"Provides the automated whole-body and organ segmentation masks from which the eleven annotation combinations are built.","marker":"[29]"},{"why":"Supports the idea that multi-class labels with surrounding tissues act as anatomical priors for segmentation.","marker":"[26]"},{"why":"Shows anatomical priors from nearby structures improving lymph node segmentation, motivating the organ-prior design.","marker":"[27]"},{"why":"Supplies the interactive segmentation tool used to create the reference tumor annotations.","marker":"[28]"},{"why":"Provides the instance-wise detection and F1 evaluation methodology with IoU matching.","marker":"[31]"},{"why":"Supplies the Dice Similarity Coefficient and Normalized Surface Distance conventions used for segmentation evaluation.","marker":"[33]"}],"fun_headline_variants":["Kidney and aorta priors boost pheochromocytoma segmentation","TKA annotation beats body prior for adrenal tumor CT","Organ priors sharpen adrenal tumor segmentation","Tumor plus kidney and aorta: best CT segmentation recipe"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the 31-scan test set is an independent confirmation of TKA's superiority, but the same test set was used to select TKA from eleven annotation strategies; if that selection invalidates the statistical test, the 'significant' claim weakens, though the descriptive ranking may still hold.","fun_headline_variants_meta":{"raw":{"variants":["Kidney and aorta priors boost pheochromocytoma segmentation","TKA annotation beats body prior for adrenal tumor CT","Organ priors sharpen adrenal tumor segmentation","Tumor plus kidney and aorta: best CT segmentation recipe"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000329,"raw_usage":{"total_tokens":1895,"prompt_tokens":1067,"completion_tokens":828,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":683,"completion_tokens_details":{"reasoning_tokens":765}},"tokens_in":683,"tokens_out":828,"duration_ms":8354,"temperature":1.0,"reasoning_tokens":765,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T15:38:30.313553+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a confirmatory study with a pre-specified primary comparison between the TKA and TB annotations on a held-out multi-center CT cohort, assigning the split before any model selection; if TKA does not beat TB on DSC, NSD, or F1 at IoU = 0.5, the paper's central claim fails. A simpler internal check: apply a Bonferroni correction for the eleven strategy comparisons to the reported p-values and see whether TKA-vs-TB remains significant.","supporting_citations":[{"cited_title":"Computerized Medical Imaging and Graphics 116, 102419 (2024)","cited_arxiv_id":null,"evidence_quote":"Defines the prior Tumor + Body annotation strategy that TKA is compared against as the previous baseline."},{"cited_title":"Radiology: Artificial Intelligence 5(5), 230024 (2023)","cited_arxiv_id":null,"evidence_quote":"Provides the automated whole-body and organ segmentation masks from which the eleven annotation combinations are built."},{"cited_title":"International Journal of Computer Assisted Radiology and Surgery, 1–7 (2024)","cited_arxiv_id":null,"evidence_quote":"Supports the idea that multi-class labels with surrounding tissues act as anatomical priors for segmentation."},{"cited_title":"International journal of computer assisted radiology and surgery 19(8), 1537–1544 (2024)","cited_arxiv_id":null,"evidence_quote":"Shows anatomical priors from nearby structures improving lymph node segmentation, motivating the organ-prior design."},{"cited_title":"Neuroimage 31(3), 1116–1128 (2006)","cited_arxiv_id":null,"evidence_quote":"Supplies the interactive segmentation tool used to create the reference tumor annotations."},{"cited_title":"Medical image analysis 84, 102680 (2023)","cited_arxiv_id":null,"evidence_quote":"Provides the instance-wise detection and F1 evaluation methodology with IoU matching."},{"cited_title":"Nature communications 13(1), 4128 (2022)","cited_arxiv_id":null,"evidence_quote":"Supplies the Dice Similarity Coefficient and Normalized Surface Distance conventions used for segmentation evaluation."}],"review_version":1}