{"id":"dd328870-64d6-47f6-b098-92b00412ae38","arxiv_id":"2509.01554","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A volumetric vision-language model trained jointly on classification labels and segmentation masks from three CT datasets reaches 83% AUROC on CT-RATE and shows cross-dataset zero-shot behavior.","lead":"This paper presents Uniferum, a 3D medical imaging model that trains on both disease labels and organ segmentation masks from different CT datasets at once. On the CT-RATE benchmark it reports a 7% AUROC gain over CLIP-based models, and shows what it calls zero-shot performance on two unseen datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Potential image-level leakage in +SEG zero-shot evaluation: manuscript does not state whether the 10% segmentation-task sample is split-restricted; if it includes validation/test volumes, the body-segmentation enhancement claim is invalid.","rationale":"The reader's weakest_assumption identifies the same risk I consider most load-bearing, so I marked partial agreement: I agree this is the point on which the paper's generalization claims hinge, but I do not treat contamination as established because the text is ambiguous. The base zero-shot claim (model trained only on CT-RATE, evaluated on unique INSPECT/RADCHEST tasks) is not affected by the +SEG issue, and it is independently supported by Table 2 (0.5808 and 0.5559 AUROC), so the paper's strongest claim does not collapse entirely. What does depend on the ambiguous sampling is the more novel assertion that body segmentation is a universal performance enhancer—repeated in the abstract and Sec. 5.2. Given the code is public, a targeted check can settle this quickly. I also note the main results table lacks confidence intervals, but that is secondary. The verdict should remain CONDITIONAL pending the leakage check; no change from the reader's verdict.","tokens_in":12590,"tokens_out":12186,"duration_ms":126969,"concrete_test":"In the publicly available GitHub repository, inspect the data-loading script that constructs segmentation tasks. Verify that the 'extra 10%' sample for organ segmentation is drawn from the training split only, and cross-check the volume IDs in the segmentation task list against the INSPECT validation/test (1000/3000) and RADCHEST validation/test (984/360) volume IDs. Any overlap confirms contamination; zero overlap resolves the concern. As a supplementary check, retrain CTRA TE+SEG with segmentation tasks restricted to CT-RATE training volumes and confirm whether the zero-shot gains on INSPECT/RADCHEST persist.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim that 'body segmentation provides a universal strategy for enhancing performance' (Abstract, Sec. 5.2) rests on the +SEG columns of Table 2 and Figures 3–4. In the +SEG condition, organ-segmentation tasks are generated by 'randomly sampl[ing] an extra 10% samples from the dataset' (Sec. 3.3). The manuscript never states that this 10% is drawn exclusively from the training split of each dataset. If INSPECT or RADCHEST validation/test volumes are included as segmentation targets, then the model has been trained on the exact volumes later used for 'zero-shot' evaluation, so the observed +SEG gains (e.g., INSPECT unique 0.5808→0.6130; RADCHEST unique 0.5559→0.5768, Table 2) may reflect image-level domain adaptation rather than a transferable benefit of segmentation. The preprocessing pipeline (Sec. 3.5) additionally applies TotalSegmentator to all non-CT-RATE scans, but that alone is not trainable leakage; the risk is specifically whether those masks are used as training targets for evaluation volumes. This is the load-bearing assumption: if violated, the 'body segmentation improves zero-shot' contribution collapses, though the base CTRA TE-only zero-shot results (0.5808/0.5559) would remain.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Uniferum, a vision-language model for 3D CT that unifies classification labels and segmentation masks through task-conditioned training on CT-RATE, RAD-CHEST, and INSPECT. The model is an encoder-only transformer with a 3D EfficientNet vision encoder and text embeddings, trained with binary cross-entropy for classification and focal loss for segmentation. The authors report a 7% AUROC improvement over CT-CLIP on CT-RATE, zero-shot performance on INSPECT and RAD-CHEST unique tasks, and consistent 1–3% gains when body-segmentation tasks are added. The manuscript includes public code and comparisons with multiple baselines.","tokens_in":12986,"tokens_out":2657,"duration_ms":31531,"significance":"If the claims hold, the paper makes a useful contribution: it shows how heterogeneous public CT annotations can be harmonized in a single VLM, and it provides evidence that segmentation supervision can improve classification in a task-conditioned framework. The use of multiple datasets, public code, and some confidence-interval reporting are strengths. However, the central generalization and segmentation-benefit claims depend on experimental hygiene that is not fully described: the segmentation sampling may overlap with evaluation volumes, and some evaluation labels are machine-generated. These issues are fixable but need to be resolved before the conclusions can be accepted.","major_comments":[{"comment":"The segmentation-task construction is described as 'randomly sampled an extra 10% samples from the dataset' without stating that this 10% is restricted to the training split. If INSPECT or RAD-CHEST validation/test volumes are included as segmentation targets, then the model has been trained on the exact volumes later used for 'zero-shot' evaluation. The +SEG improvements (e.g., INSPECT unique 0.5808→0.6130; RADCHEST unique 0.5559→0.5768; shared-task gains) could then reflect image-level domain adaptation rather than a transferable benefit of segmentation. This is load-bearing for the 'universal strategy' claim and the zero-shot claims. Please state explicitly that segmentation tasks are drawn only from training splits and, ideally, re-run the +SEG comparison with all evaluation volumes excluded from segmentation targets.","section":"§3.3, Table 2"},{"comment":"The 12 INSPECT diagnosis labels used for INSPECT-unique zero-shot evaluation are generated by keyword matching followed by GPT-4o-mini validation, with only 120 samples (10 per class) radiologist-verified (96% accuracy). The paper itself notes that labels such as 'nodules' may include false negatives. Because the zero-shot AUROC values are computed against these machine-generated labels, label noise could bias the reported comparisons. Please provide an analysis of label-noise sensitivity or restrict zero-shot evaluation to labels with stronger verification.","section":"§3.4.3, §5.4"},{"comment":"The main AUROC table reports point estimates without confidence intervals or significance tests. This is particularly important for the 1–3% segmentation gains, which are small relative to likely variability, and for the headline 7% CT-RATE improvement. Some zero-shot examples include DeLong intervals, but the main claims and the 'consistently improved' statement rely on unquantified point estimates. Please add confidence intervals or significance tests for the key comparisons in Table 2.","section":"Table 2 and §5.2"},{"comment":"The p-values reported (e.g., p=0.03, p=0.02 for mortality improvements) are not corrected for multiple comparisons, and several confidence intervals overlap (e.g., honeycombing 69.2±11.0 vs. 65.0±13 after +SEG). The claim that body segmentation 'could be raised' by adding segmentation tasks is stated more strongly than the evidence supports. Please temper the wording or provide multiple-comparison-corrected statistics.","section":"§5.4"}],"minor_comments":[{"comment":"The conclusion states 'achieving 72% and 79% AUROC on out-of-distribution datasets RAD-CHEST and INSPECT respectively,' but Section 5.3 reports 72.1% on INSPECT shared and 79.6% on RAD-CHEST shared. The dataset names are swapped; please correct.","section":"Conclusion"},{"comment":"There are several typographical and formatting issues (e.g., 'plumary emblosim', 'V alidation', 'W e', 'T o'). A careful proofread is needed.","section":"Throughout"},{"comment":"The CT-RATE validation description is slightly confusing: 'Scans of 100 patients from the official training set were withheld for model selection' and then the official validation set of 3002 volumes is used. Please clarify the relationship between the internal withheld set and the official validation set.","section":"§3.4.1"},{"comment":"The table includes dashes for some baseline cells. It would aid reproducibility to state in the caption which baseline results are from prior papers and which were produced by the authors.","section":"Table 2, baseline rows"}],"recommendation":"major_revision","confidential_remarks":"The central ambiguity in §3.3 is the kind of issue that should be resolved before publication: if the 10% segmentation sample includes evaluation volumes, the zero-shot and segmentation-benefit claims are invalidated. Because the manuscript does not currently rule this out, major revision is appropriate. The machine-generated INSPECT labels are a secondary concern that also needs attention. The paper is otherwise within scope for the journal and the core architecture is interesting."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper is worth a serious look: it shows a clean way to combine classification and segmentation supervision in a single 3D CT VLM, and it ships code and public data. The headline result on CT-RATE is credible, though the zero-shot generalization claim needs tightening.\n\nWhat's new: Uniferum reformulates multi-label classification as task-conditioned predictions, multiplying effective training samples, and trains on a mix of labels and body-segmentation masks from three public datasets. The CT-RATE AUROC of 0.83 beats listed CLIP baselines and a CNN. That's a real, if incremental, advance over existing 3D VLMs like CT-CLIP, Merlin, and UMIT.\n\nThe main soft spot is the split ambiguity. In Section 3.3, \"randomly sampled an extra 10% samples from the dataset\" is ambiguous; if that sample includes validation or test volumes, the +SEG gains in Table 2 are contaminated. The paper needs to state explicitly that the segmentation tasks are drawn from the training split only. This is a one-line fix, but it's load-bearing for the generalization claim. Relatedly, the INSPECT \"unique\" evaluation labels are generated by GPT-4o-mini with only 120 samples manually checked at 96%; noisy reference labels can inflate or deflate AUROC, so the zero-shot numbers on those tasks should be read with caution. The main table also lacks error bars; confidence intervals appear only for a few selected tasks.\n\nFor a reader working on medical VLMs or data-efficient 3D CT, this paper is a useful empirical data point. The combination of tasks is new, and the code is a plus. The flaws are addressable, not fatal. I'd send it to a strong reviewer.","headline":"Useful empirical contribution on 3D CT VLMs, but the zero-shot generalization claim needs a clearer statement on data splitting and better label validation.","tokens_in":13419,"tokens_out":3563,"would_cite":true,"duration_ms":39861,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A vision-language model for 3D CT that trains on both diagnosis labels and organ masks, reaching 83.0% AUROC on CT-RATE (7 points above CLIP-based baselines) and showing unexpected zero-shot accuracy on withheld datasets.","keywords":["vision-language model","3D computed tomography","multi-task learning","segmentation","classification","zero-shot","radiology","AUROC"],"falsifier":"Hold out segmentation entirely for the RAD-CHEST and INSPECT evaluation sets (do not run TotalSegmentator on those volumes, or remove their masks from segmentation training), then compare zero-shot AUROC. If the gains on unique labels drop to chance or shrink substantially, the zero-shot transfer claim is an artifact of domain adaptation to the segmentation outputs, not language-driven generalization.","tokens_in":12558,"feed_emoji":"🩻","tokens_out":3908,"duration_ms":42300,"temperature":0.7,"pith_summary":"The paper claims that a single vision-language model can absorb both classification labels and segmentation masks from heterogeneous 3D CT datasets into one task-conditioned training framework, and that doing so yields higher accuracy than training on any single annotation type. On the CT-RATE benchmark, the model—called Uniferum—reaches 83.0% AUROC, beating a CLIP-based model by 7 points and a conventional multi-label CNN by 20 points. The authors also report out-of-distribution and zero-shot performance on two held-out datasets, and find that adding body-segmentation tasks improves classification by 1–3%. The motivation is that public volumetric CT data are scarce and fragmented, so a framework that reuses every image across many tasks could be a practical path to clinically reliable radiology VLMs.","feed_headline":"Unified training on labels plus masks beats CLIP by 7% AUROC","feed_subtitle":"A task-conditioned VLM mixes classification and segmentation from three public CT datasets, with unexpected zero-shot gains.","key_machinery":"The task-conditioned fusion transformer: each training pair includes a natural-language task description, and the model predicts either a classification logit (from the CLS token) or a low-resolution patch-wise segmentation mask (from the per-patch embeddings) depending on which supervision the task supplies. This lets one backbone reuse every volume across many labels, mix classification and segmentation losses in a single batch, and transfer knowledge between datasets without requiring paired labels or masks.","core_discovery":"Uniferum reformulates every classification and segmentation label as a vision-language task: a CT volume is paired with a natural-language description such as 'Diagnose the presence of pleural effusion around the lungs' or 'Segment the lungs in the image,' and the model is trained to produce either a classification logit or a patch-wise segmentation prediction from the same fused embedding. A 3D EfficientNet encodes the volume; a small transformer fuses those features with the task text; the CLS token drives binary classification while per-patch embeddings predict downsampled masks. Because labels are decoupled, one volume participates in many tasks, and datasets with incompatible annotation","pith_inferences":["The segmentation gains may be anatomy-aware regularization rather than true multimodal reasoning; a cleaner ablation would hold out segmentation masks for the evaluation scans.","The same task-conditioning trick could apply to other 3D modalities (MRI, PET) where annotations are similarly fragmented.","The low-resolution mask objective (patch-wise u^3 prediction) may already provide enough spatial grounding, implying fine-grained voxel segmentation is unnecessary for classification gains.","If zero-shot claims survive decontamination, they suggest that radiograph-level text prompts carry enough semantics to initialize unseen diagnostic categories, arguing for scaling task descriptions rather than datasets."],"forward_implications":["If the claim holds, fragmented public CT datasets can be combined without re-annotation, making data-efficient 3D VLMs practical.","Adding generic body segmentation (60 organ-level tasks) is a low-cost, universal way to improve classification AUROC by 1–3%.","Task-conditioned evaluation lets a single model answer many diagnostic and prognostic questions, decoupling label count from model count.","The observed zero-shot performance on unshared labels suggests language-conditioned representations generalize to unseen abnormalities, though this is the claim most sensitive to segmentation leakage."],"supporting_citations":[{"why":"Supplies the CT-RATE dataset, its 18 pathology labels, and the CT-CLIP baseline results (zero-shot, vocabFine, classFine) that Uniferum outperforms by 7% AUROC.","marker":"[6]"},{"why":"Supplies the RAD-CHEST dataset, 84 abnormality labels, and the CNN baseline (83.5% AUROC on shared tasks) used for cross-dataset comparison.","marker":"[4]"},{"why":"Supplies the INSPECT dataset with pulmonary embolism and prognostic labels, plus the image-only baseline (65.9% AUROC) Uniferum compares against.","marker":"[9]"},{"why":"Supplies TotalSegmentator, the source of the 53 organ segmentation tasks used in the +SEG training conditions.","marker":"[22]"},{"why":"Supplies LUNA16 nodule segmentation masks used as an additional fine-grained segmentation task.","marker":"[17]"},{"why":"Defines CLIP, the contrastive language-image pretraining approach whose 3D adaptation serves as the primary baseline and conceptual starting point.","marker":"[18]"},{"why":"Supplies the EfficientNet backbone that the vision encoder is initialized from and inflated to 3D.","marker":"[20]"}],"fun_headline_variants":["Unified CT model joins labels and masks, beats CLIP by 7% AUROC","One vision-language model for CT diagnosis and segmentation, zero-shot gains","Uniferum fuses classification and masks, lifts CT AUROC 7% over CLIP","Task-conditioned VLM turns CT labels and masks into one framework"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The claim that adding segmentation masks is a neutral way to boost classification depends on the masks for the evaluation scans not leaking information; the reported zero-shot gains could instead come from the model having seen those same scans through their segmentation task.","fun_headline_variants_meta":{"raw":{"variants":["Unified CT model joins labels and masks, beats CLIP by 7% AUROC","One vision-language model for CT diagnosis and segmentation, zero-shot gains","Uniferum fuses classification and masks, lifts CT AUROC 7% over CLIP","Task-conditioned VLM turns CT labels and masks into one framework"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000215,"raw_usage":{"total_tokens":1258,"prompt_tokens":730,"completion_tokens":528,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":474,"completion_tokens_details":{"reasoning_tokens":442}},"tokens_in":474,"tokens_out":528,"duration_ms":6303,"temperature":1.0,"reasoning_tokens":442,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T12:24:39.622488+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Hold out segmentation entirely for the RAD-CHEST and INSPECT evaluation sets (do not run TotalSegmentator on those volumes, or remove their masks from segmentation training), then compare zero-shot AUROC. If the gains on unique labels drop to chance or shrink substantially, the zero-shot transfer claim is an artifact of domain adaptation to the segmentation outputs, not language-driven generalization.","supporting_citations":[{"cited_title":"Machine-learning-based multiple abnormality prediction with large-scale chest computed tomography volumes.Medical image analysis, 67:101857, 2021","cited_arxiv_id":null,"evidence_quote":"Supplies the RAD-CHEST dataset, 84 abnormality labels, and the CNN baseline (83.5% AUROC on shared tasks) used for cross-dataset comparison."},{"cited_title":"Inspect: a multimodal dataset for patient outcome prediction of pulmonary embolisms.Advances in Neural Information Processing Systems, 36:17742–17772, 2023","cited_arxiv_id":null,"evidence_quote":"Supplies the INSPECT dataset with pulmonary embolism and prognostic labels, plus the image-only baseline (65.9% AUROC) Uniferum compares against."},{"cited_title":"T otalseg- mentator: robust segmentation of 104 anatomic structures in ct images","cited_arxiv_id":null,"evidence_quote":"Supplies TotalSegmentator, the source of the 53 organ segmentation tasks used in the +SEG training conditions."},{"cited_title":"A large-scale evaluation of automatic pulmonary nodule detection in chest ct using local image features and k-nearest-neighbour classification","cited_arxiv_id":null,"evidence_quote":"Supplies LUNA16 nodule segmentation masks used as an additional fine-grained segmentation task."},{"cited_title":"Learning transferable visual models from natural language supervision","cited_arxiv_id":null,"evidence_quote":"Defines CLIP, the contrastive language-image pretraining approach whose 3D adaptation serves as the primary baseline and conceptual starting point."},{"cited_title":"Efficientnet: Rethinking model scaling for convolutional neural networks","cited_arxiv_id":null,"evidence_quote":"Supplies the EfficientNet backbone that the vision encoder is initialized from and inflated to 3D."}],"review_version":1}