{"id":"e6dba7d4-cb56-4b31-bf6c-de801929e79a","arxiv_id":"2507.05582","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"R-Super converts tumor count, size, and location information from radiology reports into voxel-wise losses that improve CT tumor segmentation beyond training with masks alone.","lead":"This paper presents R-Super, a training method that turns written radiology reports into supervision for tumor segmentation in CT scans, improving F1 scores by up to 16% over mask-only training. It combines a private dataset of 6,718 CT-report pairs with public mask datasets, showing gains with as few as 50 masks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No control trains on the added UCSF-Train CTs without report losses, so the reported F1/DSC gains may reflect more training data rather than the report text itself.","rationale":"The paper's strongest claim is that radiology reports directly supervise tumor segmentation and improve F1 by up to 16% over mask-only training. The most direct threat to that claim is not whether the report-extracted attributes are accurate, but whether the comparison actually isolates the report text. Every R-Super experiment adds UCSF-Train CT-Report pairs, so the model sees thousands of additional CT volumes compared to the 'segmentation' baseline. If those CTs alone provide a benefit (via better feature representations, domain adaptation, or implicit regularization), the reported gains could occur even if the reports were entirely ignored. The paper does not include any baseline that trains on the same additional CTs without report losses. The ablations compare R-Super with only Volume Loss vs only Ball Loss, both of which still use reports, so they do not control for the presence of the extra data. Comparisons to CLIP-Like and multi-task methods are also uninformative on this point because those methods also consume the UCSF-Train reports and CTs. The external JHH-Test DSC improvements are real evidence, but they suffer from the same confound: R-Super was fine-tuned on UCSF-Train CTs, and the baseline was not. The concrete test I propose would settle this: train with the same CTs but drop the report losses. If the control matches R-Super, the central claim is unsupported; if it is much worse, the report text is the active ingredient. This is a feasible experiment for the authors because they have the UCSF-Train data and the training pipeline. I keep the verdict as CONDITIONAL because the claim may be true, but it requires this missing control to be believed. The reader's focus on report attribute accuracy is reasonable but not the most load-bearing issue, hence my disagreement.","tokens_in":1008,"tokens_out":948,"duration_ms":68871,"concrete_test":"Train MedFormer on the same AbdomenAtlas masks plus all UCSF-Train CT volumes, but without any report losses or report text, using the identical fine-tuning schedule, data augmentation, and training/validation split as R-Super. Evaluate on UCSF-Test and JHH-Test. If this no-report control achieves F1/DSC within a small margin (e.g., <2 points) of R-Super, the headline claim that report text itself improves segmentation must be weakened. If the control is clearly worse (e.g., >5 points), the report supervision is causal. Also verify patient-level disjointness between UCSF-Train and UCSF-Test to rule out a second confound.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that report text provides voxel-level supervision that improves segmentation. The experiments supporting this compare R-Super (masks + UCSF-Train CT-report pairs) to a segmentation baseline trained only on masks. This design confounds two variables: the addition of ~2.2K/2.7K CT volumes from UCSF-Train and the use of report-derived losses. A model trained on the same CTs without any report signal (e.g., standard fine-tuning on the masks, or a semi-supervised method) would likely also improve, since the extra CTs include many tumor cases that regularize the feature space. The paper's ablations remove either the Volume Loss or the Ball Loss but never remove the report supervision entirely; the comparisons to CLIP-Like, Multi-task learning, and RG pseudo-labels all use the same reports and CTs, so they do not provide a no-report control. Consequently, the attribution 'reports improve segmentation up to +16% F1' is not yet established; the active ingredient may be the additional CT images alone. This is load-bearing because if it fails, the method's novelty reduces to semi-supervised learning on unlabeled CTs, and the specific report-supervision mechanism is unsupported. The reader's concern about report attribute accuracy is valid but secondary: even perfect report extraction would not justify the claim if the gains are driven by data quantity rather than the report text.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces R-Super, a training loss that converts radiology reports into voxel-level supervision for tumor segmentation. An LLM extracts tumor count, organ/sub-segment location, and diameters from reports; the Volume Loss and Ball Loss enforce agreement between the predicted segmentation and tumor volumes, locations, and counts estimated from reports. Training combines public CT-mask pairs (AbdomenAtlas) with a large private UCSF CT-report dataset. The method is evaluated for pancreatic and kidney tumor segmentation, internally on UCSF-Test (no masks, so detection metrics) and externally on JHH-Test (with manual masks). R-Super reports F1 gains up to +16% internally and Dice gains of +8 to +11 points externally over mask-only training, and it outperforms several report-based and self-supervised baselines.","tokens_in":12232,"tokens_out":9246,"duration_ms":105176,"significance":"If the central claim holds, the contribution is significant: it offers a practical way to scale tumor segmentation by exploiting routinely available radiology reports, and it includes external validation with real masks. The paper provides ablations of the two proposed losses, comparisons with five state-of-the-art methods, and public code. The main caveats are that the few-mask result lacks a no-report control using the same extra CT volumes, and the internal headline metric measures tumor detection rather than voxel-level segmentation.","major_comments":[{"comment":"The few-mask experiments (50 masks) compare R-Super trained with 2.2K/2.7K additional CT-report pairs against a segmentation baseline trained with only 50 masks. No baseline uses the same additional CT volumes without report supervision at the 50-mask setting; Models Genesis, the only method that learns from CTs without reports, is reported only at 344/1.7K masks. Consequently, the +9.7% F1 gain at 50 masks could be caused by the additional CT volumes rather than by the report-derived losses. Please add a no-report baseline at 50 masks using the same UCSF-Train CTs (e.g., Models Genesis pretraining or a pseudo-label semi-supervised method), or clearly restrict the few-mask claim to acknowledge this confound.","section":"Section 3, Fig. 4, Table 2"},{"comment":"The headline '+16% F1' is measured on UCSF-Test, which has no segmentation masks, so F1 and AUC there evaluate tumor detection, not voxel-level segmentation. The abstract's phrase 'strongly improved tumor segmentation in internal and external validation' conflates detection and segmentation. Please rephrase the abstract to state that internal gains are in detection F1 while segmentation gains are demonstrated on JHH-Test via DSC/NSD, or obtain manual masks on a subset of UCSF-Test and report a segmentation metric there.","section":"Abstract, Section 3, Table 2"},{"comment":"The comparison to Models Genesis is intended to control for learning from CT volumes without reports, but the paper does not state explicitly whether the Models Genesis baseline was pre-trained on the same UCSF-Train CT volumes used by R-Super. Since the central claim is that report text, not merely extra CT data, drives the improvement, please specify the exact unlabeled CT data used for the Models Genesis baseline and, if it does not include UCSF-Train, add a control that does.","section":"Training R-Super, Table 1"}],"minor_comments":[{"comment":"The right panel of Figure 1 would be easier to read with labeled axes (for example, segmented volume V_s on the x-axis and loss value on the y-axis) and a clear indication of the tolerance region.","section":"Figure 1"},{"comment":"No sensitivity analysis is provided for the tolerance parameter τ, the stabilization constant E, or the relative loss weight of 0.1; a brief ablation or discussion of these choices would help readers understand the robustness of the method to hyperparameter settings.","section":"Section 2.1 and Section 3"},{"comment":"The text reports that pancreas sub-segment masks improve R-Super and gives numerical results without them, but this comparison is not shown as a table row; consider adding a row or a supplementary table for this ablation.","section":"Table 2"},{"comment":"The model name is written as 'LLama' in the text; it should be 'LLaMA' for consistency with the reference.","section":"Section 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is likely acceptable after revision if the authors can add a 50-mask no-report control and clarify the detection-versus-segmentation metric issue. The private UCSF data are not released, but the code is public and the external validation with real masks is a genuine strength. The main risk is that the few-mask claim, as currently presented, is confounded by additional CT data."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. The idea is genuinely new: report-extracted count, location, and diameters are turned into voxel-level losses, so the report supervises segmentation directly rather than via a side task. The external result is real: on JHH-Test, DSC for pancreatic tumors goes from 51 to 59 with 344 masks, and from 38 to 49 with 50 masks. That is a meaningful signal.\n\nThe soft spot is the attribution. R-Super trains on masks plus 2.2K/2.7K UCSF CT-report pairs, while the 'segmentation' baseline uses only masks. So the +16% F1 includes the effect of simply adding thousands of CTs, not just the report text. The paper does compare against Models Genesis, which also uses unlabeled CTs, and R-Super beats it, which suggests reports help beyond the extra images. But Models Genesis is a different pretraining scheme, not the same MedFormer fine-tuning recipe with the same CTs and no report losses. That leaves the central claim plausible but not fully isolated.\n\nOther issues are secondary. UCSF-Test has no masks, so its F1 gains measure detection, not segmentation quality. The UCSF data is private, and the report-parsing details are described only briefly. The Volume and Ball losses assume diameters convert to ellipsoid volumes and that reported locations are accurate; the paper excludes <11% of reports without sizes rather than modeling them. No patient-level split is stated, so leakage within the private hospital data is a question the authors should answer.\n\nStill, the paper is clearly written and the method is reproducible in principle: the losses are well-specified and the code is linked. The external Dice improvement is the strongest evidence and it is the right kind of evidence. I would send it to peer review, but the authors should add a control that trains on the UCSF CTs with the same architecture and masks but without report-derived losses, and report patient-level splits. If that control shows most of the gain remains, the claim is solid.","headline":"Novel report-to-voxel supervision with a real external Dice gain, but the headline numbers are confounded by adding thousands of unlabeled CTs.","tokens_in":12776,"tokens_out":2599,"would_cite":false,"duration_ms":27538,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Radiology reports can directly supervise tumor segmentation, lifting F1 by up to 16 percentage points.","keywords":["tumor segmentation","radiology reports","weak supervision","loss function","CT imaging","large language model","pancreatic tumor","kidney tumor"],"falsifier":"Train R-Super on a dataset where ground-truth masks and paired reports are both available and compare report-estimated volumes against mask-derived volumes; if the losses improve segmentation only when the two agree, then on a cohort where reported diameters systematically underestimate true tumor volume (for example, single-slice measurements of elongated tumors), R-Super should underperform mask-only training. Alternatively, construct test reports that omit one of several visible tumors: if R-Super's false negatives match the omitted tumor, that confirms the losses enforce report completeness rather than true tumor burden.","tokens_in":11739,"feed_emoji":"🩻","tokens_out":9739,"duration_ms":91599,"temperature":0.7,"pith_summary":"Radiology reports, not just segmentation masks, can directly supervise tumor segmentation in CT scans. The paper proposes R-Super, a training paradigm that extracts tumor count, location, and diameters from free-text reports with a large language model and converts them into voxel-wise supervision through two new losses: Volume Loss and Ball Loss. Training with these report-derived losses alongside mask supervision raised F1 score by up to 16 percentage points over mask-only training, even when only 50 masks were available, and improved results on an external hospital not seen during training. The implication is that the huge archives of CT-report pairs in hospitals can be repurposed to scale segmentation AI.","feed_headline":"Radiology reports lift tumor segmentation F1 by 16 points","feed_subtitle":"CT-report pairs plus a few masks beat mask-only training on both internal and external hospitals.","key_machinery":"The central machinery is the pair of report-supervision losses. Volume Loss is applied as deep supervision: it sums predicted tumor probabilities inside pre-saved organ or sub-segment masks, multiplies by voxel volume to obtain a segmented volume, and penalizes the relative difference from the report-estimated volume with a 10% tolerance and a background cross-entropy term. Ball Loss is applied to the final layer: a fixed spherical kernel matching the reported tumor diameter is convolved over the output to find the highest-probability ball, the top-N voxels inside it (N derived from the report-estimated volume) are maximized, and all voxels not assigned to any reported tumor are minimized, repeating from largest to smallest tumor. Together these losses transform text-wise report information into voxel-wise labels.","core_discovery":"The central claim is that radiology reports contain enough quantitative detail—tumor count, organ or sub-segment location, and one-to-three diameters—to act as voxel-level training signal for tumor segmentation. The Volume Loss compares, per organ or sub-segment, the total segmented tumor volume against a volume estimated from report diameters using ball or ellipsoid formulas, with a 10% tolerance and a background penalty; the Ball Loss localizes each reported tumor with a spherical convolution whose kernel matches the reported diameter, then maximizes the top-N most probable voxels inside the highest-probability ball and minimizes all unassigned voxels, iterating from largest to smallest tumor. In internal and external validation, R-Super surpassed mask-only segmentation and four report-using baselines, with F1 gains up to 16 points with 344 masks, about 10 points with 50 masks, and 4.3 points with 1.7K masks, including gains for small (diameter at most 2 cm) and large tumors.","pith_inferences":["If report diameters are systematically recorded from a single axial slice, the ellipsoid volume estimates may be biased low, and the losses could steer the segmenter toward under-segmentation; a calibration study comparing report-derived volumes with mask-derived volumes on a paired dataset would test this.","The LLM extraction step is a potential bottleneck: prompt errors in diameter or location propagate directly into the losses, so end-to-end performance is upper-bounded by extraction accuracy; perturbing extracted attributes in a sensitivity analysis would quantify this.","The approach may transfer to other imaging modalities and lesion types where reports state dimensions and location, such as MRI or ultrasound, and to other anatomical regions beyond the abdomen."],"forward_implications":["Hospitals can contribute existing CT-report archives to segmentation training without manual mask annotation, increasing data scale and diversity across centers, scanners, and contrast phases.","Tumor types with scarce public masks, including pancreatic and kidney tumors, can gain large F1 improvements from report supervision even when only 50 masks exist.","The method is architecture-agnostic and does not require perfect report-derived labels: the tolerance in Volume Loss and the dynamic mask refinement in Ball Loss accommodate imperfect diameter estimates.","Gains persist when mask counts are large (1.7K), so report supervision complements rather than replaces manual masks.","External validation on an unseen hospital shows the report-supervised model generalizes across institutions."],"supporting_citations":[{"why":"Source of the public CT-mask dataset used for mask supervision and of the radiologist-designed LLM prompts with 96% reported accuracy for tumor presence and location.","marker":"[3]"},{"why":"The LLM used for zero-shot extraction of tumor count, locations, and diameters from radiology reports.","marker":"[10]"},{"why":"The transformer-CNN segmentation architecture used for R-Super and all baselines.","marker":"[11]"},{"why":"The self-configuring segmentation method trained to provide the organ and sub-segment masks used in the two losses.","marker":"[14]"},{"why":"Report-using contrastive pretraining baseline that R-Super surpasses in the comparisons.","marker":"[5]"},{"why":"Report-guided pseudo-label baseline that R-Super surpasses in the comparisons.","marker":"[6]"},{"why":"Multi-task learning baseline that uses reports as an auxiliary task and that R-Super surpasses.","marker":"[7]"}],"fun_headline_variants":["Radiology reports as supervision: +16 F1 for tumor segmentation","Report-based loss boosts CT tumor segmentation by 16 F1","Tumor AI learns from reports, not just masks, +16 F1","Fewer masks, more reports: tumor segmentation improves 16 points","R-Super: radiology reports turn voxels into training signal"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The report-derived tumor attributes—count, location, and diameters converted to ellipsoid volumes—are accurate and complete enough to serve as voxel-level supervision, and the paper excludes the small fraction (under 11%) of reported tumors without size rather than modeling them.","fun_headline_variants_meta":{"raw":{"variants":["Radiology reports as supervision: +16 F1 for tumor segmentation","Report-based loss boosts CT tumor segmentation by 16 F1","Tumor AI learns from reports, not just masks, +16 F1","Fewer masks, more reports: tumor segmentation improves 16 points","R-Super: radiology reports turn voxels into training signal"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00023,"raw_usage":{"total_tokens":1492,"prompt_tokens":964,"completion_tokens":528,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":580,"completion_tokens_details":{"reasoning_tokens":435}},"tokens_in":580,"tokens_out":528,"duration_ms":6499,"temperature":1.0,"reasoning_tokens":435,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:23:29.793771+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train R-Super on a dataset where ground-truth masks and paired reports are both available and compare report-estimated volumes against mask-derived volumes; if the losses improve segmentation only when the two agree, then on a cohort where reported diameters systematically underestimate true tumor volume (for example, single-slice measurements of elongated tumors), R-Super should underperform mask-only training. Alternatively, construct test reports that omit one of several visible tumors: if R-Super's false negatives match the omitted tumor, that confirms the losses enforce report completeness rather than true tumor burden.","supporting_citations":[{"cited_title":"Nature Methods18(2), 203–211 (2021)","cited_arxiv_id":null,"evidence_quote":"The self-configuring segmentation method trained to provide the organ and sub-segment masks used in the two losses."},{"cited_title":"Radiology: Artificial Intelligence 5(5), e230031 (2023)","cited_arxiv_id":null,"evidence_quote":"Report-guided pseudo-label baseline that R-Super surpasses in the comparisons."},{"cited_title":"In: 2019 IEEE 16th In- ternational Symposium on Biomedical Imaging (ISBI 2019)","cited_arxiv_id":null,"evidence_quote":"Multi-task learning baseline that uses reports as an auxiliary task and that R-Super surpasses."}],"review_version":1}