{"id":"65e2ffe0-96f0-4802-9213-f9dfc9cfd3b4","arxiv_id":"2607.25860","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A text-promptable CT segmenter trained with weak slice labels mined from radiology reports beats strong-only training by 8–22% relative dice, with larger gains when expert masks are scarce.","lead":"The authors train a text-conditioned CT segmentation model using expert masks plus weak slice labels automatically extracted from radiology reports. When only 250–1000 expert-annotated volumes are available, adding report-derived labels boosts dice score by 8–22% relative.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Low-data gains may be inflated by an unstable strong-only baseline; the 22% relative gain at 250 volumes rests on a baseline that still produced NaN loss after mitigation attempts.","rationale":"The reader identified weak-label construction noise as the weakest assumption; this is valid and worth checking, but the NaN-baseline issue is more directly load-bearing because it affects the headline numbers at the low-data regime that the abstract emphasizes. The manuscript itself flags the baseline instability in Section 4.3, yet the comparison still uses those runs as the strong-only control. Even if the extracted weak labels are perfectly clean, the reported improvement at 250/500 volumes could be an artifact of an under-optimized or restart-degraded baseline. The 1000-volume comparison is cleaner, and the 8% gain there provides some support, but the paper's main selling point is the larger gain when strong labels are scarce. My recommendation remains CONDITIONAL: the method is plausible and the 8% gain at 1000 volumes is suggestive, but the low-data claim needs a stable baseline before it can be considered established. I disagree with the reader only in emphasis, not substance.","tokens_in":12642,"tokens_out":5582,"duration_ms":57541,"concrete_test":"Re-run the strong-only baseline at 250 and 500 volumes using a configuration designed to avoid NaNs, e.g., gradient clipping, a lower peak learning rate with longer warmup, or mixed-precision training, and repeat with at least 3 seeds, selecting checkpoints by validation dice. Compare the resulting dice to the strong+weak numbers in Figure 2. If the stable strong-only baseline matches or exceeds the reported strong+weak dice, the claimed low-data gains are artifacts of baseline instability. If the stable baseline remains below strong+weak by the same margin, the central claim is supported. Also report the step at which NaN occurs and the validation dice of the checkpoint used for restart.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline quantitative claim—that weak supervision yields a 22% relative gain when only 250 fully labeled volumes are available—depends on a strong-only baseline whose training was unstable. Section 4.3's footnote states that the 250- and 500-volume baseline runs encountered NaN loss; even after halving learning rates and restarting from the latest checkpoint, 'NaN loss still occurred.' A model that has suffered NaNs and been restarted is not a reliable control for measuring the benefit of weak supervision. The strong+weak method uses a different batch composition (1 strong + 4 weak) and may be less prone to this instability, so the observed gap could reflect optimization stability rather than the information provided by the weak labels. The 1000-volume result (8% relative gain) does not carry this caveat, but the paper's central narrative is the larger gain precisely in the low-data regime where the baseline is least trustworthy. This is a concrete methodological concern about the comparison itself, independent of whether the extracted weak labels are noisy.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a method for open-ended CT volume segmentation that augments scarce voxel-level strong supervision with slice-level weak supervision extracted from radiology reports. A finetuned SAM3 model is trained with the standard SAM3 losses on strongly labeled slices and a presence classification loss on weakly labeled slices, where the weak labels are obtained by prompting an LLM to extract finding descriptions and series/image locators from a large internal database of chest CT reports. The authors evaluate on ReXGroundingCT and report relative Dice gains of 8% with 1000 strongly labeled volumes and 22% with 250 strongly labeled volumes, plus per-category analysis and a comparison with VoxTell.","tokens_in":12952,"tokens_out":3820,"duration_ms":38707,"significance":"If the empirical result holds, the paper makes a useful and timely contribution: it demonstrates a scalable way to leverage widely available radiology reports to reduce the annotation burden for text-conditioned CT segmentation. The evaluation is largely sound in design: it uses an external test set (ReXGroundingCT), varies the amount of strong supervision across three regimes, includes a per-category analysis that supports the proposed mechanism, and is transparent about the difficulties encountered during training. The main risks are that the weak-label extraction pipeline is never validated, the low-data headline result rests on a strong-only baseline that exhibited NaN loss, and no ablation isolates whether the gains come from the report-derived locator information or simply from additional presence-label training data.","major_comments":[{"comment":"The headline claim of a 22% relative gain at 250 strongly labeled volumes is based on a strong-only baseline that, according to the footnote, encountered NaN loss; even after halving learning rates and restarting from the latest checkpoint, 'NaN loss still occurred.' A control that has gone through NaN loss and restart is not a reliable measure of the benefit of weak supervision. The strong+weak method also uses a different batch composition (1 strong + 4 weak), so the observed gap may reflect optimization stability rather than the information in the weak labels. Please provide stable strong-only baselines for the 250- and 500-volume regimes (e.g., with gradient clipping, loss scaling, or a more robust recipe) or explicitly restrict the central claim to the 1000-volume result, which does not carry this caveat.","section":"§4.3 footnote and Fig. 2"},{"comment":"The weak-label construction is never validated. The labeling rule in §3.4 labels reported slices (plus immediate neighbors) as positive and all slices farther than g=5 as negative. This is known to be inexact for diffuse findings such as emphysema or large pleural effusions, and the paper acknowledges that the slice set is non-exhaustive, but it never quantifies the resulting label noise. The LLM extraction pipeline in Algorithm 1 is also unmeasured: no precision, recall, or manual audit of the extracted finding descriptions and slice indices is reported. Please validate the extracted weak labels against ground-truth masks on a subset of ReXGroundingCT (or another manually annotated set) and report the accuracy of the slice-locator matching.","section":"§3.3, Algorithm 1 and §3.4"},{"comment":"The strong:weak ratio ablation shows that adding more weak examples per batch improves Dice, but it does not isolate the information source of the weak labels. The observed improvement could come from the extra presence-classification training signal provided by the report-derived locators, or simply from more data / regularization. An ablation with permuted or randomized weak slice labels—e.g., assigning the positive slice indices to the wrong volumes—would test whether the gain is actually attributable to the language-derived locator signal. Without such an ablation, the causal claim that report-based weak supervision drives the improvement is not fully established.","section":"§4.4.3, Fig. 6"}],"minor_comments":[{"comment":"Error bars are described in the text as 95% bootstrap confidence intervals, but the figure appears to show them only inconsistently across panels. Please make the presence/absence of error bars explicit in the figure caption and ensure all reported metrics carry the same uncertainty treatment.","section":"§4.3, Fig. 2"},{"comment":"The dataset name is written inconsistently as 'ReXGroundingCT' and 'RexGroundingCT' (e.g., §4.2 vs. §4.4.2). Please standardize.","section":"General"},{"comment":"The strong:weak ratio ablation should specify the total batch size, the number of weak examples per batch, and whether the number of strong examples per update is held constant. The current description is ambiguous and makes the ablation difficult to reproduce.","section":"§4.4.3, Fig. 6"},{"comment":"The phrase 'RA VE cache' appears with an odd spacing throughout; presumably a formatting artifact from 'RAVE'. Please correct.","section":"Appendix A.1"}],"recommendation":"major_revision","confidential_remarks":"The core idea is promising and the external evaluation is a strength. However, the two main weaknesses—unvalidated weak-label extraction and an unstable low-data baseline—are load-bearing for the paper's central quantitative claim. Both are fixable within the scope of the manuscript: validate a sample of extracted labels, and either stabilize the strong-only baselines or reframe the headline claim. I would be willing to look at a revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a legitimate, well-scoped contribution—the first demonstration that slice-level weak labels extracted from radiology reports can improve a text-promptable CT segmenter when strong labels are scarce. It deserves a serious referee. But the headline low-data gains (22% at 250 volumes) are tied to a baseline that went NaN, and the weak-label pipeline is not validated at all. Those two things need to be fixed before I'd fully trust the quantitative claims.\n\nWhat's actually new: the extraction pipeline (regex + LLM, producing 93k (finding, volume, slice-set) triplets from 244k reports) and the training recipe that adds a presence loss on weak slices to SAM3 finetuning. Combining strong and weak supervision is not new, and SAM3 is not new, but using report-derived slice indices as weak supervision for open-ended segmentation is a real contribution. The experiments are mostly well done: three strong-label regimes, a per-category analysis showing correlation with weak-data availability, a strong:weak ratio ablation, bootstrap error bars, and a full-data comparison with VoxTell that stays within error bars. The footnote admitting NaN on the 250/500 baselines is honest, and the appendix gives enough detail to reproduce the method.\n\nSoft spots, in order of severity. First, the baseline issue. The stress-test note is right: the 22% gain—and presumably the 500-volume gain—is measured against a control that went NaN and had to be restarted. The strong+weak method uses a different batch structure and may simply be more stable, so part of the gap could be optimization robustness rather than label information. This is not fatal, because the 1000-volume result (8%) doesn't carry that caveat and the per-category trend is suggestive, but the abstract should not sell the 22% without a stable baseline. Second, the weak labels themselves are never validated. The assumption that a finding appears only near report-mentioned slices plus a margin g=5 is wrong for diffuse findings like emphysema; the paper should at least quantify false-negative rates on a few manually labeled volumes. Third, there is no ablation separating language-derived slice indices from a simpler 'finding present somewhere in this volume' label. That control is important to support the claimed value of location information. Fourth, the paper does not state whether the internal 244k-report corpus overlaps CT-RATE/ReXGroundingCT; if the weak-training volumes include evaluation volumes, that is leakage.\n\nBottom line: a promising recipe with honest reporting, but not yet a fully supported quantitative story. Worth peer review and worth citing if you work on CT segmentation or report-derived supervision. A revised version needs a stable baseline, weak-label validation, and the presence-vs-location ablation.","headline":"First real demonstration that report-derived slice-level weak supervision helps open-ended CT segmentation, but the headline low-data gains rest on an unstable baseline and the weak-label pipeline is unvalidated.","tokens_in":13341,"tokens_out":2361,"would_cite":true,"duration_ms":23790,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Training a text-conditioned CT segmentation model on slice-level labels mined from radiology reports improves accuracy, with the largest gains when dense ground-truth masks are scarce.","keywords":["CT segmentation","text-conditioned segmentation","weak supervision","radiology reports","slice-level labels","open-ended segmentation","medical image analysis"],"falsifier":"Compare extracted slice indices against ground-truth masks on a sample of volumes, measuring how often a reported slice contains the finding and how often far-away slices actually contain it; if diffuse findings frequently have positive slices far from the reported locator, then the strong negative labels are corrupting training, and removing or fixing those labels should change results.","tokens_in":12591,"feed_emoji":"🩻","tokens_out":4524,"duration_ms":39919,"temperature":0.7,"pith_summary":"Open-ended CT segmentation usually requires dense voxel-level masks, which are expensive to produce. The paper shows that cheap slice-level labels—binary indicators of which slices contain a described finding—can be extracted automatically from radiology reports and combined with a small amount of dense supervision. This weak supervision improves segmentation performance, yielding an 8% relative dice gain with 1000 fully labeled volumes and a 22% gain with only 250 fully labeled volumes. The work suggests that widely available report text can partially offset the annotation cost that currently limits scaling of CT segmentation models.","feed_headline":"CT segmentation gains 22% from report-mined weak labels","feed_subtitle":"Adding slice-level labels scraped from radiology reports beats dense-mask-only training when labeled volumes are scarce.","key_machinery":"The load-bearing mechanism is the slice-level presence loss applied to weakly labeled slices. For each finding, a report snippet is parsed into a standalone description plus explicit slice locators (e.g., 'series 2, section 206'). Slices within a 5-slice margin of the reported location are treated as positive, slices farther away as negative, and slices in an ambiguous buffer between these zones are excluded from the loss. This per-slice binary presence signal is combined with the standard dense segmentation losses from strongly labeled slices in a single finetuning run.","core_discovery":"The paper's central claim is that training a text-conditioned CT segmentation model on a mixture of densely labeled slices (full masks) and weakly labeled slices (binary presence labels derived from radiology reports) yields better open-ended segmentation than training on dense labels alone. On a benchmark of chest CT volumes, relative dice-score gains from adding the weak supervision are 8% when 1000 strongly labeled volumes are available, 22% when 250 are available, with intermediate gains at 500. The paper further argues that the weak supervision is most effective for finding categories that appear frequently in the report-derived weak data.","pith_inferences":["Possible extension: validating the extracted slice indices against a small set of manually labeled volumes would quantify label noise; if agreement is low for diffuse findings, a denoising or interval-aware labeling step could further improve gains.","Possible extension: the approach could be paired with active learning, spending dense annotation effort only on slices or findings the weak model is least confident about.","Possible extension: the presence-loss margin of 5 slices is a free parameter; sweeping it or making it category-dependent may improve performance for diffuse findings like emphysema.","Possible extension: since the gains are largest at low strong-label counts, the method points toward a regime where institutions with limited annotation budgets could rely primarily on report text."],"forward_implications":["With only 250 fully labeled volumes, adding weak slice labels improves dice by 22% relative; with 1000 volumes, it still improves by 8%.","Categories better represented in the weak training data tend to show larger gains from weak supervision.","Increasing the ratio of weak to strong examples in a batch improves performance up to a point of diminishing returns.","The method does not rely on anatomy-specific assumptions, so the recipe could transfer to other CT anatomies or imaging modalities with report locators."],"fun_headline_variants":["Report-mined weak labels improve CT segmentation by 22%","Slice-level report labels raise CT dice by 22% when masks are scarce","Weak supervision from radiology reports adds 22% to CT segmentation","Language-guided CT segmentation gains 22% from report-mined slice labels","Report-mined weak labels: 22% relative dice gain for CT segmentation"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The weak labels assume a finding appears only on the slices explicitly mentioned in the report (plus a 5-slice margin), so every slice farther away is treated as negative; for diffuse findings this produces false negatives, and the paper does not validate extracted slice indices against manual labels.","fun_headline_variants_meta":{"raw":{"variants":["Report-mined weak labels improve CT segmentation by 22%","Slice-level report labels raise CT dice by 22% when masks are scarce","Weak supervision from radiology reports adds 22% to CT segmentation","Language-guided CT segmentation gains 22% from report-mined slice labels","Report-mined weak labels: 22% relative dice gain for CT segmentation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000992,"raw_usage":{"total_tokens":3978,"prompt_tokens":622,"completion_tokens":3356,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":366,"completion_tokens_details":{"reasoning_tokens":3271}},"tokens_in":366,"tokens_out":3356,"duration_ms":21741,"temperature":1.0,"reasoning_tokens":3271,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T01:16:25.696716+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare extracted slice indices against ground-truth masks on a sample of volumes, measuring how often a reported slice contains the finding and how often far-away slices actually contain it; if diffuse findings frequently have positive slices far from the reported locator, then the strong negative labels are corrupting training, and removing or fixing those labels should change results.","supporting_citations":[],"review_version":1}