Pith. sign in

REVIEW 3 major objections 4 minor 32 references

Open-Ended CT Volume Segmentation with Weak Supervision from Language

T0 review · 3 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Training a text-conditioned CT segmentation model on slice-level labels mined from radiology reports improves accuracy, with the largest gains when dense ground-truth masks are scarce.

desk verdict First real demonstration that report-derived slice-level weak supervision helps open-ended CT segmentation, but the headline low-data gains rest on an unstable baseline and the weak-label pipeline is unvalidated. read the letter →

arxiv 2607.25860 v1 pith:RN7IUMAB submitted 2026-07-28 cs.CV

Sanjay Subramanian , Junwei Yu , Zirui Wang , Rohil Malpani , Maggie Chung , Adam Yala , Dan Klein , Trevor Darrell This is my paper
classification cs.CV
keywords CTsegmentationtext-conditionedweaksupervisionradiologyreportsslice-levellabelsopen-endedmedicalimageanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Open-ended CT segmentation usually requires dense voxel-level masks, which are expensive to produce. The paper shows that cheap slice-level labels—binary indicators of which slices contain a described finding—can be extracted automatically from radiology reports and combined with a small amount of dense supervision. This weak supervision improves segmentation performance, yielding an 8% relative dice gain with 1000 fully labeled volumes and a 22% gain with only 250 fully labeled volumes. The work suggests that widely available report text can partially offset the annotation cost that currently limits scaling of CT segmentation models.

What carries the argument

The load-bearing mechanism is the slice-level presence loss applied to weakly labeled slices. For each finding, a report snippet is parsed into a standalone description plus explicit slice locators (e.g., 'series 2, section 206'). Slices within a 5-slice margin of the reported location are treated as positive, slices farther away as negative, and slices in an ambiguous buffer between these zones are excluded from the loss. This per-slice binary presence signal is combined with the standard dense segmentation losses from strongly labeled slices in a single finetuning run.

What would settle it

Compare extracted slice indices against ground-truth masks on a sample of volumes, measuring how often a reported slice contains the finding and how often far-away slices actually contain it; if diffuse findings frequently have positive slices far from the reported locator, then the strong negative labels are corrupting training, and removing or fixing those labels should change results.

Watch

Extended reading notes

Core claim

The paper's central claim is that training a text-conditioned CT segmentation model on a mixture of densely labeled slices (full masks) and weakly labeled slices (binary presence labels derived from radiology reports) yields better open-ended segmentation than training on dense labels alone. On a benchmark of chest CT volumes, relative dice-score gains from adding the weak supervision are 8% when 1000 strongly labeled volumes are available, 22% when 250 are available, with intermediate gains at 500. The paper further argues that the weak supervision is most effective for finding categories that appear frequently in the report-derived weak data.

Load-bearing premise

The weak labels assume a finding appears only on the slices explicitly mentioned in the report (plus a 5-slice margin), so every slice farther away is treated as negative; for diffuse findings this produces false negatives, and the paper does not validate extracted slice indices against manual labels.

Editorial extensions

If this is right

  • With only 250 fully labeled volumes, adding weak slice labels improves dice by 22% relative; with 1000 volumes, it still improves by 8%.
  • Categories better represented in the weak training data tend to show larger gains from weak supervision.
  • Increasing the ratio of weak to strong examples in a batch improves performance up to a point of diminishing returns.
  • The method does not rely on anatomy-specific assumptions, so the recipe could transfer to other CT anatomies or imaging modalities with report locators.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Possible extension: validating the extracted slice indices against a small set of manually labeled volumes would quantify label noise; if agreement is low for diffuse findings, a denoising or interval-aware labeling step could further improve gains.
  • Possible extension: the approach could be paired with active learning, spending dense annotation effort only on slices or findings the weak model is least confident about.
  • Possible extension: the presence-loss margin of 5 slices is a free parameter; sweeping it or making it category-dependent may improve performance for diffuse findings like emphysema.
  • Possible extension: since the gains are largest at low strong-label counts, the method points toward a regime where institutions with limited annotation budgets could rely primarily on report text.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a method for open-ended CT volume segmentation that augments scarce voxel-level strong supervision with slice-level weak supervision extracted from radiology reports. A finetuned SAM3 model is trained with the standard SAM3 losses on strongly labeled slices and a presence classification loss on weakly labeled slices, where the weak labels are obtained by prompting an LLM to extract finding descriptions and series/image locators from a large internal database of chest CT reports. The authors evaluate on ReXGroundingCT and report relative Dice gains of 8% with 1000 strongly labeled volumes and 22% with 250 strongly labeled volumes, plus per-category analysis and a comparison with VoxTell.

Significance. If the empirical result holds, the paper makes a useful and timely contribution: it demonstrates a scalable way to leverage widely available radiology reports to reduce the annotation burden for text-conditioned CT segmentation. The evaluation is largely sound in design: it uses an external test set (ReXGroundingCT), varies the amount of strong supervision across three regimes, includes a per-category analysis that supports the proposed mechanism, and is transparent about the difficulties encountered during training. The main risks are that the weak-label extraction pipeline is never validated, the low-data headline result rests on a strong-only baseline that exhibited NaN loss, and no ablation isolates whether the gains come from the report-derived locator information or simply from additional presence-label training data.

major comments (3)
  1. [§4.3 footnote and Fig. 2] The headline claim of a 22% relative gain at 250 strongly labeled volumes is based on a strong-only baseline that, according to the footnote, encountered NaN loss; even after halving learning rates and restarting from the latest checkpoint, 'NaN loss still occurred.' A control that has gone through NaN loss and restart is not a reliable measure of the benefit of weak supervision. The strong+weak method also uses a different batch composition (1 strong + 4 weak), so the observed gap may reflect optimization stability rather than the information in the weak labels. Please provide stable strong-only baselines for the 250- and 500-volume regimes (e.g., with gradient clipping, loss scaling, or a more robust recipe) or explicitly restrict the central claim to the 1000-volume result, which does not carry this caveat.
  2. [§3.3, Algorithm 1 and §3.4] The weak-label construction is never validated. The labeling rule in §3.4 labels reported slices (plus immediate neighbors) as positive and all slices farther than g=5 as negative. This is known to be inexact for diffuse findings such as emphysema or large pleural effusions, and the paper acknowledges that the slice set is non-exhaustive, but it never quantifies the resulting label noise. The LLM extraction pipeline in Algorithm 1 is also unmeasured: no precision, recall, or manual audit of the extracted finding descriptions and slice indices is reported. Please validate the extracted weak labels against ground-truth masks on a subset of ReXGroundingCT (or another manually annotated set) and report the accuracy of the slice-locator matching.
  3. [§4.4.3, Fig. 6] The strong:weak ratio ablation shows that adding more weak examples per batch improves Dice, but it does not isolate the information source of the weak labels. The observed improvement could come from the extra presence-classification training signal provided by the report-derived locators, or simply from more data / regularization. An ablation with permuted or randomized weak slice labels—e.g., assigning the positive slice indices to the wrong volumes—would test whether the gain is actually attributable to the language-derived locator signal. Without such an ablation, the causal claim that report-based weak supervision drives the improvement is not fully established.
minor comments (4)
  1. [§4.3, Fig. 2] Error bars are described in the text as 95% bootstrap confidence intervals, but the figure appears to show them only inconsistently across panels. Please make the presence/absence of error bars explicit in the figure caption and ensure all reported metrics carry the same uncertainty treatment.
  2. [General] The dataset name is written inconsistently as 'ReXGroundingCT' and 'RexGroundingCT' (e.g., §4.2 vs. §4.4.2). Please standardize.
  3. [§4.4.3, Fig. 6] The strong:weak ratio ablation should specify the total batch size, the number of weak examples per batch, and whether the number of strong examples per update is held constant. The current description is ambiguous and makes the ablation difficult to reproduce.
  4. [Appendix A.1] The phrase 'RA VE cache' appears with an odd spacing throughout; presumably a formatting artifact from 'RAVE'. Please correct.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: external benchmark, no fitted-to-test quantities, and no load-bearing self-citation.

full rationale

The central claim is an empirical comparison of SAM3 finetuned with strong-only vs strong+weak supervision on the external ReXGroundingCT benchmark. The train/val/test split is fixed, hyperparameters are chosen on validation, and the reported dice/ROC-AUC numbers are computed on the held-out test set; no parameter is fitted to the test predictions, so the improvement is not forced by construction. The weak-slice labels are produced by a described extraction pipeline (regex locators plus GPT-5.4-mini parsing, Algorithm 1) and are not defined in terms of the model's outputs; the presence-loss labels are a function of report slice indices, not of the model's predictions, so there is no self-definitional loop. The only author self-citation (Yu et al. 2025, UnSAMv2) appears in a general related-work sentence about combining strong and weak supervision and is not load-bearing for any derivation. The main caveats—the unvalidated assumption that findings appear only on mentioned slices plus margin g=5, the possible overlap between the internal report corpus and CT-RATE/ReXGroundingCT, and the NaN-loss instability of the 250/500-volume strong-only baseline—are data-quality and comparison-validity concerns, not circularity: the manuscript does not exhibit an equation or fitted quantity that reduces to its own input. Accordingly, no circular step is identified.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central empirical claim relies on the quality of the LLM-extracted weak labels and on the negative-labeling heuristic; these are not validated against manual labels in the paper.

free parameters (4)
  • weak-label margin g = 5
    Defines which slices are treated as positive/negative in the weak presence loss (Section 3.4); not tuned, but affects label noise.
  • strong:weak batch ratio = 1:4
    Chosen via ablation (Section 4.4.3); diminishing returns observed, but affects results.
  • presence loss alpha (alpha_p) = 0.7
    Focal loss weight for presence; standard SAM3 default, not carefully tuned here.
  • slice window size = not reported
    Batch sampling uses a fixed slice subset size around an anchor (Section 3.4); the value is omitted from the paper.
assumptions (4)
  • domain assumption Radiology reports contain explicit, accurate slice indices for findings that can serve as weak labels.
    The extraction procedure (Section 3.3) relies on regex and LLM parsing of 'series X image Y' locators without manual validation of a random sample.
  • domain assumption Findings occur on a contiguous or small set of slices, so labeling slices farther than g=5 from the mentioned slice as negative is valid.
    Section 3.4 defines the weak label assignment; this assumption is violated by diffuse findings like emphysema or large effusions.
  • domain assumption The internal 244k-report database is independent of the ReXGroundingCT evaluation set.
    No overlap analysis is provided; both datasets are chest CT with reports, creating a leakage risk.
  • ad hoc to paper The LLM (GPT-5.4-mini) extracts complete and correct lung findings from report snippets.
    The extraction pipeline in Algorithm 1 uses an LLM without any manual evaluation of extraction quality; errors propagate directly to weak labels.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Open-Ended CT Volume Segmentation with Weak Supervision from Language." pith.science (2026). https://pith.science/paper/RN7IUMAB

@misc{pith2026260725860,
  author       = {Pith},
  title        = {Pith review of: Open-Ended CT Volume Segmentation with Weak Supervision from Language},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RN7IUMAB}},
  note         = {Machine review of arXiv:2607.25860}
}
read the original abstract

We introduce a method for training a text-conditioned segmentation model for CT scans, which combines voxel-level supervision with coarse but scalable slice-level supervision from reports. We extract, from a large database of scan-report pairs, descriptions of findings with indices of slices where those findings occur. We then finetune a general-purpose 2D image segmentation model, SAM3, with standard segmentation losses from strongly labeled data and with a slice-level classification loss from the extracted weak supervision. Our results on the ReXGroundingCT dataset illustrate that this strategy improves the segmentation dice score: from an 8% relative gain when there are 1000 fully labeled volumes to 22% when there are 250 fully labeled volumes.

Figures

Figures reproduced from arXiv: 2607.25860 by the authors.

Figure 1
Figure 1. Open-ended CT segmentation with weak supervision. In our method, we train a per-slice segmentation model with both strong and weak supervision. The model takes as input a single 2D slice and a finding description and produces as output a presence (i.e. slice-level existence) probability and segmentation mask. Strongly labeled (volume, finding) pairs include both presence labels and mask labels for every slice but ar… view at source ↗
Figure 2
Figure 2. Strong vs. Strong+Weak supervision. ReXGroundingCT test set results with different numbers of strongly labeled volumes. Weak supervision improves segmentation performance, especially when the number of available strongly labeled volumes is modest [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Per-category results. Finding categories with more representation in the weak training data tend to exhibit greater improvement from weakly supervised training. These results are from training on 500 strongly labeled volumes with and without the weak supervision. inverse square root decay. For both the baseline and the weakly supervised method, we experiment with scheduler timescales of 20 and 320 and find that 320 … view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Examples from different categories. Each 4-image panel shows (from left to right): a slice from the CT volume, the prediction of the strong model, the prediction of the strong+weak model, the ground-truth mask. For each panel, the finding text corresponding to the exam…
Figure 5
Figure 5. Figure 5: Comparison with state-of-the-art. Our SAM3 models have dice scores which are slightly lower than and hit rates which are slightly greater than those of the state-of-the-art VoxTell model [Rokuss et al., 2026]. For this experiment, we train on the full official training…
Figure 6
Figure 6. Figure 6: Ablating strong:weak ratio. Adding more weak supervision per batch improves the dice score, though there are diminishing returns. the system prompt in Box 2 and the user prompt in Box 3. The resulting set of categories is provided in [PITH_FULL_IMAGE:figures/full_fig_…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

32 extracted references · 4 linked inside Pith

  1. [1]

    arXiv preprint arXiv:2603.02026 , year=

    Learning to Read Where to Look: Disease-Aware Vision-Language Pretraining for 3D CT , author=. arXiv preprint arXiv:2603.02026 , year=

    arXiv

  2. [2]

    arXiv preprint arXiv:2511.16719 , year=

    Sam 3: Segment anything with concepts , author=. arXiv preprint arXiv:2511.16719 , year=

    arXiv

  3. [3]

    arXiv preprint arXiv:2405.12971 , year=

    Biomedparse: a biomedical foundation model for image parsing of everything everywhere all at once , author=. arXiv preprint arXiv:2405.12971 , year=

    arXiv

  4. [4]

    arXiv preprint arXiv:2511.19046 , year=

    MedSAM3: Delving into Segment Anything with Medical Concepts , author=. arXiv preprint arXiv:2511.19046 , year=

    arXiv

  5. [5]

    arXiv preprint arXiv:2601.10880 , year=

    Medical SAM3: A Foundation Model for Universal Prompt-Driven Medical Image Segmentation , author=. arXiv preprint arXiv:2601.10880 , year=

    arXiv

  6. [6]

    CVPR 2025: Foundation Models for 3D Biomedical Image Segmentation , year=

    BiomedParse-V: Scaling Foundation Model for Universal Text-guided Volumetric Biomedical Image Segmentation , author=. CVPR 2025: Foundation Models for 3D Biomedical Image Segmentation , year=

    2025

  7. [7]

    NPJ Digital Medicine , volume=

    Large-vocabulary segmentation for medical images with text prompts , author=. NPJ Digital Medicine , volume=. 2025 , publisher=

    2025

  8. [8]

    Advances in Neural Information Processing Systems , volume=

    Segvol: Universal and interactive volumetric medical image segmentation , author=. Advances in Neural Information Processing Systems , volume=

Show all 32 references
  1. [9]

    CVPR 2025: Foundation Models for 3D Biomedical Image Segmentation , year=

    Text3DSAM: Text-Guided 3D Medical Image Segmentation Using SAM-Inspired Architecture , author=. CVPR 2025: Foundation Models for 3D Biomedical Image Segmentation , year=

    2025

  2. [10]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Voxtell: Free-text promptable universal 3d medical image segmentation , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  3. [11]

    arXiv preprint arXiv:2510.14803 , year=

    Scaling artificial intelligence for multi-tumor early detection with more reports, fewer masks , author=. arXiv preprint arXiv:2510.14803 , year=

  4. [12]

    2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI) , pages=

    Weakly supervised lesion co-segmentation on ct scans , author=. 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI) , pages=. 2020 , organization=

    2020

  5. [13]

    Journal of medical imaging , volume=

    DeepLesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning , author=. Journal of medical imaging , volume=. 2018 , publisher=

    2018

  6. [14]

    Advances in neural information processing systems , volume=

    Decoupled deep neural network for semi-supervised semantic segmentation , author=. Advances in neural information processing systems , volume=

  7. [15]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Ficklenet: Weakly and semi-supervised semantic image segmentation using stochastic inference , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  8. [16]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Weakly-supervised semantic segmentation via sub-category exploration , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  9. [17]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Weakly supervised learning of instance segmentation with inter-pixel relations , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  10. [18]

    arXiv preprint arXiv:2010.09713 , year=

    Pseudoseg: Designing pseudo labels for semantic segmentation , author=. arXiv preprint arXiv:2010.09713 , year=

    2010 arXiv

  11. [19]

    arXiv preprint arXiv:2010.11929 , year=

    An image is worth 16x16 words: Transformers for image recognition at scale , author=. arXiv preprint arXiv:2010.11929 , year=

    2010 arXiv

  12. [20]

    arXiv preprint arXiv:2403.17834 , year=

    Developing generalist foundation models from a multimodal dataset for 3d computed tomography , author=. arXiv preprint arXiv:2403.17834 , year=

  13. [21]

    arXiv preprint arXiv:2507.22030 , year=

    Rexgroundingct: A 3d chest ct dataset for segmentation of findings from free-text reports , author=. arXiv preprint arXiv:2507.22030 , year=

  14. [22]

    European conference on computer vision , pages=

    Exploring plain vision transformer backbones for object detection , author=. European conference on computer vision , pages=. 2022 , organization=

    2022

  15. [23]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Groupvit: Semantic segmentation emerges from text supervision , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  16. [24]

    arXiv preprint arXiv:2511.17803 , year=

    Pillar-0: A new frontier for radiology foundation models , author=. arXiv preprint arXiv:2511.17803 , year=

  17. [25]

    Proceedings of SPIE--the International Society for Optical Engineering , volume=

    Improving Small Lesion Segmentation in CT Scans using Intensity Distribution Supervision: Application to Small Bowel Carcinoid Tumor , author=. Proceedings of SPIE--the International Society for Optical Engineering , volume=

  18. [26]

    American Journal of Roentgenology , volume=

    Impact of artificial intelligence assistance on chest CT interpretation times: a prospective randomized study , author=. American Journal of Roentgenology , volume=. 2022 , publisher=

    2022

  19. [27]

    British Journal of Radiology , volume=

    Artificial intelligence-based tools with automated segmentation and measurement on CT images to assist accurate and fast diagnosis in acute pancreatitis , author=. British Journal of Radiology , volume=. 2024 , publisher=

    2024

  20. [28]

    Academic radiology , volume=

    Automated segmentation of tissues using CT and MRI: a systematic review , author=. Academic radiology , volume=. 2019 , publisher=

    2019

  21. [29]

    Medical Image Analysis , volume=

    A comprehensive survey on deep active learning in medical image analysis , author=. Medical Image Analysis , volume=. 2024 , publisher=

    2024

  22. [30]

    Nature medicine , volume=

    AI in health and medicine , author=. Nature medicine , volume=. 2022 , publisher=

    2022

  23. [31]

    Quantitative Imaging in Medicine and Surgery , volume=

    Detection-guided deep learning-based model with spatial regularization for lung nodule segmentation , author=. Quantitative Imaging in Medicine and Surgery , volume=. 2025 , publisher=

    2025

  24. [32]

    arXiv preprint arXiv:2511.13714 , year=

    UnSAMv2: Self-Supervised Learning Enables Segment Anything at Any Granularity , author=. arXiv preprint arXiv:2511.13714 , year=

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.