REVIEW 4 major objections 6 minor 21 references
Radiologist dictation drives brain-tumor segmentation through a modular chain of pretrained speech, language, and vision models, reaching 98.6% of full fine-tuning while training under 5% of parameters.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-02 17:55 UTC pith:ZSYWDKJV
load-bearing objection A sensible modular speech-to-segmentation pipeline with real parameter-efficiency numbers, but the headline Dice is measured with clean text prompts, not with the speech-derived prompts the system is named for. the 4 major comments →
LoGSAM: Parameter-Efficient Cross-Modal Grounding for MRI Segmentation
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper's central discovery is that a fully modular chain — speech transcription and translation, negation-aware clinical NLP, a LoRA-tuned Grounding DINO detector for text-conditioned box prediction, and a frozen MedSAM for mask generation — can turn radiologist dictation into brain-tumor segmentations while adapting fewer than 5% of the parameters of the localization model. On BRISC 2025 it reaches a mean Dice of 0.8032, 98.6% of the fully fine-tuned GDINO+MedSAM baseline, and on 12 real German dictations it extracts the correct tumor class in 11 of 12 cases. The intended upshot is that dense pixel annotations and full model fine-tuning are not prerequisites for clinically useful segment
What carries the argument
The load-bearing mechanism is the detection-driven prompt chain. A negation-aware NLP stage turns transcribed dictation into a short class prompt (e.g., 'glioma'); a LoRA-augmented Grounding DINO detector (low-rank adaptation inserted in attention, cross-attention, feed-forward, and text-encoder layers) aligns that prompt to image regions and outputs bounding boxes; a frozen MedSAM turns each box into a pixel mask. LoRA is the parameter-efficiency device: fewer than 5% of model weights are updated, while the pretrained cross-modal knowledge stays fixed.
Load-bearing premise
The load-bearing premise is that healthy MRIs will not routinely produce tumor masks; the paper's qualitative note that on healthy scans the detector flags the entire brain region, with no false-positive metrics reported, leaves this assumption unquantified.
What would settle it
Count the fraction of healthy BRISC 2025 test images for which LoGSAM returns a non-empty mask; a high fraction would directly contradict the tumor-segmentation utility claimed for the pipeline.
If this is right
- New segmentation tasks could be tackled with box-level annotations and frozen foundation models, reducing the need for dense pixel labels.
- Non-English dictation can drive segmentation: the German-dictation evaluation yields correct tumor-class extraction in 11 of 12 unseen cases.
- Out-of-distribution detection remains serviceable: LoRA-augmented GDINO retains 0.7955 mAP@50 on an unseen brain-MRI set, enough for MedSAM to produce usable masks.
- The localization and segmentation stages are decoupled, so either module can be swapped without retraining the other.
- The speech-to-prompt stage can suppress many false positives through negation-aware NLP, which matters because radiology reports routinely contain negated findings.
Where Pith is reading between the lines
- If healthy MRIs often trigger whole-brain masks (as the qualitative note suggests), clinical use would require a no-tumor gate; measuring specificity on healthy slices is the natural next experiment.
- The same chain may transfer to other text-guided imaging tasks—chest X-ray findings, ultrasound organ localization—wherever a grounding model can align report phrases to image regions.
- End-to-end Dice from the German dictations, not just class-extraction accuracy, would strengthen the speech-to-mask claim; the paper reports class accuracy separately from segmentation Dice.
- The LoRA rank sweet spot (r=64, with degradation at r=128) hints that low-rank adaptation has a capacity ceiling for grounding models, which could inform parameter budgets in other medical domains.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes LoGSAM, a modular pipeline that converts radiologist dictation into text prompts via Whisper ASR and spaCy/negspaCy NLP, uses a LoRA-adapted Grounding DINO to localize brain tumors from these prompts, and feeds the predicted bounding boxes to a frozen MedSAM for pixel-level segmentation. On the BRISC 2025 test set, the text-prompted pipeline attains a mean Dice of 0.8032, reaching 98.6% of a fully fine-tuned GDINO+MedSAM baseline while updating under 5% of parameters. The same pipeline is evaluated on 12 German dictations, where the class-extraction stage is correct in 11/12 cases. Detection performance is also reported on an out-of-distribution Kaggle BBOX dataset.
Significance. If the full speech-to-segmentation chain were validated end-to-end, the paper would be a useful demonstration that large pretrained vision-language and segmentation models can be assembled with minimal trainable parameters for a clinically oriented task. The public code, the modular decomposition, and the explicit LoRA ablation are strengths. However, the current evidence supports a modular proof-of-concept rather than the claimed integrated speech-to-segmentation system, and the absent healthy-case metrics materially weaken the clinical utility claim.
major comments (4)
- [§3, Tables 1 and 3] The headline Dice result of 0.8032 is computed with clean text prompts, not with prompts produced by the speech/NLP module. The German-dictation experiment is reported only as 11/12 class-extraction accuracy, with no bounding boxes or segmentation masks evaluated on those 12 cases. Because the abstract calls LoGSAM a speech-to-segmentation pipeline and reports the 80.32% Dice as its score, the integrated claim is not directly tested. The authors should provide end-to-end Dice, or at least localization box quality, on the dictation-driven cases; alternatively, they should explicitly state that the segmentation score is measured under clean-text prompting.
- [§3, Qualitative Assessment] The paper acknowledges that for healthy MRIs 'the detection head identifies the entire brain region,' yet no healthy-case metrics (false-positive rate, specificity, or per-class Dice) are reported. Since healthy images constitute ~20% of BRISC and healthy dictations are interpreted via default negation, a system that regularly masks the whole brain in healthy scans would not be clinically useful. The authors should quantify how often healthy slices receive tumor masks and report the corresponding segmentation-level false-positive behavior.
- [Table 3 and Abstract] The 'state-of-the-art' label for the 0.8032 Dice is unsupported as written. The only comparator in Table 3 is Trust-Refined U-Net (0.7980) and an in-house fully fine-tuned GDINO+MedSAM baseline (0.8145); no external BRISC leaderboard comparison, confidence intervals, or significance tests are provided. The authors should either provide a rigorous comparison with existing BRISC methods and statistical uncertainty or temper the claim to 'competitive with a fully fine-tuned baseline.'
- [Table 2, OOD evaluation] The out-of-distribution evaluation is limited to detection on Kaggle BBOX and does not include segmentation results on that dataset. The LoRA-augmented model drops from 0.8428 to 0.7955 mAP@50 relative to the fully fine-tuned baseline in the OOD setting, which is a larger degradation than in-distribution. The claim of 'robust generalization' should be scoped to detection and should explicitly acknowledge this gap, ideally with confidence intervals and an analysis of failure cases.
minor comments (6)
- [§3, first paragraph of Detection-to-Segmentation] Typos: 'performstronglyonBRISC' and 'introducesamodest reduction' should be separated.
- [Table 1 caption and text] The text says the speech module was tested on 12 cases, but Table 1 shows only 6; this should be made consistent and the full confusion matrix or per-case outcomes for all 12 cases should be shown.
- [§3, Speech-to-prompt generation] The reported WER of 5.7% is cited from the Whisper model's general benchmark, not measured on the 12 radiologist dictations. The authors should report WER on their own test dictations.
- [§3, Speech-to-prompt generation] The 11/12 success rate on n=12 has a wide confidence interval; reporting only a raw percentage overstates precision.
- [Abstract] The phrase 'full pipeline using German dictations ... achieving 91.7% case-level accuracy' should be clarified as case-level class-extraction accuracy, not end-to-end segmentation accuracy.
- [§2.2, Implementation Details] The authors mention 'negspaCy gating mechanism' but the reference list cites the original NegEx paper [18]; the distinction between the implemented component and the original algorithm should be stated explicitly.
Circularity Check
No significant circularity: LoGSAM's headline Dice is an empirical test-set measurement with train/test separation; the speech stage is separately evaluated and no load-bearing self-citation or definitional reduction appears.
full rationale
The paper's central derivation is an empirical pipeline evaluation, not a definitional or self-referential chain. The headline result (mean Dice 0.8032 on BRISC 2025, Table 3) is obtained by training LoRA-augmented GDINO on BRISC bounding-box annotations and then measuring segmentation quality on the BRISC test split with frozen MedSAM. This is a standard train/test protocol: the LoRA parameters are fitted to training data, while the reported Dice is computed on held-out test images. The confidence threshold tau=0.3 and LoRA rank r=64 are ordinary hyperparameters selected by ablation; neither defines the target metric, nor is the Dice score a renamed version of any fitted quantity. The speech-to-prompt module is evaluated separately on 12 German dictations (Table 1, 11/12 case-level class-extraction accuracy), and this evaluation does not presuppose the segmentation scores; conversely, the Dice measurement uses clean textual prompts. The fact that the two stages are not evaluated end-to-end in a single numeric metric is an evaluation-coverage gap, not circularity. There is no load-bearing self-citation: references to Whisper, spaCy, negspaCy, GDINO, and MedSAM point to external pretrained systems, and no 'uniqueness theorem' or prior author result is invoked to force the design. The qualitative statement that for healthy MRIs 'the detection head identifies the entire brain region' identifies a potential false-positive risk, but this is a correctness/validation concern, not a circular-reasoning concern. Because the main claims are supported by an independent empirical benchmark with a train/test split and an OOD evaluation set, the appropriate circularity score is 0.
Axiom & Free-Parameter Ledger
free parameters (3)
- LoRA rank =
r=64
- Detection confidence threshold τ =
0.3
- LoRA training schedule =
125 epochs, AdamW lr=2e-4, OneCycle 10% warmup, gradient clip 5.0
axioms (4)
- domain assumption Pretrained component reliability
- domain assumption Curated clinical vocabulary coverage
- domain assumption Dataset label reliability
- domain assumption Healthy-case behavior is benign
read the original abstract
Precise localization and delineation of brain tumors using magnetic resonance imaging (MRI) are essential for planning therapy and guiding surgical decisions. To address this, we propose LoGSAM, a parameter-efficient, detection-driven framework that transforms radiologist dictation into text prompts for foundation-model-based localization and segmentation. Radiologist speech is first transcribed and translated using a pretrained Whisper ASR model, followed by negation-aware clinical NLP to extract tumor-specific textual prompts. These prompts guide text-conditioned tumor localization via a LoRA-adapted vision-language detection model, Grounding DINO (GDINO). The predicted bounding boxes are used as prompts for MedSAM to generate pixel-level tumor masks without any additional fine-tuning. On BRISC 2025, LoGSAM attains a Dice score of 80.32%, reaching 98.6% of a fully fine-tuned GDINO + MedSAM baseline while training fewer than 5\% of its parameters, indicating a favorable accuracy/parameter trade-off. In addition, we evaluate the full pipeline using German dictations from a board-certified radiologist on unseen MRI scans, achieving 91.7\% case-level class-extraction accuracy. These results highlight the feasibility of constructing a modular speech-to-segmentation pipeline from pretrained foundation models with minimal parameter updates.
Figures
Reference graph
Works this paper leans on
-
[1]
In: European Conference on Computer Vision (ECCV), pp
Liu, S., Zeng, Z., Ren, T., et al.: Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection. In: European Conference on Computer Vision (ECCV), pp. 38–55. Springer, Heidelberg (2024)
2024
-
[2]
In: Medical Image Computing and Computer-Assisted Intervention (MICCAI), pp
Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional Networks for Biomed- ical Image Segmentation. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI), pp. 234–241 (2015)
2015
-
[3]
W., Xu, T., Brockman, G., McLeavey, C., Sutskever, I.: Ro- bust Speech Recognition via Large-Scale Weak Supervision
Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., Sutskever, I.: Ro- bust Speech Recognition via Large-Scale Weak Supervision. In: Int. Conference on Machine Learning (ICML), pp. 28492–28518. PMLR (2023)
2023
-
[4]
Nature Communications,15(1), 654 (2024)
Ma, J., He, Y., Li, F., Han, L., You, C., Wang, B.: Segment anything in medical images. Nature Communications,15(1), 654 (2024)
2024
-
[5]
IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control (2025)
Rasaee, H., Koleilat, T., Rivaz, H.: Grounding DINO-US-SAM: Text-Prompted Multi-Organ Segmentation in Ultrasound with LoRA-Tuned Vision-Language Mod- els. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control (2025)
2025
-
[6]
J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. In: International Conference on Learning Representations (ICLR),1(2), 3 (2022)
2022
-
[7]
Acta Neuropathologica 131(6), 803–820 (2016)
Louis,D.N., Perry, A.,Reifenberger,G., etal.:The 2016World HealthOrganization Classification of Tumors of the Central Nervous System. Acta Neuropathologica 131(6), 803–820 (2016)
2016
-
[8]
Physics in Medicine & Biology58(13), R97 (2013)
Bauer, S., Wiest, R., Nolte, L.-P., Reyes, M.: A Survey of MRI-Based Medical Image Analysis for Brain Tumor Studies. Physics in Medicine & Biology58(13), R97 (2013)
2013
-
[9]
Frontiers in Oncology15, 1643208 (2025)
Han, W., Dong, X., Wang, G., Ding, Y., Yang, A.: Application and improvement of YOLO11 for brain tumor detection in medical images. Frontiers in Oncology15, 1643208 (2025)
2025
-
[10]
arXiv preprint arXiv:1906.07155 (2019)
Chen, K., Wang, J., Pang, J., Cao, Y., Xiong, Y., Li, X., Sun, S., Feng, W., Liu, Z., Xu, J., Zhang, Z.: MMDetection: Open mmlab detection toolbox and benchmark. arXiv preprint arXiv:1906.07155 (2019)
Pith/arXiv arXiv 1906
-
[11]
Insights into Imaging14(1), 47 (2023)
Jörg, T., Kämpgen, B., Feiler, D., Müller, L., Düber, C., Mildenberger, P., Jung- mann, F.: Efficient structured reporting in radiology using an intelligent dialogue system based on speech recognition and natural language processing. Insights into Imaging14(1), 47 (2023)
2023
-
[12]
In: International Conference on Medical Imaging and Computer-Aided Diagnosis, pp
Hu, J., Xu, X., Zou, Z.: LoRA-MedSAM: Efficient medical image segmentation. In: International Conference on Medical Imaging and Computer-Aided Diagnosis, pp. 154–164. Springer, Singapore (2024)
2024
-
[13]
L., et al.: Radiotherapy Planning and Evaluation
Chang, E. L., et al.: Radiotherapy Planning and Evaluation. Neurosurgery Clinics of North America18(1), 43–53 (2007)
2007
-
[14]
D., Langlotz, C
Zhang, Y., Jiang, H., Miura, Y., Manning, C. D., Langlotz, C. P.: Contrastive learning of medical visual representations from paired images and text. In: Machine Learning for Healthcare Conference, pp. 2–25. PMLR (2022)
2022
-
[15]
In: Proceedings of the Conference on Empirical Methods in NLP (EMNLP), vol
Wang, Z., Wu, Z., Agarwal, D., Sun, J.: MedCLIP: Contrastive learning from unpaired medical images and text. In: Proceedings of the Conference on Empirical Methods in NLP (EMNLP), vol. 2022, p. 3876 (2022)
2022
-
[16]
K., Mahala, S., Kumar, Y., Shekhawat, N
Bhamboo, A. K., Mahala, S., Kumar, Y., Shekhawat, N. S., Sharma, K.: Trust- Refined U-Net Ensemble for Uncertainty-Aware Brain Tumor Segmentation. Au- thorea Preprints (2025) 10 M.R.I. Bhuiyan et. al
2025
-
[17]
C., Lo, W.-Y., Dollár, P.: Segment Anything
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., Dollár, P.: Segment Anything. In: Proceed- ings of the IEEE/CVF Int. Conference on Computer Vision (ICCV), pp. 4015–4026 (2023)
2023
-
[18]
W., Bridewell, W., Hanbury, P., Cooper, G
Chapman, W. W., Bridewell, W., Hanbury, P., Cooper, G. F., Buchanan, B. G.: A simple algorithm for identifying negated findings and diseases in discharge sum- maries. Journal of Biomedical Informatics34(5), 301–310 (2001)
2001
-
[19]
Zenodo (2020)
Honnibal, M., Montani, I., Van Landeghem, S., Boyd, A.: spaCy: Industrial- Strength Natural Language Processing in Python. Zenodo (2020)
2020
-
[20]
Scientific Data (2026)
Fateh,A.,Rezvani,Y.,Moayedi,S.,Rezvani,S.,Fateh,F.,Fateh,M.,Abolghasemi, V.: BRISC: Annotated dataset for brain tumor segmentation and classification. Scientific Data (2026)
2026
-
[21]
MRI for Brain Tumor with Bounding Boxes (Kag- gle dataset),https://www.kaggle.com/datasets/ahmedsorour1/ mri-for-brain-tumor-with-bounding-boxes, last accessed 2026/02/07
2026
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.