Pith. sign in

REVIEW 3 major objections 5 minor 21 references

Radiologist speech can drive MRI tumor segmentation that matches a fully fine-tuned system while training under 5% of its parameters.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 23:05 UTC pith:ZSYWDKJV

load-bearing objection Solid parameter-efficient detection-to-segmentation result on BRISC; the speech-to-segmentation story is still preliminary because of n=12 and unquantified NLP rules. the 3 major comments →

arxiv 2603.17576 v3 pith:ZSYWDKJV submitted 2026-03-18 cs.CV

LoGSAM: Parameter-Efficient Cross-Modal Grounding for MRI Segmentation

Mohammad Robaitul Islam Bhuiyan , Sheethal Bhat , Melika Qahqaie This is my paper
classification cs.CV
keywords MRI segmentationLoRAGrounding DINOMedSAMspeech-to-segmentationvision-language groundingbrain tumorparameter-efficient fine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that brain-tumor localization and segmentation on MRI can be driven by radiologist dictation rather than dense pixel labels. Speech is transcribed, cleaned of negation, and turned into short text prompts that steer a lightly adapted vision-language detector; the resulting boxes then prompt a frozen medical segmenter. On a public multi-class brain MRI set the pipeline reaches a Dice score of 80.32 percent—about 98.6 percent of a fully fine-tuned counterpart—while updating fewer than five percent of parameters. A small test with real German dictations yields 91.7 percent case-level class accuracy. The practical claim is that modular, speech-conditioned foundation models can deliver clinically usable masks with far less annotation and compute than conventional supervised pipelines.

Core claim

LoGSAM shows that a speech-to-prompt chain (Whisper plus negation-aware clinical NLP), a LoRA-adapted Grounding DINO localizer, and a frozen MedSAM segmenter together produce tumor masks whose mean Dice of 0.8032 is 98.61 percent of a fully fine-tuned GDINO+MedSAM baseline while training only about 4.96 percent of the detector parameters, and that the same pipeline recovers the correct tumor class from real German radiologist dictations on twelve unseen scans with 91.7 percent case-level accuracy.

What carries the argument

LoRA-adapted text-conditioned Grounding DINO: low-rank adapters inserted into selected attention, cross-attention, feed-forward, and text-encoder layers so that spoken tumor cues become bounding-box prompts that condition frozen MedSAM without pixel-level fine-tuning.

Load-bearing premise

The claim that the full speech-to-segmentation pipeline works in practice rests on a twelve-case German dictation test and a hand-curated synonym and negation vocabulary applied to multi-source 2D JPEG slices that lack full clinical acquisition metadata.

What would settle it

Run the identical speech-to-prompt and LoRA-GDINO+MedSAM pipeline on a larger set of real multilingual radiologist dictations paired with independent 3D multi-sequence clinical MRI cases whose patient and scanner provenance is known; a sharp drop in class-extraction accuracy or Dice relative to the reported 91.7 percent / 0.8032 would falsify the modular speech-to-segmentation claim.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Speech-derived text prompts can replace manual boxes at inference for MedSAM-style medical segmentation.
  • LoRA adaptation of fewer than 5 percent of Grounding DINO parameters is enough to keep localization quality high enough for downstream Dice near a fully fine-tuned baseline.
  • Negation-aware clinical NLP plus a controlled tumor vocabulary can turn free-form radiologist speech into usable class prompts.
  • The same modular stack can be evaluated out-of-distribution on other 2D brain-MRI collections without retraining the segmenter.
  • Future 3D multi-sequence extensions become feasible because most foundation-model weights stay frozen.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the speech module generalizes, departments could log dictation audio as a free supervisory signal and reduce the need for dedicated pixel-label campaigns.
  • The same detection-then-prompt pattern could be tried on other organs where radiologists already dictate findings and SAM-style medical models exist.
  • Failure modes will likely concentrate on rare synonyms, heavy accents, or multi-lesion reports that the current controlled vocabulary and single-box filtering do not cover.
  • Parameter-efficient grounding may let hospitals keep a single shared MedSAM and only swap lightweight LoRA adapters per site or scanner.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. LoGSAM is a modular speech-to-segmentation pipeline for brain MRI tumors. Radiologist dictation is transcribed/translated with Whisper, then mapped by negation-aware spaCy/negspaCy NLP (with a curated synonym dictionary) into a controlled class prompt. A LoRA-adapted Grounding DINO (Swin-T) localizes the tumor from the image+prompt; predicted boxes prompt a frozen MedSAM for pixel masks. On BRISC 2025, LoRA-GDINO + MedSAM reaches mean Dice 0.8032 (98.6% of a fully fine-tuned GDINO+MedSAM baseline of 0.8145) while training ~4.96% of parameters; detection mAP/IoU and LoRA rank/injection ablations are reported, with OOD results on a Kaggle BBOX set. A separate 12-case German dictation evaluation reports 91.7% case-level class-extraction accuracy.

Significance. If the accuracy/parameter trade-off holds, the work is a useful engineering demonstration that PEFT (LoRA) can adapt open-set grounding models to MRI with modest localization loss, and that detection-driven boxes can drive frozen MedSAM to near fully supervised Dice without pixel-level fine-tuning. The modular composition of ASR, clinical NLP, grounding, and medical SAM is practically relevant for reducing dense annotation burden. Strengths include clear ablations on LoRA rank and injection location (Table 4), explicit comparison to a fully fine-tuned baseline and a U-Net ensemble (Tables 2–3), OOD detection numbers, and released code. The speech-to-segmentation framing is the more ambitious claim and would matter clinically if better supported.

major comments (3)
  1. [Abstract, §1, §2.1–2.2, Table 1, §3] Abstract, §1 contributions, and §4 frame the central result as a modular speech-to-segmentation pipeline. Detection-to-segmentation (LoRA-GDINO + frozen MedSAM) is supported by Tables 2–4. The speech claim, however, rests on n=12 German dictations (three per class; §2.1) with 91.7% case-level class-extraction accuracy and a hand-curated synonym/negation lexicon (§2.2). Table 1 already shows an ASR failure that flips the class. The paper never reports Dice or box mAP when prompts come from real ASR+NLP output rather than oracle class labels, so the end-to-end speech-to-mask claim is under-supported relative to how it is stated.
  2. [Tables 2–3, §3] Tables 2–3 report only point estimates (mAP, mean IoU, mean Dice) with no standard deviations, confidence intervals, or statistical comparison to the fully fine-tuned baseline or the Trust-Refined U-Net ensemble. Given mild class imbalance and multi-source JPEG slices without patient/scanner metadata (§2.1), the 1.13-point Dice gap and the “98.6% of baseline / SOTA” language cannot be assessed for reliability. At minimum, report variability (e.g., over seeds or folds) and a simple significance test for the key Dice comparison.
  3. [§2.1 Datasets] BRISC is described as multi-source 2D JPEGs lacking patient, slice, and scanner metadata, with only manual review against patient overlap (§2.1). The OOD set is another public JPEG collection. Without patient-level splits or acquisition metadata, the generalization claims (including the 12-case speech test on “unseen” scans) risk slice- or source-level leakage. Clarify split construction and, if possible, report patient-level metrics or a stronger external clinical cohort.
minor comments (5)
  1. [Abstract, Table 3] Abstract and Table 3 use both “80.32%” and “0.8032” for Dice; standardize to one convention and state whether the metric is mean over cases or over classes.
  2. [Fig. 2, §3] Fig. 2 caption and body text should state whether boxes/masks shown use oracle class prompts or speech-derived prompts, and whether healthy cases produce empty masks by design.
  3. [§2.2 Implementation Details] Implementation details: justify τ=0.3 and the choice of r=64 beyond the ablation table; note whether MedSAM box prompts are expanded or used as-is.
  4. [Throughout] Minor language issues: “using<5%” spacing; “gg(227)” case ID in Table 1; “BBOX” vs full dataset name consistency; “state-of-the-art dice score” capitalization.
  5. [§1] Related work could more clearly position against other LoRA+SAM medical pipelines (e.g., cited [5,12]) and against pure detection baselines that lack text conditioning.

Circularity Check

0 steps flagged

No circularity: empirical modular pipeline with standard supervised training and held-out evaluation; results do not reduce by construction to inputs.

full rationale

LoGSAM is an engineering composition of pretrained foundation models (Whisper, spaCy/negspaCy, LoRA-adapted Grounding DINO, frozen MedSAM) rather than a first-principles derivation. Localization is trained with ordinary bounding-box supervision on BRISC train splits (AdamW, 125 epochs, r=64 LoRA); segmentation Dice (0.8032) and mAP are measured on held-out BRISC test and an OOD Kaggle set; the speech-to-prompt module is evaluated separately on 12 external German dictations for class-extraction accuracy. None of the reported numbers is obtained by fitting a parameter to a quantity and then re-labeling that same quantity as a prediction, nor is any uniqueness or ansatz imported via self-citation. Ablations (rank, injection location) and the fully-fine-tuned baseline comparison are independent controls. Minor engineering choices (curated synonym list, τ=0.3) are explicit and do not force the Dice or accuracy figures by definition. The derivation chain is therefore self-contained against external benchmarks and exhibits no circular steps.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 2 invented entities

The central accuracy/parameter claim rests on standard PEFT and foundation-model assumptions plus several engineering choices (LoRA rank, score threshold, curated clinical vocabulary) and dataset premises (BRISC split integrity, adequacy of 2D JPEG multi-source data). The speech-to-segmentation feasibility claim further rests on Whisper/NLP reliability and a 12-case radiologist dictation sample. No new physical entities are postulated; the free parameters are training/inference hyperparameters selected by ablation or default.

free parameters (4)
  • LoRA rank r = 64
    Chosen by ablation among 32/64/128; r=64 yields best mAP with 4.96% trainable parameters and is used for all main results.
  • detection confidence threshold τ = 0.3
    Filters GDINO boxes before MedSAM; set to 0.3 at inference without reported sensitivity analysis in the main tables.
  • AdamW learning rate and schedule = 2e-4, 125 epochs, OneCycle
    2e-4 with OneCycle (10% warmup), 125 epochs, grad clip 5.0—standard but hand-chosen training hyperparameters that affect the reported LoRA performance.
  • curated tumor synonym dictionary and controlled class vocabulary = hand-curated
    Manually defined mapping (e.g., glioblastoma→glioma) and C={glioma,meningioma,pituitary,healthy} determine prompt extraction success; not learned from data in the paper.
axioms (5)
  • domain assumption Pretrained Whisper large-v3 transcription/translation is accurate enough for clinical entity extraction (cited German WER 5.7%).
    Invoked in §2.2 speech-to-prompt chain; one shown failure is an ASR misrecognition (“glioplaston”).
  • domain assumption Negation-aware spaCy/negspaCy plus global negative cues correctly decide tumor vs healthy prompts.
    §2.2 two-stage negation analysis; NLP accuracy cited from spaCy literature (92.6% entity extraction), not re-validated at scale on radiology speech.
  • domain assumption Predicted bounding boxes from LoRA-GDINO are sufficient prompts for frozen MedSAM to produce clinically useful masks without MedSAM fine-tuning.
    Core of the detection-driven design in §2.2 and Eqs. (1)–(3); Dice remains close to the fully fine-tuned detector baseline.
  • domain assumption BRISC train/val/test splits have negligible patient overlap and are representative enough for the reported in-distribution claims despite missing patient/scanner metadata.
    §2.1 states multi-source JPEG data and manual review to minimize overlap; residual leakage risk is unquantified.
  • domain assumption Low-rank LoRA updates in selected attention/FFN/cross-attention/text-encoder layers preserve pretrained cross-modal knowledge while adapting to MRI.
    Standard LoRA premise (Hu et al.) applied in §2.2; supported by ablations but not proven for all MRI distributions.
invented entities (2)
  • LoGSAM modular speech-to-segmentation pipeline no independent evidence
    purpose: Names the end-to-end composition of ASR, clinical NLP, LoRA-GDINO, and frozen MedSAM as a single framework.
    System label rather than a new physical or mathematical object; independent evidence is the empirical tables, not an external falsifiable entity.
  • structured tumor prompt mapping p = g(c, n) no independent evidence
    purpose: Converts extracted class and negation into the text string that conditions GDINO.
    Paper-specific interface between NLP and grounding; depends on the hand-built vocabulary and rules.

pith-pipeline@v1.1.0-grok45 · 12617 in / 4038 out tokens · 46125 ms · 2026-07-13T23:05:06.917704+00:00 · methodology

0 comments
read the original abstract

Precise localization and delineation of brain tumors using magnetic resonance imaging (MRI) are essential for planning therapy and guiding surgical decisions. To address this, we propose LoGSAM, a parameter-efficient, detection-driven framework that transforms radiologist dictation into text prompts for foundation-model-based localization and segmentation. Radiologist speech is first transcribed and translated using a pretrained Whisper ASR model, followed by negation-aware clinical NLP to extract tumor-specific textual prompts. These prompts guide text-conditioned tumor localization via a LoRA-adapted vision-language detection model, Grounding DINO (GDINO). The predicted bounding boxes are used as prompts for MedSAM to generate pixel-level tumor masks without any additional fine-tuning. On BRISC 2025, LoGSAM attains a Dice score of 80.32\%, reaching 98.6\% of a fully fine-tuned GDINO + MedSAM baseline while training fewer than 5\% of its parameters, indicating a favorable accuracy/parameter trade-off. In addition, we evaluate the full pipeline using German dictations from a board-certified radiologist on unseen MRI scans, achieving 91.7\% case-level class-extraction accuracy. These results highlight the feasibility of constructing a modular speech-to-segmentation pipeline from pretrained foundation models with minimal parameter updates.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

21 extracted references · 1 linked inside Pith

  1. [1]

    In: European Conference on Computer Vision (ECCV), pp

    Liu, S., Zeng, Z., Ren, T., et al.: Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection. In: European Conference on Computer Vision (ECCV), pp. 38–55. Springer, Heidelberg (2024)

    2024

  2. [2]

    In: Medical Image Computing and Computer-Assisted Intervention (MICCAI), pp

    Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional Networks for Biomed- ical Image Segmentation. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI), pp. 234–241 (2015)

    2015

  3. [3]

    W., Xu, T., Brockman, G., McLeavey, C., Sutskever, I.: Ro- bust Speech Recognition via Large-Scale Weak Supervision

    Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., Sutskever, I.: Ro- bust Speech Recognition via Large-Scale Weak Supervision. In: Int. Conference on Machine Learning (ICML), pp. 28492–28518. PMLR (2023)

    2023

  4. [4]

    Nature Communications,15(1), 654 (2024)

    Ma, J., He, Y., Li, F., Han, L., You, C., Wang, B.: Segment anything in medical images. Nature Communications,15(1), 654 (2024)

    2024

  5. [5]

    IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control (2025)

    Rasaee, H., Koleilat, T., Rivaz, H.: Grounding DINO-US-SAM: Text-Prompted Multi-Organ Segmentation in Ultrasound with LoRA-Tuned Vision-Language Mod- els. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control (2025)

    2025

  6. [6]

    J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models

    Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. In: International Conference on Learning Representations (ICLR),1(2), 3 (2022)

    2022

  7. [7]

    Acta Neuropathologica 131(6), 803–820 (2016)

    Louis,D.N.,Perry,A.,Reifenberger,G.,etal.:The2016WorldHealthOrganization Classification of Tumors of the Central Nervous System. Acta Neuropathologica 131(6), 803–820 (2016)

    2016

  8. [8]

    Physics in Medicine & Biology58(13), R97 (2013)

    Bauer, S., Wiest, R., Nolte, L.-P., Reyes, M.: A Survey of MRI-Based Medical Image Analysis for Brain Tumor Studies. Physics in Medicine & Biology58(13), R97 (2013)

    2013

  9. [9]

    Frontiers in Oncology15, 1643208 (2025)

    Han, W., Dong, X., Wang, G., Ding, Y., Yang, A.: Application and improvement of YOLO11 for brain tumor detection in medical images. Frontiers in Oncology15, 1643208 (2025)

    2025

  10. [10]

    arXiv preprint arXiv:1906.07155 (2019)

    Chen, K., Wang, J., Pang, J., Cao, Y., Xiong, Y., Li, X., Sun, S., Feng, W., Liu, Z., Xu, J., Zhang, Z.: MMDetection: Open mmlab detection toolbox and benchmark. arXiv preprint arXiv:1906.07155 (2019)

    Pith/arXiv arXiv 1906

  11. [11]

    Insights into Imaging14(1), 47 (2023)

    Jörg, T., Kämpgen, B., Feiler, D., Müller, L., Düber, C., Mildenberger, P., Jung- mann, F.: Efficient structured reporting in radiology using an intelligent dialogue system based on speech recognition and natural language processing. Insights into Imaging14(1), 47 (2023)

    2023

  12. [12]

    In: International Conference on Medical Imaging and Computer-Aided Diagnosis, pp

    Hu, J., Xu, X., Zou, Z.: LoRA-MedSAM: Efficient medical image segmentation. In: International Conference on Medical Imaging and Computer-Aided Diagnosis, pp. 154–164. Springer, Singapore (2024)

    2024

  13. [13]

    L., et al.: Radiotherapy Planning and Evaluation

    Chang, E. L., et al.: Radiotherapy Planning and Evaluation. Neurosurgery Clinics of North America18(1), 43–53 (2007)

    2007

  14. [14]

    D., Langlotz, C

    Zhang, Y., Jiang, H., Miura, Y., Manning, C. D., Langlotz, C. P.: Contrastive learning of medical visual representations from paired images and text. In: Machine Learning for Healthcare Conference, pp. 2–25. PMLR (2022)

    2022

  15. [15]

    In: Proceedings of the Conference on Empirical Methods in NLP (EMNLP), vol

    Wang, Z., Wu, Z., Agarwal, D., Sun, J.: MedCLIP: Contrastive learning from unpaired medical images and text. In: Proceedings of the Conference on Empirical Methods in NLP (EMNLP), vol. 2022, p. 3876 (2022)

    2022

  16. [16]

    K., Mahala, S., Kumar, Y., Shekhawat, N

    Bhamboo, A. K., Mahala, S., Kumar, Y., Shekhawat, N. S., Sharma, K.: Trust- Refined U-Net Ensemble for Uncertainty-Aware Brain Tumor Segmentation. Au- thorea Preprints (2025) 10 M.R.I. Bhuiyan et. al

    2025

  17. [17]

    C., Lo, W.-Y., Dollár, P.: Segment Anything

    Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., Dollár, P.: Segment Anything. In: Proceed- ings of the IEEE/CVF Int. Conference on Computer Vision (ICCV), pp. 4015–4026 (2023)

    2023

  18. [18]

    W., Bridewell, W., Hanbury, P., Cooper, G

    Chapman, W. W., Bridewell, W., Hanbury, P., Cooper, G. F., Buchanan, B. G.: A simple algorithm for identifying negated findings and diseases in discharge sum- maries. Journal of Biomedical Informatics34(5), 301–310 (2001)

    2001

  19. [19]

    Zenodo (2020)

    Honnibal, M., Montani, I., Van Landeghem, S., Boyd, A.: spaCy: Industrial- Strength Natural Language Processing in Python. Zenodo (2020)

    2020

  20. [20]

    Scientific Data (2026)

    Fateh,A.,Rezvani,Y.,Moayedi,S.,Rezvani,S.,Fateh,F.,Fateh,M.,Abolghasemi, V.: BRISC: Annotated dataset for brain tumor segmentation and classification. Scientific Data (2026)

    2026

  21. [21]

    MRI for Brain Tumor with Bounding Boxes (Kag- gle dataset),https://www.kaggle.com/datasets/ahmedsorour1/ mri-for-brain-tumor-with-bounding-boxes, last accessed 2026/02/07

    2026