Pith. sign in

REVIEW 4 major objections 5 minor 26 references

Uterine Ultrasound Image Captioning Using Deep Learning Techniques

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A hybrid CNN-BiGRU model that fuses features from InceptionV3 and DenseNet201 with text embeddings produces the best captions for uterine ultrasound images, outperforming UniLSTM, UniGRU, and BiLSTM baselines on BLEU and ROUGE metrics.

desk verdict The new uterine ultrasound dataset is a real asset, but the model as described cannot generate multi-word captions — the central result is not supported. read the letter →

arxiv 2411.14039 v1 pith:2EPVDDDY submitted 2024-11-21 cs.CV cs.AI

Abdennour Boulesnane , Boutheina Mokhtari , Oumnia Rana Segueni , Slimane Segueni This is my paper · ORCID
classification cs.CVcs.AI
keywords medicalimagecaptioninguterineultrasoundbidirectionalGRUInceptionV3DenseNet201BLEUscoreROUGEdeeplearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a hybrid deep-learning model can write short descriptive captions for uterine ultrasound images, a task harder than captioning ordinary photos because ultrasound frames are noisy, low-contrast, and full of artefacts. The authors assembled a private set of 505 uterine ultrasound images from their own scanner and two public collections, annotated them with expert descriptions, and compared four architectures. Their CNN-BiGRU model, which joins features from InceptionV3 and DenseNet201 with a bidirectional gated recurrent unit, reports the best scores on the held-out test split: BLEU-4 of 0.55 and ROUGE-L of 0.78, ahead of UniLSTM, UniGRU, and BiLSTM baselines. If the claim holds, automated captioning of uterine ultrasound could give clinicians and trainees a fast, consistent first-pass description of what a scan shows, supporting documentation and triage in obstetrics and gynaecology.

What carries the argument

The load-bearing mechanism is a two-stream feature-fusion module feeding a bidirectional GRU, a recurrent network that reads sequences both left-to-right and right-to-left. Image side: DenseNet201 features of shape (None, 1920) and InceptionV3 features of shape (None, 4800) each pass through a dense layer to 256 dimensions, are reshaped to (None, 1, 256), and concatenated with a 256-dimensional token embedding of the caption sequence of shape (None, 54, 256). The combined sequence goes through the BiGRU, producing one 256-dimensional vector; dropout at rate 0.5, a dense 128 layer, another dropout, and a final softmax over the 626-word vocabulary yield the predicted caption-word probabilities. The concatenation is what lets visual and textual information influence one another before the recurrent layer integrates temporal structure.

What would settle it

Run the model as specified on a held-out ultrasound image: if the forward pass returns a single probability vector over 626 words instead of a sequence of words, the reported BLEU and ROUGE values cannot be reproduced. Alternatively, add the missing word-by-word decoding loop and re-run the same 85/15 split; BLEU-4 must come out at 0.55 and ROUGE-L at 0.78 for the paper's central claim to stand.

Watch

Extended reading notes

Core claim

The paper's central claim is that feeding a uterine ultrasound image through two pretrained CNNs, InceptionV3 and DenseNet201, and combining their 256-dimensional feature vectors with embedded caption tokens inside a bidirectional GRU produces accurate, informative captions, and does so more reliably than unidirectional LSTM, unidirectional GRU, or bidirectional LSTM counterparts. On a held-out 15% of 505 annotated scans, the authors report BLEU-4 0.55 and ROUGE-L 0.78, with training loss 1.64 and validation loss 1.86 after early stopping at epoch 39. The paper presents this as evidence that bidirectional context is especially useful for medical captioning, because the relationships between image structures and word choices span the whole caption.

Load-bearing premise

The whole comparison depends on the model being able to output a full sentence word by word; the architecture as drawn ends in one softmax over the 626-word dictionary, and the paper never says how successive words are produced, so the claimed sentence-level scores rest on that unspecified decoding step.

Editorial extensions

If this is right

  • If the reported scores hold, the CNN-BiGRU recipe offers a workable path for captioning small, private ultrasound corpora using only pretrained CNNs plus a single BiGRU.
  • The reported BLEU-4 of 0.55 and ROUGE-L of 0.78 establish a quantitative benchmark that future uterine-ultrasound captioning systems can be compared against on the same 85/15 split.
  • Bidirectional sequence processing is the paper's stated explanation for the gain, implying that unidirectional recurrent decoders are a weaker choice for noisy medical-imaging captions.
  • The authors' proposed next steps, attention mechanisms, transformer-based decoders, larger multi-source datasets, and real-time clinical integration, are the direct follow-ons if the current claim is accepted.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's architecture section is underspecified on decoding: a single softmax over 626 vocabulary items is described, so a reader cannot tell from the text alone how a multi-word caption is produced; making that loop explicit is a prerequisite for reproducing the reported scores.
  • A testable extension would be to apply the same two-stream CNN + BiGRU recipe to a public fetal-ultrasound dataset with published reference numbers, which would reveal whether the 0.55/0.78 advantage transfers beyond the private 505-image set.
  • The ROI-cropping preprocessing, which detects intensity drops from the image center to isolate the ultrasound field, is independent of the captioning network and could be reused as a generic cleaning step for other low-quality ultrasound sources.
  • If the missing decoding loop turns out to be the only gap, replacing the final single-vector softmax with a per-timestep recurrent or attention decoder trained with teacher forcing is the natural correction, and the reported BLEU-4 and ROUGE-L figures are the targets such a corrected model should be re-measured against.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a CNN-BiGRU hybrid model for generating descriptive captions for uterine ultrasound images. It uses pre-trained InceptionV3 and DenseNet201 features concatenated with an embedding of the text sequence, processed by a bidirectional GRU, and claims state-of-the-art BLEU/ROUGE performance over UniLSTM, UniGRU, and BiLSTM baselines on a private dataset of 505 images. The paper includes data collection, preprocessing, feature extraction, model architecture, and comparative experiments.

Significance. If valid, the result would be of moderate practical significance: automated captioning of uterine ultrasound images could assist clinicians in documentation and diagnostic workflows. The paper also addresses a relatively underexplored domain (ultrasound image captioning) and contributes a private annotated dataset. However, the central empirical claim rests on a model that, as described in Section III-D, is a single-token classifier rather than a caption generator. The absence of a decoding mechanism, combined with unspecified inference-time text input and potential patient-level leakage in the data split, makes the reported BLEU/ROUGE scores uninterpretable. The paper does provide a clear comparison among four recurrent architectures and uses standard metrics, but those strengths are undermined by the architectural inconsistency.

major comments (4)
  1. [Section III-D, Figure 3] The architecture as written cannot generate a caption. The model takes tokenized text (None, 54), concatenates it with reshaped image features, feeds it to a BiGRU that 'produces an output shape of (None, 256)', and applies a final Dense(626) softmax to produce one probability vector over the vocabulary. This is a sequence-to-one classifier; without an autoregressive decoding loop that feeds predicted tokens back and stops at an <END> token, it cannot emit the multi-word captions reported in Table I and evaluated in Section IV. The paper does not specify any such loop, nor a time-distributed decoding layer, nor a masking/teacher-forcing scheme. This is a load-bearing internal inconsistency between the proposed model and the claimed results.
  2. [Section III-D, Section IV] The paper does not state what text input is provided to the model at inference time. Since the third input layer is the tokenized text sequence, the model cannot generate a caption without either (a) using the ground-truth reference caption as input, which would make the evaluation circular and not a captioning task, or (b) an autoregressive loop starting from <START>, which is not described. Without specifying this, the reported BLEU and ROUGE scores have no clear meaning.
  3. [Section III-A, Section IV] The random 85/15 split is likely to mix images from the same patient between training and test sets. The paper states that Fetal_Planes_DB provides four images per patient, and the Mendeley data may also contain multiple images per patient. A patient-independent split is required to avoid information leakage that inflates the reported scores. The authors should report the number of unique patients in each split and re-evaluate with a patient-level partition.
  4. [Section IV] The evaluation protocol is under-specified: the paper does not state whether BLEU and ROUGE are computed at the corpus or sentence level, how many references are used, how the test set size (15% of 505) is defined, or whether multiple runs with different seeds were averaged. The single-point scores (BLEU-4 0.55, ROUGE-L 0.78) are not accompanied by confidence intervals or significance tests, so the claimed superiority of BiGRU over the baselines is not statistically supported.
minor comments (5)
  1. [Section III (first paragraph)] The proposed model is inconsistently named: Section III refers to 'CNN-BiLGRU' while the rest of the paper uses 'CNN-BiGRU'. Please unify.
  2. [Figure 5a] The metric names are misspelled as 'BLUE1, BLUE2...' in the legend; they should read 'BLEU1, BLEU2, etc.'.
  3. [Section IV, Figure 5b] The label 'ROUGEL' should be 'ROUGE-L' for consistency with the text.
  4. [Section IV (first paragraph)] The sentence 'we divided the dataset, allocating 85% for training and 15% for testing (validation)' is ambiguous because early stopping is described using a validation loss, but no separate validation subset is defined anywhere in the experimental setup.
  5. [Section III-B] The ROI cropping threshold is described as a predefined value ('5% of peak value') without justification or sensitivity analysis; a brief rationale or ablation would strengthen the methodology.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the reported BLEU/ROUGE results are empirical evaluations on a held-out test split, and no claim reduces to its own inputs by construction.

full rationale

The paper makes no first-principles derivation; its central claim is an empirical comparison of captioning models on a private 505-image dataset with an 85/15 train/test split (Section IV). The BLEU and ROUGE scores in Figure 5 are computed on the held-out test portion against expert reference captions, so the reported performance is an external evaluation rather than a quantity forced by construction from the training inputs. No fitted parameter is renamed as a prediction: the model weights are trained on the training split and scored on unseen data, and the stated hyperparameters (Adam optimizer, batch size 16, early stopping patience 10) are not shown to be tuned to the test outputs. The only author self-citations ([21], [22]) support generic preprocessing statements (data quality; NLP text cleaning) and carry no load for the captioning claim; neither supplies a uniqueness theorem nor an ansatz that forecloses alternatives. The proposed architecture combines InceptionV3 and DenseNet201 features, whose pretraining is the external ImageNet benchmark [24], with a BiGRU, and is presented directly rather than imported from prior author work. The reviewer-flagged gap in Section III-D — a single 626-way softmax output with no described autoregressive decoding loop — is a correctness and reproducibility risk about whether multiword generation is actually implemented, not circularity: no exhibited equation or fitted value makes the reported BLEU/ROUGE numbers equal the inputs by construction. Therefore the circularity score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper rests entirely on empirical training and evaluation. It introduces no new mathematical framework and no new entities. The central claim depends on hand-chosen hyperparameters, a private dataset, and an unstated decoding assumption. The most serious axiom is that the described network can generate word sequences, which the architecture section does not support.

free parameters (5)
  • CNN/BiGRU trainable weights = not reported (millions)
    All network parameters are fit to the 505-image training set; no parameter-free or first-principles component underlies the captioning claim.
  • Hyperparameters (batch size, dropout, dense widths, early stopping) = 16, 0.5, 256/128, patience 10
    Chosen by hand without ablation; the reported BLEU/ROUGE scores depend on these choices.
  • ROI crop intensity threshold = 5% of peak intensity
    Ad hoc threshold used to define crop boundaries in preprocessing.
  • Max caption length = 54 tokens
    Set by the longest caption in the private dataset; affects padding and BiGRU input shape.
  • Train/test split ratio = 85/15
    Arbitrary split; with 505 images this gives 76 test images and no patient-level separation.
assumptions (5)
  • domain assumption Expert-provided captions are accurate ground truth for uterine ultrasound findings
    Annotation quality is asserted but no inter-annotator agreement or clinical validation is reported in Section III-A.
  • domain assumption ImageNet-pretrained InceptionV3 and DenseNet201 features transfer to grayscale ultrasound ROI crops
    The models are pretrained on natural images, not on ultrasound; no domain adaptation or fine-tuning analysis is presented in Section III-C.
  • domain assumption BLEU and ROUGE are appropriate measures of caption quality for this clinical task
    These n-gram overlap metrics do not capture clinical correctness, and no clinician evaluation is reported in Section IV.
  • ad hoc to paper Random 85/15 split keeps patients independent across train and test
    Fetal Planes DB has four images per patient; the paper does not state a patient-level split, so leakage is possible in Sections III-A and IV.
  • ad hoc to paper The described model with output shape (None,626) can generate multi-word captions
    No decoding loop is specified; as written, the model emits a single vocabulary-sized vector in Section III-D.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Uterine Ultrasound Image Captioning Using Deep Learning Techniques." pith.science (2026). https://pith.science/paper/2EPVDDDY

@misc{pith2026241114039,
  author       = {Pith},
  title        = {Pith review of: Uterine Ultrasound Image Captioning Using Deep Learning Techniques},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2EPVDDDY}},
  note         = {Machine review of arXiv:2411.14039}
}
read the original abstract

Medical imaging has significantly revolutionized medical diagnostics and treatment planning, progressing from early X-ray usage to sophisticated methods like MRIs, CT scans, and ultrasounds. This paper investigates the use of deep learning for medical image captioning, with a particular focus on uterine ultrasound images. These images are vital in obstetrics and gynecology for diagnosing and monitoring various conditions across different age groups. However, their interpretation is often challenging due to their complexity and variability. To address this, a deep learning-based medical image captioning system was developed, integrating Convolutional Neural Networks with a Bidirectional Gated Recurrent Unit network. This hybrid model processes both image and text features to generate descriptive captions for uterine ultrasound images. Our experimental results demonstrate the effectiveness of this approach over baseline methods, with the proposed model achieving superior performance in generating accurate and informative captions, as indicated by higher BLEU and ROUGE scores. By enhancing the interpretation of uterine ultrasound images, our research aims to assist medical professionals in making timely and accurate diagnoses, ultimately contributing to improved patient care.

Figures

Figures reproduced from arXiv: 2411.14039 by the authors.

Figure 1
Figure 1. Proportion of used uterine ultrasound images by source. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Original image to the region of interest. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Architecture of the proposed CNN-BiGRU model. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Loss curve of our proposed CNN-BiGRU model. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Comparative performance analysis of the proposed model and the baseline models. (a) Comparison of BLEU scores. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 24 canonical work pages

  1. [1]

    M. A. Haidekker, Medical Imaging Technology . Springer New York, 2013

    work page 2013

  2. [2]

    History of medical imaging,

    W. G. Bradley, “History of medical imaging,” Proceedings of the American Philosophical Society , vol. 152, no. 3, pp. 349–361, 2008

    work page 2008

  3. [3]

    Obuchowicz, M

    R. Obuchowicz, M. Strzelecki, and A. Pi ´orkowski, Artificial Intelligence in Medical Imaging and Image Processing . MDPI, May 2024

    work page 2024

  4. [4]

    Overview of deep learning in medical imaging,

    K. Suzuki, “Overview of deep learning in medical imaging,” Radiolog- ical Physics and Technology , vol. 10, no. 3, p. 257–273, Jul. 2017

    work page 2017

  5. [5]

    D. Kaul, H. Raju, and B. K. Tripathy, Deep Learning in Healthcare . Springer International Publishing, Aug. 2021, p. 97–115

    work page 2021

  6. [6]

    Automatic captioning for medical imaging (mic): a rapid review of literature,

    D.-R. Beddiar, M. Oussalah, and T. Sepp ¨anen, “Automatic captioning for medical imaging (mic): a rapid review of literature,” Artificial Intelligence Review, vol. 56, no. 5, p. 4019–4076, Sep. 2022

    work page 2022

  7. [7]

    Deep image captioning: A review of methods, trends and future challenges,

    L. Xu, Q. Tang, J. Lv, B. Zheng, X. Zeng, and W. Li, “Deep image captioning: A review of methods, trends and future challenges,” Neuro- computing, vol. 546, p. 126287, Aug. 2023

    work page 2023

  8. [8]

    Adaptive segmentation of ultrasound images,

    B. Levienaise-Obadia and A. Gee, “Adaptive segmentation of ultrasound images,” Image and Vision Computing , vol. 17, no. 8, p. 583–588, Jun. 1999

    work page 1999

Show all 26 references
  1. [9]

    H. Chen, Y . Zheng, J.-H. Park, P.-A. Heng, and S. K. Zhou, Iterative Multi-domain Regularized Deep Learning for Anatomical Structure Detection and Segmentation from Ultrasound Images . Springer In- ternational Publishing, 2016, p. 487–495

    2016

  2. [10]

    Understanding and generating ultrasound image description,

    X.-H. Zeng, B.-G. Liu, and M. Zhou, “Understanding and generating ultrasound image description,” Journal of Computer Science and Tech- nology, vol. 33, no. 5, p. 1086–1100, Sep. 2018

    2018

  3. [11]

    A survey of deep learning-based radiology report generation using multimodal data,

    X. Wang, G. Figueredo, R. Li, W. E. Zhang, W. Chen, and X. Chen, “A survey of deep learning-based radiology report generation using multimodal data,” 2024. [Online]. Available: https: //arxiv.org/abs/2405.12833

    2024 arXiv

  4. [12]

    Alsharid, H

    M. Alsharid, H. Sharma, L. Drukker, P. Chatelain, A. T. Papageorghiou, and J. A. Noble, Captioning Ultrasound Images Automatically. Springer International Publishing, 2019, p. 338–346

    2019

  5. [13]

    Deep learning for ultrasound image caption generation based on object detection,

    X. Zeng, L. Wen, B. Liu, and X. Qi, “Deep learning for ultrasound image caption generation based on object detection,” Neurocomputing, vol. 392, p. 132–141, Jun. 2020

    2020

  6. [14]

    Generating diagnostic report for medical image by high-middle-level visual information incorporation on double deep learning models,

    X. Zeng, L. Wen, Y . Xu, and C. Ji, “Generating diagnostic report for medical image by high-middle-level visual information incorporation on double deep learning models,” Computer Methods and Programs in Biomedicine, vol. 197, p. 105700, Dec. 2020

    2020

  7. [15]

    Automatic ultrasound image report generation with adaptive multimodal attention mechanism,

    S. Yang, J. Niu, J. Wu, Y . Wang, X. Liu, and Q. Li, “Automatic ultrasound image report generation with adaptive multimodal attention mechanism,” Neurocomputing, vol. 427, p. 40–49, Feb. 2021

    2021

  8. [16]

    Alsharid, H

    M. Alsharid, H. Sharma, L. Drukker, A. T. Papageorgiou, and J. A. Noble, Weakly Supervised Captioning of Ultrasound Images . Springer International Publishing, 2022, p. 187–198

    2022

  9. [17]

    Generating lymphoma ultrasound image description with transformer model,

    J. Deng, D. Chen, C. Zhang, and Y . Dong, “Generating lymphoma ultrasound image description with transformer model,” Computers in Biology and Medicine , vol. 174, p. 108409, May 2024

    2024

  10. [18]

    Ultrasound report generation with cross-modality feature alignment via unsupervised guidance,

    J. Li, T. Su, B. Zhao, F. Lv, Q. Wang, N. Navab, Y . Hu, and Z. Jiang, “Ultrasound report generation with cross-modality feature alignment via unsupervised guidance,” 2024

    2024

  11. [19]

    Uterine fibroid ultrasound images,

    T. Yang, “Uterine fibroid ultrasound images,” 2023

    2023

  12. [20]

    FE- TAL PLANES DB: Common maternal-fetal ultrasound images,

    X. P. Burgos-Artizzu, D. Coronado-Gutierrez, B. Valenzuela-Alcaraz, E. Bonet-Carne, E. Eixarch, F. Crispi, and E. Gratac ´os, “FE- TAL PLANES DB: Common maternal-fetal ultrasound images,” Jun. 2020

    2020

  13. [21]

    Influenza-like illness detection from arabic facebook posts based on sentiment analysis and 1d convolutional neural network,

    A. Boulesnane, S. Meshoul, and K. Aouissi, “Influenza-like illness detection from arabic facebook posts based on sentiment analysis and 1d convolutional neural network,” Mathematics, vol. 10, no. 21, p. 4089, Nov. 2022

    2022

  14. [22]

    Dzchatbot: A medical assistant chatbot in the algerian arabic dialect using seq2seq model,

    A. Boulesnane, Y . Saidi, O. Kamel, M. M. Bouhamed, and R. Mennour, “Dzchatbot: A medical assistant chatbot in the algerian arabic dialect using seq2seq model,” in 2022 4th International Conference on Pattern Analysis and Intelligent Systems (PAIS) . IEEE, Oct. 2022

    2022

  15. [23]

    Feature extraction: A survey of the types, techniques, applications,

    A. O. Salau and S. Jain, “Feature extraction: A survey of the types, techniques, applications,” in 2019 International Conference on Signal Processing and Communication (ICSC) . IEEE, Mar. 2019

    2019

  16. [24]

    Imagenet classification with deep convolutional neural networks,

    A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Communications of the ACM, vol. 60, no. 6, p. 84–90, May 2017

    2017

  17. [25]

    Bleu: a method for automatic evaluation of machine translation,

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in Proceedings of the 40th annual meeting of the Association for Computational Linguistics , 2002, pp. 311–318

    2002

  18. [26]

    Rouge: A package for automatic evaluation of summaries,

    C.-Y . Lin, “Rouge: A package for automatic evaluation of summaries,” in Text summarization branches out , 2004, pp. 74–81. 8 Abdennour Boulesnane received his B.S. degree from Mentouri Constantine University (UMC) and M.S. degree from the Department of Computer Science and Ap...

    2004

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.