RECORD
Unsupervised Contrastive Learning of Image Representations from Ultrasound Videos with Hard Negative Mining
Image and Video Processing · Submitted 26 Jul 2022
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim We show that the standard ResNet50 backbone trained with our framework improves the accuracy of models pretrained with SOTA UCL techniques as well as supervised pretrained models on ImageNet for the GB malignancy detection task by 2-6%.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Rich temporal information and variations in viewpoints make video data an attractive choice for learning image representations using unsupervised contrastive learning (UCL) techniques. State-of-the-art (SOTA) contrastive learning techniques consider frames within a video as positives in the embedding space, whereas the frames from other videos are considered negatives. We observe that unlike multiple views of an object in natural scene videos, an Ultrasound (US) video captures different 2D slices of an organ. Hence, there is almost no similarity between the temporally distant frames of even the same US video. In this paper we propose to instead utilize such frames as hard negatives. We advocate mining both intra-video and cross-video negatives in a hardness-sensitive negative mining curriculum in a UCL framework to learn rich image representations. We deploy our framework to learn the representations of Gallbladder (GB) malignancy from US videos. We also construct the first large-scale US video dataset containing 64 videos and 15,800 frames for learning GB representations. We show that the standard ResNet50 backbone trained with our framework improves the accuracy of models pretrained with SOTA UCL techniques as well as supervised pretrained models on ImageNet for the GB malignancy detection task by 2-6%. We further validate the generalizability of our method on a publicly available lung US image dataset of COVID-19 pathologies and show an improvement of 1.5% compared to SOTA. Source code, dataset, and models are available at https://gbc-iitd.github.io/usucl.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Rich temporal information and variations in viewpoints make video data an attractive choice for learning image representations using unsupervised contrastive learning (UCL) techniques. | background 1.00 | 0.08 | 0.02 | asserted 0.96 |
| s1 | State-of-the-art (SOTA) contrastive learning techniques consider frames within a video as positives in the embedding space, whereas the frames from other videos are considered negatives. | background 1.00 | 0.05 | 0.03 | asserted 1.00 |
| s2 | We observe that unlike multiple views of an object in natural scene videos, an Ultrasound (US) video captures different 2D slices of an organ. | background 0.40 | 0.58 | 0.03 | asserted 1.00 |
| s3 | Hence, there is almost no similarity between the temporally distant frames of even the same US video. | background 0.43 | 0.68 | 0.04 | asserted 0.96 |
| s4 | In this paper we propose to instead utilize such frames as hard negatives. | contribution 0.59 | 0.64 | 0.13 | asserted 0.94 |
| s5 | We advocate mining both intra-video and cross-video negatives in a hardness-sensitive negative mining curriculum in a UCL framework to learn rich image representations. | method 0.82 | 0.75 | 0.18 | asserted 0.93 |
| s6 | We deploy our framework to learn the representations of Gallbladder (GB) malignancy from US videos. | contribution 0.68 | 0.95 | 0.04 | asserted 0.97 |
| s7 | We also construct the first large-scale US video dataset containing 64 videos and 15,800 frames for learning GB representations. | contribution 0.90 | 0.97 | 0.05 | asserted 1.00 |
| s8 | We show that the standard ResNet50 backbone trained with our framework improves the accuracy of models pretrained with SOTA UCL techniques as well as supervised pretrained models on ImageNet for the GB malignancy detection task by 2-6%. | result 1.00 | 0.97 | 0.47 | asserted 1.00 |
| s9 | We further validate the generalizability of our method on a publicly available lung US image dataset of COVID-19 pathologies and show an improvement of 1.5% compared to SOTA. | result 1.00 | 0.96 | 0.12 | asserted 1.00 |
| s10 | Source code, dataset, and models are available at https://gbc-iitd.github.io/usucl. | contribution 0.90 | 0.48 | 0.02 | asserted 0.98 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Rich temporal information and variations in viewpoints make video data an attractive choice for learning image representations using unsupervised contrastive learning (UCL) techniques. State-of-the-art (SOTA) contrastive learning techniques consider frames within a video as positives in the embedding space, whereas the frames from other videos are considered negatives. We observe that unlike multiple views of an object in natural scene videos, an Ultrasound (US) video captures different 2D slices of an organ. Hence, there is almost no similarity between the temporally distant frames of even the same US video. In this paper we propose to instead utilize such frames as hard negatives. We advocate mining both intra-video and cross-video negatives in a hardness-sensitive negative mining curriculum in a UCL framework to learn rich image representations. We deploy our framework to learn the representations of Gallbladder (GB) malignancy from US videos. We also construct the first large-scale US video dataset containing 64 videos and 15,800 frames for learning GB representations. We show that the standard ResNet50 backbone trained with our framework improves the accuracy of models pretrained with SOTA UCL techniques as well as supervised pretrained models on ImageNet for the GB malignancy detection task by 2-6%. We further validate the generalizability of our method on a publicly available lung US image dataset of COVID-19 pathologies and show an improvement of 1.5% compared to SOTA. Source code, dataset, and models are available at https://gbc-iitd.github.io/usucl.
Human review
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites
Uses data or values from 2 papers
-
We use the publicly available POCUS [ref] dataset consisting of a total of 2116 lung US images, of which 655, 349, and 1112 images are of COVID-19, bacterial pneumonia, and healthy control, respectively.
POCOVID-Net: Automatic Detection of COVID-19 From a New Lung Ultrasound Imaging Dataset (POCUS) · Public Lung US Dataset for COVID-19 · 1.00
-
We also performed preliminary analysis on the performance of a ResNet50 classifier in detecting COVID-19 from a public CT dataset [ref] .
COVID-CT-Dataset: A CT Scan Dataset about COVID-19 · Experiments and Results · 0.84
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.