Pith. sign in

REVIEW 1 major objections 5 minor 49 references

AGA: An adaptive group alignment framework for structured medical cross-modal representation learning

T0 review · 1 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read AGA learns structured medical visual representations by grouping each report token with its most similar image patches and each patch with its most related tokens, and reports consistent gains over baselines on retrieval and classification.

desk verdict AGA is a sensible, genuinely new extension of fine-grained medical VLP, but the empirical case is weaker than the paper claims and needs variance reporting, code, and a more careful headline. read the letter →

arxiv 2507.23402 v1 pith:HB3TIOG2 submitted 2025-07-31 cs.CV cs.AIcs.LG

Wei Li , Xun Gong , Jiao Li , Xiaobin Sun This is my paper
classification cs.CVcs.AIcs.LG
keywords medicalvision-languagepretrainingcross-modalrepresentationlearninggroupedalignmentsparsesimilaritymatrixcontrastiveimage-textretrievalzero-shotclassificationradiologyreportsupervision
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AGA proposes a way to learn medical image representations from paired radiology reports by aligning at the level of semantic groups rather than whole images or single words. For each image-report pair, text tokens gather the image patches most similar to them into visual groups, and image patches gather the most related tokens into language groups; group membership is controlled by threshold gates that adapt during training. An alignment loss keeps each token or patch close to its own group and away from other groups inside the same pair, so no external negative pairs are needed. The paper reports that this scheme beats global and sentence-level contrastive baselines on image-to-text retrieval and on classification under both fine-tuning and zero-shot evaluation, on public chest X-ray data and on a private endoscopic ultrasound dataset.

What carries the argument

The load-bearing object is the sparsified cross-modal similarity matrix $S$ with row-wise min-max normalization and adaptive thresholds. Language-grouped and Vision-grouped Threshold Gates maintain momentum-updated running averages $\sigma_{tg}$ and $\sigma_{vg}$; entries below the threshold are zeroed, so each token keeps only its top patches and each patch keeps only its top tokens. Group embeddings are weighted averages of the surviving elements, the IGA loss aligns each element to its own group within the pair, and the BCGA module applies cross-attention between the two sets of group embeddings. This combination converts a single image-report pair into many internal alignment targets, which is what removes the dependence on external hard negatives.

What would settle it

Take a held-out set of reports whose findings mention rare or small lesions, and compare AGA's group assignments against a per-token oracle threshold chosen to maximize group purity. If the single running-average threshold leaves rare-finding tokens with near-empty or noisy groups while the per-token oracle gives clean ones, and AGA's retrieval gains disappear when groups are cleaned, then the reported gains are not caused by coherent semantic grouping.

Watch

Extended reading notes

Core claim

The paper's central claim is that structured, group-level alignment produces better medical visual representations than global or fragmented token-level alignment. Concretely, AGA builds a sparse similarity matrix between every report token and every image patch, uses momentum-updated threshold gates to decide which similarities count as group memberships, and forms a Token-Grouped Visual embedding for each token and a Patch-Grouped Language embedding for each patch as similarity-weighted averages of their group members. The Instance-aware Group Alignment loss then maximizes agreement between each token or patch and its own group within the same image-report pair, and the Bidirectional Cross-modal Grouped Alignment module aligns group representations across modalities. If the reported experiments hold, the framework gives consistent gains in retrieval precision and classification accuracy, including zero-shot, and does so without requiring large hard-negative batches.

Load-bearing premise

The method assumes that one scalar threshold per modality, maintained as a momentum running average over all pairs in the batch, yields semantically coherent token-to-patch and patch-to-token groups for every report, including rare findings and long multi-sentence descriptions.

Editorial extensions

If this is right

  • Training on paired medical data no longer needs large batches of hard negatives, since the alignment loss is computed within each image-report pair; this makes pretraining feasible on small clinical datasets.
  • Rare findings that are described across multiple sentences can be captured as a group of patches rather than as one word-to-one-patch correspondence, giving the model a structured representation of a single abnormality.
  • Adaptive thresholds give the model a way to match report styles: diffuse, multi-sentence chest X-ray descriptions converge to different threshold values than focused endoscopic ultrasound texts.
  • If the reported results transfer, downstream tasks with very little labeled data, such as 1% or 10% fine-tuning, and zero-shot classification should inherit better structured representations without extra annotation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the same grouping mechanism could be applied to other weakly paired modalities, such as pathology images with free-text reports or video with narration, where sample sizes are small and hard negatives are scarce; the paper only tests medical images and reports.
  • Beyond the paper: the reported gains should be largest on reports with multiple independent findings, because that is exactly the case where sentence-level and token-level alignment mix distinct entities; a dataset stratified by number of findings would test this prediction.
  • Beyond the paper: the single momentum threshold per modality could be replaced by per-token adaptive thresholds or learned gates conditioned on token frequency, and the paper's threshold curves suggest the optimal threshold varies with dataset style.
  • Beyond the paper: since the paper does not evaluate text-side tasks, a natural next test is whether the group representations also improve report generation or medical visual question answering.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 5 minor

Summary. The paper proposes Adaptive Grouped Alignment (AGA), a vision-language pretraining framework for medical images and reports. For each image-report pair, AGA computes a token-patch similarity matrix, sparsifies it with momentum-updated thresholds, and forms token-grouped visual (TGV) and patch-grouped language (PGL) embeddings as similarity-weighted averages. It then trains with three objectives: a global contrastive loss, an instance-aware group alignment (IGA) loss that aligns tokens with TGV embeddings and patches with PGL embeddings within each pair, and a bidirectional cross-modal grouped alignment (BCGA) loss that aligns TGV and PGL embeddings through cross-attention. The authors pretrain on MIMIC-CXR and a private endoscopic ultrasound dataset (SMTs) and evaluate on image-to-text retrieval, supervised classification, and zero-shot classification on CheXpert 5x200, RSNA Pneumonia, and private SMTs collections. The paper claims that AGA consistently outperforms ConVIRT, GLoRIA, MGCA, and SPARC across these tasks.

Significance. The grouping-based alignment idea is coherent and potentially useful: constructing cross-modal groups from sparse similarity matrices is a reasonable way to capture structured semantics without relying on large hard-negative batches for the auxiliary group losses. The method is clearly specified, and the ablation study in Tables 7-8 gives some evidence that the group alignment and BCGA components matter. If the claims of consistent superiority were supported, this would be a practical contribution to medical vision-language pretraining, especially for small private datasets. However, the current evaluation does not establish that superiority: several reported numbers are worse than baselines, no variance or significance is reported, and the retrieval metric is category-based rather than exact report matching. The stated contribution of 'eliminating the need for external negatives' is also overstated because the global loss still uses batch negatives and the IGA denominators contain within-pair negatives.

major comments (1)
  1. [Section 3.3, Eqs. 5-7] The description of 'learnable Threshold Gates' is not supported by the equations. In Eqs. (5) and (7), sigma_tg and sigma_vg are updated by an exponential moving average with fixed momentum coefficients gamma_tg and gamma_vg; there are no learnable parameters in these modules. Additionally, the symbol S in Eq. (5) is not precisely defined: it is called a 'running average of the similarity matrix', but the paper does not specify over which samples or with what initialization this average is computed. Please clarify the mechanism and avoid calling the thresholds 'learnable' if they contain no parameters that receive gradients.
minor comments (5)
  1. [Section 3.2, Eq. (3)] There is a typographical error in the definition of alpha_jk: the numerator should be e_{s_jk} (or a clearly indexed version), but the text currently writes 'esjr' with mismatched indices. Please correct the notation.
  2. [Section 3.4, Eq. (8)] In Eq. (8), the index j is used both for the positive pair and as the summation index in the denominator, which is confusing. Use separate indices for the positive term and the summation over the batch.
  3. [Figure 2 and Figure 6 captions] The captions contain stray punctuation: 'Figure 2: . Overview' and 'Figure 6: . T-SNE visualizations' should be cleaned up.
  4. [Section 4.1.1] The numbers in parentheses for MIMIC-CXR, e.g., '270,742 (152142)', are difficult to parse. Please format with commas and provide a sentence that clearly explains what the parenthetical numbers count (e.g., unique studies versus image-report pairs).
  5. [Section 3.1] The notation Vi,l = (vi,1, ..., vi,N) with 'Vi,N in R^d' is inconsistent: the subscript N on a vector suggests the N-th vector, while the text means each patch embedding has dimension d. Please use v_{i,n} for the n-th patch embedding and clarify the dimension.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: AGA's losses are architecture-defined and evaluated against external baselines.

full rationale

The central derivation is self-contained. The proposed losses (global contrastive Eq. 8, IGA Eqs. 9-10, BCGA Eqs. 11-13, total Eq. 14) are defined from the proposed architecture and empirically benchmarked against ConVIRT, GLoRIA, MGCA, and SPARC, which are external baselines reproduced on the same data. The adaptive threshold gate (Eqs. 5-7) is a training component, not a predicted output: using a momentum-averaged similarity statistic to sparsify the same similarity matrix is a standard adaptive sparsification mechanism and does not make the downstream claim follow by construction. The IGA loss aligns each token to a group representation that is a similarity-weighted average of patches selected by that token; this is a training objective, and any self-reference in the grouping is validated only through downstream retrieval/classification on held-out benchmarks, not presented as a derived prediction. The claim that IGA 'eliminates the need for external negatives' is an overstatement because denominators in Eqs. 9-10 contain within-pair negatives, but this is a precision-of-claim issue rather than circularity. The paper contains one self-citation (Fan et al. 2025, sharing authors Xun Gong and Jiao Li), used only as a related-work example; it is not load-bearing. The 'consistently outperforms' wording is undercut by some of the paper's own tables (e.g., MGCA P@100 44.58 vs AGA 43.16 in Table 2; ConVIRT ROC 53.0 vs AGA 52.4 in Table 5) and by the absence of variance reporting, but that is a correctness/evidentiary concern, not circular logic. No load-bearing step reduces to its own inputs by definition or by self-citation.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim rests on several hand-set hyperparameters and a domain assumption about structured medical reports. The global threshold assumption (Eq. 5) is the most fragile because a single scalar must serve all tokens in all reports. The TGV/PGL representations are derived quantities, not invented entities.

free parameters (4)
  • Momentum hyperparameters γ_tg, γ_vg = 0.99 (SMTs), 0.999 (MIMIC-CXR)
    Control how quickly the grouping threshold adapts; set per dataset in Section 4.2.1, no sensitivity analysis.
  • Temperatures τ1, τ2, τ3 = 0.3, 0.3, 0.1
    Set by hand; affect sharpness of all contrastive losses, Section 4.2.1.
  • Loss weights λ1, λ2, λ3 = 0.5 each
    Default weighting of global, IGA, and BCGA losses; no ablation for these.
  • Grouping thresholds σ_tg, σ_vg = learned via momentum (initial values not reported)
    Central to grouping; updated as running averages of similarities, Eqs. 5-7; effectively a learned scalar but the initialization is unspecified.
assumptions (3)
  • standard math Inner-product similarity and min-max normalization produce a meaningful cross-modal affinity matrix.
    Eqs. 1-2 assume cosine-like inner products between text tokens and image patches are comparable; standard in cross-modal alignment.
  • domain assumption Clinical reports contain structured semantic units (findings, locations, attributes) that can be captured by grouping tokens with patches.
    Motivates the whole approach in Section 1; if reports are not structured this way, grouping adds little.
  • ad hoc to paper A single global threshold (momentum-updated average similarity) yields coherent groups for all tokens and patches.
    Eqs. 5-7; the paper provides a threshold curve but no per-token quality analysis; this is the weakest premise.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AGA: An adaptive group alignment framework for structured medical cross-modal representation learning." pith.science (2026). https://pith.science/paper/HB3TIOG2

@misc{pith2026250723402,
  author       = {Pith},
  title        = {Pith review of: AGA: An adaptive group alignment framework for structured medical cross-modal representation learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HB3TIOG2}},
  note         = {Machine review of arXiv:2507.23402}
}
read the original abstract

Learning medical visual representations from paired images and reports is a promising direction in representation learning. However, current vision-language pretraining methods in the medical domain often simplify clinical reports into single entities or fragmented tokens, ignoring their inherent structure. In addition, contrastive learning frameworks typically depend on large quantities of hard negative samples, which is impractical for small-scale medical datasets. To tackle these challenges, we propose Adaptive Grouped Alignment (AGA), a new framework that captures structured semantics from paired medical images and reports. AGA introduces a bidirectional grouping mechanism based on a sparse similarity matrix. For each image-report pair, we compute fine-grained similarities between text tokens and image patches. Each token selects its top-matching patches to form a visual group, and each patch selects its most related tokens to form a language group. To enable adaptive grouping, we design two threshold gating modules, called Language Grouped Threshold Gate and Vision Grouped Threshold Gate, which learn grouping thresholds dynamically. Group representations are computed as weighted averages based on similarity scores. To align each token with its group representation, we introduce an Instance Aware Group Alignment loss that operates within each image-text pair, removing the need for external negatives. Finally, a Bidirectional Cross-modal Grouped Alignment module is applied to enhance fine-grained alignment between visual and linguistic group representations. Extensive experiments on public and private datasets show that our method achieves strong performance on image-text retrieval and classification tasks under both fine-tuning and zero-shot settings.

Figures

Figures reproduced from arXiv: 2507.23402 by the authors.

Figure 1
Figure 1. Fine-grained alignment. (a) illustrates the conventional word-to-patch alignment approach. (b) shows our [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed adaptive group alignment (AGA) framework. Dynamic grouping is achieved [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Line plot of language grouping threshold [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Line plot of language grouping threshold [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Visualization of learned token correspondence by our AGA. Highlighted pixels represent higher activation [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: . T-SNE visualizations of encoded image representations. Colors represent the ground truth disease types. [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

49 extracted references · 43 canonical work pages

  1. [1]

    Joint modeling of chest radiographs and radiology reports for pulmonary edema assessment

    Geeticka Chauhan, Ruizhi Liao, William Wells, Jacob Andreas, Xin Wang, Seth Berkowitz, Steven Horng, Peter Szolovits, and Polina Golland. Joint modeling of chest radiographs and radiology reports for pulmonary edema assessment. Medical Image Computing and Computer Assisted Intervention-MICCAI 2020: 23rd International Conference, Lima, Peru, October 4-8, 2...

    work page 2020

  2. [2]

    Tandemnet: Distilling knowledge from medical images using diagnostic reports as optional semantic references

    Zizhao Zhang, Pingjun Chen, Manish Sapkota, and Lin Yang. Tandemnet: Distilling knowledge from medical images using diagnostic reports as optional semantic references. Medical Image Computing and Computer Assisted Intervention-MICCAI 2017: 20th International Conference, Quebec City, QC, Canada, September 11-13, 2017, Proceedings, Part III 20. Springer Int...

    work page 2017

  3. [3]

    Contrastive learning of medical visual representations from paired images and text

    Yuhao Zhang, Hang Jiang, Yasuhide Miura, Christopher D Manning, and Curtis P Langlotz. Contrastive learning of medical visual representations from paired images and text. Machine learning for healthcare conference. PMLR, pages 2--25, 2022 a

    work page 2022

  4. [4]

    Learning to exploit temporal structure for biomedical vision-language processing

    Shruthi Bannur, Stephanie Hyland, Qianchu Liu, Fernando Pérez-García, Maximilian Ilse, Daniel C Castro, Benedikt Boecking, Harshita Sharma, Kenza Bouzid, Anja Thieme, Anton Schwaighofer, Maria Wetscherek, Matthew P Lungren, Aditya Nori, Javier Alvarez-Valle, and Ozan Oktay. Learning to exploit temporal structure for biomedical vision-language processing. ...

    work page 2023

  5. [5]

    Gloria: A multimodal global-local representation learning framework for label-efficient medical image recognition

    Shih-Cheng Huang, Liyue Shen, Matthew P Lungren, and Serena Yeung. Gloria: A multimodal global-local representation learning framework for label-efficient medical image recognition. Proceedings of the IEEE/CVF international conference on computer vision, pages 3942--3951, 2021

    work page 2021

  6. [6]

    Multi-granularity cross-modal alignment for generalized medical visual representation learning

    Fuying Wang, Yuyin Zhou, Shujun Wang, Varut Vardhanabhuti, and Lequan Yu. Multi-granularity cross-modal alignment for generalized medical visual representation learning. Advances in Neural Information Processing Systems, 35: 0 33536--33549, 2022

    work page 2022

  7. [7]

    Abnormal-region-aware multi-modal feature fusion for medical report generation

    Yan Gao, Zhiwei Ni, Wentao Liu, Liping Ni, Ling Xin, Linbo Hu, and Li Zhang. Abnormal-region-aware multi-modal feature fusion for medical report generation. Knowledge-Based Systems, 318: 0 113538, 2025

    work page 2025

  8. [8]

    Prior: Prototype representation joint learning from medical images and reports

    Pujin Cheng, Li Lin, Junyan Lyu, Yijin Huang, Wenhan Luo, and Xiaoying Tang. Prior: Prototype representation joint learning from medical images and reports. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 21361--21371, 2023

    work page 2023

Show all 49 references
  1. [9]

    Multi-task paired masking with alignment modeling for medical vision-language pre-training

    Ke Zhang, Yan Yang, Jun Yu, Hanliang Jiang, Jianping Fan, and Qingming Huang. Multi-task paired masking with alignment modeling for medical vision-language pre-training. IEEE Transactions on Multimedia, 26: 0 4706--4721, 2023 a

    2023

  2. [10]

    Hard negative mixing for contrastive learning

    Yannis Kalantidis, Mert Bulent Sariyildiz, Noe Pion, Philippe Weinzaepfel, and Diane Larlus. Hard negative mixing for contrastive learning. Advances in neural information processing systems, 33: 0 21798--21809, 2020

    2020

  3. [11]

    Learning deep representations by mutual information estimation and maximization

    R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representations by mutual information estimation and maximization. arXiv preprint arXiv:1808.06670, 2018

    2018 arXiv

  4. [12]

    Reliable few-shot learning under dual noises

    Ji Zhang, Jingkuan Song, Lianli Gao, Nicu Sebe, and Hengtao Shen. Reliable few-shot learning under dual noises. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1--17, 2025

    2025

  5. [13]

    Progressive meta-learning with curriculum

    Ji Zhang, Jingkuan Song, Lianli Gao, Ye Liu, and Hengtao Shen. Progressive meta-learning with curriculum. IEEE Transactions on Circuits and Systems for Video Technology, 32 0 (9): 0 5916--5930, 2022 b

    2022

  6. [14]

    Prototype contrastive consistency learning for semi-supervised medical image segmentation

    Shihuan He, Zhihui Lai, Ruxin Wang, and Heng Kong. Prototype contrastive consistency learning for semi-supervised medical image segmentation. arXiv preprint arXiv:2502.06650, 2025

    2025 arXiv

  7. [15]

    Prototype correlation matching and class-relation reasoning for few-shot medical image segmentation

    Yumin Zhang, Hongliu Li, Yajun Gao, Haoran Duan, Yawen Huang, and Yefeng Zheng. Prototype correlation matching and class-relation reasoning for few-shot medical image segmentation. IEEE Transactions on Medical Imaging, 43 0 (11): 0 4041--4054, 2024

    2024

  8. [16]

    Diagllm: multimodal reasoning with large language model for explainable bearing fault diagnosis

    Jie Wang, Tianrui Li, Yan Yang, Shiqian Chen, and Wanming Zhai. Diagllm: multimodal reasoning with large language model for explainable bearing fault diagnosis. Science China Information Sciences, 68 0 (6): 0 160103, 2025 a

    2025

  9. [17]

    Improving fine-grained understanding in image-text pre-training

    Ioana Bica, Anastasija Ilić, Matthias Bauer, Goker Erdogan, Matko Bošnjak, Christos Kaplanis, Alexey A Gritsenko, Matthias Minderer, Charles Blundell, Razvan Pascanu, and Jovana Mitrović. Improving fine-grained understanding in image-text pre-training. arXiv preprint arXiv:240...

    2024 arXiv

  10. [18]

    Amam: An attention-based multimodal alignment model for medical visual question answering

    Haiwei Pan, Shuning He, Kejia Zhang, Bo Qu, Chunling Chen, and Kun Shi. Amam: An attention-based multimodal alignment model for medical visual question answering. Knowledge-Based Systems, 255: 0 109763, 2022

    2022

  11. [19]

    Context-enhanced framework for medical image report generation using multimodal contexts

    Hongzhao Li, Hongyu Wang, Xia Sun, Hua He, and Jun Feng. Context-enhanced framework for medical image report generation using multimodal contexts. Knowledge-Based Systems, 310: 0 112913, 2025

    2025

  12. [20]

    Cycle-vqa: A cycle-consistent framework for robust medical visual question answering

    Lin Fan, Xun Gong, Cenyang Zheng, Xuli Tan, Jiao Li, and Yafei Ou. Cycle-vqa: A cycle-consistent framework for robust medical visual question answering. Pattern Recognition, 165: 0 111609, 2025

    2025

  13. [21]

    From global to local: Multi-scale out-of-distribution detection

    Ji Zhang, Lianli Gao, Bingguang Hao, Hao Huang, Jingkuan Song, and Hengtao Shen. From global to local: Multi-scale out-of-distribution detection. IEEE Transactions on Image Processing, 32: 0 6115--6128, 2023 b

    2023

  14. [22]

    Multimodal representation learning via maximization of local mutual information

    Ruizhi Liao, Daniel Moyer, Miriam Cha, Keegan Quigley, Seth Berkowitz, Steven Horng, Polina Golland, and William M Wells. Multimodal representation learning via maximization of local mutual information. Medical Image Computing and Computer Assisted Intervention-MICCAI 2021: 24...

    2021

  15. [23]

    Breaking with fixed set pathology recognition through report-guided contrastive training

    Constantin Seibold, Simon Reiß, M Saquib Sarfraz, Rainer Stiefelhagen, and Jens Kleesiek. Breaking with fixed set pathology recognition through report-guided contrastive training. International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Spr...

    2022

  16. [24]

    Tier: Text-image entropy regularization for medical clip-style models

    Anil Palepu and Andrew Beam. Tier: Text-image entropy regularization for medical clip-style models. Machine Learning for Healthcare Conference. PMLR, 219: 0 548--564, 2023

    2023

  17. [25]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. in international conference on machine learning. International conference on machine learning. PMLR, 119: 0 1597--1607, 2020

    2020

  18. [26]

    Momentum contrast for unsupervised visual representation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729--9738, 2020

    2020

  19. [27]

    Parts2whole: Self-supervised contrastive learning via reconstruction

    Ruibin Feng, Zongwei Zhou, Michael B Gotway, and Jianming Liang. Parts2whole: Self-supervised contrastive learning via reconstruction. Domain Adaptation and Representation Transfer, and Distributed and Collaborative Learning: Second MICCAI Workshop, DART 2020, and First MICCAI...

    2020

  20. [28]

    Bootstrap your own latent-a new approach to self-supervised learning

    Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, Bilal Piot, koray kavukcuoglu, Remi Munos, and Michal Valko. Bootstrap your own latent-a new appr...

    2020

  21. [29]

    Contrastive learning of global and local features for medical image segmentation with limited annotations

    Krishna Chaitanya, Ertunc Erdil, Neerav Karani, and Ender Konukoglu. Contrastive learning of global and local features for medical image segmentation with limited annotations. Advances in neural information processing systems, 33: 0 12546--12558, 2020

    2020

  22. [30]

    Pneumonia detection on chest x-ray using radiomic features and contrastive learning

    Yan Han, Chongyan Chen, Ahmed Tewfik, Ying Ding, and Yifan Peng. Pneumonia detection on chest x-ray using radiomic features and contrastive learning. 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pages 247--251, 2021

    2021

  23. [31]

    Contig: Self-supervised multimodal contrastive learning for medical imaging with genetics

    Aiham Taleb, Matthias Kirchler, Remo Monti, and Christoph Lippert. Contig: Self-supervised multimodal contrastive learning for medical imaging with genetics. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 20908--20921, 2022

    2022

  24. [32]

    Lspcl: Label-specific supervised prototype contrastive learning for multi-label text classification

    Gang Wang, Yajun Du, and Yurui Jiang. Lspcl: Label-specific supervised prototype contrastive learning for multi-label text classification. Knowledge-Based Systems, 309: 0 112887, 2025 b

    2025

  25. [33]

    Unsupervised feature learning by cross-level instance-group discrimination

    Xudong Wang, Ziwei Liu, and Stella X Yu. Unsupervised feature learning by cross-level instance-group discrimination. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12586--12595, 2021

    2021

  26. [34]

    Hcsc: Hierarchical contrastive selective coding

    Yuanfan Guo, Minghao Xu, Jiawen Li, Bingbing Ni, Xuanyu Zhu, Zhenbang Sun, and Yi Xu. Hcsc: Hierarchical contrastive selective coding. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9706--9715, 2022

    2022

  27. [35]

    Contrastive clustering

    Yunfan Li, Peng Hu, Zitao Liu, Dezhong Peng, Joey Tianyi Zhou, and Xi Peng. Contrastive clustering. Proceedings of the AAAI conference on artificial intelligence, 35 0 (10): 0 8547--8555, 2021

    2021

  28. [36]

    Unsupervised learning of visual features by contrasting cluster assignments

    Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. Advances in neural information processing systems, 33: 0 9912--9924, 2020

    2020

  29. [37]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770--778, 2016

    2016

  30. [38]

    Imagenet large scale visual recognition challenge

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C Berg, and Feifei Li. Imagenet large scale visual recognition challenge. International journal of computer vision, 115: ...

    2015

  31. [39]

    Bert: pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language...

    2019

  32. [40]

    Scaling up visual and vision-language representation learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. International conference on machine learning. PMLR, 139: 0 4904--...

    2021

  33. [41]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. International confer...

    2021

  34. [42]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

  35. [43]

    Representation learning with contrastive predictive coding

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018

    2018 arXiv

  36. [44]

    W Johnson, Tom J Pollard, Seth J Berkowitz, Nathaniel R Greenbaum, Matthew P Lungren, Chih-ying Deng, Roger G Mark, and Steven Horng

    Alistair E. W Johnson, Tom J Pollard, Seth J Berkowitz, Nathaniel R Greenbaum, Matthew P Lungren, Chih-ying Deng, Roger G Mark, and Steven Horng. Mimic-cxr, a de-identified publicly available database of chest radiographs with free-text reports. Scientific data, 6 0 (1): 0 317, 2019

    2019

  37. [45]

    Dynamic graph enhanced contrastive learning for chest x-ray report generation

    Mingjie Li, Bingqian Lin, Zicong Chen, Haokun Lin, Xiaodan Liang, and Xiaojun Chang. Dynamic graph enhanced contrastive learning for chest x-ray report generation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3334--3343, 2023

    2023

  38. [46]

    Interactive and explainable regionguided radiology report generation

    Tim Tanida, Philip Müller, Georgios Kaissis, and Daniel Rueckert. Interactive and explainable regionguided radiology report generation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7433--7442, 2023

    2023

  39. [47]

    Knowledge matters: chest radiology report generation with general and specific knowledge

    Shuxin Yang, Xian Wu, Shen Ge, S Kevin Zhou, and Li Xiao. Knowledge matters: chest radiology report generation with general and specific knowledge. Medical image analysis, 80: 0 102510, 2022

    2022

  40. [48]

    Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison

    Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-Ilcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, Jayne Seekins, David A Mong, Safwan S Halabi, Jesse K Sandberg, Ricky Jones, David B Larson, Curtis P Langlotz, Bhavik N Pat...

    2019

  41. [49]

    Augmenting the national institutes of health chest radiograph dataset with expert annotations of possible pneumonia

    George Shih, Carol C Wu, Safwan S Halabi, Marc D Kohli, Luciano M Prevedello, Tessa S Cook, Arjun Sharma, Judith K Amorosa, Veronica Arteaga, Maya Galperin-Aizenberg, Ritu R Gill, Myrna C.B Godoy, Stephen Hobbs, Jean Jeudy, Archana Laroia, Palmi N Shah, Dharshan Vummidi, Kavit...

    2019

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.