Maximal Matching Matters: Preventing Representation Collapse for Robust Cross-Modal Retrieval

Andrew Zhang; Anushka Sivakumar; Chris Thomas; Hani Alomari

REVIEW 4 major objections 5 minor 1 cited by

Maximal Matching Matters: Preventing Representation Collapse for Robust Cross-Modal Retrieval

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that representing each image and caption as a set of embeddings, scored by an optimal one-to-one assignment between the sets rather than a max or average, prevents representation collapse and yields state-of-the-art…

desk verdict A natural set-based retrieval idea with strong reported numbers, but the appendix pseudocode does not implement the core matching step, so the results are not yet reproducible. read the letter →

arxiv 2506.21538 v1 pith:CK47PLWL submitted 2025-06-26 cs.CV cs.IRcs.LG

Hani Alomari , Anushka Sivakumar , Andrew Zhang , Chris Thomas This is my paper · ORCID

classification cs.CVcs.IRcs.LG

keywords cross-modalretrievalset-basedembeddingsrepresentationcollapsemaximalpairassignmentHungarianalgorithmimage-textdiversitylosssemantic

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that set-based image-text embeddings, where each sample is represented by several vectors instead of one, have been held back by two training failures: sparse supervision, where only one embedding in a set receives gradient signal, and set collapse, where all embeddings in a set converge to the same point. It proposes Maximal Pair Assignment Similarity, which computes similarity between two sets as the optimal one-to-one matching of their elements, so every embedding participates in the score. Two additional losses, Global Discriminative and Intra-Set Divergence, push embeddings away from the set's global summary and from each other. If correct, MaxMatch would make set-based retrieval both more accurate and more semantically diverse, with state-of-the-art results on MS-COCO and Flickr30k obtained without external data or cross-attention.

What carries the argument

The load-bearing object is the Maximal Pair Assignment Similarity score: given two sets $S_1=\{x_1,\dots,x_K\}$ and $S_2=\{y_1,\dots,y_K\}$ with cosine similarity matrix $S_{mn}(S_1,S_2)=x_m\cdot y_n/(\|x_m\|\|y_n\|)$, the Hungarian algorithm finds the permutation $\pi^* = \arg\max_\pi \operatorname{tr}(S(S_1,\pi(S_2)))$, and the matching mask selects those one-to-one pairs whose similarities are then exponentially scaled and averaged. This replaces both max-pooling (MIL) and mean-pooling (smooth-Chamfer) and is what distributes gradient signal across all embeddings while penalizing collapse. It is supported by the Global Discriminative Loss, which pushes each set element away from the sample's global embedding vector, and the Intra-Set Divergence Loss, which penalizes pairwise cosine similarity within a set.

What would settle it

Train the model using the argmax-based masking in the paper's Listing 3 in place of the described Hungarian optimal assignment; if the COCO 5K RSUM does not fall below the reported 446.53, or if per-slot retrieval diversity does not exceed SetDiv's, then the central claim that optimal one-to-one matching is what prevents collapse would be falsified.

Watch

Extended reading notes

Core claim

The central claim is that the choice of set-level similarity function determines whether multi-embedding representations degenerate. Multiple Instance Learning similarity, which takes the maximum over all pairs, leaves most embeddings undertrained, while smooth-Chamfer similarity averages all pairs and mathematically encourages all embeddings in a set to become identical, because the objective is convex in each embedding and minimized at the mean. Maximal Pair Assignment Similarity instead solves the assignment problem with the Hungarian algorithm on the cosine similarity matrix, selecting the permutation that maximizes total one-to-one similarity between the K image embeddings and K text embeddings. With every embedding forced to participate in the matching, training keeps sets diverse; the two new losses reinforce this by penalizing closeness to the global embedding and closeness among set members. The paper reports that this combination prevents degenerate embedding sets and achieves state-of-the-art Recall@K and RSUM scores on MS-COCO and Flickr30k.

Load-bearing premise

The reported results assume a correct implementation of the optimal one-to-one assignment step, yet the pseudocode in Section 3.3 and Appendix H does not actually execute the Hungarian assignment it describes, instead selecting similarities with an argmax over generated index permutations, so the experimental gains are tied to an unstated implementation.

Editorial extensions

If this is right

Set-based image-text retrieval can be trained without choosing between unused embeddings and collapsed embeddings: MaxMatch keeps all K elements active and distinct, as shown by circular-variance and per-embedding retrieval analyses.
The method reaches state-of-the-art Recall@K and RSUM on MS-COCO (1K and 5K) and Flickr30k across ResNet152, Faster R-CNN, and BERT configurations, without external data or cross-attention at inference.
MaxMatch transfers across architectures: applying it to the PVSE architecture improves its RSUM by 18 points on Flickr30k, indicating the similarity and loss mechanism rather than a specific encoder does the work.
The combination of MaxMatch with the two diversity losses contributes more than either component alone, with the full configuration reaching the best RSUM in the ablation study.
Because inference uses top-k set elements without matching, the diversity gains come without the computational cost of the training-time assignment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The collapse argument against smooth-Chamfer, shown in Appendix C via Jensen's inequality, suggests any mean-pooling-style set similarity will tend to collapse embeddings; one testable extension is whether replacing the mean with a median or a trimmed average would avoid collapse without needing a matching step.
The Hungarian assignment is a discrete operation during training; a natural next step, not explored in the paper, is to use a differentiable assignment such as Sinkhorn iterations or a soft permutation so the matching itself can be trained end-to-end rather than combined with the extra diversity losses.
The paper evaluates only image-text alignment; the same one-to-one assignment idea could be extended to three or more modalities by matching across multiple sets, though the paper states that substantial modifications would be required.
A direct test of the diversity claim beyond retrieval accuracy would be to quantify how often the K slots retrieve distinct captions or images, using the per-slot retrieval setup of Figure 5, rather than relying only on qualitative examples.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know about arXiv:2506.21538. First, the core proposal is a sensible next step for set-based cross-modal retrieval: replace max and smooth-Chamfer set similarities with an exact one-to-one assignment (Hungarian) and add two diversity-promoting losses. On paper, that directly addresses the sparse-supervision and set-collapse problems the paper documents. Second, the paper's own pseudocode does not implement that proposal. Appendix H's mask_max_similarity does an argmax over enumerated rows, not an optimal assignment, and the index tensors it creates have incompatible shapes for batch sizes above 1. So the reported SOTA numbers cannot currently be tied to the method as written. That is not a fatal flaw in the idea, but it is a fatal flaw in the manuscript as evidence.

What is genuinely good: the breadth of the evaluation is above average for this area—three visual/text encoder combinations, two benchmarks, ablations over the similarity function and each loss, a per-embedding usefulness analysis, variance-based collapse metrics, and a port of the method to PVSE that shows it is not architecture-specific. The appendix proof that smooth-Chamfer encourages collapse is a standard Jensen/convexity argument and it checks out for the affine similarity case it assumes (though the model uses cosine, so the proof does not directly apply). The qualitative figures are illustrative.

The soft spots beyond the pseudocode: Eq. 4 is malformed—it sums over i,j in a way that conflicts with the per-pair definition of MaxSim, and the indices start at 0 with a 1/K normalization that does not match anything. The paper says inference uses top-k without the assignment step, which is fine for speed, but it means training and inference use different similarities; that deserves a clear statement of what exactly is optimized. There are no error bars, and several tables have formatting artifacts. None of these are evidence that the method is wrong, but they are exactly the things that need fixing before the claimed gains can be believed.

Bottom line: if the Hungarian assignment is actually what is implemented, this is a plausible contribution—maybe not a field-changer, but a useful recipe for multi-embedding models. If the argmax variant is what was actually used, then the paper's own motivation (full participation, no sparse supervision) is undermined, because that variant does not enforce one-to-one matching.

Who is this for? Anyone working on set-based cross-modal retrieval or multi-embedding representations. It deserves a serious referee round, but the review must make code release and a corrected, runnable specification a hard condition for acceptance. I would not cite it until code is out.

Recommendation: send to peer review, with a strong request for code and a corrected Appendix H. My own verdict is revise-and-verify rather than accept-as-is.

Referee Report

4 major / 5 minor

Summary. The paper proposes MaxMatch, a set-based cross-modal image-text retrieval method. Each image/caption is represented by K embeddings produced by an encoder plus a set prediction module. The claimed novelty is a Maximal Pair Assignment Similarity that matches embeddings between sets via an optimal permutation (Hungarian algorithm), together with two new losses: a Global Discriminative Loss pushing set embeddings away from a global reference, and an Intra-Set Divergence Loss penalizing high pairwise similarity within a set. The method is evaluated on Flickr30k and MS-COCO across multiple feature extractor/encoder combinations, with ablations, embedding-diversity metrics, and qualitative comparisons against SetDiv, PVSE, and other baselines. The central claims are state-of-the-art retrieval accuracy without external data and prevention of set collapse.

Significance. If the claims hold, the paper makes a useful contribution: it identifies sparse supervision and set collapse as failure modes of existing set-based retrieval similarities, proposes a principled replacement (optimal one-to-one matching), and supports it with broad experiments across two datasets, three backbone configurations, ablations, and qualitative analysis. The inclusion of pseudocode, an appendix proof about smooth-Chamfer collapse, and an explicit limitations section is commendable. However, the significance is currently conditional because the only executable specification of the central similarity, Appendix H, does not implement the optimal assignment described in Eq. (2), and the appendix proof does not actually prove the paper's headline claim that MaxMatch prevents degenerate embedding sets. These issues affect reproducibility and the attribution of the reported gains, so the contribution cannot be fully assessed in its present form.

major comments (4)

[§3.3 and Appendix H] The pseudocode in Appendix H does not implement the Hungarian optimal assignment defined in Eq. (2). Listing 2 returns row_indices of shape [num_perms * batch, K] and col_indices of shape [num_perms * batch, col_size], which are incompatible for batch size > 1; Listing 1 then performs an argmax over selected_similarities along dim=1 and writes ones into the mask, which selects per-permutation maximum pairs rather than solving a global one-to-one assignment, does not enforce a permutation matrix, and does not maximize the trace objective of Eq. (2). Listing 3 calls this on detached similarities. Consequently, if the pseudocode reflects the implementation actually used, the evaluated similarity is not MaxMatch as defined; if a correct Hungarian solver was used, that code is missing. The reported state-of-the-art results therefore cannot be tied to the proposed mechanism as written, and the central claim of the paper is not reproducible from the manuscript.
[§3.3, Eq. (4)] Equation (4) conflates per-pair and batch-level indexing. MaxSim is defined for a single image-text set pair (S^V_i, S^T_j), but then Eq. (4) sums over i and j from 0 to K as if these were batch indices, while K is elsewhere the number of embeddings per set. The notation also mixes 0-based and 1-based indexing inconsistently with Eq. (2)-(3). This makes it unclear whether S_H is a per-pair similarity or an aggregated batch score, and how it is used in the triplet loss of Eq. (6). The loss definitions and all experimental results depend on this quantity, so the ambiguity is load-bearing and needs a precise, consistent formulation.
[Appendix C and §3.3] Appendix C proves only that smooth-Chamfer similarity encourages all embeddings in a set to become identical under the stated convexity assumptions. It does not prove that MaxMatch prevents degenerate embedding sets, which is a central claim in the Abstract and Section 1 ('We show that MaxMatch prevents degenerate embedding sets'). The only direct evidence for this claim is qualitative (Figures 3-5) and an empirical variance measure in Table 4. To support the paper's theoretical framing, the authors should either provide a formal argument for MaxMatch or explicitly reframe the claim as an empirical observation, ideally with quantitative collapse metrics across training.
[Tables 1-4] Several tables contain formatting and annotation errors that hinder evaluation. In Table 2, the COCO 5K columns for MaxMatch are run together (e.g., '51.8479.8687.6236.35 66 77.28398.95'), making individual R@K values unreadable. In Table 4, the column structure collapses the evaluation-condition labels (SV(1)...ST(4)) into a single row, so it is impossible to tell which subset of embeddings is used in each row. Table 3 repeats the configuration 'Smooth-Chamfer ✓ ✓ ✓' twice with different RSUM values and omits explicit column headers for which losses are active, so the ablation cannot be checked. These should be corrected with clear per-cell alignment and loss-configuration labels.

minor comments (5)

[Tables 1-4] Several tables contain formatting and annotation errors that hinder evaluation: Table 2 has run-together numbers in the COCO 5K sections, Table 4 loses the row labels for which embeddings are removed, and Table 3 repeats identical checkmark patterns with different RSUM values and lacks explicit column headers. Please correct these so results can be read and verified.
[§3.3, Eq. (2)] The notation π(T_j) is not defined; it should be clarified that π permutes the indices of T_j (or columns of the similarity matrix) so that the trace objective is unambiguously specified.
[§3.4, Eqs. (7)-(8)] The symbols vi and ti are used both for individual set elements and for the full set, and Eq. (8) uses M where K is used elsewhere for the number of embeddings; please unify notation and state clearly over which indices the losses are summed.
[Appendix B] The contrastive loss is listed as part of the total loss in Eq. (5) but is described as not included in some configurations; please state explicitly for each configuration whether λ_CON is zero or the term is omitted.
[§4.2] The discussion of CORA and 3SHNet is more detailed in Appendix D than in the main text; please ensure the main-text comparison makes clear which methods use external data, since the abstract claims 'without relying on external data' and the comparison fairness depends on this distinction.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claims are benchmarked externally and the proposed losses/matching are design choices, not assumptions restated as conclusions.

full rationale

MaxMatch's central performance claim is measured on external benchmarks (MS-COCO and Flickr30k) against prior published methods, with no target result assumed in the construction of Eq. (2) or Eqs. (5)-(8). The maximal pair assignment similarity is defined as the optimal permutation trace (Eq. 2), which is independent of the reported outcome; if anything, Appendix H's pseudocode fails to implement that definition (Listing 1 performs an argmax over selected entries rather than a global assignment), so the implementation may not match the method, but this is a correctness/reproducibility gap, not circularity. The diversity results are enforced by explicit losses (Global Discriminative Loss Eq. 7 and Intra-Set Divergence Loss Eq. 8) and then measured by circular variance and qualitative retrieval; reporting that the trained model is diverse is an empirical observation, and Appendix G honestly documents failure cases where diversity still collapses. Appendix C's proof that smooth-Chamfer encourages collapse is a self-contained convexity argument. Citations to Song and Soleymani (2019) and Kim et al. (2023) are external prior work used as baselines and architecture source, not self-citations carrying the argument. No fitted parameter is renamed as a prediction, and no uniqueness theorem is imported from the authors' own prior work. Hyperparameters are tuned, but tuning is not circular reasoning. Therefore no step in the derivation reduces to its own input; circularity score is 0.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the set prediction architecture from Kim et al. (2023), pretrained feature extractors, and a set of hand-chosen hyperparameters; the method itself introduces no new physical entities, particles, or unobserved mechanisms.

free parameters (7)

K (number of embeddings per set) = 4
Set size used in all experiments, following SetDiv and PVSE; no sensitivity analysis is provided.
Triplet margin delta_1 = 0.2 (ResNet152+BiGRU), 0.3 (Faster R-CNN+BiGRU), 0.15 (Faster R-CNN+BERT), 0.1 (ResNeXT101)
Chosen per configuration without reported sensitivity.
Global Discriminative margin delta_2 = 0.6 or 0.8 depending on configuration
Hand-selected margin; affects how far embeddings are pushed from the global reference.
Intra-Set Divergence margin delta_3 = 0.6 or 0.8 depending on configuration
Hand-selected margin; controls within-set diversity.
Scaling factor s = 0.5
Smooths gradients in both exponential losses; chosen by hand.
Loss weights lambda_GD and lambda_ISD = 0.1 or 0.05 per configuration
Tuned per dataset and backbone combination.
Regularization weights lambda_MMD, lambda_Div, lambda_CON = 0.01, 0.01, and 0.001 or 0 depending on configuration
Inherited from prior work or tuned per configuration.

assumptions (5)

standard math The Hungarian algorithm (or full permutation enumeration for K=4) returns the maximum-weight one-to-one assignment for the cosine similarity matrix.
Used in Section 3.3 to define Maximal Pair Assignment Similarity.
standard math The log-sum-exp objective for smooth-Chamfer is convex when c(x,y) is affine in x and S2 is fixed.
Appendix C, used to argue smooth-Chamfer encourages collapse.
domain assumption Pretrained visual (ResNet, Faster R-CNN, ResNeXT) and text (GloVe, BERT) features provide a useful shared basis for retrieval.
All experiments use features from external pretraining; the method does not learn from raw pixels or raw words.
domain assumption The set prediction module from Kim et al. (2023) produces meaningful slot embeddings when trained with the proposed losses.
The architecture is adopted unchanged from SetDiv and is not independently analyzed.
ad hoc to paper Gradients through the similarity can ignore the assignment mask (detached argmax) and still train effectively.
The pseudocode detaches the similarity matrix before selecting matches; no analysis of the resulting gradient bias is provided.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Maximal Matching Matters: Preventing Representation Collapse for Robust Cross-Modal Retrieval." pith.science (2026). https://pith.science/paper/CK47PLWL

@misc{pith2026250621538,
  author       = {Pith},
  title        = {Pith review of: Maximal Matching Matters: Preventing Representation Collapse for Robust Cross-Modal Retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CK47PLWL}},
  note         = {Machine review of arXiv:2506.21538}
}

read the original abstract

Cross-modal image-text retrieval is challenging because of the diverse possible associations between content from different modalities. Traditional methods learn a single-vector embedding to represent semantics of each sample, but struggle to capture nuanced and diverse relationships that can exist across modalities. Set-based approaches, which represent each sample with multiple embeddings, offer a promising alternative, as they can capture richer and more diverse relationships. In this paper, we show that, despite their promise, these set-based representations continue to face issues including sparse supervision and set collapse, which limits their effectiveness. To address these challenges, we propose Maximal Pair Assignment Similarity to optimize one-to-one matching between embedding sets which preserve semantic diversity within the set. We also introduce two loss functions to further enhance the representations: Global Discriminative Loss to enhance distinction among embeddings, and Intra-Set Divergence Loss to prevent collapse within each set. Our method achieves state-of-the-art performance on MS-COCO and Flickr30k without relying on external data.

Figures

Figures reproduced from arXiv: 2506.21538 by the authors.

**Figure 1.** We illustrate how the same image can be paired with multiple captions. Color-coded regions link visual elements to corresponding textual descriptions. Our method produces multiple embeddings for each sample (image or caption) in a shared space where similar visual and textual elements cluster (indicated by colored groups and dashed boundaries). Icons on embeddings illustrate the model’s capacity to map multiple im… view at source ↗

**Figure 2.** Left: An overview of the model architecture based on [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** t-SNE visualization of learned embedding spaces, where colors indicate different elements within each set, and markers differentiate modalities (dots for image embeddings, crosses for text embeddings). MIL produces scattered embeddings, Smooth Chamfer collapses them, while MaxMatch preserves semantic distinctiveness and strong alignment. feature ϕ T (y) ∈ R D. 3.2 Embedding Set Prediction Module We next describe t… view at source ↗

Figures from the paper (2 more)

**Figure 5.** Figure 5: Visual comparison of image retrieval results [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

**Figure 6.** Figure 6: Visual comparison of image retrieval results [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Modality-Aware Feature Matching in Visual and Vision-Language Applications: A Comprehensive Survey
cs.CV 2025-07 conditional novelty 2.0 of 10

A survey organizing feature matching research by modality, from SIFT to transformer-based dense matchers and vision-language models.

Reference graph

Works this paper leans on

39 extracted references · 30 canonical work pages · cited by 1 Pith paper

[1]

Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. 2018. Bottom-up and top-down attention for image captioning and visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6077--6086

work page 2018
[2]

Galen Andrew and Jianfeng Gao. 2007. Scalable training of L1 -regularized log-linear models. In Proceedings of the 24th International Conference on Machine Learning, pages 33--40

work page 2007
[3]

Jiacheng Chen, Hexiang Hu, Hao Wu, Yuning Jiang, and Changhu Wang. 2021. Learning the best pooling strategy for visual semantic embedding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15789--15798

work page 2021
[4]

Sanghyuk Chun. 2024. Improved probabilistic image-text representations. In International Conference on Learning Representations (ICLR)

work page 2024
[5]

Sanghyuk Chun, Seong Joon Oh, Rafael Sampaio De Rezende, Yannis Kalantidis, and Diane Larlus. 2021. Probabilistic embeddings for cross-modal retrieval. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8415--8424

work page 2021
[6]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171--4186

2019
[7]

Haiwen Diao, Ying Zhang, Lin Ma, and Huchuan Lu. 2021. https://doi.org/10.1609/aaai.v35i2.16209 Similarity reasoning and filtration for image-text matching . Proceedings of the AAAI Conference on Artificial Intelligence, 35(2):1218--1226

work page doi:10.1609/aaai.v35i2.16209 2021
[8]

Fleet, Jamie Ryan Kiros, and Sanja Fidler

Fartash Faghri, David J. Fleet, Jamie Ryan Kiros, and Sanja Fidler. 2018. https://github.com/fartashf/vsepp Vse++: Improving visual-semantic embeddings with hard negatives . In Proceedings of the British Machine Vision Conference (BMVC)

work page 2018

Show all 39 references

[9]

Andrea Frome, Greg S Corrado, Jon Shlens, Samy Bengio, Jeff Dean, Marc Aurelio Ranzato, and Tomas Mikolov. 2013. https://proceedings.neurips.cc/paper_files/paper/2013/file/7cce53cf90577442771720a370c3c723-Paper.pdf Devise: A deep visual-semantic embedding model . In Advances i...

2013
[10]

Zheren Fu, Zhendong Mao, Yan Song, and Yongdong Zhang. 2023. Learning semantic relationship among instances for image-text matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

2023
[11]

Xuri Ge, Songpei Xu, Fuhai Chen, Jie Wang, Guoxin Wang, Shan An, and Joemon M Jose. 2024. 3shnet: Boosting image--sentence retrieval via visual semantic--spatial self-highlighting. Information Processing & Management, 61(4):103716

2024
[12]

Arthur Gretton, Karsten Borgwardt, Malte Rasch, Bernhard Sch \"o lkopf, and Alex Smola. 2006. A kernel method for the two-sample-problem. Advances in neural information processing systems, 19

2006
[13]

Jiuxiang Gu, Jianfei Cai, Shafiq R Joty, Li Niu, and Gang Wang. 2018. Look, imagine and match: Improving textual-visual cross-modal retrieval with generative models. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7181--7189

2018
[14]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770--778

2016
[15]

Dongwon Kim, Namyup Kim, and Suha Kwak. 2023. Improving cross-modal retrieval with set of diverse embeddings. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 23422--23431

2023
[16]

Kuang-Huei Lee, Xi Chen, Gang Hua, Houdong Hu, and Xiaodong He. 2018. Stacked cross attention for image-text matching. In Proceedings of the European conference on computer vision (ECCV), pages 201--216

2018
[17]

Kunpeng Li, Yulun Zhang, Kai Li, Yuanyuan Li, and Yun Fu. 2019. Visual semantic reasoning for image-text matching. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4654--4662

2019
[18]

Zhuang Li, Yuyang Chai, Terry Yue Zhuo, Lizhen Qu, Gholamreza Haffari, Fei Li, Donghong Ji, and Quan Hung Tran. 2023. Factual: A benchmark for faithful and consistent textual scene graph parsing. In Findings of the Association for Computational Linguistics: ACL 2023, pages 6377--6390

2023
[19]

Lawrence Zitnick

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll \'a r, and C. Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In Computer Vision -- ECCV 2014, pages 740--755, Cham. Springer International Publishing

2014
[20]

Ilya Loshchilov and Frank Hutter. 2017. https://openreview.net/forum?id=Skq89Scxx SGDR: stochastic gradient descent with warm restarts . In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings . Op...

2017
[21]

Nicola Messina, Giuseppe Amato, Andrea Esuli, Fabrizio Falchi, Claudio Gennaro, and St\' e phane Marchand-Maillet. 2021. https://doi.org/10.1145/3451390 Fine-grained visual textual alignment for cross-modal retrieval using transformer encoders . ACM Trans. Multimedia Comput. C...

2021 doi
[22]

Antoine Miech, Jean-Baptiste Alayrac, Ivan Laptev, Josef Sivic, and Andrew Zisserman. 2021. Thinking fast and slow: Efficient text-to-visual retrieval with transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9826--9836

2021
[23]

Zhengxin Pan, Fangyu Wu, and Bailing Zhang. 2023. https://doi.org/10.1109/CVPR52729.2023.01847 Fine-grained image-text matching by cross-modal hard aligning network . In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19275--19284

2023
[24]

Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in pytorch. In NIPS-W: Workshop on Automatic Differentiation

2017
[25]

Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. https://doi.org/10.3115/v1/D14-1162 G lo V e: Global vectors for word representation . In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing ( EMNLP ) , pages 1532--1543. Ass...

2014 doi
[26]

Khoi Pham, Chuong Huynh, Ser-Nam Lim, and Abhinav Shrivastava. 2024. Composing object relations and attributes for image-text matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14354--14363

2024
[27]

Bryan A Plummer, Liwei Wang, Chris M Cervantes, Juan C Caicedo, Julia Hockenmaier, and Svetlana Lazebnik. 2015. Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models. In Proceedings of the IEEE international conference on computer ...

2015
[28]

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

2021
[29]

Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems, 28

2015
[30]

Yale Song and Mohammad Soleymani. 2019. Polysemous visual-semantic embedding for cross-modal retrieval. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1979--1988

2019
[31]

Christopher Thomas and Adriana Kovashka. 2020. Preserving semantic neighborhoods for robust cross-modal retrieval. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XVIII 16, pages 317--335. Springer

2020
[32]

Christopher Thomas and Adriana Kovashka. 2022. Emphasizing complementary samples for non-literal cross-modal retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4632--4641

2022
[33]

Haoran Wang, Dongliang He, Wenhao Wu, Boyang Xia, Min Yang, Fu Li, Yunlong Yu, Zhong Ji, Errui Ding, and Jingdong Wang. 2022. Coder: Coupled diversity-sensitive momentum contrastive learning for image-text retrieval. In European Conference on Computer Vision, pages 700--716. Springer

2022
[34]

Jiwei Wei, Xing Xu, Yang Yang, Yanli Ji, Zheng Wang, and Heng Tao Shen. 2020 a . Universal weighting metric learning for cross-modal matching. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13005--13014

2020
[36]

Keyu Wen, Xiaodong Gu, and Qingrong Cheng. 2021. https://doi.org/10.1109/tcsvt.2020.3030656 Learning dual semantic relations with graph attention for image-text matching . IEEE Transactions on Circuits and Systems for Video Technology, 31(7):2866–2879

2021
[37]

Saining Xie, Ross Girshick, Piotr Doll \'a r, Zhuowen Tu, and Kaiming He. 2017. Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1492--1500

2017
[38]

Fei Yan and Krystian Mikolajczyk. 2015. Deep correlation for matching images and text. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

2015
[39]

Kun Zhang, Zhendong Mao, Quan Wang, and Yongdong Zhang. 2022. https://doi.org/10.1109/CVPR52688.2022.01521 Negative-aware attention framework for image-text matching . In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

2022
[40]

Qi Zhang, Zhen Lei, Zhaoxiang Zhang, and Stan Z. Li. 2020. https://doi.org/10.1109/CVPR42600.2020.00359 Context-aware attention network for image-text retrieval . In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3533--3542

2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. 2018. Bottom-up and top-down attention for image captioning and visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6077--6086

work page 2018

[2] [2]

Galen Andrew and Jianfeng Gao. 2007. Scalable training of L1 -regularized log-linear models. In Proceedings of the 24th International Conference on Machine Learning, pages 33--40

work page 2007

[3] [3]

Jiacheng Chen, Hexiang Hu, Hao Wu, Yuning Jiang, and Changhu Wang. 2021. Learning the best pooling strategy for visual semantic embedding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15789--15798

work page 2021

[4] [4]

Sanghyuk Chun. 2024. Improved probabilistic image-text representations. In International Conference on Learning Representations (ICLR)

work page 2024

[5] [5]

Sanghyuk Chun, Seong Joon Oh, Rafael Sampaio De Rezende, Yannis Kalantidis, and Diane Larlus. 2021. Probabilistic embeddings for cross-modal retrieval. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8415--8424

work page 2021

[6] [6]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171--4186

2019

[7] [7]

Haiwen Diao, Ying Zhang, Lin Ma, and Huchuan Lu. 2021. https://doi.org/10.1609/aaai.v35i2.16209 Similarity reasoning and filtration for image-text matching . Proceedings of the AAAI Conference on Artificial Intelligence, 35(2):1218--1226

work page doi:10.1609/aaai.v35i2.16209 2021

[8] [8]

Fleet, Jamie Ryan Kiros, and Sanja Fidler

Fartash Faghri, David J. Fleet, Jamie Ryan Kiros, and Sanja Fidler. 2018. https://github.com/fartashf/vsepp Vse++: Improving visual-semantic embeddings with hard negatives . In Proceedings of the British Machine Vision Conference (BMVC)

work page 2018