Pith. sign in

REVIEW 4 major objections 5 minor 76 references

DALR: Dual-level Alignment Learning for Multimodal Sentence Representation Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that two alignment losses — a cross-modal consistency loss and an intra-modal ranking-distillation loss — jointly cure the biases that limit multimodal sentence embeddings, improving semantic similarity and transfer…

desk verdict The paper's central intra-modal loss is degenerate as printed, but the overall design is plausible and the equation is likely a fixable typo. read the letter →

arxiv 2506.21096 v2 pith:I4BTBGLK submitted 2025-06-26 cs.CL

Kang He , Yuzhe Ding , Haining Wang , Fei Li , Chong Teng , Donghong Ji This is my paper · ORCID
classification cs.CL
keywords multimodalsentencerepresentationcross-modalalignmentintra-modalsemanticdivergencerankingdistillationtextualsimilaritycontrastivelearningknowledgeKL
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Multimodal sentence embeddings learned by contrastive alignment alone inherit two distortions: a caption and its image can be paired even when the caption describes only a minor part of the visual content (cross-modal misalignment bias), and two captions of the same image can be semantically far apart while the pairing treats them as interchangeable (intra-modal semantic divergence). DALR argues that both distortions should be corrected at the loss level, by softening negatives with a consistency task and by replacing binary positive-negative labels with a continuous ranking structure distilled from text teachers. The paper's evidence is a consistent improvement over MCSE and KDMCSE on seven STS datasets and seven transfer tasks with both BERT and RoBERTa, for example DALR-BERT reaches 79.5 average Spearman on wiki+flickr versus KDMCSE's 78.6. If the claim holds, a lightweight unsupervised recipe can improve sentence embeddings without extra data or large-language-model supervision.

What carries the argument

The carrying mechanism is the dual-level loss composition in Eq. 14, which stacks three complementary objectives on the same student encoder. The cross-modal part, $L_{\mathrm{CML}} = L_{\mathrm{cons}} + L_{\mathrm{CMA}}$, uses a cosine-embedding consistency loss on shuffled image-text pairs to soften negatives and a KL divergence that pulls the student's normalized text-to-image similarity distribution toward teacher text-text and image-image similarity distributions. The intra-modal part, $L_{\mathrm{IML}} = L_{\mathrm{rank}} + L_{\mathrm{IMA}}$, combines ListMLE, a listwise ranking loss that assigns high probability to the teacher's permutation of similarity scores, with a KL divergence between the teacher text distribution and the student's text distribution under different dropout masks. This machinery converts binary contrastive labels into distributional targets, which is what lets the model carry fine-grained semantic structure across and within modalities.

What would settle it

Run DALR with the text-teacher rankings replaced by random permutations while keeping all other components fixed; if average STS Spearman and transfer accuracy stay within seed noise of the reported 79.5 and 86.92, then ranking distillation is not carrying the claimed weight. A complementary check is to measure DALR's text-text similarity on contradictory pairs such as 'A man is skating' versus 'A man is gliding' and require a separation gain over MCSE and KDMCSE larger than the reported standard deviations.

Watch

Extended reading notes

Core claim

The central claim is that the quality ceiling of multimodal sentence representation is set not by the amount of paired image-text data but by two alignment failures: cross-modal misalignment bias and intra-modal semantic divergence. DALR addresses the first by coupling the InfoNCE contrastive objective with an auxiliary binary consistency classifier over shuffled pairs, whose cosine embedding loss produces a soft similarity matrix, and by minimizing the KL divergence between the student's image-to-text distribution and the teacher's text-text and image-image distributions. It addresses the second by treating sentence relationships as a ranking rather than binary positive-negative labels: a weighted ensemble of SimCSE and DiffCSE produces pseudo-ranking labels, the listwise loss ListMLE transfers the ordering, and a KL term aligns the global similarity distribution between teacher and student. The joint objective is $L_{\mathrm{total}} = L_{\mathrm{Info}} + \lambda L_{\mathrm{CML}} + \mu L_{\mathrm{IML}}$ (Eq. 14), trained with mixed alternating batches of Wikipedia text and Flickr or MSCOCO image-text pairs, and on the paper's evidence it produces state-of-the-art STS and transfer scores, with DALR-BERT at 79.5 average Spearman on wiki+flickr and transfer accuracy rising from KDMCSE's 86.53 to 86.92.

Load-bearing premise

The load-bearing premise is that the frozen teacher similarities (SimCSE, DiffCSE, and CLIP) are trustworthy enough to act as ranking ground truth, since Appendix B specifies only 'weighted aggregation' for the text teachers without giving the weights, and if those teachers rank divergent captions of one image as near-neighbors, DALR inherits exactly the intra-modal divergence it is meant to remove.

Editorial extensions

If this is right

  • Because the two modules are losses rather than data pipelines, they can be dropped onto any contrastive sentence encoder that has a frozen visual teacher; the paper shows that adding the cross-modal alignment module to KDMCSE alone improves Flickr30k retrieval, with image-to-text R@1 rising from 17.9 to 19.1 on wiki+flickr.
  • Teacher quality directly bounds student quality: Appendix E.2 reports that replacing SimCSE with RankCSE as a text teacher raises DALR's results further, implying that any future improvement in text or image encoders will compound into this method.
  • The ablation ordering indicates that cross-modal alignment is the primary driver, while intra-modal ranking is a complementary fine-grained regularizer; on wiki+coco, removing $L_{\mathrm{CML}}$ costs the most in both STS and transfer accuracy.
  • The gains on reranking, retrieval, classification, and STS shown in Table 5 imply that the learned embeddings transfer beyond the training objective, not just to the two standard benchmarks.
  • The method's training time is reported as roughly four hours on a single A100 for wiki+flickr with BERT, which is shorter than prior multimodal baselines, so the improvements do not come at a large compute premium.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the same two-bias recipe should transfer to other modality pairs such as audio-text or video-text, because image redundancy and caption selectivity are instances of a general dense-modality versus selective-modality asymmetry.
  • A testable extension is iterative self-distillation: since the teacher-model table shows that stronger text teachers produce better student embeddings, allowing the student's own improved embeddings to replace the frozen text teacher after a few epochs could compound the gains.
  • The paper specifies only 'weighted aggregation' for the SimCSE and DiffCSE ensemble in Appendix B without giving the weights, so an immediate empirical question is whether a learned or validation-tuned weighting lifts the reported 79.5 average Spearman further; nothing in the method forbids it.
  • A sharper test of the intra-modal divergence claim than overall STS averages would be human intra-modal similarity judgments on captions that share an image but diverge in focus; DALR should show larger text-text separation for such pairs than MCSE and KDMCSE.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes DALR, a dual-level alignment framework for multimodal sentence representation learning. The model fine-tunes BERT/RoBERTa students with an image-text InfoNCE loss (Eq. 4), a cross-modal alignment module composed of an auxiliary binary consistency task and KL divergence against teacher vision/text similarity distributions (Eqs. 5-9), and an intra-modal alignment module that combines ListMLE ranking distillation with KL divergence from a weighted SimCSE/DiffCSE teacher distribution (Eqs. 10-13). The joint objective is Eq. (14). Experiments on STS and transfer tasks report that DALR outperforms SimCSE, MCSE, and KDMCSE on BERT and RoBERTa under wiki+flickr and wiki+coco settings; ablations, retrieval evaluations, and MTEB-style downstream tasks are used to support the contribution of each module.

Significance. If the results hold after correction and controlled re-evaluation, DALR is a reasonable incremental contribution: it combines an auxiliary cross-modal consistency task, KL-based distribution matching, and ranking distillation from frozen teachers, and it reports consistent gains over SimCSE, MCSE, and KDMCSE on STS and transfer benchmarks. The paper's strengths include five-seed repetitions for the main STS and ablation tables, evaluation on two multimodal datasets, and ablations that separate the contributions of each loss term. The significance is currently conditional, however, because Eq. (11) as printed makes the intra-modal KL loss degenerate, the training objective omits the loss used for text-only batches, and the baseline comparisons are not controlled under a common seed protocol.

major comments (4)
  1. [Section 3.3, Eq. (11)] As printed, the numerator exp(sim(s_i^z, s_i^{z'})) and the denominator sum_{j=1}^N exp(sim(s_i^z, s_j^{z'})) in Eq. (11) are both independent of the output index j, so every entry of P_i^{t2t} is identical and the student distribution is uniform. Consequently L_IML in Eq. (12) equals sum_i KL(Q_i^{t2t} || uniform), which is constant with respect to the student parameters. This contradicts Table 3, where removing IML changes the STS average by about 0.7 points and the TR average by about 0.5 points. The numerator should presumably read exp(sim(s_i^z, s_j^{z'})) by analogy with Eq. (6); please correct the equation and confirm against the implementation, since as written the intra-modal alignment loss cannot produce the reported gains. Because no commit hash or exact code is provided, the reader cannot check which form was actually used.
  2. [Section 3.4, Eq. (14); Section 4.1] The final objective in Eq. (14) contains only L_Info, L_CML, and L_IML, but Section 4.1 describes a mixed alternating sampling strategy in which pure-text batches from Wiki1M are also loaded and updated. If those batches are trained with the unsupervised SimCSE objective L_text from Eq. (1), then Eq. (14) omits a term that is active for a large fraction of the training updates; if they are trained with L_Info, the image-text InfoNCE loss is undefined for text-only batches. This omission affects the main results and the ablations (e.g., w/o L_Info in Table 3), so the full training objective must be stated explicitly.
  3. [Section 4.2, Tables 1 and 2] The STS baselines in Table 1 are taken from the original papers (marks ♡, †, ‡) rather than re-run under the same code, seeds, and evaluation pipeline as DALR, while DALR is averaged over five seeds with its own standard deviations. The reported improvements over KDMCSE are 0.8-0.9 points with DALR standard deviations of 0.5-0.7, so without paired runs or significance tests the claimed superiority is not established. Table 2 reports transfer accuracy without any variance or significance testing, although the gains over KDMCSE are as small as 0.39 points (BERT wiki+flickr). Please rerun all baselines in the same harness with the same seeds and report paired differences or significance tests.
  4. [Appendix B, Teacher Text Model] The final text teacher representation is described only as a "weighted aggregation" of SimCSE and DiffCSE embeddings, but the weights and any normalization steps are not given. Since the teacher rankings in Eq. (10) and the teacher distribution Q^{t2t} in Eq. (7) are computed from this representation, the missing weights prevent exact reproduction of the pseudo-labels and therefore of the reported results. Please specify the weights and the aggregation procedure, or provide code with a commit hash.
minor comments (5)
  1. [Section 3.2, Eq. (6)] The text introduces P^{v2t}_{ij} but Eq. (6) defines P^{t2v}_{ij}; please unify the notation for the text-to-image and image-to-text probability names.
  2. [Section 3.3, Eq. (13)] The left-hand side of Eq. (13) is L_IML while the right-hand side uses L_IMA; the subscripts should be consistent.
  3. [Appendix B and references] There are several typographical errors: "preform experiments" should be "perform experiments", the reference to Voorhees and Tice has a stray space in "V oorhees", and Section 4.2 says "dedicate the effectiveness" where "indicate the effectiveness" is meant.
  4. [Figure 3 and Table 10] The name DALR is misspelled as DLAR in the caption of Figure 3 and in the column headers of Table 10; please correct these.
  5. [Appendix E.2] The text says "As illustrated in Figure 9" but the referenced content is Table 9; please correct the cross-reference.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; the DALR objectives distill from frozen external teachers and are evaluated on external benchmarks, with the only flagged issue being a non-circular Eq. 11 degeneracy.

full rationale

The DALR derivation is self-contained rather than circular. The student objective in Eq. 14 combines standard InfoNCE (Eq. 4), a cross-modal consistency term built from frozen CLIP image features and a constructed match/mismatch label (Eqs. 5-8), and an intra-modal distillation term whose targets are rankings and similarity distributions from frozen external teachers, SimCSE and DiffCSE, with the teacher setup attributed to RankCSE (Appendix B). None of these targets is fitted to the STS or transfer evaluation scores, and the evaluation benchmarks are external to the training data, so the reported gains over MCSE and KDMCSE are not forced by construction. The only self-citation, He et al. (2025), is a routine related-work mention and is not load-bearing; SimCSE doubling as teacher and baseline makes the comparison conservative rather than circular. A correctness flaw should be flagged separately: Eq. 11's numerator is j-independent, making P_i^{t2t} uniform and Eq. 12's KL term constant with respect to the student parameters as written, which contradicts the IML ablation in Table 3; this is a non-circular typographical or derivation error, not an equivalence between a prediction and its inputs.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new physical or mathematical entities are introduced. CMB and ISD are descriptive names for known annotation and alignment phenomena (false negatives, caption diversity), not new postulates. The central load is carried by external teacher models and the hyperparameters listed above.

free parameters (4)
  • lambda (loss weight for L_CML) = 0.1
    Set in Eq. 14 and Appendix B as a balance weight; no sensitivity analysis is reported.
  • mu (loss weight for L_IML) = 0.2
    Set in Eq. 14 and Appendix B as a balance weight; no sensitivity analysis is reported.
  • m (margin in consistency loss) = 0.2
    Margin for negative pairs in Eq. 5, described as 'typically set to 0.2 based on empirical findings.'
  • tau (temperature) = 0.05
    Temperature used in all contrastive and KL distributions; stated in Appendix B without tuning details.
assumptions (5)
  • domain assumption Cosine similarity of embeddings is a valid semantic similarity metric for sentence and image features.
    All contrastive and KL objectives in Eqs. 6-12 use cosine similarity as the semantic measure.
  • domain assumption Teacher text models (SimCSE, DiffCSE) and the CLIP image encoder provide reliable pseudo-ranking labels for semantic structure.
    Ranking distillation Eq. 10 and KL losses Eqs. 8 and 12 treat teacher similarity rankings as ground truth.
  • ad hoc to paper Randomly shuffled image-text pairs in D' are genuinely mismatched (y'=0) for the auxiliary consistency task.
    Section 3.2 constructs negatives by shuffling images, but captions and images in the same batch are not guaranteed dissimilar, so the binary labels may be noisy.
  • domain assumption Representations from text and image encoders can be projected into a shared 256-dimensional space where cross-modal similarity is comparable.
    Section 3 and Appendix B project all modalities with separate MLPs into one 256-dimensional space and compare with cosine similarity.
  • standard math ListMLE (Xia et al., 2008) is a valid listwise ranking objective for this setting.
    Used in Eq. 10 without proof; standard learning-to-rank result.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DALR: Dual-level Alignment Learning for Multimodal Sentence Representation Learning." pith.science (2026). https://pith.science/paper/I4BTBGLK

@misc{pith2026250621096,
  author       = {Pith},
  title        = {Pith review of: DALR: Dual-level Alignment Learning for Multimodal Sentence Representation Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I4BTBGLK}},
  note         = {Machine review of arXiv:2506.21096}
}
read the original abstract

Previous multimodal sentence representation learning methods have achieved impressive performance. However, most approaches focus on aligning images and text at a coarse level, facing two critical challenges:cross-modal misalignment bias and intra-modal semantic divergence, which significantly degrade sentence representation quality. To address these challenges, we propose DALR (Dual-level Alignment Learning for Multimodal Sentence Representation). For cross-modal alignment, we propose a consistency learning module that softens negative samples and utilizes semantic similarity from an auxiliary task to achieve fine-grained cross-modal alignment. Additionally, we contend that sentence relationships go beyond binary positive-negative labels, exhibiting a more intricate ranking structure. To better capture these relationships and enhance representation quality, we integrate ranking distillation with global intra-modal alignment learning. Comprehensive experiments on semantic textual similarity (STS) and transfer (TR) tasks validate the effectiveness of our approach, consistently demonstrating its superiority over state-of-the-art baselines.

Figures

Figures reproduced from arXiv: 2506.21096 by the authors.

Figure 1
Figure 1. Illustration of a batch image-caption pairs [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The illustration of our proposed framework DALR, consisting of three components: (a) the multimodal [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The t-SNE of sentence representations learned by DLAR and its three deviants (w/o specific component) [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Scatter plot of the ground truth similarity scores (x-axis) and the cosine similarities (y-axis) between [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

76 extracted references · 63 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, I \ n igo Lopez-Gazpio, Montse Maritxalar, Rada Mihalcea, German Rigau, Larraitz Uria, and Janyce Wiebe. 2015. S em E val-2015 task 2: Semantic textual similarity, E nglish, S panish and pilot on interpretability. In Proceedings of the 9th International Wo...

    work page 2015

  4. [4]

    Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2014. S em E val-2014 task 10: Multilingual semantic textual similarity. In Proceedings of the 8th International Workshop on Semantic Evaluation ( S em E val 2014) , pages 81--91

    work page 2014

  5. [5]

    Eneko Agirre, Carmen Banea, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2016. S em E val-2016 task 1: Semantic textual similarity, monolingual and cross-lingual evaluation. In Proceedings of the 10th International Workshop on Semantic Evaluation ( S em E val-2016) , pages 497--511

    work page 2016

  6. [6]

    Eneko Agirre, Daniel Cer, Mona Diab, and Aitor Gonzalez-Agirre. 2012. S em E val-2012 task 6: A pilot on semantic textual similarity. In * SEM 2012: The First Joint Conference on Lexical and Computational Semantics -- Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic...

    work page 2012

  7. [7]

    Eneko Agirre, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, and Weiwei Guo. 2013. * SEM 2013 shared task: Semantic textual similarity. In Second Joint Conference on Lexical and Computational Semantics (* SEM ), Volume 1: Proceedings of the Main Conference and the Shared Task: Semantic Textual Similarity , pages 32--43

    work page 2013

  8. [8]

    Patrick Bordes, Eloi Zablocki, Laure Soulier, Benjamin Piwowarski, and Patrick Gallinari. 2020. Incorporating visual semantics into sentence representations within a grounded space. arXiv preprint arXiv:2002.02734

    work page Pith review arXiv 2020

Show all 76 references
  1. [9]

    Daniel Cer, Mona Diab, Eneko Agirre, I \ n igo Lopez-Gazpio, and Lucia Specia. 2017. S em E val-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation. In Proceedings of the 11th International Workshop on Semantic Evaluation ( S em E val-2017...

    2017

  2. [10]

    Qinyuan Cheng, Xiaogui Yang, Tianxiang Sun, Linyang Li, and Xipeng Qiu. 2023 a . Improving contrastive learning of sentence embeddings from ai feedback. In Findings of the Association for Computational Linguistics: ACL 2023, pages 11122--11138

    2023

  3. [11]

    Xize Cheng, Tao Jin, Linjun Li, Wang Lin, Xinyu Duan, and Zhou Zhao. 2023 b . O pen SR : Open-modality speech recognition via maintaining multi-modality alignment. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers...

    2023

  4. [12]

    Yung-Sung Chuang, Rumen Dangovski, Hongyin Luo, Yang Zhang, Shiyu Chang, Marin Soljacic, Shang-Wen Li, Scott Yih, Yoon Kim, and James Glass. 2022. D iff CSE : Difference-based contrastive learning for sentence embeddings. In Proceedings of the 2022 Conference of the North Amer...

    2022

  5. [13]

    Sanghyuk Chun, Wonjae Kim, Song Park, Minsuk Chang, and Seong Joon Oh. 2022. Eccv caption: Correcting false negatives by collecting machine-and-human-verified image-caption associations for ms-coco. In European Conference on Computer Vision, pages 1--19. Springer

    2022

  6. [14]

    Sanghyuk Chun, Seong Joon Oh, Rafael Sampaio De Rezende, Yannis Kalantidis, and Diane Larlus. 2021. Probabilistic embeddings for cross-modal retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8415--8424

    2021

  7. [15]

    Alexis Conneau and Douwe Kiela. 2018. S ent E val: An evaluation toolkit for universal sentence representations. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation ( LREC 2018)

    2018

  8. [16]

    Jinghao Deng, Fanqi Wan, Tao Yang, Xiaojun Quan, and Rui Wang. 2023. Clustering-aware negative sampling for unsupervised sentence representation. In Findings of the Association for Computational Linguistics: ACL 2023, pages 8713--8729

    2023

  9. [17]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT : Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Hum...

    2019

  10. [18]

    Dolan and Chris Brockett

    William B. Dolan and Chris Brockett. 2005. Automatically constructing a corpus of sentential paraphrases. In Proceedings of the Third International Workshop on Paraphrasing ( IWP 2005)

    2005

  11. [19]

    Kawin Ethayarajh. 2019. How contextual are contextualized word representations? C omparing the geometry of BERT , ELM o, and GPT -2 embeddings. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference ...

    2019

  12. [20]

    Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. S im CSE : Simple contrastive learning of sentence embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894--6910

    2021

  13. [21]

    Jiaming Han, Kaixiong Gong, Yiyuan Zhang, Jiaqi Wang, Kaipeng Zhang, Dahua Lin, Yu Qiao, Peng Gao, and Xiangyu Yue. 2024. Onellm: One framework to align all modalities with language. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26...

    2024

  14. [22]

    Hongliang He, Junlei Zhang, Zhenzhong Lan, and Yue Zhang. 2023. Instance smoothed contrastive learning for unsupervised sentence embedding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 12863--12871

    2023

  15. [23]

    Kang He, Yuzhe Ding, Bobo Li, Haining Wang, Fei Li, Chong Teng, and Donghong Ji. 2025. Harnessing dimensional contrast and information compensation for sentence embedding enhancement. In ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processin...

    2025

  16. [24]

    Felix Hill, Kyunghyun Cho, and Anna Korhonen. 2016. Learning distributed representations of sentences from unlabelled data. In Proceedings of the 2016 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies , page...

    2016

  17. [25]

    Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 168--177

    2004

  18. [26]

    Jian Huang, Yanli Ji, Yang Yang, and Heng Tao Shen. 2023 a . Cross-modality representation interactive learning for multimodal sentiment analysis. In Proceedings of the 31st ACM International Conference on Multimedia, pages 426--434

    2023

  19. [27]

    Jingjia Huang, Yinan Li, Jiashi Feng, Xinglong Wu, Xiaoshuai Sun, and Rongrong Ji. 2023 b . Clover: Towards a unified video-language alignment and fusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14856--14866

    2023

  20. [28]

    Ryan Kiros, Yukun Zhu, Russ R Salakhutdinov, Richard Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Skip-thought vectors. Advances in neural information processing systems, 28

    2015

  21. [29]

    Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. 2021. Align before fuse: Vision and language representation learning with momentum distillation. Advances in neural information processing systems, 34:9694--9705

    2021

  22. [30]

    Linjie Li, Zhe Gan, Kevin Lin, Chung-Ching Lin, Zicheng Liu, Ce Liu, and Lijuan Wang. 2023. Lavender: Unifying video-language understanding as masked language modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23119--23129

    2023

  23. [31]

    Mingxin Li, Richong Zhang, Zhijie Nie, and Yongyi Mao. 2024. Narrowing the gap between supervised and unsupervised sentence representation learning with large language model. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 13590--13599

    2024

  24. [32]

    Zejun Li, Zhihao Fan, Huaixiao Tou, Jingjing Chen, Zhongyu Wei, and Xuanjing Huang. 2022. Mvptr: Multi-level semantic alignment for vision-language pre-training via multi-stage learning. In Proceedings of the 30th ACM International Conference on Multimedia, pages 4395--4405

    2022

  25. [33]

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll \'a r, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, ...

    2014

  26. [34]

    Yan Ling, Jianfei Yu, and Rui Xia. 2022. Vision-language pre-training for multimodal aspect-based sentiment analysis. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2149--2159

    2022

  27. [35]

    Jiduan Liu, Jiahao Liu, Qifan Wang, Jingang Wang, Wei Wu, Yunsen Xian, Dongyan Zhao, Kai Chen, and Rui Yan. 2023 a . R ank CSE : Unsupervised sentence representations learning via learning to rank. In Proceedings of the 61st Annual Meeting of the Association for Computational ...

    2023

  28. [36]

    Ye Liu, Lingfeng Qiao, Changchong Lu, Di Yin, Chen Lin, Haoyuan Peng, and Bo Ren. 2023 b . Osan: A one-stage alignment network to unify multimodal alignment and unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition...

    2023

  29. [37]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692

    2019 arXiv

  30. [38]

    Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, and Roberto Zamparelli. 2014. A sick cure for the evaluation of compositional distributional semantic models. In Proceedings of the Ninth International Conference on Language Resources and Evalu...

    2014

  31. [39]

    Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023. MTEB : Massive text embedding benchmark. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2014--2037

    2023

  32. [40]

    Cong-Duy Nguyen, Thong Nguyen, Duc Vu, and Anh Luu. 2023. Improving multimodal sentiment analysis: Supervised angular margin-based contrastive learning for enhanced fusion representation. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 14714--14724

    2023

  33. [41]

    Cong-Duy Nguyen, Thong Nguyen, Xiaobao Wu, and Anh Tuan Luu. 2024. KDMCSE : Knowledge distillation multimodal sentence embeddings with adaptive angular margin contrastive learning. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Compu...

    2024

  34. [42]

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748

    2018 arXiv

  35. [43]

    Bo Pang and Lillian Lee. 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics ( ACL -04) , pages 271--278

    2004

  36. [44]

    Bo Pang and Lillian Lee. 2005. Seeing stars: exploiting class relationships for sentiment categorization with respect to rating scales. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, page 115–124

    2005

  37. [45]

    Zarana Parekh, Jason Baldridge, Daniel Cer, Austin Waters, and Yinfei Yang. 2021. Crisscrossed captions: Extended intramodal and intermodal semantic similarity judgments for ms-coco. In Proceedings of the 16th Conference of the European Chapter of the Association for Computati...

    2021

  38. [46]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

    2021

  39. [47]

    Emily Reif, Ann Yuan, Martin Wattenberg, Fernanda B Viegas, Andy Coenen, Adam Pearce, and Been Kim. 2019. Visualizing and measuring the geometry of bert. Advances in Neural Information Processing Systems, 32

    2019

  40. [48]

    Yeon Seonwoo, Guoyin Wang, Changmin Seo, Sajal Choudhary, Jiwei Li, Xiang Li, Puyang Xu, Sunghyun Park, and Alice Oh. 2023. Ranking-enhanced unsupervised sentence representation learning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistic...

    2023

  41. [49]

    Zhan Shi, Guoyin Wang, Ke Bai, Jiwei Li, Xiang Li, Qingjun Cui, Belinda Zeng, Trishul Chilimbi, and Xiaodan Zhu. 2023. Osscse: Overcoming surface structure bias in contrastive learning for unsupervised sentence embedding. In Proceedings of the 2023 Conference on Empirical Meth...

    2023

  42. [50]

    Manning, Andrew Ng, and Christopher Potts

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language ...

    2013

  43. [51]

    Kim Sung-Bin, Arda Senocak, Hyunwoo Ha, Andrew Owens, and Tae-Hyun Oh. 2023. Sound to visual scene generation by audio-to-visual latent alignment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6430--6440

    2023

  44. [52]

    Zineng Tang, Jaemin Cho, Hao Tan, and Mohit Bansal. 2021. Vidlankd: Improving language understanding via video-distilled knowledge transfer. Advances in Neural Information Processing Systems, 34:24468--24481

    2021

  45. [53]

    Zhiliang Tian, Zheng Xie, Fuqiang Lin, and Yiping Song. 2023. A multi-view meta-learning approach for multi-modal response generation. In Proceedings of the ACM Web Conference 2023, pages 1938--1947

    2023

  46. [54]

    Ellen M Voorhees and Dawn M Tice. 2000. Building a question answering test collection. In Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, pages 200--207

    2000

  47. [55]

    Haining Wang, Kang He, Bobo Li, Lei Chen, Fei Li, Xu Han, Chong Teng, and Donghong Ji. 2024. Refining and synthesis: A simple yet effective data augmentation framework for cross-domain aspect-based sentiment analysis. In Findings of the Association for Computational Linguistic...

    2024

  48. [56]

    Qian Wang, Weiqi Zhang, Tianyi Lei, Yu Cao, Dezhong Peng, and Xu Wang. 2023. Clsep: Contrastive learning of sentence embedding with prompt. Knowledge-Based Systems, 266:110381

    2023

  49. [57]

    Qifan Wang, Yi Fang, Anirudh Ravula, Fuli Feng, Xiaojun Quan, and Dongfang Liu. 2022 a . Webformer: The web-page transformer for structure information extraction. In Proceedings of the ACM Web Conference 2022, pages 3124--3133

    2022

  50. [58]

    Tongzhou Wang and Phillip Isola. 2020. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In Proceedings of International conference on machine learning, pages 9929--9939

    2020

  51. [59]

    Weizhi Wang, Li Dong, Hao Cheng, Haoyu Song, Xiaodong Liu, Xifeng Yan, Jianfeng Gao, and Furu Wei. 2022 b . Visually-augmented language modeling. arXiv preprint arXiv:2205.10178

    2022 arXiv

  52. [60]

    Janyce Wiebe, Theresa Wilson, and Claire Cardie. 2005. Annotating expressions of opinions and emotions in language. Language resources and evaluation, 39:165--210

    2005

  53. [61]

    Qiyu Wu, Chongyang Tao, Tao Shen, Can Xu, Xiubo Geng, and Daxin Jiang. 2022 a . PCL : Peer-contrastive learning with diverse augmentations for unsupervised sentence embeddings. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 120...

    2022

  54. [62]

    Xing Wu, Chaochen Gao, Liangjun Zang, Jizhong Han, Zhongyuan Wang, and Songlin Hu. 2022 b . ES im CSE : Enhanced sample building method for contrastive learning of unsupervised sentence embedding. In Proceedings of the 29th International Conference on Computational Linguistics...

    2022

  55. [63]

    Fen Xia, Tie-Yan Liu, Jue Wang, Wensheng Zhang, and Hang Li. 2008. Listwise approach to learning to rank: theory and algorithm. In Proceedings of the 25th international conference on Machine learning, pages 1192--1199

    2008

  56. [64]

    Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian-Yun Nie. 2024. C-pack: Packed resources for general chinese embeddings. In Proceedings of the 47th international ACM SIGIR conference on research and development in information retrieval, pages 641--649

    2024

  57. [65]

    Jiaming Xu, Bo Xu, Peng Wang, Suncong Zheng, Guanhua Tian, and Jun Zhao. 2017. Self-taught convolutional neural networks for short text clustering. Neural Networks, 88:22--31

    2017

  58. [66]

    Yuanmeng Yan, Rumei Li, Sirui Wang, Fuzheng Zhang, Wei Wu, and Weiran Xu. 2021. C on SERT : A contrastive framework for self-supervised sentence representation transfer. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Int...

    2021

  59. [67]

    Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. 2014. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the Association for Computational Linguistics, 2:67--78

    2014

  60. [68]

    Tianshu Yu, Haoyu Gao, Ting-En Lin, Min Yang, Yuchuan Wu, Wentao Ma, Chao Wang, Fei Huang, and Yongbin Li. 2023. Speech-text pre-training for spoken dialog understanding with explicit cross-modal alignment. In Proceedings of the 61st Annual Meeting of the Association for Compu...

    2023

  61. [69]

    Chunhui Zhang, Xin Sun, Yiqian Yang, Li Liu, Qiong Liu, Xi Zhou, and Yanfeng Wang. 2023. All in one: Exploring unified vision-language tracking with multi-modal alignment. In Proceedings of the 31st ACM International Conference on Multimedia, page 5552–5561

    2023

  62. [70]

    Miaoran Zhang, Marius Mosbach, David Adelani, Michael Hedderich, and Dietrich Klakow. 2022 a . MCSE : M ultimodal contrastive learning of sentence embeddings. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: ...

    2022

  63. [71]

    Yanzhao Zhang, Richong Zhang, Samuel Mensah, Xudong Liu, and Yongyi Mao. 2022 b . Unsupervised sentence representation via contrastive learning with mixing negatives. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 11730--11738

    2022

  64. [72]

    Li Zheng, Boyu Chen, Hao Fei, Fei Li, Shengqiong Wu, Lizi Liao, Donghong Ji, and Chong Teng. 2024. Self-adaptive fine-grained multi-modal data augmentation for semi-supervised muti-modal coreference resolution. In Proceedings of the 32nd ACM International Conference on Multime...

    2024

  65. [73]

    Li Zheng, Hao Fei, Ting Dai, Zuquan Peng, Fei Li, Huisheng Ma, Chong Teng, and Donghong Ji. 2025. Multi-granular multimodal clue fusion for meme understanding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 26057--26065

    2025

  66. [74]

    Kun Zhou, Beichen Zhang, Xin Zhao, and Ji-Rong Wen. 2022. Debiased contrastive learning of unsupervised sentence representations. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6120--6130

    2022

  67. [75]

    Minghao Zhu, Xiao Lin, Ronghao Dang, Chengju Liu, and Qijun Chen. 2023. Fine-grained spatiotemporal motion alignment for contrastive video representation learning. In Proceedings of the 31st ACM International Conference on Multimedia, page 4725–4736

    2023

  68. [76]

    Wenjie Zhuo, Yifan Sun, Xiaohan Wang, Linchao Zhu, and Yi Yang. 2023. W hitened CSE : Whitening-based contrastive learning of sentence embeddings. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12135--12148

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.