pith. sign in

arxiv: 2605.16608 · v1 · pith:XRQACSIDnew · submitted 2026-05-15 · 💻 cs.LG · cs.CL

To MRL or not to MRL: Text Embeddings are Robust to Truncation Without Matryoshka Embeddings, Except In Heavy Truncation Scenarios

Sotaro Takeshita , Yurina Takeshita , Simone Paolo Ponzetto , Daniel Ruffinelli This is my paper

Pith reviewed 2026-05-20 20:12 UTC · model grok-4.3

classification 💻 cs.LG cs.CL
keywords text embeddingstruncation robustnessMatryoshka Representation LearningMRLembedding reductiondownstream tasksmodel training
0
0 comments X

The pith

Text embeddings from standard models stay competitive when truncated unless reduced by 80 percent or more, so MRL training is often unnecessary.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tests whether Matryoshka Representation Learning is required for text embeddings to remain useful after truncation to smaller sizes. It runs the same truncation schedule on both MRL-trained models and ordinary models across multiple encoders and tasks. Results show non-MRL embeddings perform as well as or better than MRL ones until truncation reaches at least 80 percent size reduction. A reader would care because this implies the extra training cost of MRL may not be justified unless very small vectors are required.

Core claim

By applying identical truncation schedules from MRL training to models trained with and without MRL, the experiments demonstrate that non-MRL embeddings are competitive with and frequently outperform MRL embeddings on downstream tasks when size reduction stays below 80 percent, indicating that truncation robustness arises from standard embedding training rather than from the MRL procedure itself.

What carries the argument

Identical truncation schedule taken from MRL training and applied to both MRL and non-MRL text embedding vectors.

If this is right

  • Standard embedding training suffices for most truncation levels without added MRL cost.
  • MRL training becomes relevant only when applications demand very heavy truncation.
  • Truncation robustness appears to be a general property of text embeddings rather than something MRL must instill.
  • Model selection can prioritize standard objectives when moderate-sized vectors meet needs.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Deployers of embedding systems could save training compute by skipping MRL unless extreme size reduction is planned.
  • The result invites similar tests on image or multimodal embeddings to check if robustness is modality-specific.
  • Practitioners might experiment with even simpler truncation methods on existing models to confirm the pattern holds.

Load-bearing premise

That applying the truncation sizes and method chosen for MRL creates a fair test of whether MRL training itself is needed for robustness.

What would settle it

Finding that non-MRL models underperform MRL models by a large margin at truncation levels below 80 percent reduction on the same tasks would disprove the central result.

Figures

Figures reproduced from arXiv: 2605.16608 by Daniel Ruffinelli, Simone Paolo Ponzetto, Sotaro Takeshita, Yurina Takeshita.

Figure 1
Figure 1. Figure 1: (Top) Robustness of open text encoders as truncation levels increase looks the same whether trained with or without MRL. (Bottom) When models differ only in their use of MRL, truncation on non-MRL models is superior unless heavy truncation is applied. more flexibility in this regard, Matryoshka Rep￾resentation Learning (MRL) (Kusupati et al., 2022) is an approach that adds additional terms to the training … view at source ↗
Figure 2
Figure 2. Figure 2: Performance on NanoBEIR (top) and MTEB (bottom) of text embeddings truncated at various sizes, relative to the performance of the corresponding full￾size embeddings. et al. (2025), as other aspects typically differen￾tiate new models from prior work, e.g. training recipe (Neelakantan et al., 2022; Sturua et al., 2024). This makes a proper comparison prohib￾itely expensive. However, we do conduct a more con… view at source ↗
Figure 4
Figure 4. Figure 4: Standard deviation across embedding dimen [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Validation loss curve for contrastive learning with and without MRL for all model pairs. Our training [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Absolute performance on NanoBEIR (top) and MTEB (bottom) of text embeddings by smaller models [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Absolute performance on NanoBEIR (top) and MTEB (bottom) of text embeddings by larger models [PITH_FULL_IMAGE:figures/full_fig_p011_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Performance on BEIR and MTEB benchmarks of five pairs of encoders trained with and without MRL. [PITH_FULL_IMAGE:figures/full_fig_p012_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Standard deviations of values taken by each dimension when encoding different texts. We observe that [PITH_FULL_IMAGE:figures/full_fig_p012_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Performance of smaller open text encoders in NanoBEIR datasets. [PITH_FULL_IMAGE:figures/full_fig_p013_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Performance of larger open text encoders in NanoBEIR datasets. [PITH_FULL_IMAGE:figures/full_fig_p013_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Performance of smaller open text encoders in MTEB datasets. [PITH_FULL_IMAGE:figures/full_fig_p014_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Performance of larger open text encoders in MTEB datasets. [PITH_FULL_IMAGE:figures/full_fig_p014_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: BERT base performance on each of the BEIR datasets. [PITH_FULL_IMAGE:figures/full_fig_p015_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: BERT large performance on each of the NanoBEIR datasets. [PITH_FULL_IMAGE:figures/full_fig_p015_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: RoBERTa base performance on each of the BEIR datasets. [PITH_FULL_IMAGE:figures/full_fig_p015_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: RoBERTa large performance on each of the NanoBEIR datasets. [PITH_FULL_IMAGE:figures/full_fig_p016_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: T5 base performance on each of the NanoBEIR datasets. [PITH_FULL_IMAGE:figures/full_fig_p016_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: BERT base performance on each of the MTEB datasets. [PITH_FULL_IMAGE:figures/full_fig_p016_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: BERT large performance on each of the MTEB datasets. [PITH_FULL_IMAGE:figures/full_fig_p017_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: RoBERTa base performance on each of the MTEB datasets. [PITH_FULL_IMAGE:figures/full_fig_p017_21.png] view at source ↗
Figure 22
Figure 22. Figure 22: RoBERTa large performance on each of the MTEB datasets. [PITH_FULL_IMAGE:figures/full_fig_p018_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: T5 base performance on each of the MTEB datasets. [PITH_FULL_IMAGE:figures/full_fig_p018_23.png] view at source ↗
read the original abstract

Matryoshka Representation Learning (MRL) is a widely adopted approach for training text encoders so they provide useful text representations at various sizes, available by simply truncating the resulting vectors at sizes pre-determined at training time. Recent works have shown that randomly truncating text embeddings has minimal impact in downstream performance unless vectors are reduced in size by at least 70%, suggesting that embeddings are already robust to truncation without the use of MRL. However, no prior work has compared random truncation to MRL, so it is unclear how the two methods compare as effective embedding reduction methods. In this paper, we study this by applying the same truncation used by MRL to models trained with and without MRL. Our results across several models and downstream tasks show that, unless heavily truncating embeddings (i.e. reducing their size by at least 80%), truncated embeddings of non-MRL models are competitive with, and often outperform models trained with MRL. This suggests that truncation robustness may not necessarily come from MRL, and that the choice of spending the additional training cost of MRL depends on whether heavy truncation is desired.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript examines whether Matryoshka Representation Learning (MRL) is required to produce truncation-robust text embeddings. By applying the identical truncation schedule used during MRL training to both MRL-trained and standard (non-MRL) text encoders across multiple models and downstream tasks, the authors report that non-MRL embeddings remain competitive with—and frequently outperform—MRL embeddings unless the embedding dimension is reduced by at least 80%. The central conclusion is that the extra training cost of MRL is justified only in heavy-truncation regimes.

Significance. If the empirical comparison holds after addressing the noted experimental gaps, the result would have clear practical value for embedding-model training pipelines: it indicates that standard contrastive or masked-language-model training already yields sufficient robustness for moderate truncation, thereby questioning the routine adoption of MRL when only modest size reduction is needed. The work also supplies a useful baseline for future studies on embedding compression and dimensionality.

major comments (2)
  1. [Section 3 (Experimental Setup) and Section 4 (Results)] The experimental design applies the MRL-derived truncation points (prefix cuts at the sizes chosen during MRL training) directly to non-MRL embeddings without an ablation that tests alternative dimension-selection strategies (e.g., variance-ranked or random selection) at the same target sizes. Because MRL explicitly optimizes nested representations for precisely those cutoffs, the observed competitiveness of non-MRL models could be an artifact of the schedule rather than intrinsic robustness; this directly affects the claim that MRL training itself is not required.
  2. [Section 4 and associated tables/figures] The abstract and results sections state that non-MRL truncated embeddings “often outperform” MRL models, yet the manuscript provides neither error bars nor statistical significance tests for the pairwise comparisons. Without these, it is difficult to assess whether the reported outperformance is reliable or within the noise of the evaluation.
minor comments (2)
  1. [Section 3.2] The description of the exact truncation percentages and the corresponding absolute dimensions (e.g., 768 → 128) should be tabulated for each model so readers can reproduce the reduction ratios precisely.
  2. [Figures 2–4] Figure captions would benefit from explicitly labeling which curves correspond to MRL versus non-MRL models and whether the plotted points reflect mean performance across seeds.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments on our work. We address the major concerns point by point below and have made revisions to the manuscript to improve clarity and rigor where appropriate.

read point-by-point responses

  1. Referee: [Section 3 (Experimental Setup) and Section 4 (Results)] The experimental design applies the MRL-derived truncation points (prefix cuts at the sizes chosen during MRL training) directly to non-MRL embeddings without an ablation that tests alternative dimension-selection strategies (e.g., variance-ranked or random selection) at the same target sizes. Because MRL explicitly optimizes nested representations for precisely those cutoffs, the observed competitiveness of non-MRL models could be an artifact of the schedule rather than intrinsic robustness; this directly affects the claim that MRL training itself is not required.

    Authors: We chose to apply the MRL truncation schedule to non-MRL embeddings precisely to perform a controlled comparison at the dimensions for which MRL provides optimized representations. This setup directly tests whether the additional MRL training objective is necessary to achieve good performance at those specific sizes. If non-MRL embeddings perform competitively even when truncated at MRL's chosen cutoffs, it suggests that the robustness is largely intrinsic to standard training rather than dependent on MRL's nested optimization. Alternative selection strategies such as variance-based ranking would address a different question—namely, how to best truncate a fixed non-MRL embedding—rather than whether MRL training is required. We have added a clarifying paragraph in Section 3 of the revised manuscript to better articulate this experimental rationale and its relation to our central claim. revision: partial

  2. Referee: [Section 4 and associated tables/figures] The abstract and results sections state that non-MRL truncated embeddings “often outperform” MRL models, yet the manuscript provides neither error bars nor statistical significance tests for the pairwise comparisons. Without these, it is difficult to assess whether the reported outperformance is reliable or within the noise of the evaluation.

    Authors: We agree that the lack of error bars and statistical tests limits the strength of the outperformance claims. In the revised manuscript, we have added error bars representing standard deviation across multiple random seeds or evaluation runs to all relevant figures and tables. Additionally, we have included results of statistical significance tests (e.g., paired t-tests) for the key comparisons between MRL and non-MRL at each truncation level. These updates confirm that the reported advantages of non-MRL embeddings in moderate truncation regimes are statistically significant in the majority of cases. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical comparison without derivation or self-referential structure

full rationale

The paper advances an empirical claim based on direct head-to-head experiments that apply the same truncation schedule to both MRL-trained and non-MRL models across multiple encoders and downstream tasks. No equations, fitted parameters renamed as predictions, or self-definitional steps appear in the abstract or described method. The central result—that non-MRL truncated embeddings remain competitive except under heavy (>80%) truncation—is presented as an observation from those comparisons rather than a quantity derived from prior outputs of the same model. Any self-citations to the original MRL work are external and non-load-bearing; the present study does not invoke uniqueness theorems or ansatzes from the authors' own prior publications to justify its conclusions. The argument is therefore self-contained against external benchmarks and receives the default non-circularity finding.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The claim rests on standard machine-learning experimental assumptions rather than new free parameters, axioms, or invented entities.

axioms (1)
  • domain assumption Standard assumptions in machine learning about fair model comparison and downstream task evaluation
    The paper relies on typical practices for training embeddings and measuring performance on downstream tasks.

pith-pipeline@v0.9.0 · 5758 in / 1181 out tokens · 71056 ms · 2026-05-20T20:12:28.042855+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

74 extracted references · 74 canonical work pages · 6 internal anchors

  1. [1]

    Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics

    LEAF: Knowledge Distillation of Text Embedding Models with Teacher-Aligned Representations , author=. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics

    work page

  2. [2]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , year=

    Context is gold to find the gold passage: Evaluating and training contextual document embeddings , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , year=

    work page 2025

  3. [3]

    Text and Code Embeddings by Contrastive Pre-Training

    Text and code embeddings by contrastive pre-training. arXiv , author=. arXiv preprint arXiv:2201.10005 , pages=

    work page internal anchor Pith review Pith/arXiv arXiv

  4. [4]

    jina-embeddings-v5-text: Task-Targeted Embedding Distillation

    jina-embeddings-v5-text: Task-Targeted Embedding Distillation , author=. arXiv preprint arXiv:2602.15547 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  5. [5]

    Findings of the Association for Computational Linguistics: EMNLP 2025 , pages=

    Do We Really Need All Those Dimensions? An Intrinsic Evaluation Framework for Compressed Embeddings , author=. Findings of the Association for Computational Linguistics: EMNLP 2025 , pages=

    work page 2025

  6. [6]

    Journal of computer and System Sciences , pages=

    Database-friendly random projections: Johnson-Lindenstrauss with binary coins , author=. Journal of computer and System Sciences , pages=

    work page

  7. [7]

    Contemporary mathematics , pages=

    Extensions of Lipschitz mappings into a Hilbert space , author=. Contemporary mathematics , pages=

    work page

  8. [8]

    Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages =

    2D Matryoshka Training for Information Retrieval , author =. Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages =

    work page

  9. [9]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    Matryoshka-adaptor: Unsupervised and supervised tuning for smaller embedding dimensions , author=. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    work page 2024

  10. [10]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    SMEC: Rethinking Matryoshka Representation Learning for Retrieval Embedding Compression , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    work page 2025

  11. [11]

    ACM Transactions on Information Systems , volume=

    Dense text retrieval based on pretrained language models: A survey , author=. ACM Transactions on Information Systems , volume=. 2024 , publisher=

    work page 2024

  12. [12]

    arXiv preprint arXiv:2310.18608 , year=

    Embedding in recommender systems: A survey , author=. arXiv preprint arXiv:2310.18608 , year=

    work page arXiv

  13. [13]

    Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , pages=

    Embedding-based retrieval in facebook search , author=. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , pages=

    work page

  14. [14]

    EmbeddingGemma: Powerful and Lightweight Text Representations

    Embeddinggemma: Powerful and lightweight text representations , author=. arXiv preprint arXiv:2509.20354 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  15. [15]

    arXiv preprint arXiv:2409.10173

    jina-embeddings-v3: Multilingual embeddings with task lora , author=. arXiv preprint arXiv:2409.10173 , year=

    work page arXiv

  16. [16]

    Advances in Neural Information Processing Systems , volume=

    Matryoshka representation learning , author=. Advances in Neural Information Processing Systems , volume=

    work page

  17. [17]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    Randomly Removing 50\ author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    work page 2025

  18. [18]

    Retrieval of the Best Counterargument without Prior Topic Knowledge

    Wachsmuth, Henning and Syed, Shahbaz and Stein, Benno. Retrieval of the Best Counterargument without Prior Topic Knowledge. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2018. doi:10.18653/v1/P18-1023

    work page doi:10.18653/v1/p18-1023 2018

  19. [19]

    ArXiv abs/2004.07180 (2020)

    Cohan, Arman and Feldman, Sergey and Beltagy, Iz and Downey, Doug and Weld, Daniel. SPECTER : Document-level Representation Learning using Citation-informed Transformers. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020. doi:10.18653/v1/2020.acl-main.207

    work page doi:10.18653/v1/2020.acl-main.207 2020

  20. [20]

    FEVER: a large-scale dataset for Fact Extraction and VERification

    Thorne, James and Vlachos, Andreas and Christodoulopoulos, Christos and Mittal, Arpit. FEVER : a Large-scale Dataset for Fact Extraction and VER ification. Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018. doi:10.18653/v1/N18-1074

    work page internal anchor Pith review doi:10.18653/v1/n18-1074 2018

  21. [21]

    Fact or Fiction: Verifying Scientific Claims

    Wadden, David and Lin, Shanchuan and Lo, Kyle and Wang, Lucy Lu and van Zuylen, Madeleine and Cohan, Arman and Hajishirzi, Hannaneh. Fact or Fiction: Verifying Scientific Claims. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020. doi:10.18653/v1/2020.emnlp-main.609

    work page doi:10.18653/v1/2020.emnlp-main.609 2020

  22. [22]

    Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov

    Kwiatkowski, Tom and Palomaki, Jennimaria and Redfield, Olivia and Collins, Michael and Parikh, Ankur and Alberti, Chris and Epstein, Danielle and Polosukhin, Illia and Devlin, Jacob and Lee, Kenton and Toutanova, Kristina and Jones, Llion and Kelcey, Matthew and Chang, Ming-Wei and Dai, Andrew M. and Uszkoreit, Jakob and Le, Quoc and Petrov, Slav. Natura...

    work page doi:10.1162/tacl_a_00276 2019

  23. [23]

    Cohen and Ruslan Salakhutdinov and Christopher D

    Yang, Zhilin and Qi, Peng and Zhang, Saizheng and Bengio, Yoshua and Cohen, William and Salakhutdinov, Ruslan and Manning, Christopher D. H otpot QA : A Dataset for Diverse, Explainable Multi-hop Question Answering. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018. doi:10.18653/v1/D18-1259

    work page doi:10.18653/v1/d18-1259 2018

  24. [24]

    Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , doi =

    O. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , doi =

    work page 2021

  25. [25]

    The Multilingual A mazon Reviews Corpus

    Keung, Phillip and Lu, Yichao and Szarvas, Gy. The Multilingual A mazon Reviews Corpus. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020. doi:10.18653/v1/2020.emnlp-main.369

    work page doi:10.18653/v1/2020.emnlp-main.369 2020

  26. [26]

    Efficient Intent Detection with Dual Sentence Encoders , url =

    Casanueva, I. Efficient Intent Detection with Dual Sentence Encoders , url =. Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI , doi =

    work page

  27. [27]

    doi:10.18653/v1/D18-1404 , editor =

    Saravia, Elvis and Liu, Hsien-Chi Toby and Huang, Yen-Hao and Wu, Junlin and Chen, Yi-Shin , booktitle =. doi:10.18653/v1/D18-1404 , editor =

    work page doi:10.18653/v1/d18-1404

  28. [28]

    and Daly, Raymond E

    Maas, Andrew L. and Daly, Raymond E. and Pham, Peter T. and Huang, Dan and Ng, Andrew Y. and Potts, Christopher , booktitle =. Learning Word Vectors for Sentiment Analysis , url =

    work page

  29. [29]

    MASSIVE : A 1 M -Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages

    FitzGerald, Jack and Hench, Christopher and Peris, Charith and Mackie, Scott and Rottmann, Kay and Sanchez, Ana and Nash, Aaron and Urbach, Liam and Kakarala, Vishesh and Singh, Richa and Ranganath, Swetha and Crist, Laurie and Britan, Misha and Leeuwis, Wouter and Tur, Gokhan and Natarajan, Prem. MASSIVE : A 1 M -Example Multilingual Natural Language Und...

    work page doi:10.18653/v1/2023.acl-long.235 2023

  30. [30]

    doi:10.18653/v1/2021.eacl-main.257 , editor =

    Li, Haoran and Arora, Abhinav and Chen, Shuohui and Gupta, Anchit and Gupta, Sonal and Mehdad, Yashar , booktitle =. doi:10.18653/v1/2021.eacl-main.257 , editor =

    work page doi:10.18653/v1/2021.eacl-main.257 2021

  31. [31]

    BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding

    Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina. BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019. doi:10.18653/v...

    work page doi:10.18653/v1/n19-1423 2019

  32. [32]

    Yinhan Liu and Myle Ott and Naman Goyal and Jingfei Du and Mandar Joshi and Danqi Chen and Omer Levy and Mike Lewis and Luke Zettlemoyer and Veselin Stoyanov , year=. Ro

    work page

  33. [33]

    , title =

    Raffel, Colin and Shazeer, Noam and Roberts, Adam and Lee, Katherine and Narang, Sharan and Matena, Michael and Zhou, Yanqi and Li, Wei and Liu, Peter J. , title =. J. Mach. Learn. Res. , month = jan, articleno =. 2020 , issue_date =

    work page 2020

  34. [34]

    Hall, Daniel Cer, and Yinfei Yang

    Ni, Jianmo and Hernandez Abrego, Gustavo and Constant, Noah and Ma, Ji and Hall, Keith and Cer, Daniel and Yang, Yinfei. Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models. Findings of the Association for Computational Linguistics: ACL 2022. 2022. doi:10.18653/v1/2022.findings-acl.146

    work page doi:10.18653/v1/2022.findings-acl.146 2022

  35. [35]

    Sentence- BERT : Sentence Embeddings using S iamese BERT -Networks

    Reimers, Nils and Gurevych, Iryna. Sentence- BERT : Sentence Embeddings using S iamese BERT -Networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019. doi:10.18653/v1/D19-1410

    work page doi:10.18653/v1/d19-1410 2019

  36. [36]

    S im CSE : Simple Contrastive Learning of Sentence Embeddings

    Gao, Tianyu and Yao, Xingcheng and Chen, Danqi. S im CSE : Simple Contrastive Learning of Sentence Embeddings. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021. doi:10.18653/v1/2021.emnlp-main.552

    work page doi:10.18653/v1/2021.emnlp-main.552 2021

  37. [37]

    Enhancing Retrieval-Augmented Generation: A Study of Best Practices

    Li, Siran and Stenzel, Linus and Eickhoff, Carsten and Bahrainian, Seyed Ali. Enhancing Retrieval-Augmented Generation: A Study of Best Practices. Proceedings of the 31st International Conference on Computational Linguistics. 2025

    work page 2025

  38. [38]

    Dense Passage Retrieval for Open-Domain Question Answering

    Karpukhin, Vladimir and Oguz, Barlas and Min, Sewon and Lewis, Patrick and Wu, Ledell and Edunov, Sergey and Chen, Danqi and Yih, Wen-tau. Dense Passage Retrieval for Open-Domain Question Answering. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020. doi:10.18653/v1/2020.emnlp-main.550

    work page doi:10.18653/v1/2020.emnlp-main.550 2020

  39. [39]

    Improving Embedding-based Large-scale Retrieval via Label Enhancement

    Liu, Peiyang and Wang, Xi and Wang, Sen and Ye, Wei and Xi, Xiangyu and Zhang, Shikun. Improving Embedding-based Large-scale Retrieval via Label Enhancement. Findings of the Association for Computational Linguistics: EMNLP 2021. 2021. doi:10.18653/v1/2021.findings-emnlp.13

    work page doi:10.18653/v1/2021.findings-emnlp.13 2021

  40. [40]

    What`s in Your Embedding, And How It Predicts Task Performance

    Rogers, Anna and Hosur Ananthakrishna, Shashwath and Rumshisky, Anna. What`s in Your Embedding, And How It Predicts Task Performance. Proceedings of the 27th International Conference on Computational Linguistics. 2018

    work page 2018

  41. [41]

    Lepori, Michael and McCoy, R. Thomas. Picking BERT `s Brain: Probing for Linguistic Dependencies in Contextualized Embeddings Using Representational Similarity Analysis. Proceedings of the 28th International Conference on Computational Linguistics. 2020. doi:10.18653/v1/2020.coling-main.325

    work page doi:10.18653/v1/2020.coling-main.325 2020

  42. [42]

    Do Neural Language Models Show Preferences for Syntactic Formalisms?

    Kulmizev, Artur and Ravishankar, Vinit and Abdou, Mostafa and Nivre, Joakim. Do Neural Language Models Show Preferences for Syntactic Formalisms?. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020. doi:10.18653/v1/2020.acl-main.375

    work page doi:10.18653/v1/2020.acl-main.375 2020

  43. [43]

    A Structural Probe for Finding Syntax in Word Representations

    Hewitt, John and Manning, Christopher D. A Structural Probe for Finding Syntax in Word Representations. Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019. doi:10.18653/v1/N19-1419

    work page doi:10.18653/v1/n19-1419 2019

  44. [44]

    On Isotropy, Contextualization and Learning Dynamics of Contrastive-based Sentence Representation Learning

    Xiao, Chenghao and Long, Yang and Al Moubayed, Noura. On Isotropy, Contextualization and Learning Dynamics of Contrastive-based Sentence Representation Learning. Findings of the Association for Computational Linguistics: ACL 2023. 2023. doi:10.18653/v1/2023.findings-acl.778

    work page doi:10.18653/v1/2023.findings-acl.778 2023

  45. [45]

    International Conference on Learning Representations , year=

    Understanding Dimensional Collapse in Contrastive Self-supervised Learning , author=. International Conference on Learning Representations , year=

    work page

  46. [46]

    Improving Text Embeddings with Large Language Models

    Wang, Liang and Yang, Nan and Huang, Xiaolong and Yang, Linjun and Majumder, Rangan and Wei, Furu. Improving Text Embeddings with Large Language Models. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024. doi:10.18653/v1/2024.acl-long.642

    work page doi:10.18653/v1/2024.acl-long.642 2024

  47. [47]

    Anisotropy is Not Inherent to Transformers

    Machina, Anemily and Mercer, Robert. Anisotropy is Not Inherent to Transformers. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024. doi:10.18653/v1/2024.naacl-long.274

    work page doi:10.18653/v1/2024.naacl-long.274 2024

  48. [48]

    Qwen2.5: A Party of Foundation Models , url =

    Qwen Team , month =. Qwen2.5: A Party of Foundation Models , url =

    work page

  49. [49]

    doi:10.5281/zenodo.12608602 , url =

    Gao, Leo and Tow, Jonathan and Abbasi, Baber and Biderman, Stella and Black, Sid and DiPofi, Anthony and Foster, Charles and Golding, Laurence and Hsu, Jeffrey and Le Noac'h, Alain and Li, Haonan and McDonell, Kyle and Muennighoff, Niklas and Ociepa, Chris and Phang, Jason and Reynolds, Laria and Schoelkopf, Hailey and Skowron, Aviya and Sutawika, Lintang...

    work page doi:10.5281/zenodo.12608602

  50. [50]

    2016 , month =

    Nguyen, Tri and Rosenberg, Mir and Song, Xia and Gao, Jianfeng and Tiwary, Saurabh and Majumder, Rangan and Deng, Li , title =. 2016 , month =

    work page 2016

  51. [51]

    and Lo, Kyle and Roberts, Kirk and Soboroff, Ian and Wang, Lucy Lu , title =

    Voorhees, Ellen and Alam, Tasmeer and Bedrick, Steven and Demner-Fushman, Dina and Hersh, William R. and Lo, Kyle and Roberts, Kirk and Soboroff, Ian and Wang, Lucy Lu , title =. SIGIR Forum , month =. 2021 , issue_date =. doi:10.1145/3451964.3451965 , abstract =

    work page doi:10.1145/3451964.3451965 2021

  52. [52]

    Advances in Information Retrieval: 38th European Conference on IR Research, ECIR 2016, Padua, Italy, March 20--23, 2016

    A full-text learning to rank dataset for medical information retrieval , author=. Advances in Information Retrieval: 38th European Conference on IR Research, ECIR 2016, Padua, Italy, March 20--23, 2016. Proceedings 38 , pages=. 2016 , organization=

    work page 2016

  53. [53]

    WWW'18 Open Challenge: Financial Opinion Mining and Question Answering , year =

    Maia, Macedo and Handschuh, Siegfried and Freitas, Andr\'. WWW'18 Open Challenge: Financial Opinion Mining and Question Answering , year =. Companion Proceedings of the The Web Conference 2018 , pages =. doi:10.1145/3184558.3192301 , abstract =

    work page doi:10.1145/3184558.3192301 2018

  54. [54]

    Overview of Touch

    Bondarenko, Alexander and Fr. Overview of Touch. Experimental IR Meets Multilinguality, Multimodality, and Interaction: 11th International Conference of the CLEF Association, CLEF 2020, Thessaloniki, Greece, September 22--25, 2020, Proceedings 11 , pages=. 2020 , organization=

    work page 2020

  55. [55]

    Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages =

    Hasibi, Faegheh and Nikolaev, Fedor and Xiong, Chenyan and Balog, Krisztian and Bratsberg, Svein Erik and Kotov, Alexander and Callan, Jamie , title =. Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages =. 2017 , isbn =. doi:10.1145/3077136.3080751 , abstract =

    work page doi:10.1145/3077136.3080751 2017

  56. [56]

    NeurIPS 2020 Workshop on Tackling Climate Change with Machine Learning , url=

    Climate-FEVER: A Dataset for Verification of Real-World Climate Claims , author=. NeurIPS 2020 Workshop on Tackling Climate Change with Machine Learning , url=

    work page 2020

  57. [57]

    Proceedings of the 7th ACM Conference on Recommender Systems , pages =

    McAuley, Julian and Leskovec, Jure , title =. Proceedings of the 7th ACM Conference on Recommender Systems , pages =. 2013 , isbn =. doi:10.1145/2507157.2507163 , abstract =

    work page doi:10.1145/2507157.2507163 2013

  58. [58]

    2019 , howpublished =

    cjadams and Daniel Borkan and inversion and Jeffrey Sorensen and Lucas Dixon and Lucy Vasserman and nithum , title =. 2019 , howpublished =

    work page 2019

  59. [59]

    2020 , howpublished =

    Maggie and Phil Culliton and Wei Chen , title =. 2020 , howpublished =

    work page 2020

  60. [60]

    Redundancy, Isotropy, and Intrinsic Dimensionality of Prompt-based Text Embeddings

    Tsukagoshi, Hayato and Sasano, Ryohei. Redundancy, Isotropy, and Intrinsic Dimensionality of Prompt-based Text Embeddings. Findings of the Association for Computational Linguistics: ACL 2025. 2025. doi:10.18653/v1/2025.findings-acl.1330

    work page doi:10.18653/v1/2025.findings-acl.1330 2025

  61. [61]

    Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2) , year=

    Nandan Thakur and Nils Reimers and Andreas R. Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2) , year=

    work page

  62. [62]

    MTEB : Massive text embedding benchmark

    Muennighoff, Niklas and Tazi, Nouamane and Magne, Loic and Reimers, Nils. MTEB : Massive Text Embedding Benchmark. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. 2023. doi:10.18653/v1/2023.eacl-main.148

    work page doi:10.18653/v1/2023.eacl-main.148 2023

  63. [63]

    2017 , eprint=

    Efficient Natural Language Response Suggestion for Smart Reply , author=. 2017 , eprint=

    work page 2017

  64. [64]

    Probing Cross-Lingual Lexical Knowledge from Multilingual Sentence Encoders

    Vuli \'c , Ivan and Glava s , Goran and Liu, Fangyu and Collier, Nigel and Ponti, Edoardo Maria and Korhonen, Anna. Probing Cross-Lingual Lexical Knowledge from Multilingual Sentence Encoders. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. 2023. doi:10.18653/v1/2023.eacl-main.153

    work page doi:10.18653/v1/2023.eacl-main.153 2023

  65. [65]

    Margins in Contrastive Learning: Evaluating Multi-task Retrieval for Sentence Embeddings

    J rgensen, Tollef Emil and Breitung, Jens. Margins in Contrastive Learning: Evaluating Multi-task Retrieval for Sentence Embeddings. Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025). 2025

    work page 2025

  66. [66]

    A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference

    Williams, Adina and Nangia, Nikita and Bowman, Samuel. A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference. Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018. doi:10.18653/v1/N18-1101

    work page internal anchor Pith review doi:10.18653/v1/n18-1101 2018

  67. [67]

    and Angeli, Gabor and Potts, Christopher and Manning, Christopher D

    Bowman, Samuel R. and Angeli, Gabor and Potts, Christopher and Manning, Christopher D. A large annotated corpus for learning natural language inference. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2015. doi:10.18653/v1/D15-1075

    work page doi:10.18653/v1/d15-1075 2015

  68. [68]

    Matryoshka Representation Learning , url =

    Kusupati, Aditya and Bhatt, Gantavya and Rege, Aniket and Wallingford, Matthew and Sinha, Aditya and Ramanujan, Vivek and Howard-Snyder, William and Chen, Kaifeng and Kakade, Sham and Jain, Prateek and Farhadi, Ali , booktitle =. Matryoshka Representation Learning , url =

    work page

  69. [69]

    Zhao, Yi Luan, Keith B

    Ni, Jianmo and Qu, Chen and Lu, Jing and Dai, Zhuyun and Hernandez Abrego, Gustavo and Ma, Ji and Zhao, Vincent and Luan, Yi and Hall, Keith and Chang, Ming-Wei and Yang, Yinfei. Large Dual Encoders Are Generalizable Retrievers. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022. doi:10.18653/v1/2022.emnlp-main.669

    work page doi:10.18653/v1/2022.emnlp-main.669 2022

  70. [70]

    2025 , eprint=

    Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models , author=. 2025 , eprint=

    work page 2025

  71. [71]

    2026 , eprint=

    Diffusion-Pretrained Dense and Contextual Embeddings , author=. 2026 , eprint=

    work page 2026

  72. [72]

    Multilingual E5 Text Embeddings: A Technical Report

    Multilingual E5 Text Embeddings: A Technical Report , author=. arXiv preprint arXiv:2402.05672 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  73. [73]

    The Thirteenth International Conference on Learning Representations , year=

    Scaling Diffusion Language Models via Adaptation from Autoregressive Models , author=. The Thirteenth International Conference on Learning Representations , year=

    work page

  74. [74]

    2025 , eprint=

    Encoder-Decoder Gemma: Improving the Quality-Efficiency Trade-Off via Adaptation , author=. 2025 , eprint=

    work page 2025