Pith. sign in

REVIEW 3 major objections 6 minor 133 references

This paper shows that late-interaction retrieval transfers across languages far better than dense retrieval when both are trained on the same translated data, and argues that token-level matching makes translate-train a general multilingual

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 01:37 UTC pith:SX2DONPT

load-bearing objection A genuinely open English retrieval recipe and two strong models, plus a striking multilingual result—but the headline claim that late interaction transfers better is not yet isolated from pooling, scoring, dimensionality, and objective confounds. the 3 major comments →

arxiv 2607.27178 v2 pith:SX2DONPT submitted 2026-07-29 cs.CL cs.IR

DenseOn with the LateOn: Fully Open Dense and Late-Interaction Models for Multilingual, Long-Context, and Code Search

Rapha\"el Sourty , Antoine Chaffin , Paulo Roberto Moura Junior , Am\'elie Chatelain This is my paper
classification cs.CL cs.IR
keywords retrievallate interactiondense retrievaltranslate-trainmultilingualcross-lingual transferBEIRMIRACL
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper builds fully open English and multilingual retrieval models and compares dense single-vector versus late-interaction (ColBERT-style) architectures under matched training data, backbone, and objectives. It finds that the dense model performs well on English and the eight languages it was translated into, but degrades sharply on unseen languages and scripts. In contrast, the late-interaction model generalizes much more strongly, scoring 67.04 versus 58.02 nDCG@10 on the full MIRACL benchmark, and 77.92 versus 51.59 on the long-document MLDR benchmark. The authors argue this shows token-level matching preserves the multilingual structure learned during language-model pre-training, turning translate-train from a target-language expansion method into a path toward broader multilingual retrieval. The paper also contributes large open datasets — 665M English pairs, 1.88M fine-tuning pairs, 2.8B multilingual pairs — and releases all models and code.

Core claim

When English retrieval supervision is translated into eight languages and used to train two models on the same mmBERT-base backbone with the same data and objectives, the single-vector dense model is strong inside the translated-language support but loses performance on languages and scripts unseen during retrieval training, whereas the late-interaction model transfers far more effectively to those unseen languages. This is the paper's central claim: late interaction generalizes beyond translate-train targets, so token-level matching can turn translate-train into a multilingual generalization recipe rather than just a data-expansion strategy.

What carries the argument

The key comparison is between dense single-vector retrieval, which pools the entire sequence into one CLS vector and scores with cosine similarity, and late-interaction retrieval, which encodes query and document per-token and scores with MeanMaxSim — the mean over query tokens of each token's maximum similarity to any document token. Because both models share the same backbone, training data, and loss, the observed transfer gap is attributed to this architectural difference.

Load-bearing premise

The paper attributes the observed generalization gap to the dense-versus-late-interaction paradigm, but the two models also differ in several training details — CLS pooling versus per-token encoding, MaxSim versus MeanMaxSim scoring, 768-dim versus 128-dim vectors, and Matryoshka training applied only to the dense model — and these details are not ablated independently; if any of them, rather than the paradigm itself, drives the difference, the central conclusion is weakened.

What would settle it

Train matched dense and late-interaction models that differ only in the retrieval paradigm: give the dense model a per-token representation with the same dimensionality and MeanMaxSim scoring, or give the late-interaction model CLS pooling and cosine scoring, while keeping all other training settings identical. If the transfer gap disappears or inverts, the paper's conclusion is falsified; if it persists, the paradigm attribution is supported.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the finding holds, multilingual retrieval systems can be built without exhaustively translating supervision into every target language; translating into a small set of languages may suffice for late-interaction models.
  • The open data recipe enables other researchers to audit, modify, and extend the training mixture without recollecting data, reducing the reproducibility gap between open and closed retrieval systems.
  • The decontamination results on BEIR, where late-interaction models hold or improve their rank after removing overlapping training examples while dense models drop, suggest that multi-vector matching may be inherently less sensitive to training-data overlap.
  • The strong code-retrieval performance from fine-tuning only, without code-specific pre-training, suggests that late interaction can be a cost-effective way to add code search capability to existing multilingual models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If late interaction preserves the multilingual structure of the backbone, a similar transfer advantage should appear in other token-level matching tasks beyond retrieval, such as clustering or classification over unseen languages — a test that does not require new data collection.
  • The paper's conclusion implies that for low-resource languages, a small amount of organic or translated data may go further for late-interaction models than for dense models, which could change how translation budgets are allocated in practice.
  • The BEIR decontamination pattern suggests the multilingual transfer gap may be a special case of a broader robustness property: multi-vector representations may be less reliant on exact lexical or surface overlap with training data, which would have implications for domain shift beyond language transfer.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper presents a fully open recipe for training dense and late-interaction retrieval models on English data, then extends it via translate-train to eight languages. It introduces DENSEON and LATEON (149M parameters) that achieve 56.20 and 57.22 nDCG@10 on BEIR, respectively, and multilingual mDENSEON and mLATEON (307M parameters) that are evaluated on BEIR, MIRACL, MLDR, and MTEB Code. The central claim is that late-interaction retrieval generalizes much more effectively than dense retrieval to languages and scripts unseen during retrieval training, based on a 9-point gap on full MIRACL (67.04 vs 58.02) and a 26-point gap on full MLDR (77.92 vs 51.59). The paper releases the 1.4B-pair annotated corpus, 665M-pair curated mixture, 1.88M-pair fine-tuning set, models, and code.

Significance. The open release of data, models, and training code is a substantial contribution to the reproducibility of retrieval systems, and the non-destructive filtering pipeline is a useful design choice. The decontamination analysis, per-language results, and mixture ablations are valuable. If the dense-versus-late-interaction transfer finding is robust, it would have clear practical implications for multilingual retrieval architecture selection. However, the comparison that supports the central claim is confounded by several model-design differences, and the English BEIR results are selected on the test benchmark. These issues require attention before the headline conclusion can be accepted.

major comments (3)
  1. [§2.3, §D Table 11, §3.2] The controlled comparison does not isolate the retrieval paradigm. Table 11 lists different objectives (Dense: Matryoshka InfoNCE + KL-div; Late: InfoNCE + KL-div), different scoring (CLS pooling + cosine over 768-dim vectors vs per-token 128-dim MeanMaxSim), and different pretraining context lengths (1,024 tokens vs 32/300 tokens). The statement in §2.3 that 'Apart from these details and the retrieval paradigm, the models share the same backbone, training data, and objectives' is contradicted by the paper's own hyperparameter table. The ablations in C.2 and C.3 test MeanMaxSim and Matryoshka only on English BEIR, not on the unseen-language MIRACL/MLDR settings where the central claim lives, and pooling is not ablated at all. Therefore the 9-point full-MIRACL gap (67.04 vs 58.02, Table 5) and the 26-point MLDR gap (77.92 vs 51.59, Table 6) cannot yet be attributed to token-level matching
  2. [§3.1] The English pre-training mixture was selected by 'a full-pipeline search over filtering thresholds and mixture compositions' in which 'each candidate recipe is evaluated after supervised fine-tuning on the full BEIR suite.' This is selection on the test benchmark: the reported BEIR averages (56.20, 57.22) are in-sample numbers, so the 'new state-of-the-art' claim for the 149M size class is not a held-out evaluation. The decontamination analysis in Table 1 does not correct for this selection bias; it only removes overlapping examples. Please report results on a held-out subset of BEIR or otherwise quantify the selection bias, and clearly label the current numbers as model-selection results.
  3. [§3.2, Table 5] The prose characterizes mDENSEON as 'strong on English and translated languages but degrades outside translate-train support.' The per-language results in Table 5 are more heterogeneous: mDENSEON scores 70.0 on Yoruba and 67.5 on Thai, both unseen during retrieval training, exceeding its scores on several target languages (French 56.5, German 54.0). The average gap is real, but the claim of a systematic single-vector bottleneck is overstated. Please qualify the generalization pattern and discuss the role of backbone coverage or other per-language factors.
minor comments (6)
  1. [§2.2.2] The footnote 'This model was not available at the time of the pre-training translation' is ambiguous: does it refer to Qwen3-32B or to an earlier translation model? Please rephrase.
  2. [Table 1] The 'BEIR 14 Rank' column is not self-explanatory. Define that the average is computed over 14 datasets with CQADupstack excluded, and explain why this exclusion is appropriate here.
  3. [Appendix C.4, Table 9] The 'Overall' column mixes pre-training and fine-tuned scores without a clear weighting; please spell out the aggregation or omit it.
  4. [Figures 1 and 2] The stacked-bar legends are dense and hard to read in print. Consider providing the same information in tabular form in the appendix.
  5. [Abstract & Introduction] The abstract reports BEIR scores for the English models without mentioning the selection procedure; a brief caveat (e.g., 'after recipe selection on the full BEIR suite') would improve accuracy.
  6. [References] Minor formatting artifacts appear in the reference list (e.g., 'V oyage', 'T ouché-2020', 'V an'). A final proofread is needed.

Circularity Check

0 steps flagged

No circularity: the paper's central claims are empirical comparisons, not derivations from fitted inputs or self-citation chains.

full rationale

The paper makes no analytical derivation whose output reduces to its inputs by construction. Its claims are empirical: DenseOn/LateOn and mDenseOn/mLateOn are trained on released data and scored on held-out or partially unseen benchmarks. The central generalization claim — that late interaction transfers better than single-vector dense retrieval to languages and scripts unseen during retrieval training — is supported by full-MIRACL and full-MLDR evaluations on languages outside the translate-train set (Tables 2, 5, 6), and those numbers are not fitted parameters implied by construction from the training objectives. The self-citations (ColBERT-Zero, GTE-ModernColBERT, PyLate, FastPlaid, LateOn-Code) appear as baselines, tools, or data sources rather than as the load-bearing evidence for the architecture comparison; the evidence is the paper's own controlled training runs. Tuning the data mixture and filtering thresholds on BEIR and MIRACL-tgt before reporting final scores is benchmark selection rather than circular derivation, and the main claim is evaluated on languages not used in that selection. The mDenseOn/mLateOn comparison is confounded by different pooling, scoring, embedding dimensions, context lengths, and Matryoshka loss (Table 11), but confounding is a validity threat, not a circularity; no quoted reduction of the conclusion to its inputs exists. Therefore no circular steps are identified.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

The paper introduces no new physical or theoretical entities. Its free parameters are ML recipe hyperparameters, several of which were tuned on the same benchmarks used for reporting. The key axioms are standard domain assumptions about benchmark validity, cross-encoder trust, translation quality, and the matched-comparison setup.

free parameters (6)
  • Cross-encoder relevance threshold (mxbai-rerank-large-v2 score) = 3.0
    Absolute threshold chosen to match the original mGTE mixture as closely as possible (Section 2.1.1); affects which pairs survive semantic filtering and hence all downstream performance.
  • FineWeb-Edu percentile threshold = ~top third (33%)
    Applied to the FineWeb-Edu component, which comprises over half the mixture; threshold set by hand without separate validation (Section 2.1.1).
  • English share in pre-training mixture = 20%
    Selected via ablation on BEIR/MIRACL/MLDR after fine-tuning (Appendix C.4). The paper notes performance differences between mixtures are small, making the choice a tuned constant.
  • Cross-lingual share within multilingual split = 25%
    Selected via ablation on the same benchmarks (Appendix C.4).
  • KL distillation temperatures (teacher/student) = teacher 0.1; student 0.001 (LI) / 0.02 (dense)
    Tuned on BEIR in an ablation (Appendix C.1).
  • NV-Retriever negative filtering threshold = 0.95
    Used to retain top-10 negatives per query in fine-tuning; set without separate validation (Section 2.3.2).
axioms (5)
  • domain assumption BEIR, MIRACL, MLDR, and MTEB Code are valid, unbiased measures of retrieval quality for the systems compared.
    All performance claims are made via these benchmarks; if benchmark overlap or task-specific artifacts drive scores, the conclusions weaken.
  • domain assumption mxbai-rerank-large-v2 cross-encoder provides reliable relevance labels for filtering and distillation.
    Used both as a filter (Section 2.1.1) and as the KD teacher (Section 2.3.2); errors or training-set contamination in the teacher would transfer into the student.
  • domain assumption Machine translation (Mistral-Small-3.1-24B and Qwen3-32B) preserves enough semantic fidelity for contrastive learning across the eight target languages.
    The entire multilingual pre-training corpus depends on this; the paper acknowledges this in the Limitations section.
  • ad hoc to paper The dense and late-interaction models are otherwise identical in backbone, data, objectives, and training protocol, so the observed differences stem from the retrieval paradigm.
    The comparison's validity rests on this; differences in pooling, scoring, Matryoshka, and vector dimensions are not independently ablated before attributing the effect to late interaction (Section 2.3).
  • domain assumption mmBERT-base provides sufficient multilingual alignment for translate-train to transfer to unseen languages.
    The success of the method relies on the backbone's pretraining coverage; the paper itself notes that Swahili and Indonesian remain weak.

pith-pipeline@v1.3.0-alltime-deepseek · 27703 in / 11211 out tokens · 105059 ms · 2026-08-03T01:37:54.635576+00:00 · methodology

0 comments
read the original abstract

State-of-the-art retrieval models increasingly rely on closed training data, creating a reproducibility gap. We present an open end-to-end recipe for training retrieval models and study how English supervision transfers to multilingual retrieval through translate-train. We first reconstruct and curate 665M English contrastive pre-training pairs from 1.4B pairs across 34 public sources and build 1.88M supervised fine-tuning pairs with mined hard negatives. Training yields two 149M-parameter models: DenseOn, a single-vector dense model, and LateOn, a ColBERT-style late-interaction model. They achieve 56.20 and 57.22 average nDCG@10 on BEIR, respectively, setting new state-of-the-art results for this size class. We then translate the validated English data into eight languages, yielding 2.8B pairs with cross-lingual samples, and train mDenseOn and mLateOn, two 307M-parameter models built on mmBERT-base. Despite sharing their backbone, data, and objectives, their representations behave differently: the dense model is strong on English and translated languages but degrades outside translate-train support, whereas the late-interaction model generalizes better to unseen languages and scripts. This suggests that token-level matching turns translate-train from a target-language expansion strategy into a multilingual generalization recipe. We publicly release the models, datasets, and training code.

Figures

Figures reproduced from arXiv: 2607.27178 by Am\'elie Chatelain, Antoine Chaffin, Paulo Roberto Moura Junior, Rapha\"el Sourty.

Figure 1
Figure 1. Figure 1: Per-dataset language mix and stored size (GB) in the multilingual contrastive pre-training data. [PITH_FULL_IMAGE:figures/full_fig_p013_1.png] view at source ↗
Figure 1
Figure 1. Figure 1: Per-dataset language mix and stored size (GB) in the multilingual contrastive pre-training data. [PITH_FULL_IMAGE:figures/full_fig_p014_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Per-dataset language mix and total number of query-positive pairs in the filtered contrastive fine-tuning [PITH_FULL_IMAGE:figures/full_fig_p014_2.png] view at source ↗
Figure 2
Figure 2. Figure 2: Per-dataset language mix and total number of query-positive pairs in the filtered contrastive fine-tuning [PITH_FULL_IMAGE:figures/full_fig_p015_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: BEIR results (average nDCG@10) for vanilla InfoNCE and Matryoshka training across different [PITH_FULL_IMAGE:figures/full_fig_p019_3.png] view at source ↗

Review history (3 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

133 extracted references · 16 linked inside Pith

  1. [2]

    https://huggingface.co/Alibaba-NLP/gte-modernbert-base gte-modernbert-base [online]

    Alibaba NLP . https://huggingface.co/Alibaba-NLP/gte-modernbert-base gte-modernbert-base [online]. 2025

    2025

  2. [3]

    Parul Awasthy, Aashka Trivedi, Yushu Yang, Ken Barker, Yulong Li, Bhavani Iyer, Martin Franz, Juergen Bross, Meet Doshi, Vignesh P, Vishwajeet Kumar, Todd Ward, Abraham Daniels, Madison Lee, Luis Lastras, Jaydeep Sen, and Radu Florian. 2026. https://arxiv.org/abs/2605.13521 Granite embedding multilingual r2 models . Preprint, arXiv:2605.13521

    Pith/arXiv arXiv 2026

  3. [4]

    Luiz Bonifacio, Vitor Jeronymo, Hugo Queiroz Abonizio, Israel Campiotti, Marzieh Fadaee, Roberto Lotufo, and Rodrigo Nogueira. 2022. https://arxiv.org/abs/2108.13897 mmarco: A multilingual version of the ms marco passage ranking dataset . Preprint, arXiv:2108.13897

    Pith/arXiv arXiv 2022

  4. [5]

    Thorsten Brants and Alex Franz. 2006. https://catalog.ldc.upenn.edu/LDC2006T13 Web 1T 5-gram Version 1 . Linguistic Data Consortium. LDC Catalog No. LDC2006T13, ISBN 1-58563-397-6

    2006

  5. [6]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert - Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, and 12 others. 2020. https://proceedings.neurips.cc/pap...

    2020

  6. [7]

    Antoine Chaffin. 2025. https://huggingface.co/lightonai/GTE-ModernColBERT-v1 Gte-moderncolbert

    2025

  7. [8]

    Antoine Chaffin. 2026. https://huggingface.co/collections/lightonai/lateon-code Lateon-code: a family of state-of-the-art late interaction code retrieval models

    2026

  8. [10]

    Antoine Chaffin and Rapha \" e l Sourty. 2025. https://doi.org/10.1145/3746252.3761608 Pylate: Flexible training and retrieval for late interaction models . In Proceedings of the 34th ACM International Conference on Information and Knowledge Management, CIKM 2025, Seoul, Republic of Korea, November 10-14, 2025 , pages 6334--6339. ACM

    arXiv 2025

  9. [12]

    https://www.answer.ai/posts/2024-08-13-small-but-mighty-colbert.html Small but mighty: Introducing answerai-colbert-small [online]

    Benjamin Clavi \'e . https://www.answer.ai/posts/2024-08-13-small-but-mighty-colbert.html Small but mighty: Introducing answerai-colbert-small [online]. 2024

    2024

  10. [13]

    Benjamin Clavi \'e . 2025. https://doi.org/10.5715/jnlp.32.176 JaColBERTv2.5 : Optimising multi-vector retrievers to create state-of-the-art Japanese retrievers with constrained resources . Journal of Natural Language Processing, 32(1):176--218

    doi:10.5715/jnlp.32.176 2025

  11. [15]

    Alexis Conneau and Guillaume Lample. 2019. https://proceedings.neurips.cc/paper/2019/hash/c04c19c2c2474dbf5f7ac4372c5b9af1-Abstract.html Cross-lingual language model pretraining . In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canad...

    2019

  12. [17]

    Kenneth Enevoldsen, Isaac Chung, Imene Kerboua, M\' a rton Kardos, Ashwin Mathur, David Stap, Jay Gala, Wissam Siblini, Dominik Krzemi\' n ski, Genta Winata, Saba Sturua, Saiteja Utpala, Mathieu Ciancone, Marion Schaeffer, Diganta Misra, Shreeya Dhakal, Jonathan Rystr m, Roman Solomatin, \" O mer C a g atan, and 63 others. 2025. https://proceedings.iclr.c...

    2025

  13. [20]

    https://blogs.bing.com/search/April-2026/Microsoft-Open-Sources-Industry-Leading-Embedding-Model Microsoft open-sources industry-leading embedding model [online]

    Xiaolong Huang, Liang Wang, Furu Wei, Jingwen Lu, Knut Risvik, and Jason Li. https://blogs.bing.com/search/April-2026/Microsoft-Open-Sources-Industry-Leading-Embedding-Model Microsoft open-sources industry-leading embedding model [online]. 2026

    2026

  14. [22]

    Weld, and Luke Zettlemoyer

    Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. 2017. https://doi.org/10.18653/v1/P17-1147 TriviaQA : A large scale distantly supervised challenge dataset for reading comprehension . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1601--1611, Vancouver, Canada. Association for Computationa...

    doi:10.18653/v1/p17-1147 2017

  15. [24]

    Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2017. Bag of tricks for efficient text classification. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 427--431. Association for Computational Linguistics

    2017

  16. [26]

    Kakade, Prateek Jain, and Ali Farhadi

    Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ramanujan, William Howard - Snyder, Kaifeng Chen, Sham M. Kakade, Prateek Jain, and Ali Farhadi. 2022. http://papers.nips.cc/paper\_files/paper/2022/hash/c32319f4868da7613d78af9993100e42-Abstract-Conference.html Matryoshka representation learning . In Advances in Neural...

    2022

  17. [27]

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Matthew Kelcey, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Ming-Wei Chang, Andrew Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. https://doi.org/10.1162/tacl_a_00276 Natural questions: A benchmark ...

    doi:10.1162/tacl_a_00276 2019

  18. [29]

    Xianming Li, Aamir Shakir, Rui Huang, Julius Lipp, Benjamin Clavi \' e , and Jing Li. 2026. https://aclanthology.org/2026.findings-acl.51/ Prorank: Prompt warmup via reinforcement learning for small language models reranking . In Findings of the Association for Computational Linguistics, ACL 2026, San Diego, California, United States, July 2-7, 2026 , pag...

    2026

  19. [30]

    https://www.liquid.ai/blog/lfm2-5-retrievers LFM2.5 Retrievers : Bi-directional LFM s for fast multilingual search [online]

    Liquid AI . https://www.liquid.ai/blog/lfm2-5-retrievers LFM2.5 Retrievers : Bi-directional LFM s for fast multilingual search [online]. 2026

    2026

  20. [31]

    Macedo Maia, Siegfried Handschuh, Andr \'e Freitas, Brian Davis, Ross McDermott, Manel Zarrouk, and Alexandra Balahur. 2018. https://doi.org/10.1145/3184558.3192301 WWW'18 open challenge: Financial opinion mining and question answering . In Companion Proceedings of The Web Conference 2018, pages 1941--1942. International World Wide Web Conferences Steerin...

    arXiv 2018

  21. [34]

    Mistral AI . 2025. https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503 Mistral-Small-3.1-24B-Instruct-2503

    2025

  22. [35]

    Moreira, Radek Osmulski, Mengyao Xu, Ronay Ak, Benedikt Schifferer, and Even Oldridge

    Gabriel de Souza P. Moreira, Radek Osmulski, Mengyao Xu, Ronay Ak, Benedikt Schifferer, and Even Oldridge. 2024. https://doi.org/10.48550/arXiv.2407.15831 NV-Retriever : Improving text embedding models with effective hard-negative mining . arXiv preprint arXiv:2407.15831

    Pith/arXiv arXiv doi:10.48550/arxiv.2407.15831 2024

  23. [36]

    Niklas Muennighoff, Qian Liu, Armel Zebaze, Qinkai Zheng, Binyuan Hui, Terry Yue Zhuo, Swayam Singh, Xiangru Tang, Leandro Von Werra, and Shayne Longpre. 2024. https://proceedings.iclr.cc/paper_files/paper/2024/file/1ec299a5229034141e58aeded0d0b9de-Paper-Conference.pdf Octopack: Instruction tuning code large language models . In International Conference o...

    2024

  24. [38]

    Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. https://ceur-ws.org/Vol-1773/CoCoNIPS\_2016\_paper9.pdf MS MARCO: A human generated machine reading comprehension dataset . In Proceedings of the Workshop on Cognitive Computation: Integrating neural and symbolic approaches 2016 co-located with the 30th ...

    2016

  25. [39]

    https://huggingface.co/nomic-ai/modernbert-embed-base modernbert-embed-base [online]

    Nomic AI . https://huggingface.co/nomic-ai/modernbert-embed-base modernbert-embed-base [online]. 2025

    2025

  26. [40]

    Zach Nussbaum, John Xavier Morris, Andriy Mulyar, and Brandon Duderstadt. 2025. https://openreview.net/forum?id=IPmzyQSiQE Nomic embed: Training a reproducible long context text embedder . Trans. Mach. Learn. Res., 2025

    2025

  27. [41]

    Guilherme Penedo, Hynek Kydl\' c ek, Loubna Ben allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, and Thomas Wolf. 2024. https://doi.org/10.52202/079017-0970 The fineweb datasets: Decanting the web for the finest text data at scale . In Advances in Neural Information Processing Systems, volume 37, pages 30811--30849. Curran Associates, Inc

    doi:10.52202/079017-0970 2024

  28. [42]

    https://huggingface.co/perplexity-ai/pplx-embed-v1-late-0.6b pplx-embed-v1-late-0.6b : Late-interaction embeddings [online]

    Perplexity AI . https://huggingface.co/perplexity-ai/pplx-embed-v1-late-0.6b pplx-embed-v1-late-0.6b : Late-interaction embeddings [online]. 2026

    2026

  29. [43]

    Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. https://doi.org/10.18653/v1/P18-2124 Know what you don't know: Unanswerable questions for SQuAD . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pages 784--789, Melbourne, Australia. Association for Computational Linguistics

    doi:10.18653/v1/p18-2124 2018

  30. [47]

    Rapha \"e l Sourty. 2025. https://github.com/lightonai/fast-plaid Fastplaid: A high-performance engine for multi-vector search

    2025

  31. [48]

    Tarun Suresh, Revanth Gangi Reddy, Yifei Xu, Zach Nussbaum, Andriy Mulyar, Brandon Duderstadt, and Heng Ji. 2025. https://openreview.net/forum?id=iyJOUELYir Cornstack: High-quality contrastive data for better code retrieval and reranking . In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025 . Ope...

    2025

  32. [50]

    Nandan Thakur, Nils Reimers, Andreas R\" u ckl\' e , Abhishek Srivastava, and Iryna Gurevych. 2021. https://datasets-benchmarks-proceedings.neurips.cc/paper_files/paper/2021/file/65b9eea6e1cc6bb9f0cd2a47751a186f-Paper-round2.pdf Beir: A heterogeneous benchmark for zero-shot evaluation of information retrieval models . In Proceedings of the Neural Informat...

    2021

  33. [51]

    James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. https://doi.org/10.18653/v1/N18-1074 FEVER : A large-scale dataset for fact extraction and VER ification . In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 809--819, New...

    doi:10.18653/v1/n18-1074 2018

  34. [52]

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2019. https://arxiv.org/abs/1807.03748 Representation learning with contrastive predictive coding . Preprint, arXiv:1807.03748

    Pith/arXiv arXiv 2019

  35. [53]

    Henrique Schechter Vera, Sahil Dua, Biao Zhang, Daniel Salz, Ryan Mullins, Sindhu Raghuram Panyam, Sara Smoot, Iftekhar Naim, Joe Zou, Feiyang Chen, Daniel Cer, Alice Lisak, Min Choi, Lucas Gonzalez, Omar Sanseviero, Glenn Cameron, Ian Ballantyne, Kat Black, Kaifeng Chen, and 70 others. 2025. https://arxiv.org/abs/2509.20354 Embeddinggemma: Powerful and l...

    Pith/arXiv arXiv 2025

  36. [54]

    https://blog.voyageai.com/2026/01/15/voyage-4/ The Voyage 4 model family: Shared embedding space with MoE architecture [online]

    Voyage AI . https://blog.voyageai.com/2026/01/15/voyage-4/ The Voyage 4 model family: Shared embedding space with MoE architecture [online]. 2026

    2026

  37. [56]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. https://arxiv.org/abs/2505.09388 Qwen3 technical report . Preprint, arXiv:2505.09388

    Pith/arXiv arXiv 2025

  38. [60]

    Cohen, Ruslan Salakhutdinov, and Christopher D

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. https://doi.org/10.18653/v1/D18-1259 HotpotQA : A dataset for diverse, explainable multi-hop question answering . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2369--2380, Brussel...

    doi:10.18653/v1/d18-1259 2018

  39. [62]

    Xin Zhang, Yanzhao Zhang, Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang, Pengjun Xie, Fei Huang, Meishan Zhang, Wenjie Li, and Min Zhang. 2024. https://aclanthology.org/2024.emnlp-industry.103/ mGTE : Generalized long-context text representation and reranking models for multilingual text retrieval . In Proceedings of the 2024 Confe...

    2024

  40. [64]

    Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. 2025. https://arxiv.org/abs/2506.05176 Qwen3 embedding: Advancing text embedding and reranking through foundation models . Preprint, arXiv:2506.05176

    Pith/arXiv arXiv 2025

  41. [65]

    arXiv preprint arXiv:2210.07316 , year =

    Muennighoff, Niklas and Tazi, Nouamane and Magne, Lo. arXiv preprint arXiv:2210.07316 , year =

    Pith/arXiv arXiv

  42. [66]

    MMTEB: Massive Multilingual Text Embedding Benchmark , url =

    Enevoldsen, Kenneth and Chung, Isaac and Kerboua, Imene and Kardos, M\'. MMTEB: Massive Multilingual Text Embedding Benchmark , url =. International Conference on Learning Representations , editor =

  43. [67]

    Advances in Information Retrieval , pages =

    Thakur, Nandan and Reimers, Nils and R. Advances in Information Retrieval , pages =. 2021 , publisher =

    2021

  44. [68]

    Companion Proceedings of The Web Conference 2018 , pages =

    Maia, Macedo and Handschuh, Siegfried and Freitas, Andr. Companion Proceedings of The Web Conference 2018 , pages =. 2018 , publisher =

    2018

  45. [69]

    and Salakhutdinov, Ruslan and Manning, Christopher D

    Yang, Zhilin and Qi, Peng and Zhang, Saizheng and Bengio, Yoshua and Cohen, William W. and Salakhutdinov, Ruslan and Manning, Christopher D. , booktitle =. 2018 , address =

    2018

  46. [70]

    Transactions of the Association for Computational Linguistics , volume =

    Natural Questions: A Benchmark for Question Answering Research , author =. Transactions of the Association for Computational Linguistics , volume =. 2019 , url =

    2019

  47. [71]

    Tri Nguyen and Mir Rosenberg and Xia Song and Jianfeng Gao and Saurabh Tiwary and Rangan Majumder and Li Deng , editor =. Proceedings of the Workshop on Cognitive Computation: Integrating neural and symbolic approaches 2016 co-located with the 30th Annual Conference on Neural Information Processing Systems. 2016 , url =

    2016

  48. [72]

    2016 , url =

    Bajaj, Payal and Campos, Daniel and Craswell, Nick and Deng, Li and Gao, Jianfeng and Liu, Xiaodong and Majumder, Rangan and McNamara, Andrew and Mitra, Bhaskar and Nguyen, Tri and Rosenberg, Mir and Song, Xia and Stoica, Alina and Tiwary, Saurabh and Wang, Tong , journal =. 2016 , url =

    2016

  49. [73]

    2018 , address =

    Thorne, James and Vlachos, Andreas and Christodoulopoulos, Christos and Mittal, Arpit , booktitle =. 2018 , address =

    2018

  50. [74]

    Know What You Don't Know: Unanswerable Questions for

    Rajpurkar, Pranav and Jia, Robin and Liang, Percy , booktitle =. Know What You Don't Know: Unanswerable Questions for. 2018 , address =

    2018

  51. [75]

    and Zettlemoyer, Luke , booktitle =

    Joshi, Mandar and Choi, Eunsol and Weld, Daniel S. and Zettlemoyer, Luke , booktitle =. 2017 , address =

    2017

  52. [76]

    MIRACL : A Multilingual Retrieval Dataset Covering 18 Diverse Languages

    Zhang, Xinyu and Thakur, Nandan and Ogundepo, Odunayo and Kamalloo, Ehsan and Alfonso-Hermelo, David and Li, Xiaoguang and Liu, Qun and Rezagholizadeh, Mehdi and Lin, Jimmy. MIRACL : A Multilingual Retrieval Dataset Covering 18 Diverse Languages. Transactions of the Association for Computational Linguistics. 2023. doi:10.1162/tacl_a_00595

    doi:10.1162/tacl_a_00595 2023

  53. [77]

    2024 , url =

    Chen, Jianlv and Xiao, Shitao and Zhang, Peitian and Luo, Kun and Lian, Defu and Liu, Zheng , journal =. 2024 , url =

    2024

  54. [78]

    M 3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation

    Chen, Jianlyu and Xiao, Shitao and Zhang, Peitian and Luo, Kun and Lian, Defu and Liu, Zheng. M 3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation. Findings of the Association for Computational Linguistics: ACL 2024. 2024. doi:10.18653/v1/2024.findings-acl.137

    doi:10.18653/v1/2024.findings-acl.137 2024

  55. [79]

    2024 , url =

    Zhang, Xin and Zhang, Yanzhao and Long, Dingkun and Xie, Wen and Dai, Ziqi and Tang, Jialong and Lin, Huan and Yang, Baosong and Xie, Pengjun and Huang, Fei and Zhang, Meishan and Li, Wenjie and Zhang, Min , booktitle =. 2024 , url =

    2024

  56. [80]

    Multilingual

    Wang, Liang and Yang, Nan and Huang, Xiaolong and Yang, Linjun and Majumder, Rangan and Wei, Furu , journal =. Multilingual. 2024 , url =

    2024

  57. [81]

    2026 , eprint=

    Granite Embedding Multilingual R2 Models , author=. 2026 , eprint=

    2026

  58. [82]

    2025 , eprint=

    EmbeddingGemma: Powerful and Lightweight Text Representations , author=. 2025 , eprint=

    2025

  59. [83]

    jina-embeddings-v5-text: Compact and Robust Text Embedding Models using Task-Targeted Distillation , booktitle =

    Mohammad Kalim Akram and Saba Sturua and Nastia Havriushenko and Quentin Herreros and Michael G. jina-embeddings-v5-text: Compact and Robust Text Embedding Models using Task-Targeted Distillation , booktitle =. 2026 , url =. doi:10.1145/3805712.3808455 , timestamp =

    arXiv 2026

  60. [84]

    Lawrie and Benjamin Van Durme , title =

    Marc Marone and Orion Weller and William Fleshman and Eugene Yang and Dawn J. Lawrie and Benjamin Van Durme , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2509.06888 , eprinttype =. 2509.06888 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2509.06888 2025

  61. [85]

    Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference

    Warner, Benjamin and Chaffin, Antoine and Clavi. Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025. doi:10.18653/v1/2025.acl-long.127

    doi:10.18653/v1/2025.acl-long.127 2025

  62. [86]

    ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over

    Omar Khattab and Matei Zaharia , editor =. ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over. Proceedings of the 43rd International. 2020 , url =. doi:10.1145/3397271.3401075 , timestamp =

    arXiv 2020

  63. [87]

    C ol BERT v2: Effective and Efficient Retrieval via Lightweight Late Interaction

    Santhanam, Keshav and Khattab, Omar and Saad-Falcon, Jon and Potts, Christopher and Zaharia, Matei. C ol BERT v2: Effective and Efficient Retrieval via Lightweight Late Interaction. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022. doi:10.18653/v1/2022.naac...

    doi:10.18653/v1/2022.naacl-main.272 2022

  64. [88]

    2022 , url =

    Santhanam, Keshav and Khattab, Omar and Potts, Christopher and Zaharia, Matei , booktitle =. 2022 , url =

    2022

  65. [89]

    Matryoshka Representation Learning , booktitle =

    Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard. Matryoshka Representation Learning , booktitle =. 2022 , url =

    2022

  66. [90]

    and Osmulski, Radek and Xu, Mengyao and Ak, Ronay and Schifferer, Benedikt and Oldridge, Even , journal =

    Moreira, Gabriel de Souza P. and Osmulski, Radek and Xu, Mengyao and Ak, Ronay and Schifferer, Benedikt and Oldridge, Even , journal =. 2024 , url =

    2024

  67. [91]

    2024 , url =

    Li, Xianming and Li, Jing , booktitle =. 2024 , url =

    2024

  68. [92]

    OctoPack: Instruction Tuning Code Large Language Models , url =

    Muennighoff, Niklas and Liu, Qian and Zebaze, Armel and Zheng, Qinkai and Hui, Binyuan and Zhuo, Terry Yue and Singh, Swayam and Tang, Xiangru and Von Werra, Leandro and Longpre, Shayne , booktitle =. OctoPack: Instruction Tuning Code Large Language Models , url =

  69. [93]

    arXiv preprint arXiv:2308.07124 , year=

    OctoPack: Instruction Tuning Code Large Language Models , author=. arXiv preprint arXiv:2308.07124 , year=

    Pith/arXiv arXiv

  70. [94]

    2019 , eprint=

    Representation Learning with Contrastive Predictive Coding , author=. 2019 , eprint=

    2019

  71. [95]

    2025 , eprint=

    Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models , author=. 2025 , eprint=

    2025

  72. [96]

    2025 , eprint=

    Qwen3 Technical Report , author=. 2025 , eprint=

    2025

  73. [97]

    arXiv preprint arXiv:2412.04506 , year =

    Arctic-Embed 2.0: Multilingual Retrieval Without Compromise , author =. arXiv preprint arXiv:2412.04506 , year =

    Pith/arXiv arXiv

  74. [98]

    Diffusion-Pretrained Dense and Contextual Embeddings

    Eslami, Sedigheh and Gaiduk, Maksim and Krimmel, Markus and Milliken, Louis Mark and Wang, Bo and Bykov, Denis. Diffusion-Pretrained Dense and Contextual Embeddings. Proceedings of the 64th Annual Meeting of the A ssociation for C omputational L inguistics (Volume 6: Industry Track). 2026. doi:10.18653/v1/2026.acl-industry.69

    doi:10.18653/v1/2026.acl-industry.69 2026

  75. [99]

    2024 , url =

    Louis, Antoine and Saxena, Vageesh and van Dijck, Gijs and Spanakis, Gerasimos , journal =. 2024 , url =

    2024

  76. [100]

    J ina- C ol BERT -v2: A General-Purpose Multilingual Late Interaction Retriever

    Jha, Rohan and Wang, Bo and G. J ina- C ol BERT -v2: A General-Purpose Multilingual Late Interaction Retriever. Proceedings of the Fourth Workshop on Multilingual Representation Learning (MRL 2024). 2024. doi:10.18653/v1/2024.mrl-1.11

    doi:10.18653/v1/2024.mrl-1.11 2024

  77. [101]

    arXiv preprint arXiv:2510.14880 , year =

    Fantastic (small) Retrievers and How to Train Them: mxbai-edge-colbert-v0 Tech Report , author =. arXiv preprint arXiv:2510.14880 , year =

    arXiv

  78. [102]

    2026 , howpublished =

    DenseOn with the LateOn: Open State-of-the-Art Single and Multi-Vector Models , author =. 2026 , howpublished =

    2026

  79. [103]

    arXiv preprint arXiv:2301.12005 , year =

    EmbedDistill: A Geometric Knowledge Distillation for Information Retrieval , author =. arXiv preprint arXiv:2301.12005 , year =

    Pith/arXiv arXiv

  80. [104]

    arXiv preprint arXiv:2412.19048 , year =

    Jasper: An Open State-of-the-Art Embedding Model with 3,938 Dimensions , author =. arXiv preprint arXiv:2412.19048 , year =

    Pith/arXiv arXiv

Showing first 80 references.