Pith. sign in

REVIEW 2 major objections 4 minor 55 references

Context is Gold to find the Gold Passage: Evaluating and Training Contextual Document Embeddings

T0 review · 2 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Chunk embeddings that can see the whole document beat independent chunk embeddings on context-dependent retrieval, and a cheap contrastive post-training method, InSeNT, delivers most of the gain.

desk verdict The paper delivers a genuinely useful benchmark for context-dependent retrieval and a cheap, effective training method, but the 'no sacrifice' claim is overstated and the training/evaluation split overlap needs to be checked. read the letter →

arxiv 2505.24782 v2 pith:BU55T2XD submitted 2025-05-30 cs.IR

Max Conti , Manuel Faysse , Gautier Viaud , Antoine Bosselut , Céline Hudelot , Pierre Colombo This is my paper
classification cs.IR
keywords contextualretrievaldocumentembeddingslatechunkingcontrastivelearningin-sequencenegativesbenchmarklong-contextRAG
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Retrieval systems typically embed each chunk of a long document in isolation, so a sentence like “He became emperor in 1804” carries no link to Napoléon. This paper argues that this missing document-wide context is a major, under-measured failure mode, and introduces ConTEB, a benchmark built from controlled and real-world tasks where answering requires context beyond the chunk itself. On ConTEB, strong off-the-shelf embedders score far below methods that propagate context, and the paper's InSeNT post-training method, applied on top of late chunking, lifts average nDCG@10 from 52.0 to 75.6 for ModernBERT-Large while leaving short-context performance essentially intact. The intended contribution is a cheap, open way to make chunk embeddings context-aware without relying on expensive LLM reformulation at indexing time.

What carries the argument

Late chunking, extended by the InSeNT objective. Late chunking runs the entire document through the encoder in one forward pass and average-pools token representations over each chunk's span, letting chunk embeddings see the rest of the document before pooling. InSeNT trains this with a weighted InfoNCE loss that combines standard in-batch negatives from other documents with “in-sequence” negatives drawn from other chunks of the same document; $\lambda_{\text{seq}}$ controls the mix, and the same loss extends to late-interaction models by substituting ColBERT's MaxSim for the dot product. The in-sequence term is what prevents all chunks of a document from collapsing into one indistinguishable representation.

What would settle it

Re-run the ConTEB evaluation on MLDR, NarrativeQA, and SQuAD using held-out splits that are provably disjoint from the InSeNT training set and compare nDCG@10; if the in-domain gains shrink to near the non-contextual baseline, the headline improvements come from train/evaluation overlap rather than from context propagation.

Watch

Extended reading notes

Core claim

The central claim is that chunk-level retrieval embeddings should be computed with access to the whole source document, and that this can be trained for efficiently. The paper shows that standard single-vector and late-interaction embedders fail on ConTEB's context-dependent tasks (e.g., Insurance nDCG@10 of 12.4 for ModernBERT-Large vs. 100.0 after InSeNT), while untrained late chunking alone already helps on controlled tasks. InSeNT adds same-document chunks as hard negatives to a contrastive objective, so each chunk absorbs document-level information while staying distinguishable from its neighbors. The authors report that this post-training costs about one H100 GPU hour per epoch, preserves NanoBEIR non-contextual performance, and makes retrieval more robust to poor chunking and larger corpora.

Load-bearing premise

The load-bearing premise is that the ConTEB evaluation sets for MLDR, NarrativeQA, and SQuAD are disjoint from the training splits used to train InSeNT; the paper does not state which splits ConTEB evaluates, so if any query-chunk pair appears in both, the reported in-domain gains would be inflated.

Editorial extensions

If this is right

  • On ConTEB's controlled tasks, context-aware retrieval can jump from near-random to near-perfect: Insurance goes from 12.4 to 100.0 nDCG@10 for ModernBERT-Large with InSeNT.
  • InSeNT-trained chunk embeddings tolerate much smaller, less self-contained chunks without the usual retrieval collapse.
  • Contextual chunk embeddings degrade more gracefully as the corpus grows with many similar documents, a common industrial condition.
  • The method adds almost no indexing cost (about 15 ms per document for ModernBERT-Large + InSeNT) and avoids the 120x slowdown of LLM-based contextual retrieval.
  • Short-context retrieval on NanoBEIR is not sacrificed when hyperparameters are tuned ($\lambda_{\text{seq}} = 0.1$).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: if chunk embeddings reliably inherit document context, then chunking strategy itself becomes less critical, and systems could use much smaller chunks to improve downstream answer granularity.
  • Editorial extension: the in-sequence negative principle should transfer to decoder-based long-context embedders, where whole-book encoding could make context propagation even stronger; the paper names this as future work.
  • Editorial extension: one testable consequence is that InSeNT should help most on corpora full of templated documents that differ by a few key fields; the robustness-to-corpus-size result suggests this, but it is not measured directly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper introduces ConTEB, a benchmark for evaluating whether text embedding models exploit document-wide context when encoding chunks, and InSeNT, a contrastive post-training objective that combines in-batch and in-sequence negatives with late chunking. On ConTEB, InSeNT substantially improves nDCG@10 over the late-chunking baseline for ModernBERT and ModernColBERT, with the largest gains on the controlled Football/Geography/Insurance tasks. The paper also reports ablations on the in-sequence weight, chunking robustness, and corpus scaling, and it releases the benchmark, models, and training data.

Significance. If the results hold, the paper makes a useful contribution: it provides a public benchmark for context-aware chunk retrieval and a cheap post-training procedure that improves contextualization without the inference cost of LLM-based reformulation. The release of all artifacts, the clear ablation isolating the role of in-sequence vs. in-batch negatives, and the demonstration that concatenated short documents do not substitute for organic long documents are concrete strengths. The main claims rest on two points that need verification: split disjointness between training and evaluation, and the statement that InSeNT improves retrieval 'without sacrificing base model performance.'

major comments (2)
  1. [§3.3 and Table 1] The paper does not establish that the ConTEB evaluation sets for MLDR, NarrativeQA, and SQuAD are disjoint from the InSeNT training data. Section 3.3 states that the released training set includes the training splits of MLDR and NarrativeQA and that SQuAD chunks are concatenated, without restricting SQuAD to a non-training split; Table 1 lists MLDR, NarrativeQA, and SQuAD among the in-domain ConTEB tasks; and the evaluation section never states which original splits are used. If any evaluation documents or query-chunk pairs appear in the training data, the reported in-domain gains (e.g., MLDR 78.4 to 88.7, SQuAD 73.4 to 80.9, NarrativeQA 77.9 to 81.3) are inflated and would propagate to the headline average. Please state the evaluation split for each dataset explicitly and, since the artifacts are released, provide an overlap check between training and evaluation instances.
  2. [Abstract and §5, Table 2] The abstract claims InSeNT improves retrieval 'without sacrificing base model performance,' but the NanoBEIR column of Table 2 shows ModernBERT Large dropping from 63.2 to 60.4 and ModernColBERT dropping from 67.7 to 59.2 after InSeNT training. The 'Short-Context Performance' paragraph in §5 asserts that 'careful hyperparameter tuning enables our best model to maintain strong performance,' but no confidence intervals or significance tests are given, so a 2.8-point drop (and an 8.5-point drop for the ColBERT variant) cannot be claimed to be non-significant. Please either soften the claim to acknowledge a measurable NanoBEIR degradation or provide statistical evidence that the difference is within noise.
minor comments (4)
  1. [References] References are duplicated: Warner et al. 2024a and 2024b are the same paper, Nussbaum et al. 2024 and 2025 share the same title, and Zhong et al. 2025a and 2025b are duplicates. Please consolidate these entries.
  2. [§3.2] The 'Sabotage' step uses LLM-reformulated paragraphs and LLM-generated questions for the controlled tasks, so the conclusion that 'state-of-the-art embedding models struggle' should be framed as a property of the constructed benchmark rather than a universal deficiency of non-contextual encoders.
  3. [Figure 4 (left)] The x-axis label 'Max Chunk Length' with a 'No chunking' category is unclear; please specify how the 'no chunking' condition is constructed and how relative nDCG@10 is computed.
  4. [Appendix C.3] The observation that naively feeding documents longer than 8,192 tokens to ModernBERT outperforms the sliding-window approach is surprising; please provide implementation details (e.g., truncation, position embeddings) given that the base model was trained with an 8,192-token context.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ConTEB is an explicitly constructed context stress test and InSeNT's central gains are supported by out-of-domain tasks and an external non-regression benchmark; the in-domain split-disjointness caveat is a data-provenance risk, not a demonstrated circular step.

full rationale

The paper's derivation chain is self-contained: InSeNT is a contrastive post-training objective defined in Equation 1, trained on a released dataset (Section 3.3) on top of pretrained embedders, and evaluated on ConTEB plus NanoBEIR. The ConTEB controlled tasks are openly constructed to require context: Section 3.2's 'Sabotage' stage reformulates chunks to 'remove explicit mentions of the original document's theme' and 'explicitly enforc[e] the need for context.' Therefore, the observation that non-contextual models struggle on those tasks is a property of the deliberately designed stress test rather than a discovered law; however, the paper presents this as a benchmark design choice, not as a prediction derived from a fitted parameter. The central contribution—InSeNT outperforming untrained late chunking and a same-data non-contextual training baseline—is not forced by construction: the non-contextual ModernBERT model trained on the same data does not improve over the untrained baseline, and the largest gains occur on out-of-domain controlled tasks (Football 19.1 to 63.9, Insurance 12.4 to 100.0). NanoBEIR provides an external non-regression check. Self-citations are incidental (e.g., Macé et al. 2025 as the ESG data source, Faysse et al. 2025 as future work) and do not carry the load-bearing argument. One caveat is data provenance: Section 3.3 says the training set 'includes the training splits of MLDR and NarrativeQA' and also 'concatenate[s] SQuAD chunks,' while Table 1 lists MLDR, NarrativeQA, and SQuAD as in-domain ConTEB tasks, and the paper never explicitly states that the evaluation splits are disjoint from the training data. This is a potential leakage or validity risk, but because the paper text does not exhibit an actual query-chunk overlap between training and evaluation, it is not counted as circularity under the hard-evidence rule.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on one tuned hyperparameter (λseq), an unreported temperature, and four assumptions about late chunking, contrastive learning, LLM supervision, and data split hygiene. The method itself introduces no new physical or conceptual entities beyond the training objective and the benchmark.

free parameters (2)
  • λseq = 0.1
    Weight balancing in-sequence vs in-batch negatives in the loss (Equation 1); tuned on the validation split of the training dataset as shown in Figure 3.
  • temperature τ
    InfoNCE temperature in the contrastive loss; defined as τ>0 in Section 4.2 but its actual value is never reported, making exact reproduction difficult.
assumptions (4)
  • domain assumption Late chunking propagates document-level context to chunk representations through a single forward pass over the concatenated document.
    The method relies on this mechanism from Günther et al. (2024), not proven in this paper.
  • domain assumption Using chunks from the same document as hard negatives in contrastive training makes representations both contextual and specific.
    This is the core training hypothesis of InSeNT; supported only by internal ablations (Figure 3), not by an external test.
  • domain assumption GPT-4o-generated queries and paragraph rephrasings are valid, unbiased supervision for context-dependent retrieval.
    The benchmark and training data rely on LLM outputs (Section 3, Appendix A) without manual verification of every item.
  • ad hoc to paper The ConTEB evaluation splits for MLDR, NarrativeQA, and SQuAD are disjoint from the training splits used for InSeNT.
    The paper does not state the split provenance, creating a leakage risk if the same query-chunk pairs appear in training and evaluation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Context is Gold to find the Gold Passage: Evaluating and Training Contextual Document Embeddings." pith.science (2026). https://pith.science/paper/BU55T2XD

@misc{pith2026250524782,
  author       = {Pith},
  title        = {Pith review of: Context is Gold to find the Gold Passage: Evaluating and Training Contextual Document Embeddings},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BU55T2XD}},
  note         = {Machine review of arXiv:2505.24782}
}
read the original abstract

A limitation of modern document retrieval embedding methods is that they typically encode passages (chunks) from the same documents independently, often overlooking crucial contextual information from the rest of the document that could greatly improve individual chunk representations. In this work, we introduce ConTEB (Context-aware Text Embedding Benchmark), a benchmark designed to evaluate retrieval models on their ability to leverage document-wide context. Our results show that state-of-the-art embedding models struggle in retrieval scenarios where context is required. To address this limitation, we propose InSeNT (In-sequence Negative Training), a novel contrastive post-training approach which combined with late chunking pooling enhances contextual representation learning while preserving computational efficiency. Our method significantly improves retrieval quality on ConTEB without sacrificing base model performance. We further find chunks embedded with our method are more robust to suboptimal chunking strategies and larger retrieval corpus sizes. We open-source all artifacts at https://github.com/illuin-tech/contextual-embeddings.

Figures

Figures reproduced from arXiv: 2505.24782 by the authors.

Figure 1
Figure 1. Importance of Contextual Information: Starting from a set of queries and mostly self-contained document paragraphs from the Football, we progres￾sively reformulate paragraphs to remove information redundant with the rest of the document. This leads to sharp performance declines in standard retrieval ap￾proaches, but not in contextual retrieval approaches. Boizard et al., 2025) along with long context em￾bedding mode… view at source ↗
Figure 2
Figure 2. Training (Left). With respect to a single query, each chunk inside a batch plays a different role, depending on its original document, and the positive chunk. Inference (Right). Traditional embedding methods (top) produce embeddings that do not include potentially essential contextual information. Contextualized embeddings (bottom) can integrate document-wide information in individual chunk representations, augmenti… view at source ↗
Figure 3
Figure 3. Importance of λseq: Results for ModernBERT-Large trained with varying λseq. Op￾timal values depend on the task, but integrating both in-sequence and in-batch negatives is crucial to perfor￾mance. thus rendering context less useful. Our results show that naively applying late chunking in this setting adds noise and leads to notable performance drops (-21 nDCG@10), which are in large part recovered through our trainin… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Contextualized models trained with InSeNT are more robust to aggressive chunking strategies that remove [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Benchmark creation process. A.1 Wiki-based Datasets Football and Geography are our two wiki-based datasets, focusing on the Sports and Geography domains. Wikipedia Data Extraction The pipeline first re￾trieves Wikipedia summaries for a given person using the wikipediaa…
Figure 6
Figure 6. Figure 6: Evaluation results for varying λseq values. Left: ModernBERT-Large. Right: GTE-ModernColBERT. Trends vary across the datasets depending on their nature. be led to better understand the dynamics underlying this extension. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

55 extracted references · 8 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Zeyuan Allen-Zhu . 2024. ICML 2024 Tutorial: Physics of Language Models . Project page: https://physics.allen-zhu.com/

    work page 2024

  4. [4]

    Anthropic. 2024. https://www.anthropic.com/news/contextual-retrieval Introducing contextual retrieval . Accessed: 2025-02-10

    work page 2024

  5. [5]

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2023. https://arxiv.org/abs/2310.11511 Self-rag: Learning to retrieve, generate, and critique through self-reflection . Preprint, arXiv:2310.11511

    arXiv 2023

  6. [6]

    Nicolas Boizard, Hippolyte Gisserot-Boukhlef, Duarte M. Alves, André Martins, Ayoub Hammal, Caio Corro, Céline Hudelot, Emmanuel Malherbe, Etienne Malaboeuf, Fanny Jourdan, Gabriel Hautreux, João Alves, Kevin El-Haddad, Manuel Faysse, Maxime Peyrard, Nuno M. Guerreiro, Patrick Fernandes, Ricardo Rei, and Pierre Colombo. 2025. https://arxiv.org/abs/2503.05...

    arXiv 2025

  7. [7]

    Antoine Chaffin. 2025 a . https://huggingface.co/lightonai/GTE-ModernColBERT-v1 Gte-moderncolbert

    work page 2025

  8. [8]

    Antoine Chaffin. 2025 b . https://huggingface.co/lightonai/modernbert-embed-large Modernbert-embed-large

    work page 2025

Show all 55 references
  1. [9]

    Antoine Chaffin and Raphaël Sourty. 2024. https://github.com/lightonai/pylate Pylate: Flexible training and retrieval for late interaction models

    2024

  2. [10]

    Harrison Chase. 2022. https://github.com/langchain-ai/langchain LangChain

    2022

  3. [11]

    Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. https://doi.org/10.48550/ARXIV.2402.03216 BGE M3 - Embedding : Multi - Lingual , Multi - Functionality , Multi - Granularity Text Embeddings Through Self - Knowledge Distillation . arXiv preprint...

    2024 doi arXiv

  4. [12]

    Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, and Jonathan Larson. 2024. https://arxiv.org/abs/2404.16130 From local to global: A graph rag approach to query-focused summarization . Preprint, arXiv:2404.16130

    2024 arXiv

  5. [13]

    Manuel Faysse, Hugues Sibille, Tony Wu, Bilel Omrani, Gautier Viaud, Céline Hudelot, and Pierre Colombo. 2025. https://arxiv.org/abs/2407.01449 Colpali: Efficient document retrieval with vision language models . Preprint, arXiv:2407.01449

    2025 arXiv

  6. [14]

    Gemini Team , Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, Soroosh Mariooryad, Yifan Ding, Xinyang Geng, Fred Alcober, Roy Frostig, Mark Omernick, Lexi Walker, Cosmin Paduraru, Christina Sorokin, ...

    2024 arXiv

  7. [15]

    Hippolyte Gisserot-Boukhlef, Manuel Faysse, Emmanuel Malherbe, Céline Hudelot, and Pierre Colombo. 2024. https://arxiv.org/abs/2402.12997 Towards trustworthy reranking: A simple yet effective abstention mechanism . Preprint, arXiv:2402.12997

    2024 arXiv

  8. [16]

    Michael Günther, Isabelle Mohr, Daniel James Williams, Bo Wang, and Han Xiao. 2024. https://arxiv.org/abs/2409.04701 Late chunking: Contextual chunk embeddings using long-context embedding models . Preprint, arXiv:2409.04701

    2024 arXiv

  9. [17]

    Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2021. https://doi.org/10.48550/ARXIV.2112.09118 Unsupervised Dense Information Retrieval with Contrastive Learning . arXiv preprint. Version Number: 4

    2021 doi arXiv

  10. [18]

    Ziyan Jiang, Xueguang Ma, and Wenhu Chen. 2024. https://arxiv.org/abs/2406.15319 Longrag: Enhancing retrieval-augmented generation with long-context llms . Preprint, arXiv:2406.15319

    2024 arXiv

  11. [19]

    Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. https://doi.org/10.48550/ARXIV.2004.04906 Dense Passage Retrieval for Open - Domain Question Answering . arXiv preprint. Version Number: 3

    2020 doi arXiv

  12. [20]

    Omar Khattab and Matei Zaharia. 2020. https://doi.org/10.48550/ARXIV.2004.12832 ColBERT : Efficient and Effective Passage Search via Contextualized Late Interaction over BERT

    2020 doi arXiv

  13. [21]

    Tomáš Kočiský, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette. 2017. https://arxiv.org/abs/1712.07040 The narrativeqa reading comprehension challenge . Preprint, arXiv:1712.07040

    2017 arXiv

  14. [22]

    Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. 2024. https://arxiv.org/abs/2405.17428 Nv-embed: Improved techniques for training llms as generalist embedding models . Preprint, arXiv:2405.17428

    2024 arXiv

  15. [23]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. https://doi.org/10.48550/ARXIV.2005.11401 Retrieval- Augmented Generation for Kno...

    2020 doi arXiv

  16. [24]

    Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023. https://arxiv.org/abs/2308.03281 Towards general text embeddings with multi-stage contrastive learning . Preprint, arXiv:2308.03281

    2023 arXiv

  17. [25]

    Jerry Liu. 2022. https://doi.org/10.5281/zenodo.1234 LlamaIndex

    2022 doi

  18. [26]

    Xueguang Ma, Sheng-Chieh Lin, Minghan Li, Wenhu Chen, and Jimmy Lin. 2024. https://arxiv.org/abs/2406.11251 Unifying multimodal retrieval via document screenshot embedding . Preprint, arXiv:2406.11251

    2024 arXiv

  19. [27]

    Quentin Macé, António Loison, and Manuel Faysse. 2025. https://arxiv.org/abs/2505.17166 Vidore benchmark v2: Raising the bar for visual retrieval . Preprint, arXiv:2505.17166

    2025

  20. [28]

    Timo M \"o ller, Anthony Reina, Raghavan Jayakumar, and Malte Pietsch. 2020. https://aclanthology.org/2020.nlpcovid19-acl.18/ COVID-QA : A question answering dataset for COVID -19 . In Proceedings of the 1st Workshop on NLP for COVID-19 at ACL 2020 , Online. Association for Co...

    2020

  21. [29]

    Morris and Alexander M

    John X. Morris and Alexander M. Rush. 2024. https://arxiv.org/abs/2410.02525 Contextual document embeddings . Preprint, arXiv:2410.02525

    2024 arXiv

  22. [30]

    Niklas Muennighoff, Nouamane Tazi, Loïc Magne, and Nils Reimers. 2022. https://doi.org/10.48550/ARXIV.2210.07316 MTEB : Massive Text Embedding Benchmark . arXiv preprint. Version Number: 3

    2022 doi arXiv

  23. [31]

    Zhao, Yi Luan, Keith B

    Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hernández Ábrego, Ji Ma, Vincent Y. Zhao, Yi Luan, Keith B. Hall, Ming-Wei Chang, and Yinfei Yang. 2021. https://arxiv.org/abs/2112.07899 Large dual encoders are generalizable retrievers . Preprint, arXiv:2112.07899

    2021 arXiv

  24. [33]

    Morris, Brandon Duderstadt, and Andriy Mulyar

    Zach Nussbaum, John X. Morris, Brandon Duderstadt, and Andriy Mulyar. 2025. https://arxiv.org/abs/2402.01613 Nomic embed: Training a reproducible long context text embedder . Preprint, arXiv:2402.01613

    2025 arXiv

  25. [34]

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. https://doi.org/10.48550/ARXIV.1807.03748 Representation Learning with Contrastive Predictive Coding . arXiv preprint. Version Number: 2

    2018 doi arXiv

  26. [35]

    Mykhailo Poliakov and Nadiya Shvai. 2024. https://arxiv.org/abs/2406.13213 Multi-meta-rag: Improving rag for multi-hop queries using database filtering with llm-extracted metadata . Preprint, arXiv:2406.13213

    2024 arXiv

  27. [36]

    Hongjin Qian, Zheng Liu, Kelong Mao, Yujia Zhou, and Zhicheng Dou. 2024. https://arxiv.org/abs/2402.09760 Grounding language model with chunking-free in-context retrieval . Preprint, arXiv:2402.09760

    2024 arXiv

  28. [37]

    Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. https://doi.org/10.18653/v1/D16-1264 SQ u AD : 100,000+ questions for machine comprehension of text . In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383-...

    2016 doi

  29. [38]

    Nils Reimers and Iryna Gurevych. 2019. https://doi.org/10.48550/ARXIV.1908.10084 Sentence- BERT : Sentence Embeddings using Siamese BERT - Networks . arXiv preprint. Version Number: 1

    2019 doi arXiv

  30. [39]

    Robertson, Steve Walker, Susan Jones, Micheline Hancock-Beaulieu, and Mike Gatford

    Stephen E. Robertson, Steve Walker, Susan Jones, Micheline Hancock-Beaulieu, and Mike Gatford. 1994. http://trec.nist.gov/pubs/trec3/papers/city.ps.gz Okapi at TREC -3 . In Proceedings of The Third Text REtrieval Conference , TREC 1994, Gaithersburg , Maryland , USA , November...

    1994

  31. [40]

    Fu, Simran Arora, Neel Guha, and Christopher Ré

    Jon Saad-Falcon, Daniel Y. Fu, Simran Arora, Neel Guha, and Christopher Ré. 2024. https://arxiv.org/abs/2402.07440 Benchmarking and building long-context retrieval models with loco and m2-bert . Preprint, arXiv:2402.07440

    2024 arXiv

  32. [41]

    Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, and Christopher D. Manning. 2024. https://arxiv.org/abs/2401.18059 Raptor: Recursive abstractive processing for tree-organized retrieval . Preprint, arXiv:2401.18059

    2024 arXiv

  33. [42]

    Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. https://doi.org/10.48550/ARXIV.1503.03832 FaceNet : A Unified Embedding for Face Recognition and Clustering . Publisher: arXiv Version Number: 3

    2015 doi arXiv

  34. [43]

    Nandan Thakur, Jimmy Lin, Sam Havens, Michael Carbin, Omar Khattab, and Andrew Drozdov. 2025. https://arxiv.org/abs/2504.13128 Freshstack: Building realistic benchmarks for evaluating retrieval on technical documents . Preprint, arXiv:2504.13128

    2025 arXiv

  35. [44]

    Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. https://doi.org/10.48550/ARXIV.2104.08663 BEIR : A Heterogenous Benchmark for Zero -shot Evaluation of Information Retrieval Models . arXiv preprint. Version Number: 4

    2021 doi arXiv

  36. [45]

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2023. https://arxiv.org/abs/2212.10509 Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions . Preprint, arXiv:2212.10509

    2023 arXiv

  37. [46]

    Ke Wang, Nikolaos Dimitriadis, Alessandro Favero, Guillermo Ortiz-Jimenez, Francois Fleuret, and Pascal Frossard. 2025. https://arxiv.org/abs/2410.17146 Lines: Post-training layer scaling prevents forgetting and enhances model merging . Preprint, arXiv:2410.17146

    2025 arXiv

  38. [47]

    Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. https://doi.org/10.48550/ARXIV.2212.03533 Text Embeddings by Weakly - Supervised Contrastive Pre -training . arXiv preprint. Version Number: 2

    2022 doi arXiv

  39. [48]

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. https://arxiv.org/abs/2401.00368 Improving text embeddings with large language models . Preprint, arXiv:2401.00368

    2024 arXiv

  40. [50]

    Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, Oskar Hallström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, Nathan Cooper, Griffin Adams, Jeremy Howard, and Iacopo Poli. 2024 b . https://arxiv.org/abs/2412.13663 Smarter, better...

    2024 arXiv

  41. [51]

    Wenhan Xiong, Xiang Lorraine Li, Srini Iyer, Jingfei Du, Patrick Lewis, William Yang Wang, Yashar Mehdad, Wen tau Yih, Sebastian Riedel, Douwe Kiela, and Barlas Oğuz. 2021. https://arxiv.org/abs/2009.12756 Answering complex open-domain questions with multi-hop dense retrieval ...

    2021 arXiv

  42. [52]

    Peng Xu, Wei Ping, Xianchao Wu, Lawrence McAfee, Chen Zhu, Zihan Liu, Sandeep Subramanian, Evelina Bakhturina, Mohammad Shoeybi, and Bryan Catanzaro. 2024. https://arxiv.org/abs/2310.03025 Retrieval meets long context large language models . Preprint, arXiv:2310.03025

    2024 arXiv

  43. [53]

    An Yang, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoyan Huang, Jiandong Jiang, Jianhong Tu, Jianwei Zhang, Jingren Zhou, Junyang Lin, Kai Dang, Kexin Yang, Le Yu, Mei Li, Minmin Sun, Qin Zhu, Rui Men, Tao He, and 9 others. 2025. https://arxiv.org/abs/2501.15383 Qwen2....

    2025 arXiv

  44. [54]

    Xin Zhang, Yanzhao Zhang, Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang, Pengjun Xie, Fei Huang, and 1 others. 2024. mgte: Generalized long-context text representation and reranking models for multilingual text retrieval. In Proceedings of the 2024 Conf...

    2024

  45. [56]

    Zijie Zhong, Hanwen Liu, Xiaoya Cui, Xiaofan Zhang, and Zengchang Qin. 2025 b . https://arxiv.org/abs/2406.00456 Mix-of-granularity: Optimize the chunking granularity for retrieval-augmented generation . Preprint, arXiv:2406.00456

    2025 arXiv

  46. [57]

    Yang Zhou, Hongyi Liu, Zhuoming Chen, Yuandong Tian, and Beidi Chen. 2025. https://arxiv.org/abs/2502.05252 Gsm-infinite: How do your llms behave over infinitely increasing context length and reasoning complexity? Preprint, arXiv:2502.05252

    2025 arXiv

  47. [58]

    Dawei Zhu, Liang Wang, Nan Yang, Yifan Song, Wenhao Wu, Furu Wei, and Sujian Li. 2024. https://arxiv.org/abs/2404.12096 Longembed: Extending embedding models for long context retrieval . Preprint, arXiv:2404.12096

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.