Pith. sign in

REVIEW 4 major objections 5 minor 37 references

This paper claims that generating negative samples at four similarity levels and weighting anchor tokens makes a 7B decoder the best text embedder on MTEB, at 67.0 average.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A new MTEB state-of-the-art for text embeddings is reported by combining multi-granularity LLM-generated hard negatives with curriculum training and an anchor-token-aware pooling method.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Useful incremental recipe for embedding training, but the ATA pooling mechanism looks mis-specified in the math, and the missing code/data make the small SOTA margins hard to trust. the 4 major comments →

arxiv 2509.00842 v1 pith:MCBTYTZD submitted 2025-08-31 cs.CL

Negative Matters: Multi-Granularity Hard-Negative Synthesis and Anchor-Token-Aware Pooling for Enhanced Text Embeddings

Tengyu Pan , Zhichao Duan , Zhenyu Li , Bowen Dong , Ning Liu , Xiuxing Li , Jianyong Wang This is my paper
classification cs.CL
keywords text embeddingshard-negative synthesiscurriculum learninganchor-token aware poolingcontrastive learningMTEBlarge language models
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that two cheap ingredients get a 7-billion-parameter language model to the top of the standard MTEB text-embedding benchmark. The first is a data-synthesis step: prompt a large language model to emit, for each query, four hard-negative documents arranged from most to least similar, then feed those negatives to the embedding model in that coarse-to-fine order during supervised contrastive training. The second is a pooling step: read the final-layer attention matrix along its Key dimension, turn those scores into log-scaled, length-normalized token weights, and use them to blend the final hidden states instead of averaging them. If the results hold, the recipe matches or beats prior systems that use extra pooling layers or far more training data, under the same 180M-token synthetic budget.

Core claim

The central claim is that a multi-granularity hard-negative (MGH) synthesis framework plus an anchor-token-aware (ATA) pooling method jointly improve English text embeddings enough to set a new state of the art on the 56-task MTEB benchmark (67.0 average with full data) under a fixed synthetic token budget, without touching the benchmark's training split. The data side works by having an LLM generate four ranked negatives per query; the training side feeds them from easiest to hardest; the pooling side reweights tokens by the summed log-scaled attention they receive along the Key dimension in the final layer. The combination is credited with both the benchmark gains and a more stable trainin

What carries the argument

Two mechanisms carry the argument. MGH is a data-generation template: the LLM is prompted to return a JSON object with four hard_negative_examples awarded similarity levels high, medium, medium-low, and low, ranked by decreasing similarity to the query; these provide a controllable difficulty ladder for curriculum learning. ATA is a pooling operation: the anchor weight of each token is wi = sum over attention heads h and Key positions j of log( a_ij^h * K + 1 ), where K is sequence length; after normalizing the weights, the sentence embedding is the weighted sum of the final hidden states. MGH supplies the coarse-to-fine curriculum signal, ATA supplies the token-salience signal, and both are

Load-bearing premise

The ATA gains rest on the premise that the anchor-token aggregation pattern observed in causal LLMs persists unchanged after the causal-to-bidirectional attention transformation, and that final-layer attention scores are a faithful proxy for how much semantic information each token carries; the paper shows one worked example, not a systematic survey.

What would settle it

Train the same model with ATA pooling but replace the computed anchor weights with random or uniform weights over final-layer tokens; if the MTEB score stays at the ATA level or does not fall below the mean-pooling baseline, the attentional reweighting is not doing the claimed work. Alternatively, evaluate the trained embedder on inputs that lack the three anchor tokens the paper highlights (initial token, punctuation, and the appended [INST] marker); if scores hold, the anchor-token explanation is incomplete.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the claimed result holds, a 7B bidirectional decoder trained with MGH+ATA on public retrieval data reaches 67.0 MTEB average, ahead of the compared E5, SPEED, and Gecko recipes under the same token budget.
  • ATA pooling adds no parameters and negligible runtime (about 32.4 hours with or without it), so it can be layered onto any bidirectional decoder embedding model without architectural change.
  • The curriculum ordering is load-bearing: reverse or random negative-level orderings produce worse and less stable final scores than the easy-to-hard progression.
  • Without using the MTEB training split, MGH with in-context learning scores 66.43, exceeding prior public-data-only systems.
  • Post-hoc similarity measurements confirm the ladder really exists in the data: the hardest negative averages 0.881 cosine with the query and the easiest 0.793.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If final-layer attention truly tracks token salience, the same ATA weights could plausibly be transferred to untrained or frozen embedders, but the paper only applies it after supervised fine-tuning; testing ATA on a zero-shot bidirectional decoder would isolate the benefit from the training signal.
  • The similarity-ladder idea could be inverted to produce multi-granularity positives or used at inference time to mine hard negatives from a retriever's own near-misses, which the paper leaves untested.
  • The length-normalization constant (multiplying attention by K) suggests raw attention magnitudes are not comparable across sequence lengths, implying ATA should matter most on long-document tasks; the paper's MTEB breakdown does not single out long-text categories to show that.
  • A follow-up could randomize the attention-weight ordering across heads while keeping the same aggregate weight; if that preserves the gain, the 'anchor token' interpretation is not the operative mechanism.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes two contributions for contrastively trained text embedding models: (1) a Multi-Granularity Hard-negative (MGH) synthesis framework, in which an LLM generates four negatives per query with explicitly ranked similarity levels, and training proceeds with a coarse-to-fine curriculum; and (2) an Anchor Token Aware (ATA) pooling method, which reweights final-layer hidden states using log-scaled attention scores in order to emphasize tokens that aggregate information. Experiments are conducted on Mistral-7B-Instruct-v0.2 converted to bidirectional attention following LLM2Vec. On MTEB, MGH reports 64.5 (synthetic only) and 67.0 (with public retrieval) versus prior synthesis methods; under the no-MTEB-training-split setting, ATA reports 65.87 versus 65.80 for NV-Embed pooling, and with in-context instructions MGH reports 66.43 versus 66.08 for bge-en-icl. Ablations address curriculum order and pooling choices.

Significance. If the results are reproducible, MGH is a practically useful and low-cost modification of LLM-based hard-negative synthesis: it keeps the token budget fixed, uses a standard backbone, and yields consistent gains in synthetic-only and combined-data settings. ATA is attractive in principle as a parameter-free pooling alternative. However, the quantitative support for the ATA-specific claim is thin: the headline margin over NV-Embed pooling is 0.07 MTEB points with no variance information, and Eq. (3) as written does not literally implement the described anchor-token weighting. The MGH contribution is better supported, including the clean difficulty-ordering verification in Table 4. The paper deserves serious consideration after the pooling mechanism is clarified and the small-margin claims are made statistically defensible.

major comments (4)
  1. [§2.2, Eq. (3)] The formula does not implement the anchor-token mechanism described in the text. With a^h_{ij} the attention from Query i to Key j, w_i = Σ_h Σ_j log(a^h_{ij}K+1) sums over the key index for each fixed query, so it scores each token by how broadly it attends to the rest of the sequence. Anchor tokens, as defined in §5.2 and Figure 5, are key positions that many queries attend to; the corresponding column sum would be Σ_h Σ_i log(a^h_{ij}K+1) for fixed key j. The scaling justification is also inconsistent: in row-normalized attention, row sums Σ_j a_{ij}=1 are constant, not column sums Σ_i a_{ij}=1. If the implementation follows Eq. (3), the reported gains are evidence for an entropy-like query-side weighting rather than anchor-token reweighting; if the implementation follows the described column-sum mechanism, the equation and text must be corrected. Please report the exact index orienta
  2. [§4.2, Table 3] ATA pooling is credited with a 0.07 MTEB point improvement over NV-Embed pooling (65.87 vs 65.80) and 0.46 over mean pooling. The table reports a single run with no variance or significance testing. Given typical MTEB run-to-run noise for 7B-scale models, a 0.07-point difference cannot by itself support the conclusion that ATA outperforms NV-Embed pooling. The same concern applies to Table 2, where the headline comparison (MGH w/ icl 66.43 vs bge-en-icl 66.08) is a 0.35-point margin. Please report multiple seeds or bootstrap confidence intervals over tasks, or temper the SOTA claims accordingly.
  3. [§4.1, Figure 3] The curriculum order and the fixed-difficulty comparisons are evaluated on a 25-task subset of MTEB, and the chosen order ('curriculum learning, as adopted in our main approach') is then used to produce the full-MTEB results in Tables 1 and 2. Because the validation signal comes from the same benchmark on which the final numbers are reported, the headline curriculum gain reflects selection on the target benchmark rather than an independent evaluation. This does not invalidate the synthesis contribution, but it weakens the curriculum claim. Please either use a non-MTEB validation set for the ordering decision, or report full-MTEB results for all four orderings, or at minimum discuss the selection risk explicitly.
  4. [§5.2, Figure 5] The claim that 'the aggregation pattern still remains when the base model is transformed from causal to bidirectional attention' and that 'most samples allocate a greater proportion of the ATA weight to the three anchor patterns' is supported by one randomly selected example plus qualitative observation. This is not sufficient to establish that final-layer attention in the bidirectional model is a reliable anchor-token signal, especially given the index-orientation issue in Eq. (3). Please provide a systematic analysis, for example the distribution of top-weight token types across the MTEB subset, or a correlation between ATA weights and attention-column mass, or temper the mechanistic claims.
minor comments (5)
  1. [§2.1 / Figure 2] The prompt template contains malformed quotation marks and the literal placeholder 'Omitted for space limitations' inside the JSON output specification. This should be cleaned and the full template provided in a readable form.
  2. [§2.1, Eq. (1)] The InfoNCE loss is typeset with missing braces around the denominator, and the exponent is written as e^{τ·φ} rather than e^{φ/τ} (with τ=1 this is not consequential, but the form should be conventional).
  3. [Table 2] The abbreviation 'icl' is used without definition in the caption or text. It should be spelled out and the relationship to bge-en-icl made explicit.
  4. [§2.1 / §4.1] The curriculum schedule is described as 'gradually feed the negative samples from {N4} to {N1}, allocating equal proportions (25% each)'. The exact schedule (which difficulty levels are active at each training stage) should be stated precisely.
  5. [Reproducibility] No code or data release is mentioned. For a data-synthesis paper, releasing the prompts, generated negatives, and training code would substantially aid reproducibility; please add a reproducibility statement.

Circularity Check

0 steps flagged

No significant circularity: MTEB results are external benchmarks; ATA pooling is a parameter-free heuristic grounded in independent prior work.

full rationale

The paper's central claims are validated by external MTEB benchmark scores, not by reductions to fitted parameters or self-referential definitions. The MGH synthesis framework generates multi-granularity hard negatives using an LLM prompt; the downstream contrastive training and MTEB evaluation are independent of the generation procedure. The similarity statistics in Table 4 verify that the LLM followed the prompt's ranking instruction, which is a sanity check rather than a circular prediction. The ATA pooling method introduces no learned parameters; it computes token weights directly from the final-layer attention matrix (Eq. 3-5) and is motivated by aggregation-pattern findings in prior work (Wang et al. 2023a; Huang et al. 2024) that are not authored by the present paper. No equation in the paper is equivalent to a fitted value, and no load-bearing claim relies on a self-citation. The possible index-orientation concern in Eq. (3) is a correctness/robustness issue about whether the implemented weighting matches the stated anchor-token mechanism; it does not make the benchmark results circular. The paper's ablations are performed on an external benchmark subset, and the full MTEB results are reported transparently. There is no self-referential derivation chain, so the circularity score is 0.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The central results rest on the reliability of LLM-generated ordered negatives, the persistence of anchor-token aggregation in bidirectional models, and the validity of MTEB subset ablations as a guide for full-benchmark choices.

free parameters (4)
  • Number of granularity levels (k) = 4
    Chosen in Section 2.1 without sensitivity analysis; all experiments use four negatives per query.
  • Curriculum proportion per level = 25% each
    Section 2.1: negatives fed from N4 to N1 with equal 25% allocation; no ablation on other proportions.
  • ATA weighting constants = K multiplier, +1 offset, log
    Equation 3 uses log(a_ij*K+1); the constants stabilize length scaling but are a hand-chosen heuristic with no theoretical derivation.
  • InfoNCE temperature tau = 1
    Equation 1 sets tau=1; a standard setting, not tuned.
axioms (4)
  • domain assumption LLMs follow the multi-granularity prompt and produce negatives with the intended ordered similarity
    Relied on in Section 2.1; Table 4 and the case study give partial evidence, but it is a capability assumption about closed models.
  • domain assumption Anchor-token aggregation holds in bidirectional-attention embedding models
    Assumed in Section 2.2 and illustrated with one example in Figure 5; based on prior work on causal models.
  • domain assumption The 25-task MTEB subset is representative for selecting the final method
    Ablations in Section 4.1 and Appendix C.2 choose curriculum order and pooling on this subset; the full MTEB is then used for the headline numbers.
  • domain assumption The public retrieval training corpora are not contaminated with MTEB training splits
    Section 3.1 and B.3 follow prior splits; overlap in dataset names (for example MS MARCO) exists but train/eval splits are distinct.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Negative Matters: Multi-Granularity Hard-Negative Synthesis and Anchor-Token-Aware Pooling for Enhanced Text Embeddings." pith.science (2026). https://pith.science/paper/MCBTYTZD

@misc{pith2026250900842,
  author       = {Pith},
  title        = {Pith review of: Negative Matters: Multi-Granularity Hard-Negative Synthesis and Anchor-Token-Aware Pooling for Enhanced Text Embeddings},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MCBTYTZD}},
  note         = {Machine review of arXiv:2509.00842}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Text embedding models are essential for various natural language processing tasks, enabling the effective encoding of semantic information into dense vector representations. These models are typically optimized using triplets of (query, positive, negative) data pairs for contrastive learning, where the negative samples play a critical role in enhancing the model's ability to discern subtle semantic distinctions. In this work, we introduce a Multi-Granularity Hard-negative (MGH) synthesis framework that leverages large language models (LLMs) to generate diverse negative samples with varying levels of similarity with the query. This approach facilitates a coarse-to-fine curriculum learning strategy during supervised training, allowing the embedding model to progressively learn more nuanced semantic representations. Meanwhile, we propose an Anchor Token Aware (ATA) pooling method that assigns higher weights to anchor tokens based on aggregation patterns observed in LLMs, improving text embedding accuracy without increasing model complexity. Comprehensive experiments on the MTEB benchmark demonstrate that our methods achieve state-of-the-art performance, surpassing existing synthesis strategies both with synthetic data and when combined with public retrieval datasets.

Figures

Figures reproduced from arXiv: 2509.00842 by Bowen Dong, Jianyong Wang, Ning Liu, Tengyu Pan, Xiuxing Li, Zhenyu Li, Zhichao Duan.

Figure 1
Figure 1. Figure 1: Illustration of our proposed multi-granularity [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The core template used for prompting LLMs [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Trend of MTEB subset scores during super [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: An example of a multi-granularity hard negative synthesis, randomly selected for illustration. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: The upper part illustrates the summed results [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Distribution of the task categories in the syn [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The individual attention matrices from all 32 attention heads in the last layer of the bidirectional model, [PITH_FULL_IMAGE:figures/full_fig_p017_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 6 canonical work pages

  1. [1]

    Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, et al. 2016. Ms marco: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268

    Pith/arXiv arXiv 2016

  2. [2]

    Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy. 2024. Llm2vec: Large language models are secretly powerful text encoders. arXiv preprint arXiv:2404.05961

    Pith/arXiv arXiv 2024

  3. [3]

    Luiz Henrique Bonifacio, Hugo Queiroz Abonizio, Marzieh Fadaee, and Rodrigo Nogueira. 2022. https://api.semanticscholar.org/CorpusID:246705967 Inpars: Data augmentation for information retrieval using large language models . ArXiv, abs/2202.05144

    Pith/arXiv arXiv 2022

  4. [4]

    Haonan Chen, Liang Wang, Nan Yang, Yutao Zhu, Ziliang Zhao, Furu Wei, and Zhicheng Dou. 2024. https://arxiv.org/abs/2410.18634 Little giants: Synthesizing high-quality embedding data at scale . Preprint, arXiv:2410.18634

    Pith/arXiv arXiv 2024

  5. [5]

    Zhao, Ji Ma, Yi Luan, Jianmo Ni, Jing Lu, Anton Bakalov, Kelvin Guu, Keith B

    Zhuyun Dai, Vincent Y. Zhao, Ji Ma, Yi Luan, Jianmo Ni, Jing Lu, Anton Bakalov, Kelvin Guu, Keith B. Hall, and Ming-Wei Chang. 2022. https://arxiv.org/abs/2209.11755 Promptagator: Few-shot dense retrieval from 8 examples . Preprint, arXiv:2209.11755

    Pith/arXiv arXiv 2022

  6. [6]

    DataCanary, hilfialkaff, Lili Jiang, Meg Risdal, Nikhil Dandekar, and tomtung. 2017. Quora question pairs. https://kaggle.com/competitions/quora-question-pairs. Kaggle

    work page 2017

  7. [7]

    Angela Fan, Yacine Jernite, Ethan Perez, David Grangier, Jason Weston, and Michael Auli. 2019. https://doi.org/10.18653/v1/P19-1346 ELI 5: Long form question answering . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3558--3567, Florence, Italy. Association for Computational Linguistics

    doi:10.18653/v1/p19-1346 2019

  8. [8]

    Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.552 S im CSE : Simple contrastive learning of sentence embeddings . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894--6910, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics

    doi:10.18653/v1/2021.emnlp-main.552 2021

  9. [9]

    Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. 2024. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13418--13427

    2024

  10. [10]

    Vitor Jeronymo, Luiz Bonifacio, Hugo Abonizio, Marzieh Fadaee, Roberto Lotufo, Jakub Zavrel, and Rodrigo Nogueira. 2023. https://arxiv.org/abs/2301.01820 Inpars-v2: Large language models as efficient dataset generators for information retrieval . Preprint, arXiv:2301.01820

    Pith/arXiv arXiv 2023

  11. [11]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825

    Pith/arXiv arXiv 2023

  12. [12]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.550 Dense passage retrieval for open-domain question answering . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769--6781, Online. Ass...

    doi:10.18653/v1/2020.emnlp-main.550 2020

  13. [13]

    Ekaterina Khramtsova, Shengyao Zhuang, Mahsa Baktashmotlagh, and Guido Zuccon. 2024. Leveraging llms for unsupervised dense retriever ranking. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1307--1317

    work page 2024

  14. [14]

    Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. 2024 a . Nv-embed: Improved techniques for training llms as generalist embedding models. arXiv preprint arXiv:2405.17428

    Pith/arXiv arXiv 2024

  15. [15]

    Jinhyuk Lee, Zhuyun Dai, Xiaoqi Ren, Blair Chen, Daniel Cer, Jeremy R Cole, Kai Hui, Michael Boratko, Rajvi Kapadia, Wen Ding, et al. 2024 b . Gecko: Versatile text embeddings distilled from large language models. arXiv preprint arXiv:2403.20327

    Pith/arXiv arXiv 2024

  16. [16]

    Chaofan Li, MingHao Qin, Shitao Xiao, Jianlyu Chen, Kun Luo, Yingxia Shao, Defu Lian, and Zheng Liu. 2024. Making text embedders few-shot learners. arXiv preprint arXiv:2409.15700

    Pith/arXiv arXiv 2024

  17. [17]

    Xueguang Ma, Liang Wang, Nan Yang, Furu Wei, and Jimmy Lin. 2024. Fine-tuning llama for multi-stage text retrieval. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2421--2425

    work page 2024

  18. [18]

    Niklas Muennighoff. 2022. https://arxiv.org/abs/2202.08904 Sgpt: Gpt sentence embeddings for semantic search . Preprint, arXiv:2202.08904

    Pith/arXiv arXiv 2022

  19. [19]

    Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023. Mteb: Massive text embedding benchmark. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2014--2037

    work page 2023

  20. [20]

    Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, Johannes Heidecke, Pranav Shyam, Boris Power, Tyna Eloundou Nekoul, Girish Sastry, Gretchen Krueger, David Schnurr, Felipe Petroski Such, Kenny Hsu, Madeleine Thompson, Tabarak Khan, Toki Sherbakov, Joanne Jang, P...

    Pith/arXiv arXiv 2022

  21. [21]

    Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hernandez Abrego, Ji Ma, Vincent Zhao, Yi Luan, Keith Hall, Ming-Wei Chang, and Yinfei Yang. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.669 Large dual encoders are generalizable retrievers . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9844--9855, A...

    doi:10.18653/v1/2022.emnlp-main.669 2022

  22. [22]

    Rodrigo Nogueira, Wei Yang, Jimmy Lin, and Kyunghyun Cho. 2019. Document expansion by query prediction. arXiv preprint arXiv:1904.08375

    Pith/arXiv arXiv 2019

  23. [23]

    Yifu Qiu, Hongyu Li, Yingqi Qu, Ying Chen, QiaoQiao She, Jing Liu, Hua Wu, and Haifeng Wang. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.357 D u R eader-retrieval: A large-scale C hinese benchmark for passage retrieval from web search engine . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5326--5338...

    work page doi:10.18653/v1/2022.emnlp-main.357 2022

  24. [24]

    Jacob Mitchell Springer, Suhas Kotha, Daniel Fried, Graham Neubig, and Aditi Raghunathan. 2024. Repetition improves language model embeddings. arXiv preprint arXiv:2402.15449

    Pith/arXiv arXiv 2024

  25. [25]

    Saba Sturua, Isabelle Mohr, Mohammad Kalim Akram, Michael Günther, Bo Wang, Markus Krimmel, Feng Wang, Georgios Mastrapas, Andreas Koukounas, Nan Wang, and Han Xiao. 2024. https://arxiv.org/abs/2409.10173 jina-embeddings-v3: Multilingual embeddings with task lora . Preprint, arXiv:2409.10173

    Pith/arXiv arXiv 2024

  26. [26]

    James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. https://doi.org/10.18653/v1/N18-1074 FEVER : a large-scale dataset for fact extraction and VER ification . In Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pap...

    doi:10.18653/v1/n18-1074 2018

  27. [27]

    Lean Wang, Lei Li, Damai Dai, Deli Chen, Hao Zhou, Fandong Meng, Jie Zhou, and Xu Sun. 2023 a . Label words are anchors: An information flow perspective for understanding in-context learning. In The 2023 Conference on Empirical Methods in Natural Language Processing

    work page 2023

  28. [28]

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. https://doi.org/10.18653/v1/2024.acl-long.642 Improving text embeddings with large language models . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11897--11916, Bangkok, Thailand. Association f...

    doi:10.18653/v1/2024.acl-long.642 2024

  29. [29]

    Liang Wang, Nan Yang, and Furu Wei. 2023 b . https://doi.org/10.18653/v1/2023.emnlp-main.585 Query2doc: Query expansion with large language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9414--9423, Singapore. Association for Computational Linguistics

    doi:10.18653/v1/2023.emnlp-main.585 2023

  30. [30]

    Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian-Yun Nie. 2024. https://doi.org/10.1145/3626772.3657878 C-pack: Packed resources for general chinese embeddings . In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '24, page 641–649, New York, NY, USA. Assoc...

    arXiv 2024

  31. [31]

    Xiaohui Xie, Qian Dong, Bingning Wang, Feiyang Lv, Ting Yao, Weinan Gan, Zhijing Wu, Xiangsheng Li, Haitao Li, Yiqun Liu, and Jin Ma. 2023. https://arxiv.org/abs/2304.03679 T2ranking: A large-scale chinese benchmark for passage ranking . Preprint, arXiv:2304.03679

    Pith/arXiv arXiv 2023

  32. [32]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. https://doi.org/10.18653/v1/D18-1259 H otpot QA : A dataset for diverse, explainable multi-hop question answering . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2369--2380, Brussels...

    doi:10.18653/v1/d18-1259 2018

  33. [33]

    Xin Zhang, Zehan Li, Yanzhao Zhang, Dingkun Long, Pengjun Xie, Meishan Zhang, and Min Zhang. 2023 a . https://arxiv.org/abs/2310.08232 Language models are universal embedders . Preprint, arXiv:2310.08232

    Pith/arXiv arXiv 2023

  34. [34]

    Xinyu Zhang, Xueguang Ma, Peng Shi, and Jimmy Lin. 2021. https://doi.org/10.18653/v1/2021.mrl-1.12 Mr. T y D i: A multi-lingual benchmark for dense retrieval . In Proceedings of the 1st Workshop on Multilingual Representation Learning, pages 127--137, Punta Cana, Dominican Republic. Association for Computational Linguistics

    doi:10.18653/v1/2021.mrl-1.12 2021

  35. [35]

    Xinyu Zhang, Nandan Thakur, Odunayo Ogundepo, Ehsan Kamalloo, David Alfonso-Hermelo, Xiaoguang Li, Qun Liu, Mehdi Rezagholizadeh, and Jimmy Lin. 2023 b . https://doi.org/10.1162/tacl_a_00595 MIRACL : A multilingual retrieval dataset covering 18 diverse languages . Transactions of the Association for Computational Linguistics, 11:1114--1131

    doi:10.1162/tacl_a_00595 2023

  36. [36]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  37. [37]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.