Pith. sign in

REVIEW 3 major objections 5 minor 79 references

Extreme low-bit quantization of LLM backbones can yield text embedders that nearly match full-precision teachers while running about twice as fast and storing vectors at 1 to 16 bits per dimension from a single checkpoint.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-02 10:09 UTC pith:U2HR7NIP

load-bearing objection Useful engineering result on 1.58-bit LLM embedders, but the missing teacher+CPT control means the reported quality gap understates the true cost of quantization. the 3 major comments →

arxiv 2606.25674 v2 pith:U2HR7NIP submitted 2026-06-24 cs.CL cs.IR

BitNet Text Embeddings

Zhen Li , Xin Huang , Liang Wang , Nan Yang , Ting Song , Yan Xia , Xun Wu , Shaohan Huang
show 3 more authors
Huishuai Zhang Furu Wei Dongyan Zhao
This is my paper
classification cs.CL cs.IR MSC 68T5068T07
keywords BitNettext embeddingsternary weightsquantization-aware trainingcontrastive pre-trainingattention distillationmulti-precision embeddingsMMTEB
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that converting an LLM into a BitNet-style 1.58-bit (ternary-weight) encoder, then adapting it with large-scale contrastive pre-training and teacher distillation, preserves most of the semantic quality of a full-precision embedder. On MMTEB (English v2), the low-bit model scores 67.60 on a Qwen3-0.6B backbone against 67.95 for the FP16 teacher, and 66.10 against 66.71 on Gemma3-270M — gaps of about half a point. The same checkpoint supports output embeddings at 1, 2, 4, 8, or 16 bits per dimension, with 4- and 8-bit nearly lossless (67.45 and 67.55 vs 67.60) and 1-bit dropping to 64.43 but still usable. It also reports roughly 2x CPU token throughput in this setup. If true, this would make LLM-based embedders practical for storage-constrained retrieval and CPU/edge deployment without retraining per memory budget.

Core claim

The central discovery is that extreme low-bit quantization is not a barrier to representation learning when the conversion is paired with a specific recovery recipe. BitNet-style ternary weights and 8-bit activations disrupt the embedding space; three ingredients restore it: continual contrastive pre-training on 1 billion text pairs, supervised fine-tuning with two distillation losses (KL over cosine-similarity distributions and KL over attention-relation distributions at one middle layer), and sub-layer normalization in each transformer block. With these, BITEMBED lands within 0.35-0.61 points of an FP16 teacher on MMTEB (eng, v2) across two backbones. Training output embeddings to be score

What carries the argument

The load-bearing mechanism is the BitNet-style quantized transformer block: weights are rounded to ternary values {-Δw, 0, Δw} with an absmax scale (1.58-bit) and activations are quantized to 8 bits per token, with straight-through estimator gradients. Sub-layer normalization (SubLN) inserted before each quantized projection stabilizes training. Recovery is driven by (1) InfoNCE continual contrastive pre-training on 1B pairs, (2) supervised InfoNCE with in-batch and hard negatives plus two KL distillation losses — similarity-distribution distillation over candidate scores and attention-relation distillation over Q/K/V matrix row-relation distributions at a chosen layer — and (3) multi-precis

Load-bearing premise

The result is measured against a full-precision teacher built from the same backbone and supervised data but without the 1B-pair continual pre-training the student receives, so 'largely comparable to the teacher' does not by itself establish competitiveness with the strongest deployed embedders; the CPU throughput figures also rely on undocumented kernel details.

What would settle it

Run the same recipe but give the FP16 teacher the 1B-pair continual pre-training before supervised fine-tuning (or compare BITEMBED directly against a top public embedder of similar size on MMTEB retrieval and multilingual tasks). If the gap widens well beyond the reported ~0.5 points — or if 1-bit/2-bit output embeddings lose more than 10 points on out-of-distribution retrieval queries in an ANN index — the central 'largely comparable' and mild-degradation claims would need qualification.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If the claim holds, LLM-based embedders can run on ternary backbones, roughly halving CPU inference cost — the main barrier to using LLMs as embedders in latency-sensitive or edge settings.
  • Storage shrinks: 4-bit and 8-bit output embeddings keep essentially all of the 16-bit quality (67.45/67.55 vs 67.60 on Qwen3-0.6B), so indexes can drop to 1/4 or 1/2 the fp16 footprint with negligible loss.
  • A single multi-precision checkpoint replaces several separately trained encoders; practitioners choose 1-, 2-, 4-, 8-, or 16-bit storage at inference time to fit the memory budget.
  • Retrieval is the most quantization-sensitive task (~8-point drop at 1-bit output for Qwen3-0.6B), so extremely low-bit vectors should be paired with re-ranking or higher-precision shortlists.
  • The full recipe — BitNet conversion, continual contrastive pre-training, and score+attention distillation — recovers 8.68 points over plain supervised fine-tuning of the quantized backbone (58.92 to 67.60), showing the adaptation pipeline, not the quantization alone, is what unlocks the result.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the paper's reference point is a same-backbone FP16 teacher that skipped the 1B-pair continual pre-training given to the student. Against a teacher that also received that stage — or against a leading deployed embedder — the ~0.5-point gap may widen; the paper does not test that comparison.
  • Editorial inference: the ~2x CPU speedup is reported at 8 threads without kernel details; real-world speedups on AVX-512, NEON, or GPU will depend on how well ternary/8-bit matmul kernels are optimized. Treat the efficiency number as setup-specific.
  • Editorial inference: multi-precision training buys its largest gains on retrieval (2.64 points at 1-bit over post-hoc quantization), suggesting the retained dimensions carry more retrievable structure. A natural extension is hybrid search: 1-bit/2-bit codes as a coarse filter, 4-bit/16-bit only on the shortlist.
  • Editorial inference: since classification and STS degrade little at 1-2 bits, low-bit storage may be a good fit for semantic-similarity applications even where it hurts retrieval; this is a testable deployment niche the paper leaves implicit.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents BITEMBED, a framework for extreme low-bit LLM-based text embeddings. It converts pretrained LLM backbones (Qwen3-0.6B, Gemma3-270M) into BitNet-style encoders with ternary weights, token-wise 8-bit activations, and SubLN refinement; then performs continual contrastive pre-training on 1B text pairs, followed by supervised contrastive fine-tuning with similarity-distribution and attention-relation distillation from an FP16 teacher. It also trains output embeddings to support 1-, 2-, 4-, 8-, and 16-bit storage precisions. Experiments on MMTEB (eng, v2) report that BITEMBED is within 0.35/0.61 points of the FP16 teacher, with roughly 2× CPU token throughput and graceful degradation under output-precision reduction.

Significance. The paper is the first systematic study, to my knowledge, of extreme low-bit quantization for LLM-based text embeddings, and it combines several nontrivial ingredients: BitNet-style backbone conversion, continual contrastive pre-training, two distillation objectives, and multi-precision output embedding training. The ablations in Table 2 are informative and support the contribution of each component, and the multilingual results in Appendix C.2 plus the MRL comparison broaden the evaluation. If the quality comparison were properly controlled, the framework would be a meaningful step toward deployable low-bit embedders. However, the central external claim—'largely comparable to full-precision teacher'—is weakened by an unmatched teacher baseline and by the absence of statistical precision measures. The authors are transparent that they do not compare against production embedders, which is a reasonable scope choice, but it makes the internal teacher comparison the only yardstick, and that yardstick must be fair.

major comments (3)
  1. [§4.1, Table 1, §3.3] The headline comparison is not apples-to-apples. BITEMBED receives continual contrastive pre-training on 1B text pairs (§3.3), while the FP16 teacher is 'fine-tuned directly on the same supervised fine-tuning data' without that stage. Table 2 shows CPT is worth 2.83 points (67.60 vs 64.77). A full-precision model that also received the 1B-pair CPT could plausibly score several points above the 67.95 teacher, which would make the true cost of extreme low-bit quantization much larger than the reported 0.35/0.61 gaps. Please add a matched FP16+CPT baseline, or otherwise quantify the quantization loss after equalizing the training data. The current wording 'largely comparable to full precision teacher' is not yet supported as a statement about quantization preservation.
  2. [§4.2, Tables 1–2] No error bars, multiple seeds, or significance tests are reported. The central claim rests on differences of 0.35 (Qwen3) and 0.61 (Gemma3) in average MMTEB points, with large per-task swings (e.g., retrieval: 55.34 vs 54.03; summarization: 31.87 vs 32.06). Without repeated training runs or paired significance tests, these differences may be within typical noise for some task categories. Report seed variance or at least paired tests over tasks to support the 'largely comparable' claim.
  3. [§4.1/§4.2 efficiency claim] The 'approximately 2× CPU token throughput' is reported for 'CPU with 8 threads' only. No CPU model, quantization kernel implementation, thread pinning, or measurement methodology is given, and no variance is reported. Since the paper's second contribution is deployment efficiency, please document the benchmark environment and, ideally, compare against the same backend in FP16. Without this, the speedup cannot be reproduced or assessed.
minor comments (5)
  1. [§2.2] 'Bitnet distllation' should be 'BitNet Distillation'.
  2. [§3.5, Eq. (10)] Define sgn+ for zero explicitly (the text says sgn+(0)=1, but the domain of e_j is not stated), and clarify whether the scale α(e) is stored as part of the code or reconstructable.
  3. [Table 7] The BITEMBED-2bit row for Qwen3-0.6B reports 64.9 instead of 64.90, inconsistent with other rows.
  4. [Appendix C.2] 'superivised' typo; the reference to 'harrier-oss-v12' is informal and should be a proper citation or link.
  5. [Figure 2] The rightmost column labeled 'BITEMBED' is a comparison to the FP16 teacher, while the other columns are within-BITEMBED precision differences; clarify in the caption to avoid confusion.

Circularity Check

0 steps flagged

No significant circularity: the teacher-student gap is partly a distillation target, but the benchmark result is emergent and all components are ablated.

full rationale

The claimed derivation chain is not circular in the hard sense. BitNet-style quantization (Eqs. 1-2), SubLN residual refinement (Eq. 3), continual contrastive pre-training (Eq. 4), supervised contrastive loss (Eq. 5), and the two distillation losses (Eqs. 6-9, combined in Eq. 12) are explicit training objectives; the MMTEB scores in Table 1 and Figure 1 are emergent evaluation quantities, not parameters fitted to those numbers. The only self-referential element is that BITEMBED is explicitly distilled to match the FP16 teacher's similarity distribution (Lscore, Eq. 7) and attention relations (Lattn, Eq. 9), so a small teacher-student gap is partly a training target rather than an independent discovery. This is not a reduction by construction: the loss does not directly optimize MMTEB averages, the extreme low-bit backbone capacity is not assumed away, and the ablations (Table 2) show each component contributes meaningfully (e.g., removing SubLN costs 2.12 points; removing CPT costs 2.83; removing all adaptation drops to 58.92). The paper's comparison baseline is intentionally a same-backbone FP16 teacher without the 1B-pair continual pre-training (§4.1), which is a legitimate external-validity limitation for the claim that low-bit training is performance-preserving, but it is a control/comparison issue, not circularity. Self-citations to BitNet, BitNet Distillation, E5, and MiniLM are used as building blocks and are internally evaluated; no load-bearing 'uniqueness' or ansatz is imported solely through self-citation.

Axiom & Free-Parameter Ledger

10 free parameters · 6 axioms · 0 invented entities

The central result is an empirical system: no derivation. It leans on standard QAT heuristics, distillation assumptions, and benchmark measurements. Free parameters are mostly hyperparameters chosen by hand, not fitted to the benchmark, but the attention-distillation layer is explicitly tuned using Figure 3. No new theoretical entities are introduced; all components (ternary weights, SubLN, distillation, Matryoshka-style quantization) come from cited prior work.

free parameters (10)
  • similarity-distillation weight λs = 0.2
    Weights the KL similarity-distribution loss in Eq. 12; chosen by hand.
  • attention-distillation weight λa = 10^4
    Weights the attention-relation KL loss in Eq. 12; chosen by hand.
  • temperature τ = 0.02
    Used in InfoNCE and distillation similarity distributions (Eqs. 4-6); set in §4.1.
  • attention distillation layer (Qwen3-0.6B) = 18
    Selected via Figure 3; performance peaks at layer 18.
  • attention distillation layer (Gemma3-270M) = 10
    Selected via Figure 3; best around layers 10-12.
  • supervised fine-tuning batch size = 128
    Set in §4.1; not swept.
  • number of hard negatives = 7
    Set in §4.1; not swept.
  • teacher / student learning rates = 5e-6 / 3e-5
    Set in §4.1; not swept.
  • output embedding bitwidth set R = {1,2,4,8,16}
    Chosen as supported storage precisions; multi-precision objective averages over them.
  • continual pre-training data size = 1B text pairs
    Following Multilingual E5; not justified by a sweep.
axioms (6)
  • domain assumption Straight-through estimator gives usable gradients through non-differentiable quantization (Eqs. 1-2).
    Relied on for all low-bit training; a known heuristic, not a theorem.
  • domain assumption Sub-layer normalization inside transformer blocks stabilizes ternary-weight training sufficiently (Eq. 3).
    Taken from prior BitNet work; the paper does not analyze failure modes.
  • domain assumption A full-precision teacher trained on the same supervised data is a valid reference for full-precision embedding quality (§4.1).
    The paper explicitly chooses this baseline over external embedders; if the teacher is weak, the comparison is favorable to the method.
  • domain assumption MMTEB (eng, v2) with the instruction templates in Table 4 is a representative evaluation of embedding quality.
    All quality conclusions rely on this benchmark.
  • domain assumption CPU token throughput with 8 threads is a meaningful measure of deployment efficiency (§4.1).
    No kernel details, memory bandwidth, or end-to-end index cost reported.
  • domain assumption The 1B-pair continual pre-training and BGE-en-ICL supervised data are sufficient to rebuild semantic structure after quantization (§3.3, §4.1).
    Data specifics and quality filters are not fully specified.

reviewed 2026-08-02 · how reviews work

0 comments
Cite this review

Pith. "Pith review of BitNet Text Embeddings." pith.science (2026). https://pith.science/paper/U2HR7NIP

@misc{pith2026260625674,
  author       = {Pith},
  title        = {Pith review of: BitNet Text Embeddings},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U2HR7NIP}},
  note         = {Machine review of arXiv:2606.25674}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

LLM-based text embedders have substantially improved retrieval and semantic representation quality, but their deployment remains costly: large backbone models slow down embedding inference, while high-dimensional full-precision embeddings impose substantial storage and bandwidth overhead on large-scale indexes. In this paper, we present BITEMBED, an extreme low-bit framework for LLM-based text embedding that jointly targets encoding efficiency and vector storage. BITEMBED converts pretrained LLM backbones into BitNet-style embedding encoders with ternary weights, quantized activations, and lightweight normalization refinement. The converted model is adapted to representation learning through continual contrastive pre-training, followed by supervised contrastive fine-tuning with both similarity-distribution distillation and attention-relation distillation from a full-precision teacher. Beyond quantizing the backbone, BITEMBED further trains output embeddings to support multiple storage precisions meeting different storage needs in various scenarios. Experiments on MMTEB (eng, v2) with Qwen3-0.6B and Gemma3-270M show that BITEMBED is largely comparable to full precision teacher embedders. Moreover, BITEMBED flexibly obtains text embeddings of various precisions, achieving a trade-off between performance and storage cost.

Figures

Figures reproduced from arXiv: 2606.25674 by Dongyan Zhao, Furu Wei, Huishuai Zhang, Liang Wang, Nan Yang, Shaohan Huang, Ting Song, Xin Huang, Xun Wu, Yan Xia, Zhen Li.

Figure 1
Figure 1. Figure 1: Performance-precision trade-off of BITEMBED on Qwen3-0.6B and Gemma3-270M. We report the average MMTEB (eng, v2) performance of 1-, 2-, 4-, 8-, and 16-bit output embeddings of BITEMBED [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Task-type sensitivity on MMTEB (eng, v2). Columns 1, 2, 4, and 8 report the performance [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Effect of the attention-relation distilla [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

79 extracted references · 1 canonical work pages

  1. [1]

    Semeval-2012 task 6: A pilot on semantic textual similarity

    Eneko Agirre, Daniel Cer, Mona Diab, and Aitor Gonzalez-Agirre. Semeval-2012 task 6: A pilot on semantic textual similarity. in* sem 2012: The first joint conference on lexical and computational semantics–volume 1: Proceedings of the main conference and the shared task, and volume 2: Proceedings of the sixth international workshop on semantic evaluation (...

    2012

  2. [2]

    Estimating or propagating gradients through stochastic neurons for conditional computation.arXiv preprint arXiv:1308.3432, 2013

    Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation.arXiv preprint arXiv:1308.3432, 2013

    Pith/arXiv arXiv 2013

  3. [3]

    Revela: Dense retriever learning via language modeling

    Fengyu Cai, Tong Chen, Xinran Zhao, Sihao Chen, Hongming Zhang, Sherry Tongshuang Wu, Iryna Gurevych, and Heinz Koeppl. Revela: Dense retriever learning via language modeling. arXiv preprint arXiv:2506.16552, 2025

    arXiv 2025

  4. [4]

    Efficient intent detection with dual sentence encoders.arXiv preprint arXiv:2003.04807, 2020

    Iñigo Casanueva, Tadas Temˇcinas, Daniela Gerz, Matthew Henderson, and Ivan Vuli´c. Efficient intent detection with dual sentence encoders.arXiv preprint arXiv:2003.04807, 2020

    Pith/arXiv arXiv 2003

  5. [5]

    Quartet: Native fp4 training can be optimal for large language models.Advances in Neural Information Processing Systems, 38:43552–43572, 2026

    Roberto Castro, Andrei Panferov, Rush Tabesh, Oliver Sieberling, Jiale Chen, Mahdi Nikdan, Saleh Ashkboos, and Dan Alistarh. Quartet: Native fp4 training can be optimal for large language models.Advances in Neural Information Processing Systems, 38:43552–43572, 2026

    2026

  6. [6]

    Semeval-2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation.arXiv preprint arXiv:1708.00055, 2017

    Daniel Cer, Mona Diab, Eneko Agirre, Inigo Lopez-Gazpio, and Lucia Specia. Semeval-2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation.arXiv preprint arXiv:1708.00055, 2017

    Pith/arXiv arXiv 2017

  7. [7]

    Open-domain question answering

    Danqi Chen and Wen-tau Yih. Open-domain question answering. InProceedings of the 58th annual meeting of the association for computational linguistics: tutorial abstracts, pages 34–37, 2020

    2020

  8. [8]

    mme5: Improving multimodal multilingual embeddings via high-quality synthetic data

    Haonan Chen, Liang Wang, Nan Yang, Yutao Zhu, Ziliang Zhao, Furu Wei, and Zhicheng Dou. mme5: Improving multimodal multilingual embeddings via high-quality synthetic data. In Findings of the Association for Computational Linguistics: ACL 2025, pages 8254–8275, 2025

    2025

  9. [9]

    Efficientqat: Efficient quantization-aware training for large language models

    Mengzhao Chen, Wenqi Shao, Peng Xu, Jiahao Wang, Peng Gao, Kaipeng Zhang, and Ping Luo. Efficientqat: Efficient quantization-aware training for large language models. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10081–10100, 2025

    2025

  10. [10]

    Semeval-2022 task 8: Multilingual news article similarity

    Xi Chen, Ali Zeynali, Chico Camargo, Fabian Flöck, Devin Gaffney, Przemyslaw Grabowicz, Scott A Hale, David Jurgens, and Mattia Samory. Semeval-2022 task 8: Multilingual news article similarity. InProceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1094–1106, 2022

    2022

  11. [11]

    Linq-embed-mistral technical report.arXiv preprint arXiv:2412.03223, 2024

    Chanyeol Choi, Junseong Kim, Seolhwa Lee, Jihoon Kwon, Sangmo Gu, Yejin Kim, Minkyung Cho, and Jy-yong Sohn. Linq-embed-mistral technical report.arXiv preprint arXiv:2412.03223, 2024. 10

    Pith/arXiv arXiv 2024

  12. [12]

    Specter: Document-level representation learning using citation-informed transformers.arXiv preprint arXiv:2004.07180, 2020

    Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, and Daniel S Weld. Specter: Document-level representation learning using citation-informed transformers.arXiv preprint arXiv:2004.07180, 2020

    Pith/arXiv arXiv 2004

  13. [13]

    Quora question pairs.https://kaggle.com/competitions/quora-question-pairs, 2017

    DataCanary, hilfialkaff, Lili Jiang, Meg Risdal, Nikhil Dandekar, and tomtung. Quora question pairs.https://kaggle.com/competitions/quora-question-pairs, 2017. Kaggle

    2017

  14. [14]

    Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale.Advances in neural information processing systems, 35: 30318–30332, 2022

    2022

  15. [15]

    BERT: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio, editors,Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volu...

    work page doi:10.18653/v1/n 2019

  16. [16]

    Bitdistiller: Unleashing the potential of sub-4-bit llms via self-distillation

    Dayou Du, Yijia Zhang, Shijie Cao, Jiaqi Guo, Ting Cao, Xiaowen Chu, and Ningyi Xu. Bitdistiller: Unleashing the potential of sub-4-bit llms via self-distillation. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 102–116, 2024

    2024

  17. [17]

    Mmteb: Massive multilingual text embedding benchmark.arXiv preprint arXiv:2502.13595, 2025

    Kenneth Enevoldsen, Isaac Chung, Imene Kerboua, Márton Kardos, Ashwin Mathur, David Stap, Jay Gala, Wissam Siblini, Dominik Krzemi ´nski, Genta Indra Winata, et al. Mmteb: Massive multilingual text embedding benchmark.arXiv preprint arXiv:2502.13595, 2025

    arXiv 2025

  18. [18]

    Eli5: Long form question answering.arXiv preprint arXiv:1907.09190, 2019

    Angela Fan, Yacine Jernite, Ethan Perez, David Grangier, Jason Weston, and Michael Auli. Eli5: Long form question answering.arXiv preprint arXiv:1907.09190, 2019

    Pith/arXiv arXiv 1907

  19. [19]

    Gptq: Accurate post-training quantization for generative pre-trained transformers.arXiv preprint arXiv:2210.17323, 2022

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. Gptq: Accurate post-training quantization for generative pre-trained transformers.arXiv preprint arXiv:2210.17323, 2022

    Pith/arXiv arXiv 2022

  20. [20]

    Simcse: Simple contrastive learning of sentence embeddings.arXiv preprint arXiv:2104.08821, 2021

    Tianyu Gao, Xingcheng Yao, and Danqi Chen. Simcse: Simple contrastive learning of sentence embeddings.arXiv preprint arXiv:2104.08821, 2021

    Pith/arXiv arXiv 2021

  21. [21]

    A survey of low-bit large language models: Basics, systems, and algorithms.Neural networks, page 107856, 2025

    Ruihao Gong, Yifu Ding, Zining Wang, Chengtao Lv, Xingyu Zheng, Jinyang Du, Yang Yong, Shiqiao Gu, Haotong Qin, Jinyang Guo, et al. A survey of low-bit large language models: Basics, systems, and algorithms.Neural networks, page 107856, 2025

    2025

  22. [22]

    Fei Huang, Fan Wu, Zeqing Zhang, Qihao Wang, Long Zhang, Grant Michael Boquet, and Hongyang Chen. Geogpt. rag technical report.arXiv preprint arXiv:2509.09686, 2025

    arXiv 2025

  23. [23]

    Quaff: Quantized parameter-efficient fine-tuning under outlier spatial stability hypothesis

    Hong Huang and Dapeng Wu. Quaff: Quantized parameter-efficient fine-tuning under outlier spatial stability hypothesis. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6481–6496, 2025

    2025

  24. [24]

    Dense passage retrieval for open-domain question answering

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick SH Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. InEMNLP (1), pages 6769–6781, 2020

    2020

  25. [25]

    Colbert: Efficient and effective passage search via contextual- ized late interaction over bert

    Omar Khattab and Matei Zaharia. Colbert: Efficient and effective passage search via contextual- ized late interaction over bert. InProceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pages 39–48, 2020

    2020

  26. [26]

    Ma- tryoshka representation learning.Advances in Neural Information Processing Systems, 35: 30233–30249, 2022

    Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ramanujan, William Howard-Snyder, Kaifeng Chen, Sham Kakade, Prateek Jain, et al. Ma- tryoshka representation learning.Advances in Neural Information Processing Systems, 35: 30233–30249, 2022

    2022

  27. [27]

    Newsweeder: Learning to filter netnews

    Ken Lang. Newsweeder: Learning to filter netnews. InMachine learning proceedings 1995, pages 331–339. Elsevier, 1995. 11

    1995

  28. [28]

    Nv-embed: Improved techniques for training llms as generalist embedding models.arXiv preprint arXiv:2405.17428, 2024

    Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. Nv-embed: Improved techniques for training llms as generalist embedding models.arXiv preprint arXiv:2405.17428, 2024

    Pith/arXiv arXiv 2024

  29. [29]

    Gecko: Versatile text embeddings distilled from large language models.arXiv preprint arXiv:2403.20327, 2024

    Jinhyuk Lee, Zhuyun Dai, Xiaoqi Ren, Blair Chen, Daniel Cer, Jeremy R Cole, Kai Hui, Michael Boratko, Rajvi Kapadia, Wen Ding, et al. Gecko: Versatile text embeddings distilled from large language models.arXiv preprint arXiv:2403.20327, 2024

    Pith/arXiv arXiv 2024

  30. [30]

    Llama2vec: Unsupervised adaptation of large language models for dense retrieval

    Chaofan Li, Zheng Liu, Shitao Xiao, Yingxia Shao, and Defu Lian. Llama2vec: Unsupervised adaptation of large language models for dense retrieval. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3490–3500, 2024

    2024

  31. [31]

    Making text embedders few-shot learners.arXiv preprint arXiv:2409.15700, 2024

    Chaofan Li, MingHao Qin, Shitao Xiao, Jianlyu Chen, Kun Luo, Yingxia Shao, Defu Lian, and Zheng Liu. Making text embedders few-shot learners.arXiv preprint arXiv:2409.15700, 2024

    Pith/arXiv arXiv 2024

  32. [32]

    Mtop: A comprehensive multilingual task-oriented semantic parsing benchmark.arXiv preprint arXiv:2008.09335, 2020

    Haoran Li, Abhinav Arora, Shuohui Chen, Anchit Gupta, Sonal Gupta, and Yashar Mehdad. Mtop: A comprehensive multilingual task-oriented semantic parsing benchmark.arXiv preprint arXiv:2008.09335, 2020

    Pith/arXiv arXiv 2008

  33. [33]

    Towards general text embeddings with multi-stage contrastive learning.arXiv preprint arXiv:2308.03281, 2023

    Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. Towards general text embeddings with multi-stage contrastive learning.arXiv preprint arXiv:2308.03281, 2023

    Pith/arXiv arXiv 2023

  34. [35]

    Quantization meets reasoning: Exploring llm low-bit quantization degradation for mathematical reasoning.arXiv preprint arXiv:2501.03035, 2025

    Zhen Li, Yupeng Su, Runming Yang, Congkai Xie, Zheng Wang, Zhongwei Xie, Ngai Wong, and Hongxia Yang. Quantization meets reasoning: Exploring llm low-bit quantization degradation for mathematical reasoning.arXiv preprint arXiv:2501.03035, 2025

    Pith/arXiv arXiv 2025

  35. [36]

    Awq: Activation-aware weight quantization for on-device llm compression and acceleration.Proceedings of machine learning and systems, 6:87–100, 2024

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. Awq: Activation-aware weight quantization for on-device llm compression and acceleration.Proceedings of machine learning and systems, 6:87–100, 2024

    2024

  36. [37]

    Linkso: a dataset for learning to retrieve similar question answer pairs on software development forums

    Xueqing Liu, Chi Wang, Yue Leng, and ChengXiang Zhai. Linkso: a dataset for learning to retrieve similar question answer pairs on software development forums. InProceedings of the 4th ACM SIGSOFT International Workshop on NLP for Software Engineering, pages 2–5, 2018

    2018

  37. [38]

    Llm-qat: Data-free quantiza- tion aware training for large language models

    Zechun Liu, Barlas Oguz, Changsheng Zhao, Ernie Chang, Pierre Stock, Yashar Mehdad, Yangyang Shi, Raghuraman Krishnamoorthi, and Vikas Chandra. Llm-qat: Data-free quantiza- tion aware training for large language models. InFindings of the Association for Computational Linguistics: ACL 2024, pages 467–484, 2024

    2024

  38. [39]

    The era of 1-bit llms: All large language models are in 1.58 bits.arXiv preprint arXiv:2402.17764, 2024

    Shuming Ma, Hongyu Wang, Lingxiao Ma, Lei Wang, Wenhui Wang, Shaohan Huang, Li Dong, Ruiping Wang, Jilong Xue, and Furu Wei. The era of 1-bit llms: All large language models are in 1.58 bits.arXiv preprint arXiv:2402.17764, 2024

    Pith/arXiv arXiv 2024

  39. [40]

    Bitnet b1

    Shuming Ma, Hongyu Wang, Shaohan Huang, Xingxing Zhang, Ying Hu, Ting Song, Yan Xia, and Furu Wei. Bitnet b1. 58 2b4t technical report.arXiv preprint arXiv:2504.12285, 2025

    Pith/arXiv arXiv 2025

  40. [41]

    Fine-tuning llama for multi-stage text retrieval

    Xueguang Ma, Liang Wang, Nan Yang, Furu Wei, and Jimmy Lin. Fine-tuning llama for multi-stage text retrieval. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2421–2425, 2024

    2024

  41. [42]

    Learning word vectors for sentiment analysis

    Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. Learning word vectors for sentiment analysis. InProceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, pages 142–150, 2011. 12

    2011

  42. [43]

    Tweet sentiment extraction

    Maggie, Phil Culliton, and Wei Chen. Tweet sentiment extraction. https://kaggle.com/ competitions/tweet-sentiment-extraction, 2020. Kaggle

    2020

  43. [44]

    Www’18 open challenge: financial opinion mining and question answering

    Macedo Maia, Siegfried Handschuh, André Freitas, Brian Davis, Ross McDermott, Manel Zarrouk, and Alexandra Balahur. Www’18 open challenge: financial opinion mining and question answering. InCompanion proceedings of the the web conference 2018, pages 1941– 1942, 2018

    2018

  44. [45]

    Hidden factors and hidden topics: understanding rating dimensions with review text

    Julian McAuley and Jure Leskovec. Hidden factors and hidden topics: understanding rating dimensions with review text. InProceedings of the 7th ACM conference on Recommender systems, pages 165–172, 2013

    2013

  45. [46]

    Sfrembedding-mistral: enhance text retrieval with transfer learning.Salesforce AI Research Blog, 3:6, 2024

    Rui Meng, Ye Liu, Shafiq Rayhan Joty, Caiming Xiong, Yingbo Zhou, and Semih Yavuz. Sfrembedding-mistral: enhance text retrieval with transfer learning.Salesforce AI Research Blog, 3:6, 2024

    2024

  46. [47]

    Sgpt: Gpt sentence embeddings for semantic search.arXiv preprint arXiv:2202.08904, 2022

    Niklas Muennighoff. Sgpt: Gpt sentence embeddings for semantic search.arXiv preprint arXiv:2202.08904, 2022

    Pith/arXiv arXiv 2022

  47. [48]

    MTEB: Massive text embedding benchmark

    Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. MTEB: Massive text embedding benchmark. In Andreas Vlachos and Isabelle Augenstein, editors,Proceedings of the 17th Conference of the European Chapter of the Association for Computational Lin- guistics, pages 2014–2037, Dubrovnik, Croatia, May 2023. Association for Computational Linguistics....

    doi:10.18653/v1/2023.eacl-main.148 2014

  48. [49]

    Generative representational instruction tuning

    Niklas Muennighoff, Hongjin Su, Liang Wang, Nan Yang, Furu Wei, Tao Yu, Amanpreet Singh, and Douwe Kiela. Generative representational instruction tuning. InInternational Conference on Learning Representations, volume 2025, pages 45544–45613, 2025

    2025

  49. [50]

    Matryoshka quantization.arXiv preprint arXiv:2502.06786, 2025

    Pranav Nair, Puranjay Datta, Jeff Dean, Prateek Jain, and Aditya Kusupati. Matryoshka quantization.arXiv preprint arXiv:2502.06786, 2025

    Pith/arXiv arXiv 2025

  50. [51]

    Ms marco: A human-generated machine reading comprehension dataset

    Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. Ms marco: A human-generated machine reading comprehension dataset. 2016

    2016

  51. [52]

    I wish i would have loved this one, but i didn’t–a multilingual dataset for counterfactual detection in product reviews.arXiv preprint arXiv:2104.06893, 2021

    James O’Neill, Polina Rozenshtein, Ryuichi Kiryo, Motoko Kubota, and Danushka Bollegala. I wish i would have loved this one, but i didn’t–a multilingual dataset for counterfactual detection in product reviews.arXiv preprint arXiv:2104.06893, 2021

    Pith/arXiv arXiv 2021

  52. [53]

    Representation learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748, 2018

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748, 2018

    Pith/arXiv arXiv 2018

  53. [54]

    Sentence-bert: Sentence embeddings using siamese bert- networks

    Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert- networks. InProceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP- IJCNLP), pages 3982–3992, 2019

    2019

  54. [55]

    Carer: Contextualized affect representations for emotion recognition

    Elvis Saravia, Hsien-Chi Toby Liu, Yen-Hao Huang, Junlin Wu, and Yi-Shin Chen. Carer: Contextualized affect representations for emotion recognition. InProceedings of the 2018 conference on empirical methods in natural language processing, pages 3687–3697, 2018

    2018

  55. [56]

    Q-rag: Long context multi-step retrieval via value-based embedder training.arXiv preprint arXiv:2511.07328, 2025

    Artyom Sorokin, Nazar Buzun, Alexander Anokhin, Oleg Inozemcev, Egor Vedernikov, Petr Anokhin, Mikhail Burtsev, Trushkov Alexey, Yin Wenshuai, and Evgeny Burnaev. Q-rag: Long context multi-step retrieval via value-based embedder training.arXiv preprint arXiv:2511.07328, 2025

    Pith/arXiv arXiv 2025

  56. [57]

    Smith, Luke Zettlemoyer, and Tao Yu

    Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A. Smith, Luke Zettlemoyer, and Tao Yu. One embedder, any task: Instruction- finetuned text embeddings. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors, Findings of the Association for Computational Linguistics: ACL 2023, pages 1102–1121, Toronto...

    doi:10.18653/v1/2023 2023

  57. [58]

    Llms are also effective embedding models: An in-depth overview

    Chongyang Tao, Tao Shen, Shen Gao, Junshuo Zhang, Zhen Li, Kai Hua, Wenpeng Hu, Zhengwei Tao, and Shuai Ma. Llms are also effective embedding models: An in-depth overview. arXiv preprint arXiv:2412.12591, 2024

    Pith/arXiv arXiv 2024

  58. [59]

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penchev, Gaël Liu, Francesco Visin, Kathleen Kenealy, Lucas Bey...

    Pith/arXiv arXiv 2025

  59. [60]

    Fever: a large-scale dataset for fact extraction and verification.arXiv preprint arXiv:1803.05355, 2018

    James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. Fever: a large-scale dataset for fact extraction and verification.arXiv preprint arXiv:1803.05355, 2018

    Pith/arXiv arXiv 2018

  60. [61]

    Retrieval of the best counterargument without prior topic knowledge

    Henning Wachsmuth, Shahbaz Syed, and Benno Stein. Retrieval of the best counterargument without prior topic knowledge. InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 241–251, 2018

    2018

  61. [62]

    Bitnet: Scaling 1-bit transformers for large language models.arXiv preprint arXiv:2310.11453, 2023

    Hongyu Wang, Shuming Ma, Li Dong, Shaohan Huang, Huaijie Wang, Lingxiao Ma, Fan Yang, Ruiping Wang, Yi Wu, and Furu Wei. Bitnet: Scaling 1-bit transformers for large language models.arXiv preprint arXiv:2310.11453, 2023

    Pith/arXiv arXiv 2023

  62. [63]

    Text embeddings by weakly-supervised contrastive pre-training

    Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.03533, 2022

    Pith/arXiv arXiv 2022

  63. [64]

    Improving text embeddings with large language models

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. Improving text embeddings with large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association for 14 Computational Linguistics (Volume 1: Long Papers), pages 11897–11916, Bangkok, Thailand, August 20...

    doi:10.18653/v1/2024.acl-long.642 2024

  64. [65]

    Multilingual e5 text embeddings: A technical report.arXiv preprint arXiv:2402.05672, 2024

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. Multilingual e5 text embeddings: A technical report.arXiv preprint arXiv:2402.05672, 2024

    Pith/arXiv arXiv 2024

  65. [66]

    Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers.Advances in neural information processing systems, 33:5776–5788, 2020

    Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers.Advances in neural information processing systems, 33:5776–5788, 2020

    2020

  66. [67]

    Minilmv2: Multi-head self-attention relation distillation for compressing pretrained transformers

    Wenhui Wang, Hangbo Bao, Shaohan Huang, Li Dong, and Furu Wei. Minilmv2: Multi-head self-attention relation distillation for compressing pretrained transformers. InFindings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2140–2151, 2021

    2021

  67. [68]

    Yutong Wang, Haiyu Wang, and Sai Qian Zhang. Qsvd: Efficient low-rank approximation for unified query-key-value weight compression in low-precision vision-language models.Advances in Neural Information Processing Systems, 38:1789–1820, 2026

    2026

  68. [69]

    Bitnet distillation.arXiv preprint arXiv:2510.13998, 2025

    Xun Wu, Shaohan Huang, Wenhui Wang, Ting Song, Li Dong, Yan Xia, and Furu Wei. Bitnet distillation.arXiv preprint arXiv:2510.13998, 2025

    arXiv 2025

  69. [70]

    Smoothquant: Accurate and efficient post-training quantization for large language models

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. Smoothquant: Accurate and efficient post-training quantization for large language models. InInternational conference on machine learning, pages 38087–38099. PMLR, 2023

    2023

  70. [71]

    Approximate nearest neighbor negative contrastive learning for dense text retrieval.arXiv preprint arXiv:2007.00808, 2020

    Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul Bennett, Junaid Ahmed, and Arnold Overwijk. Approximate nearest neighbor negative contrastive learning for dense text retrieval.arXiv preprint arXiv:2007.00808, 2020

    Pith/arXiv arXiv 2007

  71. [72]

    Onebit: Towards extremely low-bit large language models.Advances in Neural Information Processing Systems, 37:66357–66382, 2024

    Yuzhuang Xu, Xu Han, Zonghan Yang, Shuo Wang, Qingfu Zhu, Zhiyuan Liu, Weidong Liu, and Wanxiang Che. Onebit: Towards extremely low-bit large language models.Advances in Neural Information Processing Systems, 37:66357–66382, 2024

    2024

  72. [73]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, ...

    Pith/arXiv arXiv 2025

  73. [74]

    Hotpotqa: A dataset for diverse, explainable multi-hop question answering.arXiv preprint arXiv:1809.09600, 2018

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhut- dinov, and Christopher D Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering.arXiv preprint arXiv:1809.09600, 2018

    Pith/arXiv arXiv 2018

  74. [75]

    Rptq: Reorder-based post-training quantization for large language models.arXiv preprint arXiv:2304.01089, 2023

    Zhihang Yuan, Lin Niu, Jiawei Liu, Wenyu Liu, Xinggang Wang, Yuzhang Shang, Guangyu Sun, Qiang Wu, Jiaxiang Wu, and Bingzhe Wu. Rptq: Reorder-based post-training quantization for large language models.arXiv preprint arXiv:2304.01089, 2023

    Pith/arXiv arXiv 2023

  75. [76]

    Qwen3 embedding: Advancing text embedding and reranking through foundation models.arXiv preprint arXiv:2506.05176, 2025

    Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, et al. Qwen3 embedding: Advancing text embedding and reranking through foundation models.arXiv preprint arXiv:2506.05176, 2025

    Pith/arXiv arXiv 2025

  76. [77]

    Retrieval-augmented generation for ai-generated content: A survey.arXiv preprint arXiv:2402.19473, 2024

    Penghao Zhao, Hailin Zhang, Qinhan Yu, Zhengren Wang, Yunteng Geng, Fangcheng Fu, Ling Yang, Wentao Zhang, Jie Jiang, and Bin Cui. Retrieval-augmented generation for ai-generated content: A survey.arXiv preprint arXiv:2402.19473, 2024

    Pith/arXiv arXiv 2024

  77. [78]

    Dense text retrieval based on pretrained language models: A survey.ACM Transactions on Information Systems, 42(4):1–60, 2024

    Wayne Xin Zhao, Jing Liu, Ruiyang Ren, and Ji-Rong Wen. Dense text retrieval based on pretrained language models: A survey.ACM Transactions on Information Systems, 42(4):1–60, 2024. 15

    2024

  78. [79]

    Embedding in recommender systems: A survey.arXiv preprint arXiv:2310.18608, 2023

    Xiangyu Zhao, Maolin Wang, Xinjian Zhao, Jiansheng Li, Shucheng Zhou, Dawei Yin, Qing Li, Jiliang Tang, and Ruocheng Guo. Embedding in recommender systems: A survey.arXiv preprint arXiv:2310.18608, 2023

    arXiv 2023

  79. [80]

    Kalm-embedding-v2: Superior training techniques and data inspire a versatile embedding model.arXiv preprint arXiv:2506.20923, 2025

    Xinping Zhao, Xinshuo Hu, Zifei Shan, Shouzheng Huang, Yao Zhou, Xin Zhang, Zetian Sun, Zhenyu Liu, Dongfang Li, Xinyuan Wei, et al. Kalm-embedding-v2: Superior training techniques and data inspire a versatile embedding model.arXiv preprint arXiv:2506.20923, 2025. A Training Data Following BGE-en-ICL [31], our training data contains retrieval, reranking, ...

    arXiv 2025

This paper was first reviewed by deepseek-v4-flash on August 2, 2026.