pith. machine review for the scientific record. sign in

arxiv: 2509.20354 · v3 · submitted 2025-09-24 · 💻 cs.CL · cs.AI

Recognition: no theorem link

EmbeddingGemma: Powerful and Lightweight Text Representations

Henrique Schechter Vera , Sahil Dua , Biao Zhang , Daniel Salz , Ryan Mullins , Sindhu Raghuram Panyam , Sara Smoot , Iftekhar Naim
show 81 more authors
Joe Zou Feiyang Chen Daniel Cer Alice Lisak Min Choi Lucas Gonzalez Omar Sanseviero Glenn Cameron Ian Ballantyne Kat Black Kaifeng Chen Weiyi Wang Zhe Li Gus Martins Jinhyuk Lee Mark Sherwood Juyeong Ji Renjie Wu Jingxiao Zheng Jyotinder Singh Abheesht Sharma Divyashree Sreepathihalli Aashi Jain Adham Elarabawy AJ Co Andreas Doumanoglou Babak Samari Ben Hora Brian Potetz Dahun Kim Enrique Alfonseca Fedor Moiseev Feng Han Frank Palma Gomez Gustavo Hern\'andez \'Abrego Hesen Zhang Hui Hui Jay Han Karan Gill Ke Chen Koert Chen Madhuri Shanbhogue Michael Boratko Paul Suganthan Sai Meher Karthik Duddu Sandeep Mariserla Setareh Ariafar Shanfeng Zhang Shijie Zhang Simon Baumgartner Sonam Goenka Steve Qiu Tanmaya Dabral Trevor Walker Vikram Rao Waleed Khawaja Wenlei Zhou Xiaoqi Ren Ye Xia Yichang Chen Yi-Ting Chen Zhe Dong Zhongli Ding Francesco Visin Ga\"el Liu Jiageng Zhang Kathleen Kenealy Michelle Casbon Ravin Kumar Thomas Mesnard Zach Gleicher Cormac Brick Olivier Lacombe Adam Roberts Qin Yin Yunhsuan Sung Raphael Hoffmann Tris Warkentin Armand Joulin Tom Duerig Mojtaba Seyedhosseini
Authors on Pith no claims yet

Pith reviewed 2026-05-15 12:02 UTC · model grok-4.3

classification 💻 cs.CL cs.AI
keywords text embeddingslightweight modelsMTEB benchmarkmultilingual embeddingsmodel distillationon-device applicationsquantized modelsembedding regularization
0
0 comments X

The pith

A 300 million parameter model reaches state-of-the-art text embedding results on MTEB

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents EmbeddingGemma, a 300M-parameter text embedding model built from the Gemma 3 family. It applies encoder-decoder initialization to capture knowledge from larger models through geometric embedding distillation, adds a spread-out regularizer to improve embedding distribution, and merges checkpoints trained on varied data mixtures to boost generalizability. On the Massive Text Embedding Benchmark the model leads across multilingual, English-only, and code tasks, beating prior top models with under 500M parameters and matching those twice its size. The lead holds after weight quantization and output truncation, which points to strong suitability for low-latency and on-device settings.

Core claim

EmbeddingGemma is a 300M-parameter open text embedding model derived from Gemma 3. Through encoder-decoder initialization, geometric embedding distillation, a spread-out regularizer, and checkpoint merging across optimized mixtures, it achieves state-of-the-art results on MTEB in multilingual, English, and code domains. It outperforms earlier leading models with fewer than 500M parameters and delivers performance comparable to models of double the size, with the advantage preserved under quantization and embedding truncation.

What carries the argument

The training recipe that combines encoder-decoder initialization, geometric embedding distillation from larger models, a spread-out regularizer, and merging of checkpoints from different data mixtures.

If this is right

  • The model supplies a high performance-to-cost ratio for text embedding workloads.
  • It remains effective for on-device and high-throughput uses even after quantization or output truncation.
  • Ablation results isolate the contribution of each training choice to the final scores.
  • Open release allows direct reuse and further adaptation by the community.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar initialization and distillation patterns may narrow the gap between small and large models in other representation tasks.
  • On-device embedding models of this size could support private, low-latency retrieval in mobile and edge applications.
  • The approach invites tests of even smaller variants or direct integration into retrieval-augmented generation pipelines.
  • Checkpoint merging across mixtures may generalize to other fine-tuning regimes where data diversity matters.

Load-bearing premise

The reported performance gains are driven mainly by the described training steps rather than by data selection or the base model scale alone.

What would settle it

A side-by-side training run of a second 300M model on the same data but without the distillation step, spread-out regularizer, or checkpoint merging would show whether the MTEB scores drop to the level of prior models of similar size.

read the original abstract

We introduce EmbeddingGemma, a new lightweight, open text embedding model based on the Gemma 3 language model family. Our innovative training recipe strategically captures knowledge from larger models via encoder-decoder initialization and geometric embedding distillation. We improve model robustness and expressiveness with a spread-out regularizer, and ensure generalizability by merging checkpoints from varied, optimized mixtures. Evaluated on the Massive Text Embedding Benchmark (MTEB) across multilingual, English, and code domains, EmbeddingGemma (300M) achieves state-of-the-art results. Notably, it outperforms prior top models, both proprietary and open, with fewer than 500M parameters, and provides performance comparable to models double its size, offering an exceptional performance-to-cost ratio. Remarkably, this lead persists when quantizing model weights or truncating embedding outputs. This makes EmbeddingGemma particularly well-suited for low-latency and high-throughput use cases such as on-device applications. We provide ablation studies exploring our key design choices. We release EmbeddingGemma to the community to promote further research.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 3 minor

Summary. The paper introduces EmbeddingGemma, a 300M-parameter text embedding model derived from the Gemma 3 family. It employs a training recipe of encoder-decoder initialization, geometric embedding distillation, a spread-out regularizer, and checkpoint merging from varied mixtures to claim state-of-the-art performance on the MTEB benchmark across multilingual, English, and code domains. The model is reported to outperform prior top models with fewer than 500M parameters while matching the performance of models twice its size, with these gains persisting under quantization and truncation, making it suitable for low-latency applications.

Significance. If the reported MTEB results and robustness hold, this represents a meaningful contribution to efficient text embeddings by delivering high performance at reduced scale and cost. The provision of ablation studies on the training components and the open release of the model support reproducibility and further work in the area.

minor comments (3)
  1. Abstract: The SOTA claim is stated without any numerical scores, specific MTEB average values, or pointers to result tables, which reduces immediate clarity even though the full results appear later in the manuscript.
  2. Section 4 (or equivalent results section): While ablations on the training recipe are mentioned, the paper would benefit from explicit reporting of error bars or variance across multiple runs for the headline MTEB scores to strengthen the comparison claims.
  3. The description of checkpoint merging could include more precise details on the mixture weights or selection criteria used, as this is presented as key to generalizability.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their positive summary of EmbeddingGemma, recognition of its significance for efficient embeddings, and recommendation of minor revision. We are pleased that the ablation studies, open release, and performance claims were viewed favorably.

Circularity Check

0 steps flagged

No significant circularity; empirical claims rest on external benchmarks

full rationale

The paper describes an empirical training recipe (encoder-decoder initialization, geometric distillation, spread-out regularizer, checkpoint merging) and reports MTEB results plus ablations. No equations, first-principles derivations, or predictions are presented that reduce by construction to fitted parameters or self-citations. All performance claims are benchmark-driven and falsifiable against external data; ablations provide independent controls. This matches the default expectation of no circularity for non-derivational empirical work.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review provides no explicit free parameters, axioms, or invented entities; training details are described at a high level without numerical hyperparameters or unstated assumptions listed.

pith-pipeline@v0.9.0 · 5868 in / 1119 out tokens · 66891 ms · 2026-05-15T12:02:28.861005+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 21 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Test-Time Compute for Dense Retrieval: Agentic Program Generation with Frozen Embedding Models

    cs.LG 2026-05 unverdicted novelty 7.0

    Agentic program search over frozen embedding APIs yields a parameter-free inference algebra—a softmax-weighted centroid of top-K documents interpolated with the query—that lifts nDCG@10 across seven model families on ...

  2. Test-Time Compute for Dense Retrieval: Agentic Program Generation with Frozen Embedding Models

    cs.LG 2026-05 unverdicted novelty 7.0

    A softmax-weighted centroid of the local top-K documents interpolated with the query improves nDCG@10 for frozen embedding models across seven families on held-out BEIR data.

  3. TeleEmbedBench: A Multi-Corpus Embedding Benchmark for RAG in Telecommunications

    cs.LG 2026-04 unverdicted novelty 7.0

    TeleEmbedBench is the first multi-corpus benchmark showing LLM-based embedding models significantly outperform traditional sentence-transformers on telecommunications specifications and code for retrieval accuracy and...

  4. Teaching LLMs Human-Like Editing of Inappropriate Argumentation via Reinforcement Learning

    cs.CL 2026-04 unverdicted novelty 7.0

    Reinforcement learning with a multi-part reward teaches LLMs to output independent, meaning-preserving sentence edits that raise argument appropriateness close to full rewriting.

  5. LongTail Driving Scenarios with Reasoning Traces: The KITScenes LongTail Dataset

    cs.CV 2026-03 unverdicted novelty 7.0

    KITScenes LongTail supplies multimodal driving data and multilingual expert reasoning traces to benchmark models on rare scenarios beyond basic safety metrics.

  6. Spectral Tempering for Embedding Compression in Dense Passage Retrieval

    cs.IR 2026-03 unverdicted novelty 7.0

    Spectral Tempering derives an adaptive scaling factor γ(k) from the embedding eigenspectrum via local SNR analysis and knee-point normalization to achieve near-optimal compression without training or validation.

  7. LMEB: Long-horizon Memory Embedding Benchmark

    cs.CL 2026-03 unverdicted novelty 7.0

    LMEB benchmark shows that embedding models' performance on traditional retrieval does not transfer to long-horizon memory tasks, larger models do not always perform better, and LMEB measures capabilities orthogonal to MTEB.

  8. AuDirector: A Self-Reflective Closed-Loop Framework for Immersive Audio Storytelling

    cs.SD 2026-05 unverdicted novelty 6.0

    AuDirector is a self-reflective closed-loop multi-agent framework that generates immersive audio narratives with improved structural coherence, emotional expressiveness, and acoustic fidelity via identity-aware voice ...

  9. An Annotation Scheme and Classifier for Personal Facts in Dialogue

    cs.CL 2026-05 accept novelty 6.0

    An extended annotation scheme with new categories and attributes plus a Gemma-300M-based multi-head classifier achieves 81.6% macro F1 on personal fact classification, outperforming few-shot LLM baselines by nearly 9 ...

  10. MLAIRE: Multilingual Language-Aware Information Retrieval Evaluation Protocal

    cs.IR 2026-05 unverdicted novelty 6.0

    MLAIRE is a protocol that evaluates multilingual retrievers on both semantic accuracy and query-language preference using parallel passages and new metrics like LPR and Lang-nDCG, showing that standard metrics hide di...

  11. Identifier-Free Code Embedding Models for Scalable Search

    cs.CR 2026-05 unverdicted novelty 6.0

    A fine-tuned Qwen3-Embedding model with contrastive learning outperforms baselines on bidirectional source-to-decompiled code association and generalizes to constant-algorithm tasks.

  12. Iterative Definition Refinement for Zero-Shot Classification via LLM-Based Semantic Prototype Optimization

    cs.CV 2026-04 unverdicted novelty 6.0

    Iterative LLM-based refinement of category definitions improves zero-shot classification performance across 13 embedding models on a new 10-category web URL benchmark.

  13. Lost in Decoding? Reproducing and Stress-Testing the Look-Ahead Prior in Generative Retrieval

    cs.IR 2026-04 accept novelty 6.0

    Reproduction confirms PAG boosts generative retrieval effectiveness, but its look-ahead planning signal collapses under intent-preserving typos and query mismatches, reverting performance to unguided decoding.

  14. Differences in Text Generated by Diffusion and Autoregressive Language Models

    cs.CL 2026-04 unverdicted novelty 6.0

    DLMs exhibit lower n-gram entropy, higher semantic coherence, and higher semantic diversity than ARMs, primarily due to bidirectional context and remasking decoding strategies.

  15. NAVIS: Concurrent Search and Update with Low Position-Seeking Overhead in On-SSD Graph-Based Vector Search

    cs.DC 2026-05 unverdicted novelty 5.0

    NAVIS improves concurrent search and update throughput in on-SSD graph vector search by up to 2.74x for insertions and 1.37x for searches through reduced position-seeking overhead.

  16. How Does Chunking Affect Retrieval-Augmented Code Completion? A Controlled Empirical Study

    cs.SE 2026-05 conditional novelty 5.0

    Function-based chunking underperforms other strategies in RAG code completion by 3.57-5.64 points, with context length as the dominant factor.

  17. Linear-Time and Constant-Memory Text Embeddings Based on Recurrent Language Models

    cs.CL 2026-04 unverdicted novelty 5.0

    Fine-tuned recurrent models like Mamba2 produce competitive text embeddings with linear-time constant-memory inference via vertical chunking, outperforming transformers in memory use.

  18. Predicting Post-Traumatic Epilepsy from Clinical Records using Large Language Model Embeddings

    cs.LG 2026-04 unverdicted novelty 5.0

    LLM embeddings from clinical records, fused with tabular data via gradient-boosted trees, predict post-traumatic epilepsy at AUC-ROC 0.892 and AUPRC 0.798.

  19. Cross-Lingual Attention Distillation with Personality-Informed Generative Augmentation for Multilingual Personality Recognition

    cs.CL 2026-04 unverdicted novelty 5.0

    ADAM uses personality-guided LLM augmentation and cross-lingual attention distillation to raise balanced accuracy on multilingual personality recognition to 0.6332 on Essays and 0.7448 on Kaggle, outperforming standar...

  20. Granite Embedding Multilingual R2 Models

    cs.IR 2026-05 unverdicted novelty 4.0

    Granite Embedding Multilingual R2 releases 311M and 97M parameter bi-encoder models that achieve state-of-the-art retrieval performance on multilingual text, code, long-document, and reasoning datasets.

  21. Comparison of Modern Multilingual Text Embedding Techniques for Hate Speech Detection Task

    cs.CL 2026-04 unverdicted novelty 4.0

    Supervised models using embeddings like jina and e5 reach up to 92% accuracy on multilingual hate speech detection, substantially outperforming anomaly detection, while PCA to 64 dimensions preserves most performance ...

Reference graph

Works this paper leans on

27 extracted references · 27 canonical work pages · cited by 20 Pith papers · 7 internal anchors

  1. [1]

    A. Asai, J. Kasai, J. H. Clark, K. Lee, E. Choi, and H. Hajishirzi. Xor qa: Cross-lingual open-retrieval question answering. InProceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 547–564,

    work page 2021

  2. [2]

    URL https://arxiv.org/abs/2506.02153. G. de Souza P. Moreira, R. Osmulski, M. Xu, R. Ak, B. Schifferer, and E. Oldridge. Nv-retriever: Improving text embedding models with effective hard-negative mining,

    work page arXiv

  3. [3]

    URLhttps:// arxiv.org/abs/2407.15831. K. Enevoldsen, I. Chung, I. Kerboua, M. Kardos, A. Mathur, D. Stap, J. Gala, W. Siblini, D. Krzemiński, G. I. Winata, et al. Mmteb: Massive multilingual text embedding benchmark.arXiv preprint arXiv:2502.13595,

    work page arXiv

  4. [4]

    URL https://arxiv.org/abs/ 2502.13595

    doi: 10.48550/arXiv.2502.13595. URL https://arxiv.org/abs/ 2502.13595. T. Gao, X. Yao, and D. Chen. Simcse: Simple contrastive learning of sentence embeddings,

    work page doi:10.48550/arxiv.2502.13595

  5. [5]

    URL https://arxiv.org/abs/2104.08821. G. Izacard, M. Caron, L. Hosseini, S. Riedel, P. Bojanowski, A. Joulin, and E. Grave. Towards unsupervised dense information retrieval with contrastive learning.CoRR, abs/2112.09118,

    work page internal anchor Pith review Pith/arXiv arXiv

  6. [6]

    URLhttps://arxiv.org/abs/2112.09118. P. Izmailov, D. Podoprikhin, T. Garipov, D. P. Vetrov, and A. G. Wilson. Averaging weights leads to wider optima and better generalization.CoRR, abs/1803.05407,

    work page internal anchor Pith review Pith/arXiv arXiv

  7. [7]

    Averaging Weights Leads to Wider Optima and Better Generalization

    URLhttp://arxiv.org/ abs/1803.05407. B.Jacob, S.Kligys, B.Chen, M.Zhu, M.Tang, A.Howard, H.Adam, andD.Kalenichenko. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2704–2713,

    work page internal anchor Pith review Pith/arXiv arXiv

  8. [8]

    doi: 10.1109/ CVPR.2018.00286. T. Jiang, M. Song, Z. Zhang, H. Huang, W. Deng, F. Sun, Q. Zhang, D. Wang, and F. Zhuang. E5-v: Universal embeddings with multimodal large language models,

    work page arXiv 2018

  9. [9]

    URLhttps://arxiv.org/ abs/2407.12580. Z. Jiang, R. Meng, X. Yang, S. Yavuz, Y. Zhou, and W. Chen. Vlm2vec: Training vision-language models for massive multimodal embedding tasks,

    work page arXiv

  10. [10]

    URLhttps://arxiv.org/abs/2410.05160. S. Kim, A. S. Rawat, M. Zaheer, S. Jayasumana, V. Sadhanala, W. Jitkrittum, A. K. Menon, R. Fergus, and S. Kumar. Embeddistill: A geometric knowledge distillation for information retrieval,

    work page arXiv

  11. [11]

    URLhttps://arxiv.org/abs/2301.12005. A. Kusupati, G. Bhatt, A. Rege, M. Wallingford, A. Sinha, V. Ramanujan, W. Howard-Snyder, K. Chen, S. Kakade, P. Jain, and A. Farhadi. Matryoshka representation learning. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors,Advances in Neural Information Processing Systems, volume 35, pages 302...

    work page arXiv

  12. [12]

    URL https://proceedings.neurips.cc/paper_files/paper/2022/file/ c32319f4868da7613d78af9993100e42-Paper-Conference.pdf. Z. Lan, L. Niu, F. Meng, J. Zhou, and J. Su. Llave: Large language and vision embedding models with hardness-weighted contrastive learning,

    work page 2022

  13. [13]

    12 EmbeddingGemma: Powerful and Lightweight Text Representations C

    URLhttps://arxiv.org/abs/2503.04812. 12 EmbeddingGemma: Powerful and Lightweight Text Representations C. Lee, R. Roy, M. Xu, J. Raiman, M. Shoeybi, B. Catanzaro, and W. Ping. Nv-embed: Improved techniques for training llms as generalist embedding models, 2025a. URLhttps://arxiv.org/ abs/2405.17428. J. Lee, Z. Dai, X. Ren, B. Chen, D. Cer, J. R. Cole, K. H...

    work page arXiv

  14. [14]

    J. Lee, F. Chen, S. Dua, D. Cer, M. Shanbhogue, I. Naim, G. H. Ábrego, Z. Li, K. Chen, H. S. Vera, X. Ren, S. Zhang, D. Salz, M. Boratko, J. Han, B. Chen, S. Huang, V. Rao, P. Suganthan, F. Han, A. Doumanoglou, N. Gupta, F. Moiseev, C. Yip, A. Jain, S. Baumgartner, S. Shahi, F. P. Gomez, S. Mariserla, M. Choi, P. Shah, S. Goenka, K. Chen, Y. Xia, K. Chen,...

    work page internal anchor Pith review Pith/arXiv arXiv

  15. [15]

    MTEB: Massive Text Embedding Benchmark

    doi: 10.48550/ARXIV.2210.07316. URLhttps://arxiv. org/abs/2210.07316. N. Muennighoff, H. Su, L. Wang, N. Yang, F. Wei, T. Yu, A. Singh, and D. Kiela. Generative representa- tional instruction tuning,

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2210.07316

  16. [16]

    URLhttps://arxiv.org/abs/2402.09906. A. Neelakantan, T. Xu, R. Puri, A. Radford, J. M. Han, J. Tworek, Q. Yuan, N. Tezak, J. W. Kim, C. Hallacy, J. Heidecke, P. Shyam, B. Power, T. E. Nekoul, G. Sastry, G. Krueger, D. Schnurr, F. P. Such, K. Hsu, M. Thompson, T. Khan, T. Sherbakov, J. Jang, P. Welinder, and L. Weng. Text and code embeddings by contrastive...

    work page arXiv

  17. [17]

    URLhttps://arxiv.org/abs/2201.10005. J. Ni, C. Qu, J. Lu, Z. Dai, G. H. Ábrego, J. Ma, V. Y. Zhao, Y. Luan, K. B. Hall, M.-W. Chang, and Y. Yang. Large dual encoders are generalizable retrievers,

    work page arXiv

  18. [18]

    URLhttps://arxiv.org/abs/ 2112.07899. S. Ruder, J. H. Clark, A. Gutkin, M. Kale, M. Ma, M. Nicosia, S. Rijhwani, P. Riley, J.-M. Sarr, X. Wang, et al. Xtreme-up: A user-centric scarce-data benchmark for under-represented languages. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1856–1884,

    work page arXiv 2023

  19. [19]

    URLhttps://arxiv.org/abs/2112.01488. S. Sturua, I. Mohr, M. K. Akram, M. Günther, B. Wang, M. Krimmel, F. Wang, G. Mastrapas, A. Kouk- ounas, N. Wang, and H. Xiao. jina-embeddings-v3: Multilingual embeddings with task lora,

    work page arXiv

  20. [20]

    URLhttps://arxiv.org/abs/2409.10173. H. Su, W. Shi, J. Kasai, Y. Wang, Y. Hu, M. Ostendorf, W. tau Yih, N. A. Smith, L. Zettlemoyer, and T. Yu. One embedder, any task: Instruction-finetuned text embeddings,

    work page arXiv

  21. [21]

    URLhttps: //arxiv.org/abs/2212.09741. P. Suganthan, F. Moiseev, L. Yan, J. Wu, J. Ni, J. Han, I. Zitouni, E. Alfonseca, X. Wang, and Z. Dong. Adapting decoder-based language models for diverse encoder downstream tasks,

    work page arXiv

  22. [22]

    URL https://arxiv.org/abs/2503.02656. Y. Tay, M. Dehghani, V. Q. Tran, X. Garcia, J. Wei, X. Wang, H. W. Chung, S. Shakeri, D. Bahri, T. Schuster, H. S. Zheng, D. Zhou, N. Houlsby, and D. Metzler. Ul2: Unifying language learning paradigms,

    work page arXiv

  23. [23]

    13 EmbeddingGemma: Powerful and Lightweight Text Representations G

    URLhttps://arxiv.org/abs/2205.05131. 13 EmbeddingGemma: Powerful and Lightweight Text Representations G. Team. Gemma 3 technical report,

    work page arXiv

  24. [24]

    URLhttps://arxiv.org/abs/2503.19786. L. Wang, N. Yang, X. Huang, B. Jiao, L. Yang, D. Jiang, R. Majumder, and F. Wei. Text embeddings by weakly-supervised contrastive pre-training, 2024a. URLhttps://arxiv.org/abs/2212. 03533. L. Wang, N. Yang, X. Huang, L. Yang, R. Majumder, and F. Wei. Improving text embeddings with large language models, 2024b. URLhttps...

    work page internal anchor Pith review Pith/arXiv arXiv

  25. [25]

    URL https: //arxiv.org/abs/2203.05482. B. Zhang, F. Moiseev, J. Ainslie, P. Suganthan, M. Ma, S. Bhupatiraju, F. Lebron, O. Firat, A. Joulin, and Z. Dong. Encoder-decoder gemma: Improving the quality-efficiency trade-off via adaptation, 2025a. URLhttps://arxiv.org/abs/2504.06225. X. Zhang, F. X. Yu, S. Kumar, and S. Chang. Learning spread-out local featur...

    work page arXiv

  26. [26]

    URLhttp://arxiv.org/abs/1708.06320. Y. Zhang, M. Li, D. Long, X. Zhang, H. Lin, B. Yang, P. Xie, A. Yang, D. Liu, J. Lin, F. Huang, and J. Zhou. Qwen3 embedding: Advancing text embedding and reranking through foundation models, 2025b. URLhttps://arxiv.org/abs/2506.05176. D. Zhu, L. Wang, N. Yang, Y. Song, W. Wu, F. Wei, and S. Li. Longembed: Extending emb...

    work page internal anchor Pith review Pith/arXiv arXiv

  27. [27]

    14 EmbeddingGemma: Powerful and Lightweight Text Representations A

    URLhttps://arxiv.org/abs/2404.12096. 14 EmbeddingGemma: Powerful and Lightweight Text Representations A. Full Results Task Name Performance AILAStatutes 37.37AfriSentiClassification 44.47AlloProfClusteringS2S.v2 52.82AlloprofReranking 79.69AmazonCounterfactualClassification 84.23ArXivHierarchicalClusteringP2P 63.59ArXivHierarchicalClusteringS2S 59.59ArguA...

    work page arXiv