Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Tevatron Meets Megatron: Expert-Parallel LLM Reranker Training on an Academic Budget

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Expert parallelism lets a 30B-parameter MoE reranker be trained on two GPU nodes, matching dense-8B quality at less than half the active parameters.

desk verdict A genuinely useful systems paper that makes 30B MoE reranker training feasible on two nodes, with honest benchmarking, but the MoE results rest on an unverified weight-bridge conversion and no released artifacts yet. read the letter →

arxiv 2608.00916 v1 pith:D25EHJJD submitted 2026-08-02 cs.IR cs.LG

Zhichao Xu , Xueguang Ma , Shengyao Zhuang , Luyu Gao , Wenqian Ye , Yu Wang , Jamie Callan , Jimmy Lin This is my paper · ORCID
classification cs.IRcs.LG
keywords rerankingmixture-of-expertsexpertparallelismMegatrontrainingbackendTevatronLoRAknowledgedistillationBEIR-15
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the missing piece for academic-scale MoE reranker training is expert parallelism, and that a Megatron-Core training backend plugged into the Tevatron reranker toolkit supplies it without changing data, evaluation, or checkpoint formats. With that backend, a 30B-parameter Qwen3-30B-A3B reranker with 128 experts trains on a two-node 8xH200 cluster, peaking at 78.9 GiB per GPU, where the existing fully sharded data-parallel backend cannot. The same backend matches FSDP-trained dense-8B reranker quality and efficiency in matched-topology comparisons, and in its recommended configuration trains about 22% faster. The controlled comparison shows the MoE reranker lands within 0.006 NDCG@10 of dense-8B quality across three first-stage retrievers while activating less than half the parameters and serving faster. If true, it makes large MoE reranker recipes reproducible for groups without industrial-scale clusters.

What carries the argument

The load-bearing object is the Megatron-Core training backend inside Tevatron 3.0, which uses a ZeRO-1-style distributed optimizer plus tensor, pipeline, and expert parallelism, together with bidirectional weight-bridge conversion between Hugging Face-format checkpoints and Megatron parallel layouts. Expert parallelism (EP) partitions the 128 experts across ranks so that per-rank expert memory scales with experts/EP rather than the full expert count, and a named LoRA target-group registry maps adapters to attention, MoE expert, shared-expert, and router modules so that low-rank adaptation lands on the expert layers where reranking signal concentrates. Reranker scoring is the log-probability difference of yes/no answer tokens, a formulation that stays unchanged under data, tensor, pipeline, and expert parallelism.

What would settle it

Load the released Qwen3-30B-A3B MoE checkpoint, round-trip it through the bridge into Megatron layout and back, and compare expert FFN weights and router weights against the original Hugging Face checkpoint; any parameter mismatch beyond numerical tolerance, or a quality drop when training without the bridge, would show the reported memory and quality figures do not correspond to a true Qwen3-30B-A3B reranker.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that Tevatron 3.0, a Megatron-Core backend integrated into the existing Tevatron reranker pipeline, is a drop-in replacement for the Hugging Face Trainer and FSDP path for reranker training, and that its expert-parallel MoE training makes a 30B-parameter Qwen3-30B-A3B reranker trainable on two 8xH200 nodes with a peak of 78.9 GiB per GPU. Trained under one shared recipe, this MoE reranker matches dense-8B reranking quality within 0.006 NDCG@10 on BEIR-15 with three first-stage retrievers, while activating roughly 3B instead of 8B parameters and serving at higher throughput: 1.15x on the Hugging Face backend and 1.43x on vLLM. The paper presents the quality and efficiency grid as a demonstration of capability rather than as a claim that MoE is inherently better; the central assertion is that the system makes such apples-to-apples comparisons feasible.

Load-bearing premise

The central numbers assume the external weight bridge faithfully converts Hugging Face checkpoints to and from Megatron's parallel layouts for Qwen3-30B-A3B's 128 experts, including expert FFN weights and LoRA adapters, even though the paper cites but does not release or fully verify that conversion code.

Editorial extensions

If this is right

  • A standard academic two-node cluster can train and serve a 30B-parameter MoE reranker.
  • The MoE reranker can replace a dense 8B reranker with no measured quality change on BEIR-15 and better serving throughput.
  • Users switching Tevatron backends keep their dataset, evaluator, and Hugging Face/vLLM-ready checkpoints unchanged.
  • LoRA with expert-aware target groups recovers most of full-parameter fine-tuning quality on the MoE backbone.
  • Distillation and contrastive training specialize on different tasks; the aggregate comparison is not a uniform win for either loss.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the same bridge-based expert-parallel path should extend to larger MoE families with more experts on the same two nodes, though quality at that scale is untested.
  • A testable extension the paper does not run is whether EP-trained MoE rerankers keep their parity advantage on longer-context or multi-hop reranking tasks beyond BEIR-15.
  • If the throughput win holds under online batched reranking, MoE rerankers could shift latency budgets in production retriever stacks; this is our inference, not the paper's claim.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents Tevatron 3.0, a reranker training toolkit that adds a Megatron-Core backend to the existing Tevatron framework while preserving the data pipeline, evaluation interface, and Hugging Face-compatible checkpoint format. The authors profile PyTorch FSDP1 configurations for dense 8B reranker training, show that the Megatron backend matches FSDP quality on BEIR-15 and is roughly 22% faster in its recommended single-node configuration, and claim that only Megatron's expert parallelism makes it feasible to train a 30B-parameter Qwen3-30B-A3B MoE reranker on an academic budget (reported peak 78.9 GiB per GPU on two 8×H200 nodes). They then use the framework for a controlled study comparing MoE vs. dense backbones, LoRA vs. full-parameter tuning, and distillation vs. contrastive loss, reporting that the MoE reranker matches dense-8B quality within 0.006 NDCG@10 while activating fewer parameters and serving faster in both Hugging Face and vLLM backends.

Significance. If the claims hold, this is a useful systems contribution for academic groups: it demonstrates that 30B-scale MoE reranker training is possible on a dual-node cluster, and the dense-vs-MoE parity result is a practical deployment data point. The paper is careful in several respects: it presents an honest matched-topology efficiency comparison, emphasizes that the controlled study is a capability demonstration rather than a leaderboard claim, and includes a sanity check against prior rerankers with appropriate caveats. The plan to release the framework and checkpoints is valuable. However, the report's central technical risk is real: the MoE results depend on an unverified Hugging Face↔Megatron weight bridge, and the absence of error bars and the internal inconsistency in the EP/DP topology reporting currently limit confidence in the headline numbers.

major comments (4)
  1. [Section 2.2, 'Weight bridge'; Tables 5 and 6] The central MoE result depends on the NVIDIA Megatron bridge faithfully converting Qwen3-30B-A3B checkpoints between Hugging Face and Megatron layouts for all 128 expert FFN matrices, the shared expert, the router, and (for LoRA runs) the low-rank adapters. The manuscript cites the bridge documentation but does not include a round-trip conversion check, does not report expert-usage or routing statistics, and does not release the conversion code or the exact bridge version. A silent misassignment of expert weights or an incorrect TP/EP sharding pattern would make Tables 5-10 and the serving-throughput numbers describe a corrupted artifact rather than a true Qwen3-30B-A3B reranker. Please add a round-trip parameter-equality check (export to HF, re-import, compare all expert/router weights), report routing or expert-usage statistics that confirm the router operates on the intended experts, and release the conversion code and bridge version.
  2. [Section 4.1 and Tables 3 and 5] All quality numbers are single runs without seeds, error bars, or significance tests. The claim that the MoE 'matches' dense-8B quality within 0.006 NDCG@10, and the statements that differences are 'within run-to-run noise', are not statistically supported; several cells differ by only 0.002-0.006, which could easily be within seed variance. The same issue applies to the LoRA-vs-full-parameter comparisons and the distillation-vs-contrastive comparisons. Please report multiple seeds (at least three) for the headline contrastive full-parameter comparison, or otherwise provide variance estimates and a significance test, so that the parity claim is credible.
  3. [Section 3, 'Expert parallelism and topology'] The reported topology 'EP=16 and DP=16 across two 8×H200 nodes' is internally inconsistent with the paper's own statement that world size = TP×EP×DP (with PP omitted): 1×16×16 = 256 ranks, while two 8×H200 nodes provide only 16 ranks. The text then says 'the 128 experts are partitioned sixteen ways', which suggests EP=16 and DP=1 (or another combination that yields 16 ranks). Please correct the topology specification and clarify the TP setting, batch size, sequence length, and activation-recompute setting used for the 78.9 GiB peak-memory measurement, since the feasibility claim depends on this arithmetic.
  4. [Section 2.5 and Table 4] The matched-DP=8 row for Megatron (TP=1/DP=8, ZeRO-1) reports 106 GiB peak memory even though the text says activation recompute is enabled on both backends. Under ZeRO-1 with DP=8, parameters and gradients are replicated (~32 GiB for an 8B bf16 model) while optimizer states are sharded (roughly 12 GiB), so a peak well above 100 GiB is surprising unless some settings differ from those stated. The fact that both the recommended and matched rows report the same 106 GiB value further suggests the table may be misreported or the settings were not actually matched. Please clarify or remeasure this number, because the 'pays with ~2× per-rank memory' interpretation depends on it.
minor comments (5)
  1. [Appendix A] The sentence introducing the three first-stage retrievers contains an unresolved placeholder '(??????)'; please fix this reference.
  2. [Section 2.4] The text uses 'backward capability' where 'backward compatibility' appears to be intended.
  3. [Section 3] Please specify the source of the peak-memory numbers (e.g., nvidia-smi, torch.cuda.max_memory_allocated, or a profiler) for both the 8B profiling in Table 2 and the MoE measurement in Section 3.
  4. [Author list] In the full text, several author names appear with irregular spacing (e.g., 'W enqian Y e5', 'Y u W ang'); please ensure the author line is typeset correctly.
  5. [Section 4.3] The claim that 'the frozen-base forward/backward dominates' the LoRA wall-clock could be made more concrete by reporting measured times for LoRA versus full-parameter runs; as written, the reader must rely on the accompanying efficiency table without a specific row for this claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the load-bearing claims are measured against external benchmarks or standard scaling behavior, and the cited prior work is not load-bearing.

full rationale

The paper does not derive its headline results from its inputs by construction. The backend quality claim is an empirical comparison: Table 3 trains the same dense 8B reranker under PyTorch FSDP1 and the Megatron backend with matched data, recipe, and batch, then evaluates on the external BEIR-15 benchmark across three first-stage retrievers; no fitted parameter is renamed as a prediction. The efficiency claim is similarly measured: Table 4 isolates the framework by matching DP degree and activation recompute, and the paper explicitly attributes the 22% recommended-config gap to topology and recompute choices rather than to an intrinsic framework speedup. The MoE feasibility claim rests on the standard expert-parallel memory accounting that per-rank expert memory scales with experts/EP, and the reported 78.9 GiB peak is a measurement, not a fitted consequence. The MoE-versus-dense parity result is a controlled empirical finding on BEIR-15 with three first-stage retrievers; it is not derived from any assumption that MoE must match dense quality. Serving throughput is measured directly, and the active-FLOPs explanation is a post-hoc interpretation rather than the source of the numbers. The self-referential elements are limited to citations of the authors' earlier work on listwise KL distillation and LACONIC, and to the demonstration running on the authors' own Tevatron 3.0 backend; these are neither uniqueness arguments nor fitted inputs, so they do not make the derivation circular. The unverified external NVIDIA Megatron weight bridge is a reproducibility and correctness risk for the MoE checkpoint conversion, but it is an external dependency, not a circular reduction of the paper's claims.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The paper's central claims rest on standard IR evaluation assumptions (BEIR-15, RLHN-680K), on the correctness of external bridge and tooling, and on the chosen training hyperparameters. There are no invented physical or theoretical entities; the MoE models are existing open-weight models, and the LoRA target registry is a software configuration.

free parameters (6)
  • LoRA rank = 16
    LoRA rank fixed at 16 for all LoRA runs; choice affects parameter-efficiency and quality, not swept.
  • Teacher temperature = 2.0
    Listwise KL distillation teacher temperature fixed at 2.0; the paper states temperature dependence is out of scope.
  • Student temperature = 1.0
    Student temperature fixed at 1.0 for the distillation loss.
  • Train group size = 8
    Listwise group size G=8 (one positive, seven negatives) used for both losses; shapes the loss and batch.
  • Global batch size = 64 queries per step
    Held constant across runs to make wall-clock comparisons; hand-chosen, not fitted.
  • Base learning rate
    The base learning rate is not stated in the manuscript; all quality results depend on it, so it is a hidden chosen parameter.
assumptions (5)
  • domain assumption BEIR-15 mean NDCG@10 is a valid proxy for reranker quality.
    All quality conclusions are drawn from BEIR-15; standard in IR but an assumption about benchmark representativeness. Invoked in Section 4.1.
  • domain assumption RLHN-680K with one epoch and top-200 candidates yields representative rerankers.
    The controlled sweep uses one fixed recipe on RLHN-680K; the paper validates only internally via Table 7.
  • domain assumption The NVIDIA Megatron bridge correctly shards and restores Qwen3-30B-A3B's 128 experts and LoRA adapters.
    The weight bridge is external tooling (Section 2.2); correctness is assumed, not verified by the paper.
  • domain assumption PyTorch FSDP1, accessed via Hugging Face Trainer, is the relevant baseline and lacks expert parallelism.
    The paper's motivation and capability comparison depend on this; no FSDP2 or other EP-capable backend is considered.
  • domain assumption Two 8xH200 nodes with 140 GiB GPUs constitute the 'academic budget'.
    The feasibility claim is anchored to this hardware; it is a strong resource assumption for many academic groups.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Tevatron Meets Megatron: Expert-Parallel LLM Reranker Training on an Academic Budget." pith.science (2026). https://pith.science/paper/D25EHJJD

@misc{pith2026260800916,
  author       = {Pith},
  title        = {Pith review of: Tevatron Meets Megatron: Expert-Parallel LLM Reranker Training on an Academic Budget},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D25EHJJD}},
  note         = {Machine review of arXiv:2608.00916}
}
read the original abstract

Modern reranking recipes---billion-scale cross-encoders, mixture-of-experts (MoE) backbones, and distillation against strong teachers---have outpaced the training infrastructure available to most academic groups. Existing Tevatron reranker training relies on the Hugging Face Trainer with DeepSpeed or PyTorch FSDP1, but these backends lack efficient support for large-scale MoE training. We present Tevatron 3.0, which integrates a Megatron-Core training backend into Tevatron while preserving its data pipeline, evaluation workflow, and Hugging Face-compatible checkpoints. We benchmark existing distributed training configurations against the new backend, showing that Megatron matches FSDP reranker quality and training efficiency under comparable data-parallel settings, is up to 22% faster in the recommended single-node configuration, and supports both LoRA and full-parameter fine-tuning. Crucially, expert parallelism enables training a 30B-parameter Qwen3-30B-A3B MoE reranker, which is infeasible with PyTorch FSDP1. Using this framework, we conduct a controlled comparison of MoE versus dense models, LoRA versus full-parameter tuning, and distillation versus contrastive training on BEIR-15 with three first-stage retrievers, and report serving throughput for Hugging Face and vLLM. We find that the MoE reranker matches dense 8B quality while activating less than half as many parameters and achieving substantially higher inference throughput. We will release the framework and trained checkpoints.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Tevatron-Elastic: A Unified Abstraction for Training Elastic Retrievers and Rerankers

    cs.CL 2026-08 conditional novelty 7.0 of 10

    Tevatron-Elastic unifies depth, token, and width compression for retrievers and rerankers into one abstraction that reproduces prior elastic methods as special cases and adds a new multi-ratio token compression method (MLTC).

Reference graph

Works this paper leans on

18 extracted references · 3 canonical work pages · cited by 1 Pith paper

  1. [1]

    An analysis of the softmax cross entropy loss for learning-to-rank with binary relevance

    Sebastian Bruch, Xuanhui Wang, Michael Bendersky, and Marc Najork. An analysis of the softmax cross entropy loss for learning-to-rank with binary relevance. InProceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval, ICTIR ’19, page 75–78, New York, NY, USA,

    work page 2019

  2. [4]

    doi: 10.18653/v1/N19-1423

    Association for Computational Linguistics. doi: 10.18653/v1/N19-1423. URLhttps://aclanthology.org/ N19-1423/. Luyu Gao, Xueguang Ma, Jimmy Lin, and Jamie Callan. Tevatron: An efficient and flexible toolkit for neural retrieval. InProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’23, pag...

    doi:10.18653/v1/n19-1423

  3. [8]

    ISBN 9798400702297

    Association for Computing Machinery. ISBN 9798400702297. doi: 10.1145/3600006.3613165. URL https://doi.org/10.1145/3600006.3613165. Carlos Lassance, Hervé Déjean, Thibault Formal, and Stéphane Clinchant. Splade-v3: New baselines for splade.arXiv preprint arXiv:2403.06789,

    arXiv

  4. [10]

    doi: 10.18653/v1/2023.eacl-main.148

    Association for Computational Linguistics. doi: 10.18653/v1/2023.eacl-main.148. URL https://aclanthology.org/2023. eacl-main.148/. Niklas Muennighoff, Hongjin Su, Liang Wang, Nan Yang, Furu Wei, Tao Yu, Amanpreet Singh, and Douwe Kiela. Generative representational instruction tuning. InInternational Conference on Learning Representations, volume 2025, pag...

    doi:10.18653/v1/2023.eacl-main.148 2023

  5. [11]

    Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He

    URL https://arxiv.org/abs/2502.07972. Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. Zero: memory optimizations toward training trillion parameter models. InProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC ’20. IEEE Press,

    arXiv

  6. [13]

    Nandan Thakur, Crystina Zhang, Xueguang Ma, and Jimmy Lin

    URLhttps://openreview.net/forum?id= wCu6T5xFjeJ. Nandan Thakur, Crystina Zhang, Xueguang Ma, and Jimmy Lin. Hard negatives, hard lessons: Revisiting training data quality for robust information retrieval with LLMs. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Findings of the Association for Computational Lingu...

    work page 2025

  7. [14]

    ISBN 979-8-89176-335-7

    Association for Computational Linguistics. ISBN 979-8-89176-335-7. doi: 10.18653/v1/2025.findings-emnlp.481. URLhttps://aclanthology.org/2025.findings-emnlp.481/. Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. Improving text embeddings with large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,...

    doi:10.18653/v1/2025.findings-emnlp.481 2025

  8. [16]

    Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, et al

    URLhttps://arxiv.org/abs/2505.09388. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, et al. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176,

    arXiv

Show all 18 references
  1. [17]

    14 Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E

    URL https://arxiv.org/abs/2304.11277. 14 Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, and Ying Sheng. SGLang: Efficient execution of structured language model pro...

    arXiv

  2. [18]

    ISBN 9781450394086

    Association for Computing Machinery. ISBN 9781450394086. doi: 10.1145/3539618.3592047. URL https://doi.org/10.1145/3539618.3592047. Shengyao Zhuang, Zhichao Xu, and Ivano Lauriola. Layer-wise token compression for efficient document reranking. InProceedings of the 49th Interna...

  3. [2007]

    BERT: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio, editors,Proceedings of the 2019 Conference of the North American Chapter of ...

    2019

  4. [2015]

    SebastianHofstätter, SophiaAlthammer, MichaelSchröder, MeteSertkan, andAllanHanbury

    URLhttps: //arxiv.org/abs/1503.02531. SebastianHofstätter, SophiaAlthammer, MichaelSchröder, MeteSertkan, andAllanHanbury. Improvingefficientneural ranking models with cross-architecture knowledge distillation,

    arXiv

  5. [2019]

    ISBN 9781450368810

    Association for Computing Machinery. ISBN 9781450368810. doi: 10.1145/3341981.3344221. URLhttps://doi.org/10.1145/3341981.3344221. Chris Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Greg Hullender. Learning to rank using gradient descent. InPr...

  6. [2020]

    URLhttps://arxiv.org/ abs/1909.08053. Hongjin SU, Howard Yen, Mengzhou Xia, Weijia Shi, Niklas Muennighoff, Han yu Wang, Liu Haisu, Quan Shi, Zachary S Siegel, Michael Tang, Ruoxi Sun, Jinsung Yoon, Sercan O Arik, Danqi Chen, and Tao Yu. BRIGHT: A realistic and challenging ben...

    1909 arXiv

  7. [2021]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    URLhttps://arxiv.org/abs/2010.02666. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arxiv 2021.arXiv preprint arXiv:2106.09685,

    2010 arXiv

  8. [2023]

    ISBN 9781450394086

    Association for Computing Machinery. ISBN 9781450394086. doi: 10.1145/3539618.3591805. URLhttps://doi.org/10.1145/3539618.3591805. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network,

  9. [2025]

    ISBN 9798400715921

    Association for Computing Machinery. ISBN 9798400715921. doi: 10.1145/3726302.3730135. URLhttps://doi.org/10.1145/3726302.3730135. Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. MTEB: Massive text embedding benchmark. In Andreas Vlachos and Isabelle Augenstei...

  10. [2026]

    ISBN 9798400725999

    Association for Computing Machinery. ISBN 9798400725999. doi: 10.1145/3805712.3809871. URLhttps://doi.org/10.1145/3805712.3809871. 15 A Per-Dataset Evaluation Results This appendix reports the full per-dataset breakdown behind the BEIR-15 aggregates in the main text. We give N...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.