Comparative Analysis of Lion and AdamW Optimizers for Cross-Encoder Reranking with MiniLM, GTE, and ModernBERT

Anay Yatin Damle; Manu Pande; Shahil Kumar

REVIEW 3 major objections 6 minor 34 references

Comparative Analysis of Lion and AdamW Optimizers for Cross-Encoder Reranking with MiniLM, GTE, and ModernBERT

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims Lion, a sign-based optimizer, improves cross-encoder reranking for several models, with ModernBERT reaching NDCG@10 0.7225.

desk verdict A tidy empirical benchmark whose headline optimizer comparison is confounded by a 10x learning-rate difference, and whose GPU-efficiency claim inverts the meaning of utilization. read the letter →

arxiv 2506.18297 v1 pith:B6USXJOP submitted 2025-06-23 cs.IR

Shahil Kumar , Manu Pande , Anay Yatin Damle This is my paper · ORCID

classification cs.IR

keywords LionoptimizerAdamWcross-encoderpassagererankingTRECDeepLearning2019MSMARCOModernBERTGPUutilization

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that the Lion optimizer, a sign-based alternative to AdamW, is a practical choice for fine-tuning cross-encoder passage rerankers. The authors train MiniLM, GTE, and ModernBERT on MS MARCO with both optimizers and find that Lion matches or beats AdamW in several settings, with ModernBERT+Lion reaching the highest NDCG@10 (0.7225) and MAP (0.5121) on TREC DL 2019. They also report that Lion lowers GPU utilization by 2.67% to 10.33% across the three models. If right, this makes Lion a drop-in optimizer that can improve both accuracy and resource efficiency for reranking models.

What carries the argument

The central mechanism is Lion's update rule, which applies a sign operation to an exponential moving average of gradients and folds weight decay into the parameter update, so it keeps only momentum and no second-moment estimate. This is attached to a cross-encoder that concatenates query and passage, passes them through a transformer, and scores the [CLS] output with a linear layer and sigmoid, trained with binary cross-entropy on about two million MS MARCO pairs. The argument runs by varying the optimizer on three base models while holding batch size, precision, and epochs fixed, with the ModernBERT+Lion combination additionally receiving a lower learning rate and a cosine annealing scheduler.

What would settle it

Run ModernBERT-base with AdamW at the same 2e-6 learning rate with cosine annealing that the Lion run used; if NDCG@10 on TREC DL 2019 reaches or exceeds 0.7225, the claim that Lion is responsible for the gain is falsified. A complementary check is to run ModernBERT+Lion at 2e-5 with no scheduler and see whether its advantage disappears.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that the choice of optimizer interacts with model architecture and training hyperparameters in cross-encoder fine-tuning. Its headline finding is that ModernBERT-base fine-tuned with Lion, a 2e-6 learning rate, and a cosine annealing schedule reaches NDCG@10 of 0.7225 and MAP of 0.5121 on TREC DL 2019, the highest among the six model-optimizer combinations tested, and ties for the best MRR@10 (0.5988) on MS MARCO dev. The paper also claims that Lion lowers GPU utilization by 2.67% to 10.33% depending on the model, because Lion's update rule tracks only momentum and avoids the second-moment estimates that AdamW stores. The author would summarize the result as evidence that Lion is a competitive, sometimes superior alternative to AdamW for passage reranking, especially for newer long-context encoders when paired with a suitable learning-rate schedule.

Load-bearing premise

The comparison assumes the ModernBERT runs differ only in the optimizer, but the Lion run also used a ten-times-lower learning rate and a cosine annealing schedule, so that configuration difference could explain the higher score.

Editorial extensions

If this is right

Fine-tuned cross-encoders can use Lion instead of AdamW without losing effectiveness on TREC DL 2019, and can gain effectiveness for long-context encoders like ModernBERT under a low learning rate and cosine schedule.
Lion's lower GPU utilization could reduce training cost and make cloud GPU allocation easier, since the same job leaves more headroom.
Optimizer choice interacts with architecture and schedule: GTE performs better with AdamW, MiniLM is mixed, so the optimal configuration is model-specific.
Lion reaches the best MRR@10 (0.5988) on MS MARCO dev with two different models, suggesting the result is not limited to one benchmark.
Performance varies by epoch and optimizer, so validation-based checkpoint selection materially affects the reported ranking of optimizers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The headline ModernBERT result does not isolate the optimizer: the Lion run used a ten-times-lower learning rate and a cosine annealing scheduler, so the measured gain could come from that training configuration rather than from Lion; a learning-rate sweep for both optimizers is needed to settle it.
The GPU 'efficiency' claim should be read carefully: lower mean utilization can mean more idle GPU time, not higher throughput; wall-clock time or tokens-per-second would be a stronger test of Lion's practical speed advantage.
A direct extension would apply the low-learning-rate cosine schedule to MiniLM and GTE with Lion; if the pattern replicates, the paper's thesis shifts from 'Lion is better' to 'Lion benefits from lower learning rates on long-context models'.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a straightforward empirical comparison of Lion vs AdamW for cross-encoder reranking, clearly written and with code and models released. The specific comparison is new, but the headline result—Lion beating AdamW on ModernBERT—does not survive contact with the paper's own hyperparameter table.

What's good: the paper fine-tunes three different base models on MS MARCO and evaluates on TREC DL 2019 and MS MARCO dev, using standard metrics and trec_eval. The artifacts are available and the setup is reproducible. For an empirical note, that is solid. The per-epoch reporting is also honest; you can see performance wobble.

The problems are load-bearing. Section IV.B gives ModernBERT+Lion a learning rate of 2e-6 and ModernBERT+AdamW 2e-5, with a cosine schedule applied only to ModernBERT runs. Section V.B then claims AdamW used "the same LR and scheduler"—contradicting IV.B. So the 0.7225 vs 0.7105 gap attributed to Lion is really a gap in learning-rate regimes. A lower LR with annealing can easily shift the best checkpoint by that much. And GTE+AdamW, with the standard 2e-5 LR, hits 0.7224—essentially the same score. So the "Lion advantage" evaporates once you control for hyperparameters.

Second, the GPU efficiency claim. Table II labels lower mean GPU utilization as "efficiency gains" of 2.67% to 10.33%. Lower utilization means the GPU is idle more, not that the optimizer is more efficient. There may be a real memory or throughput argument for Lion (fewer state tensors), but that is not what they measured. As written, the claim inverts the standard meaning.

Minor but worth noting: single seed, post-hoc best-epoch selection on the test set, and 43 TREC queries. For a drop-in optimizer recommendation, that is thin. The paper is nevertheless readable and the experiments are transparent. It is a decent workshop submission if the authors rerun with matched LRs and fix the efficiency framing; as is, the central claims do not hold.

For a serious editor: send it to peer review? Yes, with the expectation of heavy revision. The topic is practical, the artifacts exist, and the confound is fixable. A good referee can extract a useful negative result or a properly controlled comparison.

Referee Report

3 major / 6 minor

Summary. The paper reports a comparative study of Lion versus AdamW optimizers for fine-tuning three cross-encoder reranking models (MiniLM, GTE, ModernBERT) on MS MARCO, with evaluation on TREC DL 2019 and MS MARCO dev. The abstract claims that ModernBERT with Lion achieves the best NDCG@10 (0.7225) and MAP (0.5121) on TREC DL 2019, and that Lion improves GPU utilization efficiency by 2.67% to 10.33% across models. The paper also presents per-epoch metrics, a GPU utilization table, and a comparison with published systems. The code and trained models are made publicly available.

Significance. If the claims were established, the paper would provide a practical, low-cost optimizer recommendation for cross-encoder reranking, a task where training efficiency matters. The paper has several strengths: it uses standard IR benchmarks and metrics, evaluates multiple epochs, reports a full table of effectiveness metrics, and releases code and model weights, which supports reproducibility. However, the two headline claims are not currently supported. The ModernBERT optimizer comparison is confounded by a 10x difference in learning rate and the presence of a cosine schedule only for the Lion run, and the GPU efficiency claim is based on an interpretation of lower utilization as higher efficiency, which is the opposite of the standard meaning. The single best-result difference is also within the range of noise (0.7225 vs 0.7224 for GTE+AdamW). These issues are load-bearing because they directly undermine the abstract, the contributions list, and the conclusion.

major comments (3)

[§IV.B, §V.B, Table I] The ModernBERT optimizer comparison is confounded by learning rate and scheduler. Section IV.B sets ModernBERT+Lion to learning rate 2e-6 with CosineAnnealingLR, while ModernBERT+AdamW uses 2e-5; Section I explicitly advertises this asymmetry as a 'lower learning rate (2e-6) and a Cosine Annealing scheduler for ModernBERT, contrasting with a higher rate (2e-5) and no scheduler for MiniLM and GTE.' Section V.B then states that ModernBERT+AdamW used 'the same LR and scheduler,' which is directly contradicted by Section IV.B. Consequently, the reported NDCG@10 gap (0.7225 vs 0.7105) cannot be attributed to the optimizer choice, and the central claim that Lion outperforms AdamW for ModernBERT is unsupported.
[§V.A, Table II] The GPU efficiency claim is based on an inversion of the standard meaning of utilization. The paper computes 'efficiency gain' as (AdamW mean − Lion mean)/AdamW mean on GPU utilization and states that 'lower usage indicates better efficiency,' but lower GPU utilization means the GPU is idle more of the time, not that the optimizer is more efficient. No wall-clock training time, throughput, or memory measurements are reported, so the abstract's claim that 'Lion also provides superior GPU efficiency, improving utilization by 2.67% to 10.33%' is not supported by the data presented.
[Table I, §V.C] The 'best overall' result rests on a single-seed margin of 0.0001 in NDCG@10 (ModernBERT+Lion at 0.7225 versus GTE+AdamW at 0.7224) with no significance testing or repeated runs. The best epoch is also selected post hoc for each configuration, which increases the effective number of comparisons without any correction. Given the confounded hyperparameter setup for ModernBERT and the negligible absolute difference, the paper's ranking of optimizers and models is not robust.

minor comments (6)

[§IV.B] There is a typo: 'adamW optmizer' should be 'AdamW optimizer.'
[§IV.B] The use of CosineAnnealingLR with T_max set to the total number of training steps together with warmup_ratio=0.1 is ambiguous; please specify whether warmup is applied to the cosine schedule and how T_max accounts for the warmup steps.
[Table II] The number of data points differs between optimizers for the same model (410 vs 412, 698 vs 699, 578 vs 579) without explanation; please describe the measurement window and why the counts differ.
[§IV.A and Table I footnote] The evaluation set is described as the 'MS MARCO passage dataset’s development split' in Section IV.A but as 'MS MARCO v1.1 passage dataset development split' in the Table I footnote; please clarify the exact version used.
[References] Several references have formatting issues in the provided text, particularly [23] (trec_eval) and [27] (Weights & Biases), where URLs are split across lines; please verify the final rendering.
[§III.C] The training set size is given only as 'approximately 2 million' pairs; please provide the exact count or point to the preprocessing script.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is an external empirical benchmark whose claims reduce to direct measurements on held-out test sets; the ModernBERT hyperparameter confound and GPU-utilization labeling are validity concerns, not circular reasoning.

full rationale

The paper's claims are empirical measurements, not derivations, so no step in the claimed chain reduces to its own inputs. Training uses MS MARCO triplets; every effectiveness figure (NDCG@10, MAP, MRR@10, Recall@10, R-Prec, P@10) is computed with trec_eval against the held-out TREC DL 2019 qrels and MS MARCO dev labels after reranking BM25 top-1000 candidates. No parameter is fitted to these test sets and then renamed a prediction; the best-epoch selection across three checkpoints is post-hoc selection, not a derivation. The GPU 'efficiency gain' in Table II is the arithmetic transform (AdamW Mean - Lion Mean)/AdamW Mean applied to the same measured utilization data; that is descriptive statistics, not a predicted quantity built from an assumed input. There is no self-citation: the authors cite only external prior work (Lion [11], AdamW [10], ModernBERT [8], etc.), and no uniqueness theorem or ansatz is imported from any prior publication of the present authors. Genuine weaknesses exist but are not circularity: Section IV.B assigns ModernBERT+Lion LR 2e-6 while ModernBERT+AdamW gets 2e-5 (with CosineAnnealing only for ModernBERT), so Section V.B's statement that AdamW used 'the same LR and scheduler' contradicts Section IV.B, and the claimed Lion-vs-AdamW advantage for ModernBERT is confounded by an asymmetric hyperparameter regime; Section V.A also labels lower utilization as an efficiency gain, inverting the standard meaning of utilization. Section VI's future-work admission that 'a more thorough hyperparameter search for both optimizers' is needed implicitly concedes this confound. These are correctness risks and should be weighed there, not as circularity. Therefore the appropriate score is 0.

Assumptions & free parameters 2 free parameters · 2 assumptions · 0 invented entities

The central claims rest on hyperparameter choices (LR for Lion+ModernBERT) and on an unconventional definition of GPU efficiency; no new entities are introduced.

free parameters (2)

Learning rate for ModernBERT+Lion = 2e-6
Chosen only for this configuration, 10x below the AdamW configuration, and the paper does not justify why it is the appropriate comparison point.
Best epoch (checkpoint selection) = Epoch 1-3 per configuration
Headline numbers are selected post hoc from the best epoch per metric, which inflates apparent gains.

assumptions (2)

ad hoc to paper Lower GPU utilization means higher efficiency.
Section V.A defines efficiency gain as the percentage reduction in mean GPU usage; the paper treats lower utilization as a benefit, whereas lower utilization typically means more idle GPU.
domain assumption One training seed and 43 TREC queries are enough to rank optimizer/model combinations.
The paper uses seed 12 only and makes no significance test; it nonetheless presents small NDCG@10 differences as decisive in Sections V.B and V.C.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Comparative Analysis of Lion and AdamW Optimizers for Cross-Encoder Reranking with MiniLM, GTE, and ModernBERT." pith.science (2026). https://pith.science/paper/B6USXJOP

@misc{pith2026250618297,
  author       = {Pith},
  title        = {Pith review of: Comparative Analysis of Lion and AdamW Optimizers for Cross-Encoder Reranking with MiniLM, GTE, and ModernBERT},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/B6USXJOP}},
  note         = {Machine review of arXiv:2506.18297}
}

read the original abstract

Modern information retrieval systems often employ a two-stage pipeline: an efficient initial retrieval stage followed by a computationally intensive reranking stage. Cross-encoders have shown strong effectiveness for reranking due to their deep analysis of query-document pairs. This paper studies the impact of the Lion optimizer, a recent alternative to AdamW, during fine-tuning of cross-encoder rerankers. We fine-tune three transformer models-MiniLM, GTE, and ModernBERT-on the MS MARCO passage ranking dataset using both optimizers. GTE and ModernBERT support extended context lengths (up to 8192 tokens). We evaluate effectiveness using TREC 2019 Deep Learning Track and MS MARCO dev set (MRR@10). Experiments, run on the Modal cloud platform, reveal that ModernBERT with Lion achieves the best NDCG@10 (0.7225) and MAP (0.5121) on TREC DL 2019, while MiniLM with Lion ties ModernBERT for MRR@10 (0.5988) on MS MARCO dev. Lion also provides superior GPU efficiency, improving utilization by 2.67% to 10.33% across models. We analyze performance trends using standard IR metrics and discuss the optimizer's impact on training dynamics across architectures.

Figures

Figures reproduced from arXiv: 2506.18297 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 8 canonical work pages

[1]

Retrieve, read, rerank: Towards end-to-end multi-document reading comprehension,

M. Hu, Y . Peng, Z. Huang, and D. Li, “Retrieve, read, rerank: Towards end-to-end multi-document reading comprehension,” 2019. [Online]. Available: https://arxiv.org/abs/1906.04618

arXiv 2019
[2]

The probabilistic relevance framework: Bm25 and beyond,

S. Robertson and H. Zaragoza, “The probabilistic relevance framework: Bm25 and beyond,”Foundations and Trends® in Information Retrieval, vol. 3, no. 4, pp. 333–389, 2009

work page 2009
[3]

Dense passage retrieval for open-domain question an- swering,

V . Karpukhin, B. Oguz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W.-t. Yih, “Dense passage retrieval for open-domain question an- swering,” inProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020, pp. 6769–6781

2020
[4]

Bert: Pre-training of deep bidirectional transformers for language understanding,

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 2019, pp. 4171–4186. [Online]. Available: ...

work page 2019
[5]

Passage re-ranking with bert,

R. Nogueira and K. Cho, “Passage re-ranking with bert,” 2020. [Online]. Available: https://arxiv.org/abs/1901.04085

arXiv 2020
[6]

Document ranking with a pretrained sequence-to-sequence model,

R. Nogueira, Z. Yang, K. Cho, and J. Lin, “Document ranking with a pretrained sequence-to-sequence model,” inFindings of the Association for Computational Linguistics: EMNLP 2020. Online: Association for Computational Linguistics, Nov. 2020, pp. 1124–1136. [Online]. Available: https://aclanthology.org/2020.findings-emnlp.102

work page 2020
[7]

Towards general text embeddings with multi-stage contrastive learning,

Z. Li, X. Zhang, Y . Zhang, D. Long, P. Xie, and M. Zhang, “Towards general text embeddings with multi-stage contrastive learning,” 2023. [Online]. Available: https://arxiv.org/abs/2308.03281

arXiv 2023
[8]

Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference,

B. Warner, A. Chaffin, B. Clavi ´e, O. Weller, O. Hallstr ¨om, S. Taghadouini, A. Gallagher, R. Biswas, F. Ladhak, T. Aarsen, N. Cooper, G. Adams, J. Howard, and I. Poli, “Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference,” 2024. [Online]. Available: https://arxiv.org/abs/...

arXiv 2024

Show all 34 references

[9]

Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers,

W. Wang, F. Wei, L. Dong, H. Bao, N. Yang, and M. Zhou, “Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers,” 2020. [Online]. Available: https://arxiv. org/abs/2002.10957

2020 arXiv
[10]

Decoupled weight decay regularization,

I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,”
[11]

Symbolic discovery of optimization algorithms,

X. Chen, C. Liang, D. Huang, E. Real, K. Wang, Y . Liu, H. Pham, X. Dong, T. Luong, C.-J. Hsieh, Y . Lu, and Q. V . Le, “Symbolic discovery of optimization algorithms,” 2023. [Online]. Available: https://arxiv.org/abs/2302.06675

2023 arXiv
[12]

MS MARCO: A human generated machine reading comprehension dataset,

T. Nguyen, M. Rosenberg, X. Song, J. Gao, S. Tiwary, R. Majumder, and L. Deng, “MS MARCO: A human generated machine reading comprehension dataset,”CoRR, vol. abs/1611.09268, 2016. [Online]. Available: http://arxiv.org/abs/1611.09268

2016 arXiv
[13]

Overview of the trec 2019 deep learning track,

N. Craswell, B. Mitra, E. Yilmaz, D. Campos, and E. M. V oorhees, “Overview of the trec 2019 deep learning track,” 2020. [Online]. Available: https://arxiv.org/abs/2003.07820

2019 arXiv
[14]

Modal: Serverless compute for ai and data workflows,

Modal, “Modal: Serverless compute for ai and data workflows,” https: //modal.com/, 2025, accessed: 20-04-2025

2025
[15]

Pretrained transformers for text ranking: Bert and beyond,

J. Lin, R. Nogueira, and A. Yates, “Pretrained transformers for text ranking: Bert and beyond,” inSynthesis Lectures on Human Language Technologies, 2021. [Online]. Available: https: //arxiv.org/abs/2010.06467

2021 arXiv
[16]

Complementing lexical retrieval with semantic residual embedding,

L. Gao, Z. Dai, T. Chen, Z. Fan, B. V . Durme, and J. Callan, “Complementing lexical retrieval with semantic residual embedding,”
[17]

Improving efficient neural ranking models with cross-architecture knowledge distillation,

S. Hofst ¨atter, S. Althammer, M. Schr ¨oder, M. Sertkan, and A. Hanbury, “Improving efficient neural ranking models with cross-architecture knowledge distillation,” 2021. [Online]. Available: https://arxiv.org/abs/ 2010.02666

2021 arXiv
[18]

Roformer: Enhanced transformer with rotary position embedding,

J. Su, Y . Lu, S. Pan, A. Murtadha, B. Wen, and Y . Liu, “Roformer: Enhanced transformer with rotary position embedding,” 2023. [Online]. Available: https://arxiv.org/abs/2104.09864

2023 arXiv
[19]

Flashattention: Fast and memory-efficient exact attention with io-awareness,

T. Dao, D. Y . Fu, S. Ermon, A. Rudra, and C. R ´e, “Flashattention: Fast and memory-efficient exact attention with io-awareness,” 2022. [Online]. Available: https://arxiv.org/abs/2205.14135

2022 arXiv
[20]

Sentence-bert: Sentence embeddings using siamese bert-networks,

N. Reimers and I. Gurevych, “Sentence-bert: Sentence embeddings using siamese bert-networks,” inProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 11 2...

2019 arXiv
[21]

Adam: A method for stochastic optimization,

D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,”
[22]

Overview of the trec 2020 deep learning track,

N. Craswell, B. Mitra, E. Yilmaz, and D. Campos, “Overview of the trec 2020 deep learning track,” 2021. [Online]. Available: https://arxiv.org/abs/2102.07662

2020 arXiv
[23]

trec eval information retrieval evaluation software,

National Institute of Standards and Technology (NIST), “trec eval information retrieval evaluation software,” https://github.com/usnistgov/ trec eval, 2024

2024
[24]

Transformers: State-of- the-art natural language processing,

T. Wolf, L. Debut, V . Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y . Jernite, J. Plu, C. Xu, T. L. Scao, S. Gugger, M. Drame, Q. Lhoest, and A. M. Rush, “Transformers: State-of- the-art na...

2020
[25]

Glu variants improve transformer,

N. Shazeer, “Glu variants improve transformer,” 2020. [Online]. Available: https://arxiv.org/abs/2002.05202

2020 arXiv
[26]

Pytorch: An imperative style, high-performance deep learning library,

A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. K ¨opf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “Pytorch: An imperative style, high-...

2019 arXiv
[27]

Experiment tracking with weights and biases,

L. Biewald, “Experiment tracking with weights and biases,” 2020, software available from wandb.com. [Online]. Available: https: //www.wandb.com/

2020
[28]

Pyserini: A Python toolkit for reproducible information retrieval research with sparse and dense representations,

J. Lin, X. Ma, S.-C. Lin, J.-H. Yang, R. Pradeep, and R. Nogueira, “Pyserini: A Python toolkit for reproducible information retrieval research with sparse and dense representations,” inProceedings of the 44th International ACM SIGIR Conference on Research and Development in In...

2021
[29]

Distilling dense representations for ranking using tightly-coupled teachers,

S.-C. Lin, J.-H. Yang, and J. Lin, “Distilling dense representations for ranking using tightly-coupled teachers,” 2020. [Online]. Available: https://arxiv.org/abs/2010.11386

2020 arXiv
[30]

Approximate nearest neighbor negative contrastive learning for dense text retrieval,

L. Xiong, C. Xiong, Y . Li, K.-F. Tang, J. Liu, P. Bennett, J. Ahmed, and A. Overwijk, “Approximate nearest neighbor negative contrastive learning for dense text retrieval,” 2020. [Online]. Available: https://arxiv.org/abs/2007.00808

2020 arXiv
[31]

Nboost: Neural boosting search results,

C. Thienes and J. Pertschuk, “Nboost: Neural boosting search results,” https://github.com/koursaros-ai/nboost, 2019

2019
[2017]

Available: https://arxiv.org/abs/1412.6980

[Online]. Available: https://arxiv.org/abs/1412.6980

arXiv
[2019]

Available: https://arxiv.org/abs/1711.05101

[Online]. Available: https://arxiv.org/abs/1711.05101

arXiv
[2021]

Available: https://arxiv.org/abs/2004.13969

[Online]. Available: https://arxiv.org/abs/2004.13969

2004 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Retrieve, read, rerank: Towards end-to-end multi-document reading comprehension,

M. Hu, Y . Peng, Z. Huang, and D. Li, “Retrieve, read, rerank: Towards end-to-end multi-document reading comprehension,” 2019. [Online]. Available: https://arxiv.org/abs/1906.04618

arXiv 2019

[2] [2]

The probabilistic relevance framework: Bm25 and beyond,

S. Robertson and H. Zaragoza, “The probabilistic relevance framework: Bm25 and beyond,”Foundations and Trends® in Information Retrieval, vol. 3, no. 4, pp. 333–389, 2009

work page 2009

[3] [3]

Dense passage retrieval for open-domain question an- swering,

V . Karpukhin, B. Oguz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W.-t. Yih, “Dense passage retrieval for open-domain question an- swering,” inProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020, pp. 6769–6781

2020

[4] [4]

Bert: Pre-training of deep bidirectional transformers for language understanding,

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 2019, pp. 4171–4186. [Online]. Available: ...

work page 2019

[5] [5]

Passage re-ranking with bert,

R. Nogueira and K. Cho, “Passage re-ranking with bert,” 2020. [Online]. Available: https://arxiv.org/abs/1901.04085

arXiv 2020

[6] [6]

Document ranking with a pretrained sequence-to-sequence model,

R. Nogueira, Z. Yang, K. Cho, and J. Lin, “Document ranking with a pretrained sequence-to-sequence model,” inFindings of the Association for Computational Linguistics: EMNLP 2020. Online: Association for Computational Linguistics, Nov. 2020, pp. 1124–1136. [Online]. Available: https://aclanthology.org/2020.findings-emnlp.102

work page 2020

[7] [7]

Towards general text embeddings with multi-stage contrastive learning,

Z. Li, X. Zhang, Y . Zhang, D. Long, P. Xie, and M. Zhang, “Towards general text embeddings with multi-stage contrastive learning,” 2023. [Online]. Available: https://arxiv.org/abs/2308.03281

arXiv 2023

[8] [8]

Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference,

B. Warner, A. Chaffin, B. Clavi ´e, O. Weller, O. Hallstr ¨om, S. Taghadouini, A. Gallagher, R. Biswas, F. Ladhak, T. Aarsen, N. Cooper, G. Adams, J. Howard, and I. Poli, “Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference,” 2024. [Online]. Available: https://arxiv.org/abs/...

arXiv 2024