Pith. sign in

REVIEW 3 major objections 5 minor 40 references

Leveraging Decoder Architectures for Learned Sparse Retrieval

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Multi-token decoding turns encoder-decoder transformers into the strongest learned sparse retrievers.

desk verdict Useful three-backbone LSR study with a real multi-token decoding adaptation, but the main architecture ranking is confounded by scale and pretraining and needs tighter controls. read the letter →

arxiv 2504.18151 v1 pith:3ZCU4XWH submitted 2025-04-25 cs.IR

Jingfen Qiao , Thong Nguyen , Evangelos Kanoulas , Andrew Yates This is my paper · ORCID
classification cs.IR
keywords learnedsparseretrievalrepresentationstransformerarchitecturesencoder-decodermodelsmulti-tokendecodingtermexpansionzero-shotmodelscaling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether learned sparse retrieval — encoding queries and documents into weighted vocabulary vectors for inverted-index search — can move beyond the small encoder-only transformers that currently dominate it. The authors show that zero-shot large language models produce poor sparse vectors: they expand input text with noisy terms and lose recall. Under a shared fine-tuning recipe, an encoder-decoder backbone that feeds the full input to both encoder and decoder, then pools masked-language-model logits across decoder token positions, outperforms the leading encoder-only baseline (DistilSplade-max) on MS MARCO dev and TREC Deep Learning 2019, while using fewer FLOPs. Decoder-only models lag at small scale but catch up when scaled to over a billion parameters. The practical upshot is that the architecture and the way the sparse head sees the sequence matter more than raw model size.

What carries the argument

The load-bearing mechanism is multi-tokens decoding: instead of passing only a start token to the decoder — the single-token bottleneck of encoder-decoder sentence embeddings — the full input is copied into the decoder, so the model produces one hidden state per input token. An MLM head then turns each hidden state into a vocabulary-size logit vector and a row-wise max pooling collapses them into a single sparse vector. The decoder's causal attention over the copied input lets the representation aggregate lexical evidence from the entire sequence, which the paper argues is what lets encoder-decoder outperform encoder-only and decoder-only backbones under the same training regime.

What would settle it

Train an encoder-only masked-language model with about 248M parameters, the same teacher and hard negatives, the same FLOPs regularizer, and the same 600k steps as in Table 2; if it reaches or exceeds the 43.6 NDCG@10 of EncDec.MultiTokens FlanT5-base on MS MARCO dev, the paper's architecture ranking is confounded by scale or pretraining rather than by the encoder-decoder design.

Watch

Extended reading notes

Core claim

The paper's central claim is that the encoder-decoder transformer with multi-tokens decoding yields the most effective learned sparse representations among the three backbone families, under controlled fine-tuning. Concretely, the model takes the complete token sequence as encoder input, prepends the start token and feeds the same sequence to the decoder, and then applies an MLM head to every decoder hidden state, max-pooling the resulting vocabulary logits across positions to form one sparse vector per text. In the paper's experiments this design reaches 43.6 NDCG@10 on MS MARCO dev and 71.4 on TREC DL 2019, edging out the encoder-only DistilSplade-max (43.3 and 71.0) while spending fewer FLOPs (2.8 vs 4.0). The paper also reports that zero-shot LLMs generate noisy expansions and degraded recall, and that decoder-only models need roughly a billion parameters before they match encoder-only performance on the tested benchmarks.

Load-bearing premise

The architecture ranking assumes that differences in parameter count and pretraining between the backbones (66M DistilBERT, 248M Flan-T5, 350M OPT) do not drive the results, since the paper controls training data, loss, and regularization but not model scale or pretraining objective.

Editorial extensions

If this is right

  • Encoder-decoder backbones can become the default starting point for learned sparse retrieval, since they combine bidirectional and causal attention without needing billion-scale parameters.
  • Decoding strategies matter as much as pretraining: a small change in how the decoder sees the input (full sequence vs. single token) shifts NDCG by several points, so future LSR designs should treat the sparse-representation head and decoding pattern as first-class design choices.
  • Zero-shot prompting of LLMs for sparse retrieval is not reliable as-is: expanding with all input-token logits injects noise, so retrieval-oriented fine-tuning or constrained decoding is necessary before LLMs help.
  • Scaling decoder-only models is a viable route to strong sparse retrieval: OPT-1.3B matches or exceeds the encoder-only baseline on DL2019 and DL2020, suggesting that causal LMs become effective sparsifiers only at larger scale.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to apply multi-tokens decoding to dense retrieval heads: feeding the full sequence to the decoder and pooling could similarly unify encoder-decoder backbones for both sparse and dense representations, potentially improving hybrid retrieval without separate encoders.
  • The paper's finding that a sharper teacher distribution hurts decoder-only students more than encoder-decoder students hints that distillation robustness, not just architecture, governs which backbone scales best; future work could measure how score-sharpness interacts with causal attention.
  • Because the paper does not match parameter count or pretraining objective across backbones, a direct implication is that the 1.3B decoder-only result may be driven by scale alone; the obvious next experiment is a same-size encoder comparison to isolate the architectural contribution.
  • The zero-shot expansion failure suggests that term expansion in LSR should be conditioned on retrieval objectives, not just language-model likelihood; a promising design is to restrict LLM expansions to a retrieval-specific vocabulary or to train a lightweight gating head on top of the MLM logits.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper investigates learned sparse retrieval (LSR) using transformer backbones of three architectural types: encoder-only, decoder-only, and encoder-decoder. It proposes a 'multi-tokens decoding' approach in which the full input is fed to the decoder (prepended with a start token) and an MLM head with max-pooling over token positions produces sparse lexical representations. The authors evaluate zero-shot and fine-tuned performance on MS MARCO and TREC DL 2019/2020, comparing against Splade-family baselines and various heads (MLP, MLM-single-token, MLM-multi-tokens). The main reported findings are: (1) zero-shot LLMs produce poor sparse representations, with term expansion actually hurting recall; (2) decoder-only models become competitive only when scaled to large sizes; (3) copying the encoder input to the decoder improves over single-token decoding for encoder-decoder models; and (4) the encoder-decoder backbone with multi-tokens decoding achieves the best effectiveness among the three backbone types, with lower FLOPs than DistilSplade-max. The paper releases code and reproduces baseline numbers.

Significance. If the architecture ranking were cleanly established, the paper would provide a useful recipe for applying larger pretrained models to learned sparse retrieval and would extend the LSR literature beyond small encoder-only models. The head comparison within the encoder-decoder family (MLM-MultiTokens vs. MLM-SingleToken vs. MLP, Table 3) is a clean within-model contribution with significance tests. The zero-shot analysis is falsifiable and adds evidence to the growing picture that LLM logits are not directly suited for sparse retrieval. However, the central claim about the encoder-decoder architecture being 'best among the three backbones' is currently confounded with model scale, pretraining objective, vocabulary size, and hyperparameters, as detailed in the major comments; therefore the headline conclusion is not yet established.

major comments (3)
  1. [Section 5, RQ2, Table 2] The central claim that the encoder-decoder architecture with multi-tokens decoding is the best of the three backbones is not established because the comparison confounds architecture with parameter count and pretraining. EncDec.MultiTokensFlanT5-base (Flan-T5-base, 248M parameters, span-corruption pretraining) is compared to DistilSplade-max (DistilBERT-base, 66M parameters, masked-LM distillation) and Dec.MultiTokensOPT-350M (350M parameters, causal-LM pretraining); no row in Table 2 or Table 4 isolates architecture while holding scale and pretraining fixed. Within the same T5 family, Enc.FlanT5-base uses only the encoder stack, Dec.FlanT5-base only the decoder stack, and EncDec.FlanT5-base both stacks, so parameter counts differ as well. The paper should either report controlled comparisons (e.g., matched-size T5 encoder-only and decoder-only models, or a DistilBERT-scale T5 variant) or substantially qualify the architecture ranking.
  2. [Section 5, RQ2, Table 2 row 3; Appendix Table 6] The post hoc explanation that the weakness of Enc.FlanT5-base (NDCG@10 36.0) is due to 'incompatibility' between the MLM head and T5's encoder is not tested, and Appendix Table 6 shows that zero-shot EncDec.MultiTokensFlanT5-base produces an empty representation (RR@10 0, R@1k 0), which suggests the T5 decoder's LM head may also be miscalibrated for LSR. Because row 3 is used to argue that the encoder-only backbone is inferior, the explanation for its failure must be validated (e.g., by training the MLM head from scratch on the T5 encoder, or by reporting diagnostic sparsity statistics) before the architecture comparison can be interpreted.
  3. [Section 3, Training Configuration; Table 2] The claimed 'same training configuration' is not actually identical across backbones: OPT uses a lower learning rate (1e-6) than Flan-T5 (5e-4), and the vocabulary sizes differ (DistilBERT 30k, T5 32k, OPT 50k), which can affect the FLOPs regularization and the sparse-representation search space. Moreover, no significance tests are reported for the Table 2 backbone comparisons; the daggers in Table 3 apply only to the head comparisons within EncDec.FlanT5-base. The paper should report paired significance tests or confidence intervals for the backbone differences, or explicitly state that these differences are not statistically significant.
minor comments (5)
  1. [Section 2.1] The two 'Figure??' references in the descriptions of Decoder-only multi-tokens and Encoder-Decoder multi-tokens should be replaced with the correct figure number.
  2. [Equation (7)] In the definition of the MLP head, the function name 'importanceMLM' appears to be a typo for 'importanceMLP'; please also harmonize 'ReLu' and 'ReLU' across equations.
  3. [Section 5, RQ4, Table 4] The sentence 'the nDCG@10 of the decoder-only backbone increased to 60.9 on DL2020' contradicts Table 4 row 8, which reports 69.5; this appears to be a typo.
  4. [Table 3 caption] The dagger and double-dagger footnote is repeated and ambiguous; for example, 'A † indicates paired t-test p < 0.05. († indicates test between MLM-MT and MLM-ST)' should be condensed into one clear statement.
  5. [Appendix, Table 6] The zero-shot EncDec.MultiTokensFlanT5-base row reports RR@10 0 and R@1k 0 without explanation; if this arises from an all-zero sparse vector, that should be stated and its cause discussed, as it bears on the zero-shot findings in RQ1.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper's claims are empirical benchmark comparisons with no derivation that reduces to its inputs; self-citations are contextual only.

full rationale

The paper makes no claim of deriving a result from first principles. Its central findings are empirical: EncDec.MultiTokens outperforms the other backbones on MS MARCO dev and TREC-DL 2019/2020, decoder-only models require large scale to be competitive, and zero-shot LLMs struggle with sparse retrieval. These claims are supported by direct experiments against public baselines such as Splade-max and DistilSplade-max, trained on the same MS MARCO data and evaluated with standard metrics. The architecture equations (2), (4), (6), and the sparse head equations (7) and (8) define the proposed and baseline models; none of these definitions presuppose the experimental outcome. The FLOPs regularization and MarginMSE distillation loss are standard components taken from prior work, not fitted to the conclusion. Self-citations to Nguyen et al. [23] are used for terminology, for the observation that existing LSR models use encoder backbones, and for an auxiliary EPIC finding; none of these citations is load-bearing for the new architecture comparison, and none defines the ranking in Table 2. No parameter is fitted to a subset of data and then renamed as a prediction, no uniqueness theorem is imported, and no known result is repackaged as a novelty. Concerns about confounding model scale and pretraining, or the absence of significance tests for the architecture comparison, are experimental validity issues rather than circularity. The derivation chain, such as it is, is self-contained empirical evaluation, so the circularity score is 0.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

This is an empirical benchmarking paper, so the ledger records experimental design choices rather than derivational constants. Free parameters are the sparsity regularization coefficients, the zero-shot top-1000 expansion cutoff, teacher score rescaling parameters, and per-backbone learning rates. The assumptions are standard LSR design choices inherited from SPLADE and distillation literature. No invented entities are proposed; multi-token decoding reuses existing model components.

free parameters (4)
  • FLOPs regularization weights lambda_q, lambda_d = searched over 1e-1 to 1e-4; exact per-model values not reported
    Sparsity loss coefficients in Eq. 10 are tuned per model and affect the effectiveness/efficiency tradeoff behind the architecture ranking.
  • Zero-shot expansion top-k threshold = 1000
    Table 1 keeps top 1,000 MLM logit tokens for the expansion condition; the zero-shot conclusions depend on this cutoff.
  • Affine transformation of RankLLaMA teacher scores = mean and standard deviation aligned to MiniLM-L-6-v2
    The larger teacher's scores are linearly rescaled before distillation (Section 4), and the rescaling parameters are computed from the training set.
  • Per-backbone learning rates = 1e-6 for OPT, 5e-4 for FlanT5
    Different learning rates per backbone are chosen by hand to stabilize training and could influence relative results.
assumptions (3)
  • domain assumption Max-pooling aggregation over token positions is more effective than sum pooling for MLM heads
    Adopted from Formal et al. (Splade) and applied to all MLM models; if false for decoder hidden states, the multi-token approach's advantage could be a pooling artifact.
  • domain assumption MarginMSE distillation with a cross-encoder teacher is a valid basis for comparing architectures
    All models use the same distillation loss and teacher scores; the architecture ranking is assumed to be insensitive to this choice, explicitly stated in Section 4.
  • domain assumption MLM logits from a decoder fed with the input text are a meaningful estimate of term importance for retrieval
    The entire method builds on interpreting language-model head logits as sparse term weights, following SPLADE; this is not proven for decoder hidden states.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Leveraging Decoder Architectures for Learned Sparse Retrieval." pith.science (2026). https://pith.science/paper/3ZCU4XWH

@misc{pith2026250418151,
  author       = {Pith},
  title        = {Pith review of: Leveraging Decoder Architectures for Learned Sparse Retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3ZCU4XWH}},
  note         = {Machine review of arXiv:2504.18151}
}
read the original abstract

Learned Sparse Retrieval (LSR) has traditionally focused on small-scale encoder-only transformer architectures. With the advent of large-scale pre-trained language models, their capability to generate sparse representations for retrieval tasks across different transformer-based architectures, including encoder-only, decoder-only, and encoder-decoder models, remains largely unexplored. This study investigates the effectiveness of LSR across these architectures, exploring various sparse representation heads and model scales. Our results highlight the limitations of using large language models to create effective sparse representations in zero-shot settings, identifying challenges such as inappropriate term expansions and reduced performance due to the lack of expansion. We find that the encoder-decoder architecture with multi-tokens decoding approach achieves the best performance among the three backbones. While the decoder-only model performs worse than the encoder-only model, it demonstrates the potential to outperform when scaled to a high number of parameters.

Figures

Figures reproduced from arXiv: 2504.18151 by the authors.

Figure 1
Figure 1. Output Bags of Tokens Produced by Different Sparse Representation [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Learned sparse retrieval architectures consist of (1) a transformer backbone [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Score distributions of the two teacher models on the MS MARCO training [PITH_FULL_IMAGE:figures/full_fig_p018_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 27 canonical work pages

  1. [1]

    Arpit, D., Zhou, Y., Ngo, H.Q., Govindaraju, V.: Why regularized auto-encoders learn sparse representation? In: Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48. p. 136–144. ICML’16, JMLR.org (2016)

    work page 2016

  2. [2]

    Bai, Y., Li, X., Wang, G., Zhang, C., Shang, L., Xu, J., Wang, Z., Wang, F., Liu, Q.: Sparterm: Learning term-based sparse representation for fast text retrieval (2020)

    work page 2020

  3. [3]

    Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T., Rosenberg, M., Song, X., Stoica, A., Tiwary, S., Wang, T.: MS MARCO: A human generated machine reading comprehension dataset (2018)

    work page 2018

  4. [4]

    Bruch, S., Nardini, F.M., Rulli, C., Venturini, R.: Efficient inverted indexes for approximateretrievaloverlearnedsparserepresentations.In:Proceedingsofthe47th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). pp. 152–162. ACM (2024).https://doi.org/10.1145/3626772. 3657769, https://doi.org/10.1145/3626772.3657769

    arXiv 2024

  5. [5]

    Chung, H.W., Hou, L., Longpre, S., Zoph, B., Tai, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., Webson, A., Gu, S.S., Dai, Z., Suzgun, M., Chen, X., Chowdhery, A., Castro-Ros, A., Pellat, M., Robinson, K., Valter, D., Narang, S., Mishra, G., Yu, A., Zhao, V., Huang, Y., Dai, A., Yu, H., Petrov, S., Chi, E.H., Dean, J., Devlin, J., Roberts, A....

    work page 2024

  6. [6]

    Craswell, N., Mitra, B., Yilmaz, E., Campos, D.: Overview of the trec 2020 deep learning track (2021)

    work page 2021

  7. [7]

    Craswell, N., Mitra, B., Yilmaz, E., Campos, D., Voorhees, E.M.: Overview of the trec 2019 deep learning track (2020)

    work page 2020

  8. [8]

    Dai, Z., Callan, J.: Context-aware sentence/passage term importance estimation for first stage retrieval (2019)

    work page 2019

Show all 40 references
  1. [9]

    Doshi, M., Kumar, V., Murthy, R., P, V., Sen, J.: Mistral-splade: Llms for better learned sparse retrieval (2024),https://arxiv.org/abs/2408.11119

    2024 arXiv

  2. [10]

    In: Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval

    Fang, H., Tao, T., Zhai, C.: A formal study of information retrieval heuristics. In: Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval. pp. 49–56 (2004)

    2004

  3. [11]

    In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

    Formal, T., Lassance, C., Piwowarski, B., Clinchant, S.: From distillation to hard negative sampling: Making sparse neural ir models more effective. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. p. 2353–235...

    2022

  4. [12]

    Formal, T., Lassance, C., Piwowarski, B., Clinchant, S.: Splade v2: Sparse lexical and expansion model for information retrieval (2021).https://doi.org/10.48550/ ARXIV.2109.10086, https://arxiv.org/abs/2109.10086

    2021 doi arXiv

  5. [13]

    In: Muresan, S., Nakov, P., Villavicencio, A

    Gao,L.,Callan,J.:Unsupervisedcorpusawarelanguagemodelpre-trainingfordense passage retrieval. In: Muresan, S., Nakov, P., Villavicencio, A. (eds.) Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 2843–2853. Ass...

    2022 doi

  6. [14]

    Hofstätter, S., Althammer, S., Schröder, M., Sertkan, M., Hanbury, A.: Improving efficient neural ranking models with cross-architecture knowledge distillation (2021)

    2021

  7. [15]

    Lassance, C., Déjean, H., Formal, T., Clinchant, S.: Splade-v3: New baselines for splade (2024)

    2024

  8. [16]

    Lin, J., Ma, X.: A few brief notes on deepimpact, coil, and a conceptual framework for information retrieval techniques (2021)

    2021

  9. [17]

    In: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval

    Ma, X., Wang, L., Yang, N., Wei, F., Lin, J.: Fine-tuning llama for multi-stage text retrieval. In: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. p. 2421–2425. SIGIR ’24, Association for Computing Machinery, Ne...

    2024

  10. [18]

    Ma, X., Zhang, X., Pradeep, R., Lin, J.: Zero-shot listwise document reranking with a large language model (2023)

    2023

  11. [19]

    In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval

    MacAvaney, S., Nardini, F.M., Perego, R., Tonellotto, N., Goharian, N., Frieder, O.: Expansion via prediction of importance with contextualization. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. SIGIR ’20, A...

    2020

  12. [20]

    In: Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval

    MacAvaney, S., Nardini, F.M., Perego, R., Tonellotto, N., Goharian, N., Frieder, O.: Expansion via prediction of importance with contextualization. In: Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval. pp. 1573–1576 (2020)

    2020

  13. [21]

    IEEE Trans

    Malkov, Y.A., Yashunin, D.A.: Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE Trans. Pattern Anal. Mach. Intell. 42(4), 824–836 (Apr 2020). https://doi.org/10.1109/TPAMI.2018. 2889473, https://doi.org/10.1109/TPAMI...

    2020

  14. [22]

    In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

    Mallia, A., Khattab, O., Suel, T., Tonellotto, N.: Learning passage impacts for inverted indexes. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. p. 1723–1727. SIGIR ’21, Association for Computing Machinery, ...

    2021

  15. [23]

    In: Advances in Information Retrieval: 45th European Conference on Information Retrieval, ECIR 2023, Dublin, Ireland, April 2–6, 2023, Proceedings, Part III

    Nguyen, T., MacAvaney, S., Yates, A.: A unified framework for learned sparse retrieval. In: Advances in Information Retrieval: 45th European Conference on Information Retrieval, ECIR 2023, Dublin, Ireland, April 2–6, 2023, Proceedings, Part III. pp. 101–116. Springer (2023)

    2023

  16. [24]

    In: Findings of the Association for Computational Linguistics: ACL 2022

    Ni, J., Hernandez Abrego, G., Constant, N., Ma, J., Hall, K., Cer, D., Yang, Y.: Sentence-t5: Scalable sentence encoders from pre-trained text-to-text models. In: Findings of the Association for Computational Linguistics: ACL 2022. pp. 1864–

    2022

  17. [25]

    In: Goldberg, Y., Kozareva, Z., Zhang, Y

    Ni, J., Qu, C., Lu, J., Dai, Z., Hernandez Abrego, G., Ma, J., Zhao, V., Luan, Y., Hall, K., Chang, M.W., Yang, Y.: Large dual encoders are generalizable retrievers. In: Goldberg, Y., Kozareva, Z., Zhang, Y. (eds.) Proceedings of the 2022 Conference on Empirical Methods in Nat...

    2022 doi

  18. [26]

    arXiv preprint arXiv:1904.08375 (2019) Leveraging Decoder Architectures for Learned Sparse Retrieval 17

    Nogueira, R., Yang, W., Lin, J., Cho, K.: Document expansion by query prediction. arXiv preprint arXiv:1904.08375 (2019) Leveraging Decoder Architectures for Learned Sparse Retrieval 17

    2019 arXiv

  19. [27]

    In: International Conference on Learning Representations (2020), https://openreview.net/forum?id=SygpC6Ntvr

    Paria, B., Yeh, C.K., Yen, I.E., Xu, N., Ravikumar, P., Póczos, B.: Minimizing flops to learn efficient sparse representations. In: International Conference on Learning Representations (2020), https://openreview.net/forum?id=SygpC6Ntvr

    2020

  20. [28]

    Pradeep, R., Sharifymoghaddam, S., Lin, J.: Rankvicuna: Zero-shot listwise docu- ment reranking with open-source large language models (2023)

    2023

  21. [29]

    ArXiv (2019)

    Sanh, V., Debut, L., Chaumond, J., Wolf, T.: Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. ArXiv (2019)

    2019

  22. [30]

    In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

    Sun, W., Yan, L., Ma, X., Ren, P., Yin, D., Ren, Z.: Is chatgpt good at search? investigating large language models as re-ranking agent. In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. pp. 14918–14937. Association for Computational L...

    2023

  23. [31]

    Advances in neural information processing systems33, 5776–5788 (2020)

    Wang, W., Wei, F., Dong, L., Bao, H., Yang, N., Zhou, M.: Minilm: Deep self- attention distillation for task-agnostic compression of pre-trained transformers. Advances in neural information processing systems33, 5776–5788 (2020)

    2020

  24. [32]

    In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval

    Yang, P., Fang, H., Lin, J.: Anserini: Enabling the use of lucene for information retrieval research. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. p. 1253–1256. SIGIR ’17, Association for Computing Machine...

    2017

  25. [33]

    Yang, P., Fang, H., Lin, J.: Anserini: Reproducible ranking baselines using lucene. J. Data and Information Quality10(4) (oct 2018)

    2018

  26. [34]

    In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management

    Zamani, H., Dehghani, M., Croft, W.B., Learned-Miller, E., Kamps, J.: From neural re-ranking to neural ranking: Learning a sparse representation for inverted indexing. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management. p. 497–506....

    2018

  27. [35]

    Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X.V., Mihaylov, T., Ott, M., Shleifer, S., Shuster, K., Simig, D., Koura, P.S., Sridhar, A., Wang, T., Zettlemoyer, L.: Opt: Open pre-trained transformer language models (2022)

    2022

  28. [36]

    In: Toutanova, K., Rumshisky, A., Zettlemoyer, L., Hakkani-Tur, D., Beltagy, I., Bethard, S., Cotterell, R., Chakraborty, T., Zhou, Y

    Zhao, T., Lu, X., Lee, K.: SPARTA: Efficient open-domain question answer- ing via sparse transformer matching retrieval. In: Toutanova, K., Rumshisky, A., Zettlemoyer, L., Hakkani-Tur, D., Beltagy, I., Bethard, S., Cotterell, R., Chakraborty, T., Zhou, Y. (eds.) Proceedings of...

    2021

  29. [37]

    In: Al-Onaizan, Y., Bansal, M., Chen, Y.N

    Zhuang, S., Ma, X., Koopman, B., Lin, J., Zuccon, G.: PromptReps: Prompting large languagemodelstogeneratedenseandsparserepresentationsforzero-shotdocument retrieval. In: Al-Onaizan, Y., Bansal, M., Chen, Y.N. (eds.) Proceedings of the 2024 Conference on Empirical Methods in N...

    2024

  30. [38]

    In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

    Zhuang, S., Zuccon, G.: Tilde: Term independent likelihood model for passage re-ranking. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. p. 1483–1492. SIGIR ’21, Association for Computing Machinery, New York,...

    2021

  31. [1874]

    findings-acl.146/

    Association for Computational Linguistics, Dublin, Ireland (May 2022).https: //doi.org/10.18653/v1/2022.findings-acl.146, https://aclanthology.org/2022. findings-acl.146/

    2022 doi

  32. [4391]

    https://doi.org/10.18653/v1/2024.emnlp-main.250, https://aclanthology.org/ 2024.emnlp-main.250/

    Association for Computational Linguistics, Miami, Florida, USA (Nov 2024). https://doi.org/10.18653/v1/2024.emnlp-main.250, https://aclanthology.org/ 2024.emnlp-main.250/

    2024 doi

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.