Pith. sign in

REVIEW 4 major objections 7 minor 33 references

Rational Retrieval Acts: Leveraging Pragmatic Reasoning to Improve Sparse Retrieval

T0 review · 4 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that adapting Rational Speech Acts to sparse retrieval, treating tokens as utterances and documents as meanings, consistently improves retrieval models on out-of-domain data at no extra query-time cost.

desk verdict A genuinely new RSA-for-sparse-retrieval method with a neat sparsity decomposition, but the zero-shot SOTA claim is softer than it looks because alpha is tuned per target collection on LLM-generated synthetic queries. read the letter →

arxiv 2505.03676 v1 pith:PWXSX5V3 submitted 2025-05-06 cs.IR cs.CL

Arthur Satouf , Gabriel Ben Zenou , Benjamin Piwowarski , Habiboulaye Amadou Boubacar , Pablo Piantanida This is my paper · ORCID
classification cs.IRcs.CL
keywords NeuralInformationRetrievalSparseRationalSpeechActsPragmaticReasoningLinguisticPragmaticsforBEIRbenchmarkzero-shot
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that sparse retrieval models represent each document in isolation, ignoring the rest of the collection, and that borrowing a pragmatic-communication model from linguistics fixes that blind spot. It adapts Rational Speech Acts, a framework that explains how a speaker picks words to single out an object among alternatives, into a post-processing layer that reweights token–document associations against the whole collection. Applying this layer to BM25 and four learned sparse models improves nDCG@10 on out-of-domain datasets, and the strongest configuration sets a new state of the art on the BEIR benchmark. Because the reweighting happens offline, the gain costs nothing at query time.

What carries the argument

The load-bearing object is the pragmatic listener L1(d|t), obtained by one RSA speaker–listener iteration from a sparse model's weights. The enabling identity is the factorization of the zero-weight entries: when w_{t,d} = 0, the literal-listener contribution L0(d|t) factors as a token-only factor l0_t times a document-only factor l0_d, and the pragmatic speaker and listener inherit this factored form (equations 5–8). This reduces the RSA update to computing a handful of per-token and per-document vectors instead of a dense |T| x |D| matrix, which is what lets the method scale to entire collections and score queries at retrieval time with inverted indices.

What would settle it

On a dataset with high alpha sensitivity (the paper shows TREC-COVID varies by up to 10 nDCG@10 points with alpha), generate several independent 500-query synthetic sets, tune alpha on each, and evaluate on the real queries. If the best alpha from synthetic sets yields real-query scores no better than baseline SPLADE, or if the tuned alphas disagree wildly, then the alpha-selection step—not RSA itself—is the fragile link.

Watch

Extended reading notes

Core claim

The paper introduces Rational Retrieval Acts (RRA), an adaptation of Rational Speech Acts (RSA) to sparse information retrieval. RSA models communication as a speaker choosing an utterance to identify a meaning, and a listener inferring the meaning from the utterance. RRA maps tokens to utterances and documents to meanings, then computes a literal listener from the sparse model's raw token weights, a pragmatic speaker via an exponentiated softmax controlled by a temperature alpha, and finally a pragmatic listener that becomes the document's representation. The central technical contribution is a sparsity-aware factorization: for every token–document pair with zero weight, the RSA update factors into a token-only term and a document-only term, reducing the storage footprint from |T| x |D| to |T| + |D| plus the sparse nonzeros. This makes running RSA over an entire document collection feasible, and the resulting scores are still evaluated through an inverted index.

Load-bearing premise

The reported gains rely on alpha being chosen per collection from a few hundred synthetic LLM-generated queries, and on that choice transferring to real user queries; if synthetic queries do not reflect real behavior, the improvements could shrink or vanish.

Editorial extensions

If this is right

  • Any sparse retriever, lexical or neural, can be wrapped with RRA as an offline post-processing step, preserving its efficiency profile while improving accuracy.
  • Relative gains are largest for models that do no query-side weighting, so RRA can substitute for missing contextual query term weighting.
  • The mean nDCG@10 of 53.0 on the 13 BEIR datasets gives S-RRA the top position among the compared sparse and dense baselines.
  • The dependence on alpha, tuned on synthetic queries, makes the method applicable to new domains without human relevance judgments.
  • The factorization generalizes to any sparse scoring function whose nonzero entries can be separated from zero entries, so other sparse families could benefit.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper frames RSA as a one-shot correction, but the same factored update can be iterated; the claim that alpha and depth trade off suggests iterating with a smaller alpha could yield further gains at the cost of another offline pass.
  • The observation that f(0) != 0 helps, preferring f(x)=1+x over log(1+x) or identity, suggests the zero weights are informative as 'this token is absent,' a signal most sparse models discard; a learned transformation might exploit this further.
  • The synthetic-query alpha tuning is a template for zero-shot hyperparameter selection in retrieval more broadly: if LLM-generated queries can pick alpha, they might also pick model variants or fusion weights.
  • The approach might transfer to dense retrieval by contrasting a query or document against a set of distractors sampled from the collection, though the exact factorization that makes RRA cheap would not carry over.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper adapts the Rational Speech Acts (RSA) framework to sparse neural information retrieval. It treats tokens as utterances and documents as meanings, defines a literal lexicon from sparse-model weights, and applies one RSA speaker/listener iteration to obtain a ``pragmatic'' document representation. A sparse factorization (Eqs. 5-9) reduces the memory cost from |T| x |D| to |T| + |D| plus the nonzero sparse weights. Experiments on subsets of the BEIR benchmark compare five base models (SPLADE, SPARTA, BM25, UniCOIL, DeepImpact) with their RRA-enhanced versions, report consistent nDCG@10 gains, and compare S-RRA with several strong baselines. The hyperparameter alpha is selected per target dataset using 500 LLAMA3-8B-generated synthetic queries per dataset, and the pre-transformation function f is selected on MS-Marco.

Significance. If the claims hold, the paper offers a conceptually novel and potentially useful way to make sparse retrieval representations collection-sensitive without changing the query-time scoring archetype. The sparse factorization is a genuine practical contribution, and the demonstration of consistent gains across five sparse models is encouraging. The authors also release code and include an interpretable case study. However, the state-of-the-art claim depends on a per-dataset alpha-selection protocol that has not been shown to transfer to real queries, and the evaluation covers only part of BEIR, so the empirical contribution is currently weaker than the abstract and discussion suggest.

major comments (4)
  1. [Section 3.1, Tables 1-2] The zero-shot, out-of-domain claim is not supported as presented because the RSA hyperparameter alpha is tuned separately for each target dataset using 500 synthetic queries generated from that dataset's own documents, and the reported test scores are then obtained on the same dataset with the selected alpha. This makes the comparison asymmetric: the baselines have no per-dataset hyperparameter selection, while S-RRA is given target-collection information. Figure 2 shows that alpha can change nDCG@10 by up to 10 points on TREC-COVID, so the choice is materially load-bearing. Please provide evidence that alpha values selected on synthetic queries correlate with alpha values optimal on real queries, or report results with a fixed alpha (e.g., alpha=1 or alpha chosen on MS-Marco only). Without such evidence, the mean gains in Tables 1 and 2 are not a fair zero-shot comparison.
  2. [Equations (7) and (9)] The printed equations are inconsistent with the definitions above them. In Eq. (7), the first summand is written as exp(alpha L0(d|t)), but Eq. (3) defines S1(t|d) = exp(alpha log L0(d|t)) / Z_d, so the exponent should be alpha log L0(d|t). In Eq. (9), the first sum is written as sum_{d' in D_t} S1(d|t), which has the arguments reversed; it should be S1(t|d'). Also, in the second sum of Eq. (9), s_d^1 should be s_{d'}^1 since the factor depends on the document d'. These typos sit in the core derivation of the method and need to be corrected before the factorization can be checked.
  3. [Section 3.2, Tables 1-2] The text says the method is evaluated on the BEIR benchmark, but Table 1 covers 9 datasets and Table 2 covers 13 datasets, while BEIR has 18 datasets. The reported mean nDCG@10 and the comparison with prior state-of-the-art results are therefore computed on a subset, and the subset differs between the two tables. To support the ``state-of-the-art on BEIR'' claim, the authors should either report all BEIR datasets or explicitly state that the comparison is on a subset and justify the selection. The missing datasets (e.g., BioASQ, Signal-1M, Robust04, TREC-News) could alter both the mean and the ranking of methods.
  4. [Section 4, Contributions] The paper claims that RRA enhances effectiveness ``without compromising efficiency,'' and the contributions state that gains are achieved ``without increasing inference costs.'' However, no efficiency measurements are reported: there are no latency, throughput, index-size, or memory figures. The sparse factorization is a plausible basis for the claim, but the actual retrieval-time cost of Eq. (10) and the offline cost of computing l_t^1 and l_d^1 should be measured and reported before the efficiency claim can be accepted.
minor comments (7)
  1. [Table 1 caption] The caption begins with ``Table 1: Table 1:''; please remove the duplicate label.
  2. [Equations (5)-(9)] The notation l_d^0 is defined as the constant 1 in Eq. (5), but the subsequent factorization would be clearer if l_d^0 were defined explicitly for all d and if s_d^1 in Eq. (6) were written with its full dependence on Z_d^(1).
  3. [Table 1 caption] The caption states that underlined values are significant with a paired t-test (p<0.05), but the manuscript gives no details about the unit of analysis, the number of queries or runs, or which values are underlined. Please provide these details.
  4. [Section 3.1, Pre-transformation function] The selection of f among the six listed mappings is described as ``preliminary experiments on MS-Marco,'' but no scores or protocol are reported. Since f is a component of the method, a short table of these preliminary results in an appendix would improve reproducibility.
  5. [Section 3.1, Figure 2] Figure 2 shows the sensitivity of nDCG@10 to alpha for S-RRA only. Adding similar curves for at least one other base model would strengthen the claim that alpha selection is robust across models.
  6. [Section 3.3, Case study] The sentence about functions with f(0) != 0 says they ``avoid contrasting too much the document weights L1(t|d),'' but the notation should be L1(d|t); please correct this.
  7. [References] Reference [4] for Bunt's computational pragmatics appears to have a URL that points to an unrelated journal article; please provide the correct bibliographic entry.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the RRA derivation is self-contained; per-dataset alpha selection is hyperparameter tuning, not a fitted prediction.

full rationale

The paper's derivation chain is not circular. Equations (2)-(4) apply the standard RSA framework to sparse IR weights, and equations (5)-(9) are purely algebraic sparsity decompositions of those RSA quantities; equation (10) is the resulting dot-product scoring rule. None of these equations presupposes the reported performance gains, and the reported nDCG@10 numbers are not encoded in the derivation by construction. The main potentially questionable step is the per-dataset choice of alpha in Section 3.1, where alpha is selected by maximizing nDCG@10 on synthetic query-document pairs generated by LLAMA3-8B from 500 sampled documents of the target collection. This is a hyperparameter selection on a synthetic proxy, not a fit to the real test queries or to the reported outcome; the results on the actual BEIR queries remain an independent evaluation. It is a legitimate evaluation-fairness concern that baselines do not receive the same per-collection tuning, but that is a correctness/comparison issue, not a circularity issue under the required definition. The choice of the pre-transformation f is made on MS-Marco in preliminary experiments, separate from the BEIR evaluation, so it does not reduce the out-of-domain claims to the benchmark being predicted. Self-citations, notably to SPLADE [11], refer to an external, code-reproduced model and are not load-bearing for the RRA formalism. The acknowledged limitation in Section 4 about re-applying the offline RSA phase when the collection is updated is an operational caveat, not a circular step. Overall, the derivation is self-contained against external benchmarks.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The method adds no new physical or mathematical entities; the main fitted degrees of freedom are the per-dataset alpha and the pre-transformation f. These choices, especially alpha, materially affect the reported gains.

free parameters (2)
  • alpha (RSA pragmatism parameter) = not reported per dataset; selected on synthetic queries
    Chosen per dataset to maximize nDCG@10 on 500 synthetic query-document pairs generated by LLAMA3-8B (Section 3.1). Figure 2 shows up to 10 points nDCG@10 difference with alpha, so the choice materially affects results.
  • pre-transformation function f = f(x) = 1 + x
    Selected on MS-Marco among six candidates (Section 3.1). The requirement f(0) != 0 was also identified empirically.
assumptions (4)
  • domain assumption The RSA listener-speaker derivation applies to token-document associations, treating tokens as utterances and documents as meanings.
    Section 2.1 maps RSA utterances/meanings to tokens/documents; this analogy is the foundation of the method.
  • domain assumption Uniform document prior P(d)=1/|D|.
    Section 3.1 states no prior knowledge is available; the sparsity decomposition in Eqs. (5) to (9) relies on this uniformity.
  • domain assumption A single RSA iteration is sufficient; alpha controls pragmatic depth.
    Section 2.3 cites Frank [12] to justify fixing iterations to 1.
  • ad hoc to paper LLAMA3-8B-generated synthetic queries are representative of real user queries for alpha selection.
    Section 3.1 uses this assumption to tune alpha per dataset; no evidence is provided that synthetic query behavior matches real queries.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Rational Retrieval Acts: Leveraging Pragmatic Reasoning to Improve Sparse Retrieval." pith.science (2026). https://pith.science/paper/PWXSX5V3

@misc{pith2026250503676,
  author       = {Pith},
  title        = {Pith review of: Rational Retrieval Acts: Leveraging Pragmatic Reasoning to Improve Sparse Retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PWXSX5V3}},
  note         = {Machine review of arXiv:2505.03676}
}
read the original abstract

Current sparse neural information retrieval (IR) methods, and to a lesser extent more traditional models such as BM25, do not take into account the document collection and the complex interplay between different term weights when representing a single document. In this paper, we show how the Rational Speech Acts (RSA), a linguistics framework used to minimize the number of features to be communicated when identifying an object in a set, can be adapted to the IR case -- and in particular to the high number of potential features (here, tokens). RSA dynamically modulates token-document interactions by considering the influence of other documents in the dataset, better contrasting document representations. Experiments show that incorporating RSA consistently improves multiple sparse retrieval models and achieves state-of-the-art performance on out-of-domain datasets from the BEIR benchmark. https://github.com/arthur-75/Rational-Retrieval-Acts

Figures

Figures reproduced from arXiv: 2505.03676 by the authors.

Figure 1
Figure 1. Rational Retrieval Acts (RRA) architecture. Docu [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Impact of 𝛼 on S-RRA performance (nDCG@10). Non-optimal 𝛼 values can significantly degrade results. a diverse set of queries. This process produces a synthetic infor￾mation retrieval (IR) dataset consisting of query-document pairs that are considered relevant. This in turns allows us to select the 𝛼 that maximizes nDCG@10 on the synthetic dataset. Note that this dataset is relatively small, and experiments using the… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 17 canonical work pages

  1. [1]

    Reasoning About Pragmatics with Neural Listeners and Speakers

    Jacob Andreas and Dan Klein. Reasoning about pragmatics with neural listeners and speakers, 2016. URL https://arxiv.org/abs/1604.00562

    work page Pith review arXiv 2016

  2. [2]

    Building economic models of human computer interaction

    Leif Azzopardi and Guido Zuccon. Building economic models of human computer interaction. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, CHI EA ’19, page 1–4, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 9781450359719. doi: 10.1145/3290607.3298809. URL https://doi.org/10.1145/3290607.3298809

    arXiv 2019

  3. [3]

    InPars: Unsupervised Dataset Generation for Information Retrieval

    Luiz Bonifacio, Hugo Abonizio, Marzieh Fadaee, and Rodrigo Nogueira. InPars: Unsupervised Dataset Generation for Information Retrieval. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’22, pages 2387–2392. Association for Computing Machinery, 2022. ISBN 978-1-4503-8732-3. doi: 10.11...

    arXiv 2022

  4. [4]

    H. Bunt. Computational pragmatics. Oxford University Press, 2017. URL https: //doi.org/10.4312/slo2.0.2023.1.189-217

    work page doi:10.4312/slo2.0.2023.1.189-217 2017

  5. [5]

    Cheriton

    David R. Cheriton. From doc2query to doctttttquery. In From doc2query to docTTTTTquery , 2019. URL https://api.semanticscholar.org/CorpusID:208612557

    work page 2019

  6. [6]

    Lost in Machine Translation: A Method to Reduce Meaning Loss

    Reuben Cohn-Gordon and Noah Goodman. Lost in machine translation: A method to reduce meaning loss, 2019. URL https://arxiv.org/abs/1902.09514

    work page Pith review arXiv 2019

  7. [7]

    Pragmatically Informative Image Captioning with Character-Level Inference

    Reuben Cohn-Gordon, Noah Goodman, and Christopher Potts. Pragmatically informative image captioning with character-level inference. arXiv preprint arXiv:1804.05417, 2018

    work page Pith review arXiv 2018

  8. [8]

    Wonky worlds: Listeners revise world knowledge when utterances are odd

    J Degen, M H Tessler, and N D Goodman. Wonky worlds: Listeners revise world knowledge when utterances are odd . 2015

    work page 2015

Show all 33 references
  1. [9]

    The rational speech act framework

    Judith Degen. The rational speech act framework. Annu. Rev. Linguist., 9(1): 519–540, January 2023

    2023

  2. [10]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ah- mad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sra- vankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston...

    2025 arXiv

  3. [11]

    Towards Effective and Efficient Sparse Neural Information Retrieval

    Thibault Formal, Carlos Lassance, Benjamin Piwowarski, and Stéphane Clinchant. Towards Effective and Efficient Sparse Neural Information Retrieval. ACM Trans- actions on Information Systems , page 3634912, 2024. ISSN 1046-8188, 1558-2868. doi: 10.1145/3634912. URL https://dl.a...

    2024 doi

  4. [12]

    Rational speech act models of pragmatic reasoning in reference games

    Michael C Frank. Rational speech act models of pragmatic reasoning in reference games. 2016

    2016

  5. [13]

    Frank and Noah D

    Michael C. Frank and Noah D. Goodman. Predicting pragmatic reasoning in language games. Science, 336:998 – 998, 2012. URL https://api.semanticscholar. org/CorpusID:26797288

    2012

  6. [14]

    Pragmatic language interpretation as probabilistic inference

    Noah D Goodman and Michael C Frank. Pragmatic language interpretation as probabilistic inference. Trends Cogn. Sci., 20(11):818–829, November 2016

    2016

  7. [15]

    H. P. Grice. Logic and conversation, 1975. URL https://doi.org/10.1016/j.iswa. 2022.200118

    1975

  8. [16]

    Efficiently teaching an effective dense retriever with balanced topic aware sampling, 2021

    Sebastian Hofstätter, Sheng-Chieh Lin, Jheng-Hong Yang, Jimmy Lin, and Allan Hanbury. Efficiently teaching an effective dense retriever with balanced topic aware sampling, 2021. URL https://arxiv.org/abs/2104.06967

    2021 arXiv

  9. [17]

    Will I Sound Like Me? Im- proving Persona Consistency in Dialogues through Pragmatic Self-Consciousness

    Hyunwoo Kim, Byeongchang Kim, and Gunhee Kim. Will I Sound Like Me? Im- proving Persona Consistency in Dialogues through Pragmatic Self-Consciousness. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 904–916, Online, 2020...

    2020 doi

  10. [18]

    Perspective-taking and Pragmatics for Generating Empathetic Responses Focused on Emotion Causes

    Hyunwoo Kim, Byeongchang Kim, and Gunhee Kim. Perspective-taking and Pragmatics for Generating Empathetic Responses Focused on Emotion Causes. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2227–2240, Online and Punta Cana, Dom...

    2021 doi

  11. [19]

    Dudek, Cheng Li, Mingyang Zhang, and Mike Bendersky

    Weize Kong, Jeffrey M. Dudek, Cheng Li, Mingyang Zhang, and Mike Bendersky. Sparseembed: Learning sparse lexical representations with contextual embed- dings for retrieval. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information...

    2023

  12. [20]

    A few brief notes on deepimpact, coil, and a conceptual framework for information retrieval techniques, 2021

    Jimmy Lin and Xueguang Ma. A few brief notes on deepimpact, coil, and a conceptual framework for information retrieval techniques, 2021. URL https: //arxiv.org/abs/2106.14807

    2021 arXiv

  13. [21]

    Wacky Weights in Learned Sparse Representations and the Revenge of Score-at-a-Time Query Evaluation,

    Joel Mackenzie, Andrew Trotman, and Jimmy Lin. Wacky Weights in Learned Sparse Representations and the Revenge of Score-at-a-Time Query Evaluation,

  14. [22]

    Learning passage impacts for inverted indexes, 2021

    Antonio Mallia, Omar Khattab, Nicola Tonellotto, and Torsten Suel. Learning passage impacts for inverted indexes, 2021. URL https://arxiv.org/abs/2104.12016

    2021 arXiv

  15. [23]

    Hawkins, Noah D

    Will Monroe, Robert X.D. Hawkins, Noah D. Goodman, and Christopher Potts. Colors in Context: A Pragmatic Neural Model for Grounded Language Un- derstanding. Transactions of the Association for Computational Linguistics , 5:325–338, 2017. ISSN 2307-387X. doi: 10.1162/tacl_a_000...

    2017 doi

  16. [24]

    Multi-stage document ranking with bert, 2019

    Rodrigo Nogueira, Wei Yang, Kyunghyun Cho, and Jimmy Lin. Multi-stage document ranking with bert, 2019. URL https://arxiv.org/abs/1910.14424

    2019 arXiv

  17. [25]

    A language modeling approach to information retrieval

    Jay Ponte and W Croft. A language modeling approach to information retrieval. In ACM SIGIR, 1998

    1998

  18. [26]

    The probabilistic relevance framework: Bm25 and beyond

    Stephen Robertson and Hugo Zaragoza. The probabilistic relevance framework: Bm25 and beyond. Found. Trends Inf. Retr., 3(4):333–389, April 2009. ISSN 1554-

    2009

  19. [27]

    Colbertv2: Effective and efficient retrieval via lightweight late interaction, 2022

    Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. Colbertv2: Effective and efficient retrieval via lightweight late interaction, 2022. URL https://arxiv.org/abs/2112.01488

    2022 arXiv

  20. [28]

    Pragmatically Informa- tive Text Generation, 2019

    Sheng Shen, Daniel Fried, Jacob Andreas, and Dan Klein. Pragmatically Informa- tive Text Generation, 2019

    2019

  21. [29]

    Beir: A heterogenous benchmark for zero-shot evaluation of informa- tion retrieval models, 2021

    Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. Beir: A heterogenous benchmark for zero-shot evaluation of informa- tion retrieval models, 2021. URL https://arxiv.org/abs/2104.08663

    2021 arXiv

  22. [30]

    Minilm: Deep self-attention distillation for task-agnostic compression of pre- trained transformers, 2020

    Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. Minilm: Deep self-attention distillation for task-agnostic compression of pre- trained transformers, 2020. URL https://arxiv.org/abs/2002.10957

    2020 arXiv

  23. [31]

    Sparta: Efficient open-domain question answering via sparse transformer matching retrieval, 2020

    Tiancheng Zhao, Xiaopeng Lu, and Kyusong Lee. Sparta: Efficient open-domain question answering via sparse transformer matching retrieval, 2020. URL https: //arxiv.org/abs/2009.13013. Received 23 January 2025; revised 23 January 2025; accepted 4 April 2025

    2020 arXiv

  24. [669]

    URL https://doi.org/10.1561/1500000019

    doi: 10.1561/1500000019. URL https://doi.org/10.1561/1500000019

    doi

  25. [2021]

    URL http://arxiv.org/abs/2110.11540

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.