Pith. sign in

REVIEW 4 major objections 3 minor 56 references

PLAID-PRF claims that pseudo-relevance feedback for the quantised late-interaction retriever PLAID can be done cheaply by treating the index's centroid codes as semantic terms, yielding up to 4.3% higher nDCG@10 and 7.3% higher MRR@10 at 10

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 14:47 UTC pith:FC2P3Q75

load-bearing objection A genuinely new and efficient PRF method for PLAID whose headline DL'19 gain is in-sample; the mechanism is plausible and worth serious review, but the paper needs an honest evaluation split and a toned-down consistency claim. the 4 major comments →

arxiv 2607.18626 v1 pith:FC2P3Q75 submitted 2026-07-21 cs.IR cs.CL

PLAID-PRF: Pseudo-Relevance Feedback with Centroid-like Tokens in PLAID

Xiao Wang , Sean MacAvaney , Craig Macdonald This is my paper
classification cs.IR cs.CL MSC 68P20
keywords Pseudo-Relevance FeedbackLate InteractionPLAIDColBERTCentroid QuantisationQuery ExpansionDense RetrievalTF-IDF
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces PLAID-PRF, a pseudo-relevance feedback method for the multi-vector late-interaction retriever PLAID. Instead of clustering feedback token vectors at query time, as prior dense PRF methods do, it reuses PLAID's indexing-time codebook: each feedback token's centroid ID is counted like a term, TF-IDF weights are computed from centroid occurrence statistics, and a small diverse set of reconstructed token vectors is appended to the query for a second PLAID pass. The claim is that this training-free, codebook-aware reformulation improves top-ranked effectiveness over plain PLAID—up to 4.3% nDCG@10 on TREC DL 2019 and 7.3% MRR@10 on DL-HARD—while keeping query latency near 102 ms, far below prior PRF approaches. A sympathetic reader would care because it suggests PRF can be made practical for production late-interaction search without retraining or expensive clustering.

Core claim

On the paper's terms, the central discovery is that the centroid codes stored in a ColBERTv2/PLAID index can stand in for terms in classical pseudo-relevance feedback. Tokens assigned to the same centroid live near one another in vector space, so counting centroid occurrences in the top retrieved passages and weighting them by collection-wide rarity yields a discriminative signal that aligns with the index's own structure. The query is expanded with reconstructed token vectors—centroid plus quantised residual, not bare centroids—selected greedily by usefulness and diversity, then PLAID is rerun. Experiments on MS MARCO and four BEIR sets show consistent gains, most reliably in early precisio

What carries the argument

The load-bearing object is the PLAID/ColBERTv2 codebook: a global set of centroid vectors learned at indexing time, with each document token stored as a centroid ID plus a quantised residual. PLAID-PRF treats centroid IDs as semantic terms, computes tf(centroid) over the pseudo-relevant set and a precomputed document frequency df(centroid) over the whole index to get TF-IDF weights, propagates those weights to individual reconstructed token vectors, and uses MMR (maximal marginal relevance) to pick a compact, diverse expansion set. This machinery replaces the two expensive components of prior dense PRF—query-time KMeans clustering of feedback vectors and extra transformer inference—with coun

Load-bearing premise

The method assumes that tokens sharing a centroid code are semantically similar enough that centroid co-occurrence in the top documents is a faithful proxy for which expansion tokens will help—and that those top three passages are relevant enough to trust.

What would settle it

Take a query set where the same centroid ID is frequently assigned to tokens of different senses (e.g., polysemous words in a coarse codebook), run PLAID-PRF with varying codebook sizes, and compare against full-precision ColBERT-PRF using the same feedback passages. If PLAID-PRF's gains vanish or invert when the codebook is coarsened while full-precision PRF stays strong, the centroid-proxy assumption is the culprit. Alternatively, injecting known-irrelevant documents into the top-3 feedback set and measuring nDCG drop would test the relevance assumption.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • PLAID-PRF makes PRF practical for quantised late-interaction retrieval: the feedback stage is a small fraction of total latency, so production systems can afford a second retrieval pass.
  • Code-level term weighting beats WordPiece-level weighting in these experiments, implying that index-native semantic units can capture morphology and synonyms better than surface tokens.
  • Expanding with reconstructed token vectors (centroid plus residual) yields higher MRR than expanding with raw centroids, showing that residual detail matters for moving the first relevant result to the top.
  • Gains transfer out-of-domain: on BEIR sets, Recall@1000 improves broadly and some collections also gain nDCG@10, indicating the method does not only help in-domain data.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • One extension the authors leave implicit: centroid-ID statistics could serve as a reusable semantic lexicon, so the same df(centroid) table built for PLAID-PRF could speed up other feedback or reranking methods over the same index.
  • A testable corollary is that the method's margin should narrow as the codebook becomes coarser; varying the number of centroids and measuring PLAID-PRF's gain over PLAID would directly probe whether centroid granularity is the limiting factor.
  • Because the feedback set is only three passages, PLAID-PRF inherits the standard PRF fragility: on collections where first-pass top-3 are often irrelevant, the expansion vectors could add noise. A selective application rule based on retrieval score variance is a natural next step the authors mention as adaptive expansion.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. PLAID-PRF proposes a pseudo-relevance feedback method for PLAID/ColBERTv2 that avoids query-time clustering by using the index's precomputed centroid codes: it counts centroid-code occurrences in a small top-retrieved set, weights them with TF-IDF, selects a diverse set of reconstructed token vectors via MMR, and reruns PLAID with the expanded query. The paper reports strong in-domain gains (up to +4.3% nDCG@10 on TREC DL'19 and +7.3% MRR@10 on DL-HARD), low latency (102 ms/query), and several ablations (clustering source, counting unit, expansion unit, weighting schemes, selection policies) plus out-of-domain BEIR experiments.

Significance. If the main claims hold, this is a useful contribution: it is a training-free PRF method specifically designed for quantised late-interaction retrieval, reuses PLAID's existing index structures, and avoids the expensive per-query KMeans of ColBERT-PRF. The paper ships code and a virtual appendix, reports significance tests, and includes a detailed latency breakdown. The efficiency advantage over prior dense PRF methods is convincing. My reservation is that the headline effectiveness numbers are measured on the same DL'19 set used for hyperparameter tuning, and several 'consistent improvement' statements are contradicted by the paper's own tables; these issues need to be fixed before the claims can be accepted.

major comments (4)
  1. [§5.1, §6.4, Abstract] The headline +4.3% nDCG@10 on DL'19 is an in-sample estimate. Section 5.1 states that PLAID-PRF is tuned on TREC DL'19 via grid search over f_p, f_e, beta, lambda, and the same DL'19 set is then used to report the +4.3% gain (0.7383→0.7700). Selecting the best grid point on the test set inflates both the effect size and the paired t-test. Moreover, Table 3 shows no significant nDCG@10 gain on DL'20 (0.7435 vs 0.7410, no dagger), so the claim of 'consistent improvement' is not supported by an unbiased estimate. The same issue appears in §6.7, where beta is varied per BEIR dataset (Figure 6) and the conclusions are drawn from the best beta per set. Please provide a held-out evaluation (e.g., tune on DL'19, report DL'20/DL-HARD as held-out; or use a separate validation set), and adjust the abstract and conclusions accordingly.
  2. [§6.1, §6.2, Table 2] The text says PLAID-PRF yields 'consistently higher early-precision effectiveness' than LocalCluster-PRF and 'consistently stronger retrieval effectiveness' than TokenCount-PRF. Table 2 contradicts both statements on DL'20 nDCG@10: LocalCluster-PRF scores 0.7552, TokenCount-PRF scores 0.7573, and PLAID-PRF scores 0.7435. Thus the RQ1/RQ2 conclusions are overstated. Please qualify the claims and discuss the DL'20 behavior, where the comparisons invert.
  3. [§4.2] The TF-IDF formula is written as w_tfidf(c)=tf(c)·(N+1)/(df+1), without a logarithm. This is not the standard IDF factor and can produce very large unscaled weights. If the logarithm was accidentally omitted, fix the equation; if the formula is intentional, explain why this particular variant of IDF was chosen and how it interacts with the min-max normalisation in §4.3.
  4. [§6.4, Table 3] The text reports that on DL'20 PLAID-PRF improves ColBERT-PRF by +6.3% nDCG@10 (0.6993→0.7435) and +5.5% MAP (0.4962→0.5236), but Table 3 lists ColBERT-PRF DL'20 nDCG@10 as 0.6937 and MAP as 0.4904. These numbers need to be reconciled; as written, the reader cannot tell which set of values is correct.
minor comments (3)
  1. [§6.7, Figure 6] Figure 6 shows that on DBPedia and NFCorpus nDCG@10 does not improve for any beta, while Recall@1000 improves. The conclusion that the method 'can improve both top-ranked precision and deeper retrieval coverage' is too strong for those datasets; please phrase the out-of-domain conclusion in terms of recall improvement and the datasets where precision also improves.
  2. [§4.3, Eq. (4)] The min-max normalisation of G is mentioned only in a footnote; please state it in the main text or in the equation, since it is important for the additive combination in Eq. (5).
  3. [Table 2] Several 'highest' values in the table are not bold-faced according to the caption (e.g., TokenCount-PRF MRR@10 0.9310 on DL'19, CentroidExp-PRF nDCG@10 0.7581 on DL'20). Please check the formatting.

Circularity Check

1 steps flagged

Headline DL'19 gain is in-sample from grid search; method itself is not structurally circular.

specific steps
  1. fitted input called prediction [Section 5.1 (Setup, Parameters) & Section 6.4 (RQ4) & Abstract]
    "Based on the parameter study in Section 6.5, we tune PLAID-PRF on TREC DL'19 via a grid search over fp, fe, β, λ and fix the parameters for all other test sets. We select the configuration based on nDCG@10 and set fp = 3 feedback passages, fe = 14 expansion vectors, β = 0.7 and λ = 0.3."

    The headline +4.3% nDCG@10 on DL'19 is measured on the same 43 queries used for the grid search that selected fp, fe, β, λ by nDCG@10. Hence the reported gain is the best in-sample grid point, so it is partly an artifact of the selection criterion rather than an out-of-sample prediction. The held-out DL'20 nDCG@10 gain is not significant (0.7435 vs 0.7410, no †), and BEIR nDCG@10 shows no benefit on two of four datasets, so the advertised DL'19 effect size is not independently confirmed.

full rationale

The PLAID-PRF derivation itself is not circular: it computes centroid-level TF-IDF weights from the feedback set, maps them to reconstructed token vectors, selects diverse expansions via MMR, appends them to the query (Eq. 5), and reruns PLAID. No equation or design choice is defined in terms of the reported effectiveness numbers, and ablations show that random weighting or random selection can hurt, meaning the pipeline is not guaranteed to improve by construction. The main circularity-adjacent issue is evaluation rather than derivation: the abstract and Section 6.4 advertise the DL'19 nDCG@10 improvement, but the same DL'19 queries were used to grid-search and select all four hyperparameters by nDCG@10. That makes the +4.3% headline an in-sample best-point estimate. However, the paper also reports fixed-parameter results on DL-HARD (e.g., MRR +7.3%) and BEIR recall gains, so the central claim retains some independent content. No load-bearing self-citation or uniqueness-theorem chain is present; citations to the authors' prior ColBERT-PRF/CWPRF work are contextual baselines rather than premises of the method. Overall score reflects one methodological in-sample effect, not a structurally circular derivation.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The paper's central claim rests on four hyperparameters tuned on DL'19 (f_p=3, f_e=14, β=0.7, λ=0.3) plus per-dataset β for BEIR, and on the domain assumption that centroid codes are meaningful semantic units in the PLAID codebook. It introduces no new entities; it reuses existing centroids and reconstructed vectors. The method depends on standard PRF assumptions (top passages are relevant) and on the fidelity of 2-bit residual reconstruction for query expansion.

free parameters (5)
  • f_p = 3
    Number of pseudo-relevant documents used for feedback; tuned via grid search on TREC DL'19 (Section 5.1).
  • f_e = 14
    Number of expansion vectors appended to the query; tuned via grid search on TREC DL'19 (Section 5.1).
  • beta (in-domain) = 0.7
    Scaling of expansion vectors when appended to the query; grid-searched on DL'19 (Section 5.1).
  • beta (BEIR) = 0.1 to 1.0, varies per dataset
    Tuned separately on each BEIR dataset; reported in Figure 6 (Section 6.7).
  • lambda = 0.3
    MMR diversity-usefulness trade-off; tuned via grid search on TREC DL'19 (Section 5.1).
axioms (5)
  • domain assumption Tokens sharing a centroid code are semantically similar enough for centroid-level TF-IDF to be a useful expansion signal
    Section 4.2, Stage 2: 'We observe that tokens sharing a centroid tend to be projected in a similar region of the vector space.' This is the core premise of using centroid counts as a proxy for token usefulness.
  • domain assumption The top-f_p documents from a short first pass are pseudo-relevant and provide useful expansion evidence
    Standard PRF assumption; Section 4.1 Stage 1; used to gather feedback tokens.
  • domain assumption PLAID's approximate reconstructed vectors (centroid + quantised residual) are accurate enough for effective query expansion
    PLAID-PRF uses reconstructed vectors as expansion units (Sections 4.3 and 4.5); if the 2-bit residual quantisation is too lossy, expansion quality degrades. RQ3 tests this by comparing with raw centroids.
  • domain assumption TF-IDF and related term-weighting statistics apply meaningfully to centroid-code occurrences
    Section 4.2; assumes the distribution of centroid codes over documents behaves like term distributions; the paper notes df(·) is approximately power-law over a limited range.
  • standard math Late-interaction scoring (Eq. 1) is additive in query vectors, so appending weighted expansion vectors integrates without re-encoding
    MaxSim summation in Eq. 1 allows extra query vectors to contribute directly; used in Stage 4 to form Q_new.

pith-pipeline@v1.3.0-alltime-deepseek · 22381 in / 13166 out tokens · 119405 ms · 2026-08-01T14:47:20.561703+00:00 · methodology

0 comments
read the original abstract

Multi-vector dense retrieval models, such as ColBERT, achieve strong retrieval effectiveness by modelling fine-grained token-level interactions between queries and documents. Methods such as PLAID use centroid-based quantisation of each token's vector to reduce the index size and speed up retrieval while maintaining strong effectiveness. In this work, we introduce PLAID-PRF, a method that performs Pseudo-Relevance Feedback (PRF) over PLAID to reformulate ColBERT's query vectors based on the top-retrieved results. In contrast with prior methods that perform PRF on multi-vector retrieval models, PLAID-PRF keeps computational costs low by leveraging the internal PLAID centroid vectors, treating them similarly to tokens in traditional PRF methods. The method selects a small and diverse set of high-utility expansion vectors and appends them to the original query, rerunning PLAID to refine both candidate generation and final scoring. Extensive experiments on the standard in-domain MSMARCO and four out-of-domain BEIR benchmarks show that PLAID-PRF consistently improves retrieval effectiveness over various baselines. In particular, PLAID-PRF improves over PLAID by up to 4.3% nDCG@10 and 7.3% MRR@10, while introducing substantially less computation overhead than prior PRF methods. The results demonstrate that our proposed centroid-aware PRF method offers an effective and lightweight mechanism to improve the quality of top-ranked retrieved results. Overall, this work enables effective and efficient feedback-aware late-interaction retrieval without expensive query-time document-token clustering.

Figures

Figures reproduced from arXiv: 2607.18626 by Craig Macdonald, Sean MacAvaney, Xiao Wang.

Figure 1
Figure 1. Figure 1: Effectiveness vs. efficiency trade-off on TREC Deep [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Indexing in ColBERTv1 and ColBERTv2/PLAID. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: PLAID-PRF augments PLAID’s late-interaction retrieval [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Latency Breakdown of PLAID PRF variants. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Impact of 𝑓𝑝 , 𝑓𝑒 , 𝛽 and 𝜆 on the Retrieval Effectiveness (nDCG@10 on TREC 2019 queries) of PLAID-PRF [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Effect of 𝛽 of PLAID-PRF across selected BEIR datasets, measured by nDCG@10 & Recall@1000. 6.7 RQ7: Out-of-domain Performance Experimental Setup. We now examine PLAID-PRF on the out-of￾domain retrieval BEIR benchmarks. In particular, as these datasets widely differ, we vary 𝛽, which controls the contribution of expan￾sion vectors, separately on each of the selected BEIR datasets, i.e., DBPedia, NFCorpus, T… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

56 extracted references · 4 canonical work pages

  1. [1]

    Giambattista Amati, Claudio Carpineto, and Giovanni Romano. 2004. Query Difficulty, Robustness, and Selective Application of Query Expansion. InAdvances in Information Retrieval, 26th European Conference on IR Research, ECIR 2004, Sunderland, UK, April 5-7, 2004, Proceedings (Lecture Notes in Computer Science, Vol. 2997). Springer, 127–137. doi:10.1007/97...

    work page doi:10.1007/978-3-540-24752-4 2004

  2. [2]

    Gianni Amati and C. J. van Rijsbergen. 2002. Probabilistic models of information retrieval based on measuring the divergence from randomness.ACM Trans. Inf. Syst.20, 4 (2002), 357–389. doi:10.1145/582415.582416

    arXiv 2002

  3. [3]

    Sanjeev Arora, Yingyu Liang, and Tengyu Ma. 2017. A Simple but Tough-to-Beat Baseline for Sentence Embeddings. In5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net. https://openreview.net/forum?id=SyK00v5xx

    2017

  4. [5]

    Antoine Chaffin. 2025. GTE-ModernColBERT. https://huggingface.co/lightonai/ GTE-ModernColBERT-v1

    2025

  5. [6]

    Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu

  6. [7]

    Charles L. A. Clarke, Maheedhar Kolla, Gordon V. Cormack, Olga Vechtomova, Azin Ashkan, Stefan Büttcher, and Ian MacKinnon. 2008. Novelty and diversity in information retrieval evaluation. InProceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2008, Singapore, July 20-24, 2008. ACM, ...

    arXiv 2008

  7. [8]

    Nick Craswell, Bhaskar Mitra, Emine Yilmaz, and Daniel Campos. 2020. Overview of the TREC 2020 Deep Learning Track. InProceedings of the Twenty-Ninth Text REtrieval Conference, TREC 2020, Virtual Event [Gaithersburg, Maryland, USA], November 16-20, 2020 (NIST Special Publication, Vol. 1266). National Institute of Standards and Technology (NIST). https://t...

    2020

  8. [9]

    Voorhees

    Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M. Voorhees. 2020. Overview of the TREC 2019 deep learning track.CoRR abs/2003.07820 (2020). arXiv:2003.07820 https://arxiv.org/abs/2003.07820 PLAID-PRF SIGIR ’26, July 20–24, 2026, Melbourne, VIC, Australia

    Pith/arXiv arXiv 2020

  9. [10]

    Bruce Croft and David J

    W. Bruce Croft and David J. Harper. 1979. Using Probabilistic Models of Document Retrieval without Relevance Information.J. Documentation35, 4 (1979), 285–295. doi:10.1108/EB026683

    work page doi:10.1108/eb026683 1979

  10. [11]

    Lukas Galke, Ahmed Saleh, and Ansgar Scherp. 2017. Word Embeddings for Practical Information Retrieval. In47. Jahrestagung der Gesellschaft für Informatik, Digitale Kulturen, INFORMATIK 2017, Chemnitz, Germany, September 25-29, 2017 (LNI), Maximilian Eibl and Martin Gaedke (Eds.). GI, 2155–2167. doi:10.18420/ IN2017_215

    2017

  11. [12]

    Luyu Gao, Zhuyun Dai, Tongfei Chen, Zhen Fan, Benjamin Van Durme, and Jamie Callan. 2021. Complement Lexical Retrieval Model with Semantic Residual Embeddings. InAdvances in Information Retrieval - 43rd European Conference on IR Research, ECIR 2021, Virtual Event, March 28 - April 1, 2021, Proceedings, Part I (Lecture Notes in Computer Science, Vol. 12656...

    doi:10.1007/978- 2021

  12. [13]

    Luyu Gao, Xueguang Ma, Jimmy Lin, and Jamie Callan. 2023. Precise Zero-Shot Dense Retrieval without Relevance Labels. InProceedings of the 61st Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023. Association for Computational Linguistics, 1762–1777. doi:10.18653/V1/2023.ACL-LONG.99

    doi:10.18653/v1/2023.acl-long.99 2023

  13. [14]

    Fatima Haouari, Marwa Essam, and Tamer Elsayed. 2020. bigIR at TREC 2020: Simple but Deep Retrieval of Passages and Documents. InProceedings of the Twenty-Ninth Text REtrieval Conference, TREC 2020, Virtual Event [Gaithersburg, Maryland, USA], November 16-20, 2020 (NIST Special Publication, Vol. 1266). Na- tional Institute of Standards and Technology (NIS...

    2020

  14. [15]

    Bruce Croft, Fernando Diaz, Leah S

    Nasreen Abdul Jaleel, James Allan, W. Bruce Croft, Fernando Diaz, Leah S. Larkey, Xiaoyan Li, Mark D. Smucker, and Courtney Wade. 2004. UMass at TREC 2004: Novelty and HARD. InProceedings of the Thirteenth Text REtrieval Conference, TREC 2004, Gaithersburg, Maryland, USA, November 16-19, 2004 (NIST Special Publication, Vol. 500-261). National Institute of...

    2004

  15. [16]

    Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2021. Billion-Scale Similarity Search with GPUs.IEEE Trans. Big Data7, 3 (2021), 535–547. doi:10.1109/TBDATA. 2019.2921572

    arXiv 2021

  16. [17]

    Ehsan Kamalloo, Nandan Thakur, Carlos Lassance, Xueguang Ma, Jheng-Hong Yang, and Jimmy Lin. 2023. Resources for Brewing BEIR: Reproducible Ref- erence Models and an Official Leaderboard.CoRRabs/2306.07471 (2023). arXiv:2306.07471 doi:10.48550/ARXIV.2306.07471

    Pith/arXiv arXiv doi:10.48550/arxiv.2306.07471 2023

  17. [18]

    Omar Khattab and Matei Zaharia. 2020. ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT. InProceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020, Virtual Event, China, July 25-30, 2020. ACM, 39–48. doi:10. 1145/3397271.3401075

    arXiv 2020

  18. [19]

    Hrishikesh Kulkarni, Sean MacAvaney, Nazli Goharian, and Ophir Frieder. 2023. Lexically-Accelerated Dense Retrieval. InProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2023, Taipei, Taiwan, July 23-27, 2023. ACM, 152–162. doi:10.1145/3539618.3591715

    arXiv 2023

  19. [20]

    Bruce Croft

    Victor Lavrenko and W. Bruce Croft. 2017. Relevance-Based Language Models. SIGIR Forum51, 2 (2017), 260–267. doi:10.1145/3130348.3130376

    arXiv 2017

  20. [21]

    Jinhyuk Lee, Zhuyun Dai, Sai Meher Karthik Duddu, Tao Lei, Iftekhar Naim, Ming-Wei Chang, and Vincent Zhao. 2023. Rethinking the Role of Token Retrieval in Multi-Vector Retrieval. InAdvances in Neural In- formation Processing Systems 36: Annual Conference on Neural Informa- tion Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, De- cember 10 - ...

    2023

  21. [22]

    Hang Li, Ahmed Mourad, Shengyao Zhuang, Bevan Koopman, and Guido Zuc- con. 2021. Pseudo Relevance Feedback with Deep Language Models and Dense Retrievers: Successes and Pitfalls.CoRRabs/2108.11044 (2021). arXiv:2108.11044 https://arxiv.org/abs/2108.11044

    Pith/arXiv arXiv 2021

  22. [23]

    Hang Li, Xiao Wang, Bevan Koopman, and Guido Zuccon. 2025. Pseudo Relevance Feedback is Enough to Close the Gap Between Small and Large Dense Retrieval Models.arXiv preprint arXiv:2503.14887(2025)

    Pith/arXiv arXiv 2025

  23. [24]

    Hang Li, Shengyao Zhuang, Bevan Koopman, and Guido Zuccon. 2025. LLM- VPRF: Large Language Model Based Vector Pseudo Relevance Feedback.CoRR abs/2504.01448 (2025). arXiv:2504.01448 doi:10.48550/ARXIV.2504.01448

    Pith/arXiv arXiv doi:10.48550/arxiv.2504.01448 2025

  24. [25]

    Xueguang Ma, Liang Wang, Nan Yang, Furu Wei, and Jimmy Lin. 2024. Fine- Tuning LLaMA for Multi-Stage Text Retrieval. InProceedings of the 47th Inter- national ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2024, Washington DC, USA, July 14-18, 2024. ACM, 2421–2425. doi:10.1145/3626772.3657951

    arXiv 2024

  25. [26]

    Sean MacAvaney, Antonio Mallia, and Nicola Tonellotto. 2025. Efficient Constant- Space Multi-vector Retrieval. InAdvances in Information Retrieval - 47th European Conference on Information Retrieval, ECIR 2025, Lucca, Italy, April 6-10, 2025, Proceedings, Part III (Lecture Notes in Computer Science, Vol. 15574). Springer, 237–245. doi:10.1007/978-3-031-88...

    doi:10.1007/978-3-031-88714-7 2025

  26. [27]

    Sean MacAvaney and Nicola Tonellotto. 2024. A Reproducibility Study of PLAID. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2024, Washington DC, USA, July 14-18,

    2024

  27. [28]

    Sean MacAvaney, Nicola Tonellotto, and Craig Macdonald. 2022. Adaptive Re- Ranking with a Corpus Graph. InProceedings of the 31st ACM International Conference on Information and Knowledge Management, Atlanta, GA, USA, October 17-21, 2022. ACM, 1491–1500. doi:10.1145/3511808.3557231

    arXiv 2022

  28. [29]

    Craig Macdonald, Nicola Tonellotto, Sean MacAvaney, and Iadh Ounis. 2021. PyTerrier: Declarative Experimentation in Python from BM25 to Dense Retrieval. InCIKM ’21: The 30th ACM International Conference on Information and Knowledge Management, Virtual Event, Queensland, Australia, November 1 - 5, 2021. ACM, 4526–4533. doi:10.1145/3459637.3482013

    arXiv 2021

  29. [30]

    doi:10.1145/3626772.3657856

    ACM, 1411–1419. doi:10.1145/3626772.3657856

    arXiv

  30. [31]

    Iain Mackie, Jeffrey Dalton, and Andrew Yates. 2021. How Deep is your Learning: the DL-HARD Annotated Deep Learning Dataset. InSIGIR ’21: The 44th International ACM SIGIR Conference on Research and Development in In- formation Retrieval, Virtual Event, Canada, July 11-15, 2021. ACM, 2335–2341. doi:10.1145/3404835.3463262

    arXiv 2021

  31. [32]

    Malkov and Dmitry A

    Yury A. Malkov and Dmitry A. Yashunin. 2016. Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs.CoRR abs/1603.09320 (2016). arXiv:1603.09320 http://arxiv.org/abs/1603.09320

    Pith/arXiv arXiv 2016

  32. [33]

    Iain Mackie, Shubham Chatterjee, and Jeffrey Dalton. 2023. Generative Relevance Feedback with Large Language Models. InProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2023, Taipei, Taiwan, July 23-27, 2023. ACM, 2026–2031. doi:10.1145/3539618. 3591992

    doi:10.1145/3539618 2023

  33. [34]

    Shahrzad Naseri, Jeff Dalton, Andrew Yates, and James Allan. 2021. CEQE: Contextualized Embeddings for Query Expansion. InAdvances in Information Retrieval - 43rd European Conference on IR Research, ECIR 2021, Virtual Event, March 28 - April 1, 2021, Proceedings, Part I (Lecture Notes in Computer Science, Vol. 12656). Springer, 467–482. doi:10.1007/978-3-...

    doi:10.1007/978-3-030-72113-8 2021

  34. [35]

    Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A Human Generated MAchine Reading COmprehension Dataset. InProceedings of the Workshop on Cognitive Computation: Integrating neural and symbolic approaches 2016 co-located with the 30th Annual Conference on Neural Information Processing Systems...

    2016

  35. [36]

    2009.An introduction to information retrieval

    Christopher D Manning. 2009.An introduction to information retrieval. Cambridge university press

    2009

  36. [37]

    Mandeep Rathee, Venktesh V, Sean MacAvaney, and Avishek Anand. 2025. Break- ing the Lens of the Telescope: Online Relevance Estimation over Large Retrieval Sets. InProceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2025, Padua, Italy, July 13-18,

    2025

  37. [38]

    Joseph Rocchio. 1971. Relevance feedback in information retrieval.The Smart Retrieval System-experiments in Automatic Document Processing(1971), 313–323

    1971

  38. [39]

    Mandeep Rathee, Sean MacAvaney, and Avishek Anand. 2025. Quam: Adaptive Retrieval through Query Affinity Modelling. InProceedings of the Eighteenth ACM International Conference on Web Search and Data Mining, WSDM 2025, Hannover, Germany, March 10-14, 2025. ACM, 954–962. doi:10.1145/3701551.3703584

    arXiv 2025

  39. [40]

    Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. 2022. ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction. InProceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022, Seattle, W A, United States...

    doi:10.18653/v1/2022.naacl-main.272 2022

  40. [41]

    Rodrygo L. T. Santos, Craig Macdonald, and Iadh Ounis. 2010. Exploiting query reformulations for web search result diversification. InProceedings of the 19th International Conference on World Wide Web, WWW 2010, Raleigh, North Carolina, USA, April 26-30, 2010. ACM, 881–890. doi:10.1145/1772690.1772780

    arXiv 2010

  41. [42]

    Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models.CoRRabs/2104.08663 (2021). arXiv:2104.08663 https://arxiv.org/abs/2104.08663

    Pith/arXiv arXiv 2021

  42. [43]

    Keshav Santhanam, Omar Khattab, Christopher Potts, and Matei Zaharia. 2022. PLAID: An Efficient Engine for Late Interaction Retrieval. InProceedings of the 31st ACM International Conference on Information and Knowledge Management, Atlanta, GA, USA, October 17-21, 2022. ACM, 1747–1756. doi:10.1145/3511808. 3557325

    doi:10.1145/3511808 2022

  43. [44]

    Xiao Wang, Sean MacAvaney, Craig Macdonald, and Iadh Ounis. 2023. Generative Query Reformulation for Effective Adhoc Retrieval. InProceedings of Workshop on GenIR

    2023

  44. [45]

    Xiao Wang, Craig Macdonald, and Iadh Ounis. 2022. Improving zero-shot retrieval using dense external expansion.Inf. Process. Manag.59, 5 (2022), 103026. doi:10. SIGIR ’26, July 20–24, 2026, Melbourne, VIC, Australia Xiao Wang, Sean MacAvaney, and Craig Macdonald 1016/J.IPM.2022.103026

    arXiv 2022

  45. [46]

    Xiao Wang, Craig MacDonald, Nicola Tonellotto, and Iadh Ounis. 2023. ColBERT- PRF: Semantic Pseudo-Relevance Feedback for Dense Passage and Document Retrieval.ACM Trans. Web17, 1 (2023), 3:1–3:39. doi:10.1145/3572405

    work page doi:10.1145/3572405 2023

  46. [47]

    Xiao Wang, Sean MacAvaney, Craig Macdonald, and Iadh Ounis. 2023. Effective Contrastive Weighting for Dense Query Expansion. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023. Association for Computational Linguistics, 12688–12704. doi:10.18653/V1/2...

    work page doi:10.18653/v1/2023.acl-long.710 2023

  47. [48]

    Bennett, Junaid Ahmed, and Arnold Overwijk

    Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N. Bennett, Junaid Ahmed, and Arnold Overwijk. 2021. Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval. In9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net. https://openreview.net/foru...

    2021

  48. [49]

    HongChien Yu, Chenyan Xiong, and Jamie Callan. 2021. Improving Query Rep- resentations for Dense Retrieval with Pseudo Relevance Feedback. InCIKM ’21: The 30th ACM International Conference on Information and Knowledge Manage- ment, Virtual Event, Queensland, Australia, November 1 - 5, 2021. ACM, 3592–3596. doi:10.1145/3459637.3482124

    arXiv 2021

  49. [50]

    Bruce Croft

    Hamed Zamani and W. Bruce Croft. 2016. Embedding-based Query Language Models. InProceedings of the 2016 ACM on International Conference on the Theory of Information Retrieval, ICTIR 2016, Newark, DE, USA, September 12- 6, 2016. ACM, 147–156. doi:10.1145/2970398.2970405

    arXiv 2016

  50. [51]

    Xiao Wang, Craig Macdonald, Nicola Tonellotto, and Iadh Ounis. 2023. Repro- ducibility, Replicability, and Insights into Dense Multi-Representation Retrieval Models: from ColBERT to Col★. InProceedings of SIGIR

    2023

  51. [52]

    Shengyao Zhuang, Xueguang Ma, Bevan Koopman, Jimmy Lin, and Guido Zuccon

  52. [55]

    Zhi Zheng, Kai Hui, Ben He, Xianpei Han, Le Sun, and Andrew Yates. 2020. BERT- QE: Contextualized Query Expansion for Document Re-ranking. InFindings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020 (Findings of ACL, Vol. EMNLP 2020). Association for Computational Linguistics, 4718–4728. doi:10.18653/V1/2020....

    doi:10.18653/v1/2020.findings-emnlp.424 2020

  53. [57]

    InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 12-16, 2024

    PromptReps: Prompting Large Language Models to Generate Dense and Sparse Representations for Zero-Shot Document Retrieval. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 12-16, 2024. Association for Computational Linguistics, 4375–4391. doi:10.18653/V1/2024.EMNLP-MAIN.250

    doi:10.18653/v1/2024.emnlp-main.250 2024

  54. [2024]

    InFindings of the Asso- ciation for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024

    M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation. InFindings of the Asso- ciation for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024. Association for Computational Linguistics, 2318–

    2024

  55. [2025]

    doi:10.1145/3726302.3729910

    ACM, 2287–2297. doi:10.1145/3726302.3729910

    arXiv

  56. [2335]

    doi:10.18653/V1/2024.FINDINGS-ACL.137

    doi:10.18653/v1/2024.findings-acl.137 2024