On the Robustness of Generative Information Retrieval Models

Changjiang Zhou; Jiafeng Guo; Maarten de Rijke; Ruqing Zhang; Xueqi Cheng; Yu-An Liu

REVIEW 4 major objections 5 minor 57 references

On the Robustness of Generative Information Retrieval Models

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Generative information retrieval models, despite beating dense and sparse baselines under identical distributions, suffer substantial performance drops on out-of-distribution queries, tasks, and corpora.

desk verdict First systematic OOD evaluation of generative IR models, but the corpus-expansion analysis has a baseline mismatch and a numeric error that need fixing before the paper's broader claims. read the letter →

arxiv 2412.18768 v1 pith:I5ZC5YLP submitted 2024-12-25 cs.IR

Yu-An Liu , Ruqing Zhang , Jiafeng Guo , Changjiang Zhou , Maarten de Rijke , Xueqi Cheng This is my paper · ORCID

classification cs.IR

keywords generativeinformationretrievalout-of-distributionrobustnessqueryvariationscorpusexpansiondenseKILTbenchmarkDROOD

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that generative information retrieval models, which retrieve documents by directly generating their identifiers, generalize poorly to out-of-distribution (OOD) data even though they perform well when test data resembles training data. The authors define OOD robustness through four shifts: query variations, unseen query types, unseen tasks, and corpus expansion. They evaluate two representative generative models (CorpusBrain and BART) against a dense retriever (DPR) and a sparse retriever (BM25) on the KILT benchmark. Across the four scenarios, generative models show large performance drops, especially on misspellings and word-order changes, indicating that leaderboard scores measured under identical distributions overstate the reliability of these models.

What carries the argument

The analysis is carried by four formal OOD settings defined in Section 4, each expressed as an expectation over a shifted distribution: query variations $\mathcal{G}(q_t)$, unseen query types from $G'_Q$, unseen tasks from $\tilde{G}_T$, and corpus expansion $D_n$. The quantitative carrier is DROOD, the percentage change from IID R-precision $P_{IID}$ to OOD R-precision $P_{OOD}$, defined as $\text{DROOD} = (P_{OOD} - P_{IID}) / P_{IID}$, with R-precision being the ratio of relevant pages retrieved in the top-$R$ results. For corpus expansion, the experimental machinery is a random 60/40 split of Wikipedia pages into an initial corpus and an incremental corpus, with KILT queries filtered so that all relevant documents lie exclusively in one side of the split.

What would settle it

Run a controlled corpus-expansion experiment in which new documents about a sampled set of topics are added while keeping the query types and difficulty of the test queries identical to the training queries; if previously trained generative models retrieve the new documents without a significant DROOD drop, the paper's conclusion that generative IR models perform poorly on corpus expansion would not transfer to that setting.

Watch

Extended reading notes

Core claim

The paper establishes that, overall, generative IR models are not robust to distribution shifts: on query variations such as misspelling and reordering, R-precision drops by up to 74.0% for BART and 68.0% for CorpusBrain on the FEVER dataset, and on unseen query types and unseen tasks both generative models degrade substantially, often more than BM25. In the corpus-expansion scenario, generative models can lose more than half their effectiveness, with CorpusBrain falling 58.5% on the dialogue task when the corpus grows, although they still keep a higher absolute R-precision than the baselines. The authors attribute this residual advantage to generative decoding over a prefix tree, which lets beam search probe document identifiers near those seen during training. The overall conclusion is that the strong in-distribution performance of generative retrievers does not transfer to out-of-distribution conditions and that this weakness deserves dedicated study.

Load-bearing premise

The constructed KILT scenarios in Section 5.1 faithfully realize the Section 4 OOD definitions, particularly the random 60/40 Wikipedia split for corpus expansion, which may entangle document growth with query difficulty.

Editorial extensions

If this is right

If the paper is right, IID leaderboard performance on KILT cannot be used as evidence that a generative retriever will behave reliably when deployed on new query phrasings, new question types, new tasks, or an expanding corpus.
Generative IR models should be evaluated with OOD-aware metrics such as DROOD alongside standard R-precision; a model that scores high only in-distribution should not be preferred over a more stable one.
The four-scenario taxonomy gives a reusable protocol for auditing future generative IR models, so that robustness claims can be compared across models on the same set of shifts.
Because generative models are particularly fragile to misspelling and word-order changes, improving surface-form robustness of the query-to-docid mapping is a concrete next step suggested directly by the findings.
The observation that generative models retain absolute advantages on corpus expansion, attributed to prefix-tree beam search, implies that the decoding structure itself should be examined as a robustness mechanism rather than treated as a fixed component.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension the authors do not pursue is to use naturally occurring query reformulations, typos, and new web queries rather than the four synthetic perturbation types; such data would test whether the measured fragility is an artifact of the generators.
The corpus-expansion result raises the possibility that the random 60/40 Wikipedia split confounds document growth with query difficulty, since filtering queries by whether their evidence pages fall in the initial corpus can change the difficulty distribution of the test set itself.
One concrete testable prediction from the prefix-tree explanation is that generative models should degrade gracefully when new documents are added in topics close to existing ones, but sharply on unrelated topics; ranking OOD difficulty by semantic distance to the training corpus would check this.
If robustness becomes a standard reporting requirement, model selection in generative IR is likely to shift from maximizing in-distribution accuracy toward optimizing a trade-off between accuracy and DROOD, similar to the effectiveness-robustness trade-off already studied in neural ranking models.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: the paper is the first systematic robustness evaluation of generative IR models, and the two strongest experiments are worth publishing. But the corpus-expansion leg has a baseline mismatch and a concrete arithmetic error, so the overall claim should be read with that caveat.

The new thing here is the four-way OOD taxonomy—query variations, unseen query types, unseen tasks, corpus expansion—applied to generative retrievers rather than to dense retrieval or rankers, which is what earlier robustness studies covered. The experimental design is straightforward: BART and CorpusBrain versus BM25 and DPR on KILT. The query-variation results are internally consistent and clear: generative models collapse under misspellings and word-order changes, and CorpusBrain is more resilient than BART. The unseen-task results are also coherent: all neural models lose substantially when transferred across KILT tasks, with generative models handling slot-filling better than entity linking. These two legs alone justify the paper's abstract claim that generative IR robustness is in need of improvement.

The soft spots are all in the corpus-expansion scenario. Eq. (6) defines PIID as the model trained on the training set corresponding to the test set. For Q1, that means a model trained on the full corpus, not the D0-filtered set. The paper instead uses the D0-trained model on Q0, so the reported DROOD mixes query difficulty, corpus size, and missing supervised signal. The construction of Q1 makes it worse: queries with relevant documents only partly in D1 are included, so a model can score via D0 documents. Then there is a plain numeric error in Table 7: BM25 on fact checking drops from 46.5 to 37.8, which is −18.7%, not −6.0%. The authors appear to have used the QA baseline by mistake. The text also overstates the generative advantage in this scenario: it says both BART and CorpusBrain beat DPR, but Table 7 shows CorpusBrain's average drop is worse than DPR's.

The other limitations are minor for a first study: two generative models and one benchmark, no error bars, and a random 60/40 split as a proxy for corpus expansion. The authors should rerun the DROOD calculations with the proper PIID, fix the arithmetic, and soften the narrative. Then the paper would be a solid empirical contribution.

I'd send it to peer review with a request for major revision on the corpus-expansion section. The query-variation and unseen-task results are worth refereeing, and the taxonomy is useful for future work.

Referee Report

4 major / 5 minor

Summary. The paper studies the out-of-distribution robustness of generative information retrieval models. It proposes a four-perspective taxonomy of OOD scenarios—query variations, unseen query types, unseen tasks, and corpus expansion—and evaluates BM25, DPR, BART, and CorpusBrain on KILT-based constructed datasets. The main empirical claim is that generative IR models, despite strong IID performance, degrade substantially under OOD shifts, especially query variations and unseen tasks, and that their OOD robustness is in need of improvement. The paper also analyzes differences between generative and dense models and attributes some patterns to model architecture and pretraining. The authors release code.

Significance. If the findings hold, the paper makes a useful contribution by showing that IID leaderboard scores on KILT do not reliably indicate robustness for generative retrievers. The four-scenario taxonomy is a reasonable organizing framework, and the comparison across sparse, dense, and generative families is informative. The paper is also honest about its limited model and dataset coverage in the conclusion. However, the empirical rigor is currently uneven: one of the four scenarios (corpus expansion) does not realize the paper's own definition of the OOD drop, and several reported DROOD values contain arithmetic inconsistencies. These issues must be fixed before the headline claim can be fully accepted.

major comments (4)

[Section 5.1 and Eq. (6), Table 7] The corpus-expansion experiment does not instantiate Eq. (6). Eq. (6) defines PIID as the performance of the model trained on the training set corresponding to the test set. For the OOD set Q1, the corresponding training set is the training set built on the full corpus, not the D0-filtered training set; the paper instead compares the D0-trained model's R-precision on Q0 with its R-precision on Q1. This conflates corpus growth with query-set difficulty. Moreover, Q1 is defined as the remaining dataset after excluding queries whose relevant articles are exclusively in D0, so queries with some relevant articles in D0 and some in D1 are included; a model can score substantial R-precision by retrieving only the D0 pages in the provenance, making the metric partially insensitive to new-document retrieval. The reported values such as BART +10.1 on Dialogue and CorpusBrain -58.5 therefore do not measure the ability to retrieve newly added documents. The table should be recomputed with a proper IID baseline trained on the full-corpus training set, or the experiment should be redesigned to control query difficulty.
[Table 7] There is a straightforward arithmetic error in the BM25 row for the FC column: R-precision drops from 46.5 to 37.8, which gives DROOD = (37.8 - 46.5)/46.5 = -18.7%, not -6.0%. This error, together with the baseline issue in the previous comment, means Table 7 cannot currently support the paper's corpus-expansion conclusions.
[Table 4] Several DROOD entries for BM25 are inconsistent with the reported R-precision values. For FEV Order, 22.3 vs. 50.1 should be -55.5%, not 0; for NQ Naturality, 25.4 vs. 25.8 should be approximately -1.6%, not -0.02; and for NQ Order, 31.0 vs. 25.8 should be +20.2%, not 0. These discrepancies suggest a unit or computation error in the query-variation table and should be corrected before the results are used to support the paper's claims.
[Section 5.3 and Tables 5-7] No error bars, number of runs, or significance tests are reported for the unseen query type, unseen task, and corpus expansion experiments. The text uses 'significant' claims (e.g., Section 6.3) and compares small differences, such as CorpusBrain -7.3 vs. BART -12.5 on NQ in Table 5, without any indication of variance. The authors should report multiple seeds or at least statistical tests so readers can judge whether the observed patterns are reliable.

minor comments (5)

[Section 4.3, Eq. (4)] The text describes this scenario as 'unseen tasks,' but the sentence immediately after Eq. (4) says 'unseen corpus'; please correct the mismatch.
[Section 4.4, Eq. (5)] The expectation notation in Eq. (5) uses (qt, D, yt) ~ GT, but the scenario concerns corpus-expansion samples (qn_t, Dn, yn_t); the notation should be made consistent.
[Table 1] The row label 'zxRE' should be 'zsRE' to match the dataset name Zero Shot RE.
[Section 4.2] The phrase 'every though they focus on the same task' contains a typo; it should read 'even though they focus on the same task.'
[Table 4] The BM25 Naturality DROOD values appear to be written as fractions (-0.02) rather than percentages, unlike the rest of the table; please unify the units.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the empirical OOD comparisons are computed from external benchmark measurements rather than from fitted or self-referential quantities.

full rationale

The paper's central claim—that generative IR models degrade under OOD shifts—is an empirical result, not a derivation. The DROOD metric in Eq. (6) is a descriptive normalization of measured R-precision values; POOD and PIID are read directly from benchmark runs and are not fitted parameters. The four OOD scenarios are operationalized from KILT data with standard models (BM25, DPR, BART, CorpusBrain), and the conclusions are compared against these external baselines, so the outcome is not encoded in the input. The self-citations that appear (e.g., [3] for CorpusBrain and [2] for GeRE) describe artifacts or prior analyses used as tools; none supplies a premise that is equivalent to the paper's conclusion. Two non-circular concerns are worth noting separately: the corpus-expansion scenario in Sec. 5.1 uses the D0-trained model on Q0 as PIID rather than a model trained on the full corpus, so Table 7's DROOD values may conflate query-difficulty differences with corpus growth, and the BM25 FC entry (-6.0 for 46.5 to 37.8) is arithmetically inconsistent (-18.7). These are validity and reporting issues, not circularity: correcting them would change the strength of one scenario's evidence, but would not make the claim true by construction. No equation in the paper reduces a prediction to its own input, and no load-bearing step is justified solely by an author-overlapping citation.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The study is empirical, so no mathematical derivation is being supported. The load-bearing assumptions are about benchmark validity, model representativeness, scenario operationalization, and metric validity. The corpus-expansion DROOD metric is the least secure part of the ledger.

free parameters (2)

Corpus expansion split fraction = 60% initial / 40% incremental
The D0 and D1 split in Section 5.1 is chosen by hand; all corpus-expansion DROOD results depend on it, and no sensitivity analysis is reported.
Query variation perturbation settings = not reported
The misspelling, order, and paraphrasing generators have implicit strengths and thresholds that are not specified, and the magnitude of the observed query-variation drops depends on these settings.

assumptions (4)

domain assumption KILT dev labels are a valid proxy for retrieval relevance.
All experiments evaluate on dev splits rather than test because of leaderboard submission limits (Section 5.1); label noise or split leakage would transfer to the robustness numbers.
domain assumption BART and CorpusBrain are representative of the generative IR family.
Only these two models are used (Section 5.2), and the authors note in Section 7 that more generative models are future work, so the central claim is contingent on representativeness.
domain assumption The constructed OOD scenarios in Section 5.1 match the Section 4 formal definitions.
Synthetic perturbations, cross-dataset query types, cross-task transfer, and the 60/40 corpus split are proxies for real distribution shifts; if the proxies are weak, the conclusions do not generalize.
domain assumption DROOD is a valid measure of robustness.
Equation (6) collapses robustness into a single drop rate, and its application to corpus expansion uses a different baseline than the formal definition, which weakens this axiom.

how reviews work

0 comments

Cite this review

Pith. "Pith review of On the Robustness of Generative Information Retrieval Models." pith.science (2026). https://pith.science/paper/I5ZC5YLP

@misc{pith2026241218768,
  author       = {Pith},
  title        = {Pith review of: On the Robustness of Generative Information Retrieval Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I5ZC5YLP}},
  note         = {Machine review of arXiv:2412.18768}
}

read the original abstract

Generative information retrieval methods retrieve documents by directly generating their identifiers. Much effort has been devoted to developing effective generative IR models. Less attention has been paid to the robustness of these models. It is critical to assess the out-of-distribution (OOD) generalization of generative IR models, i.e., how would such models generalize to new distributions? To answer this question, we focus on OOD scenarios from four perspectives in retrieval problems: (i)query variations; (ii)unseen query types; (iii)unseen tasks; and (iv)corpus expansion. Based on this taxonomy, we conduct empirical studies to analyze the OOD robustness of representative generative IR models against dense retrieval models. Our empirical results indicate that the OOD robustness of generative IR models is in need of improvement. By inspecting the OOD robustness of generative IR models we aim to contribute to the development of more reliable IR models. The code is available at \url{https://github.com/Davion-Liu/GR_OOD}.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 50 canonical work pages

[1]

Advances in Neural Information Processing Systems 35, 31668–31683 (2022)

Bevilacqua, M., Ottaviano, G., Lewis, P., Yih, S., Riede l, S., Petroni, F.: Autoregressive search engines: Generating substrings as d ocument identi- ﬁers. Advances in Neural Information Processing Systems 35, 31668–31683 (2022)

work page 2022
[2]

In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Informatio n Retrieval, pp

Chen, J., Zhang, R., Guo, J., Fan, Y., Cheng, X.: Gere: Gen erative evidence retrieval for fact veriﬁcation. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Informatio n Retrieval, pp. 2184–2189 (2022)

work page 2022
[3]

In: Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pp

Chen, J., Zhang, R., Guo, J., Liu, Y., Fan, Y., Cheng, X.: C orpusbrain: Pre- train a generative retrieval model for knowledge-intensiv e language tasks. In: Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pp. 191–200 (2022)

work page 2022
[4]

, Cheng, X.: A uniﬁed generative retriever for knowledge-intensive lan guage tasks via prompt learning

Chen, J., Zhang, R., Guo, J., de Rijke, M., Liu, Y., Fan, Y. , Cheng, X.: A uniﬁed generative retriever for knowledge-intensive lan guage tasks via prompt learning. In: SIGIR, pp. 1448–1457 (2023)

work page 2023
[5]

In: Proceedings of the 40th Inter- national ACM SIGIR Conference on Research and Development i n Infor- mation Retrieval, pp

Chen, R.C., Gallagher, L., Blanco, R., Culpepper, J.S.: Eﬃcient cost-aware cascade ranking in multi-stage retrieval. In: Proceedings of the 40th Inter- national ACM SIGIR Conference on Research and Development i n Infor- mation Retrieval, pp. 445–454 (2017)

work page 2017
[6]

In: Proceedings of the Thirty- First Interna- tional Joint Conference on Artiﬁcial Intelligence, IJCAI, pp

Chen, X., Luo, J., He, B., Sun, L., Sun, Y.: Towards robust dense retrieval via local ranking alignment. In: Proceedings of the Thirty- First Interna- tional Joint Conference on Artiﬁcial Intelligence, IJCAI, pp. 1980–1986 (2022)

work page 2022
[7]

In: International Conference on Learning Represe ntations (2020)

De Cao, N., Izacard, G., Riedel, S., Petroni, F.: Autoreg ressive entity re- trieval. In: International Conference on Learning Represe ntations (2020)

work page 2020
[8]

In: Proceedings of NAACL-HLT, pp

Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: Pr e-training of deep bidirectional transformers for language understandi ng. In: Proceedings of NAACL-HLT, pp. 4171–4186 (2019)

work page 2019

Show all 57 references

[9]

I n: International Conference on Learning Representations (2018)

Dinan, E., Roller, S., Shuster, K., Fan, A., Auli, M., Wes ton, J.: Wizard of wikipedia: Knowledge-powered conversational agents. I n: International Conference on Learning Representations (2018)

2018
[10]

In: LREC 2018 (2018)

Elsahar, H., Vougiouklis, P., Remaci, A., Gravier, C., Hare, J., Laforest, F., Simperl, E.: T-rex: A large scale alignment of natural la nguage with knowledge base triples. In: LREC 2018 (2018)

2018
[11]

In: Proceedings of the 57th An nual Meeting of the Association for Computational Linguistics, pp

Fan, A., Jernite, Y., Perez, E., Grangier, D., Weston, J ., Auli, M.: Eli5: Long form question answering. In: Proceedings of the 57th An nual Meeting of the Association for Computational Linguistics, pp. 3558 –3567 (2019)

2019
[12]

In: Proceedings of the 2021 Conference on Empirica l Methods in Natural Language Processing (2021) On the Robustness of Generative Information Retrieval Mode ls 15

Gao, L., Callan, J.: Condenser: a pre-training archite cture for dense re- trieval. In: Proceedings of the 2021 Conference on Empirica l Methods in Natural Language Processing (2021) On the Robustness of Generative Information Retrieval Mode ls 15

2021
[13]

ACM T ransactions on Information Systems 40(4), 1–42 (2022)

Guo, J., Cai, Y., Fan, Y., Sun, F., Zhang, R., Cheng, X.: S emantic models for the ﬁrst-stage retrieval: A comprehensive review. ACM T ransactions on Information Systems 40(4), 1–42 (2022)

2022
[14]

Semantic Web 9(4), 459–479 (2018)

Guo, Z., Barbosa, D.: Robust named entity disambiguati on with random walks. Semantic Web 9(4), 459–479 (2018)

2018
[15]

arXiv preprint a rXiv:1610.02136 (2016)

Hendrycks, D., Gimpel, K.: A baseline for detecting mis classiﬁed and out-of- distribution examples in neural networks. arXiv preprint a rXiv:1610.02136 (2016)

2016 arXiv
[16]

In: Pro- ceedings of the 58th Annual Meeting of the Association for Co mputational Linguistics, pp

Hendrycks, D., Liu, X., Wallace, E., Dziedzic, A., Kris hnan, R., Song, D.: Pretrained transformers improve out-of-distribution rob ustness. In: Pro- ceedings of the 58th Annual Meeting of the Association for Co mputational Linguistics, pp. 2744–2751 (2020)

2020
[17]

In: Proceedings of the 2011 conference on e mpirical methods in natural language processing, pp

Hoﬀart, J., Yosef, M.A., Bordino, I., Fürstenau, H., Pi nkal, M., Spaniol, M., Taneva, B., Thater, S., Weikum, G.: Robust disambiguati on of named entities in text. In: Proceedings of the 2011 conference on e mpirical methods in natural language processing, pp. 782–792 (2011)

2011
[18]

In: ACL, pp

Joshi, M., Choi, E., Weld, D., Zettlemoyer, L.: TriviaQ A: A large scale dis- tantly supervised challenge dataset for reading comprehen sion. In: ACL, pp. 1601–1611, Association for Computational Linguistics, Va ncouver, Canada (Jul 2017)

2017
[19]

In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp

Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edun ov, S., Chen, D., tau Yih, W.: Dense passage retrieval for open-domain questi on answering. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 6769–6781, Association ...

2020
[20]

Proceeding s of the 43rd In- ternational ACM SIGIR Conference on Research and Developme nt in In- formation Retrieval pp

Khattab, O., Zaharia, M.A.: ColBERT: Eﬃcient and eﬀect ive passage search via contextualized late interaction over BERT. Proceeding s of the 43rd In- ternational ACM SIGIR Conference on Research and Developme nt in In- formation Retrieval pp. 39–48 (2020)

2020
[21]

Transactions of the Association for Computational Linguistics 7, 453–466 (2019)

Kwiatkowski, T., Palomaki, J., Redﬁeld, O., Collins, M ., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., Toutan ova, K., Jones, L., Kelcey, M., Chang, M.W., Dai, A.M., Uszkoreit, J., Le, Q., Pe trov, S.: Nat- ural questions: A benchmark for question...

2019
[22]

In: Proceedings of the 21st Conf erence on Com- putational Natural Language Learning (CoNLL 2017), pp

Levy, O., Seo, M., Choi, E., Zettlemoyer, L.: Zero-shot relation extraction via reading comprehension. In: Proceedings of the 21st Conf erence on Com- putational Natural Language Learning (CoNLL 2017), pp. 333 –342 (2017)

2017
[23]

In: Proceedings of the 58th Annual Meeting of the Associatio n for Compu- tational Linguistics, pp

Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Moham ed, A., Levy, O., Stoyanov, V., Zettlemoyer, L.: Bart: Denoising sequence-t o-sequence pre- training for natural language generation, translation, an d comprehension. In: Proceedings of the 58th Annual Meeting of the Asso...

2020
[24]

arXiv preprint arXiv:2009.10270 (2020) 16 Y

Liang, D., Xu, P., Shakeri, S., dos Santos, C.N., Nallap ati, R., Huang, Z., Xiang, B.: Embedding-based zero-shot retrieval through qu ery generation. arXiv preprint arXiv:2009.10270 (2020) 16 Y. Liu et al

2020 arXiv
[25]

In: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp

Liu, Y.A., Zhang, R., Guo, J., de Rijke, M.: Robust infor mation retrieval. In: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 3009–3012 (2 024)
[26]

In: Proceedings of the 18th ACM International Conference on Web Search and Data Mining (2025)

Liu, Y.A., Zhang, R., Guo, J., de Rijke, M.: Robust infor mation retrieval. In: Proceedings of the 18th ACM International Conference on Web Search and Data Mining (2025)

2025
[27]

In: CIKM, p

Liu, Y.A., Zhang, R., Guo, J., de Rijke, M., Chen, W., Fan , Y., Cheng, X.: Black-box adversarial attacks against dense retrieval mod els: A multi-view contrastive learning method. In: CIKM, p. 1647–1656 (2023)

2023
[28]

In: SIGIR, p

Liu, Y.A., Zhang, R., Guo, J., de Rijke, M., Chen, W., Fan , Y., Cheng, X.: Topic-oriented adversarial attacks against black-box neu ral ranking models. In: SIGIR, p. 1700–1709 (2023)

2023
[29]

In: Proceedings of the AAAI Conference on Ar tiﬁcial In- telligence (2025)

Liu, Y.A., Zhang, R., Guo, J., de Rijke, M., Cheng, X.: At tack-in-the-chain: Bootstrapping large language models for attacks against bl ack-box neural ranking models. In: Proceedings of the AAAI Conference on Ar tiﬁcial In- telligence (2025)

2025
[30]

In: SIGIR (2024)

Liu, Y.A., Zhang, R., Guo, J., de Rijke, M., Fan, Y., Chen g, X.: Multi- granular adversarial attacks against black-box neural ran king models. In: SIGIR (2024)

2024
[31]

arXiv preprint arXiv:2407.06992 (2024)

Liu, Y.A., Zhang, R., Guo, J., de Rijke, M., Fan, Y., Chen g, X.: Robust neu- ral information retrieval: An adversarial and out-of-dist ribution perspective. arXiv preprint arXiv:2407.06992 (2024)

2024 arXiv
[32]

In: AAAI , vol

Liu, Y.A., Zhang, R., Zhang, M., Chen, W., de Rijke, M., G uo, J., Cheng, X.: Perturbation-invariant adversarial training for neur al ranking models: Improving the eﬀectiveness-robustness trade-oﬀ. In: AAAI , vol. 38 (2024)

2024
[33]

In: Proceedings of the 31st ACM International Conference on Information & Kn owledge Management, pp

Ma, X., Zhang, R., Guo, J., Fan, Y., Cheng, X.: A contrast ive pre-training approach to discriminative autoencoder for dense retrieva l. In: Proceedings of the 31st ACM International Conference on Information & Kn owledge Management, pp. 4314–4318 (2022)

2022
[34]

In: Proceedi ngs of the 29th annual international ACM SIGIR conference on Research and d evelopment in information retrieval, pp

Matveeva, I., Burges, C., Burkard, T., Laucius, A., Won g, L.: High ac- curacy retrieval with multiple nested ranker. In: Proceedi ngs of the 29th annual international ACM SIGIR conference on Research and d evelopment in information retrieval, pp. 437–444 (2006)

2006
[35]

ACM SIGIR Forum 55(1), 1–27 (2021)

Metzler, D., Tay, Y., Bahri, D., Najork, M.: Rethinking search: making domain experts out of dilettantes. ACM SIGIR Forum 55(1), 1–27 (2021)

2021
[36]

In: Proceedings of the 2 016 Conference of the North American Chapter of the Association for Computati onal Linguis- tics: Human Language Technologies, pp

Mrkšić, N., Séaghdha, D.Ó., Thomson, B., Gasic, M., Roj as-Barahona, L.M., Su, P.H., Vandyke, D., Wen, T.H., Young, S.: Counter-ﬁ tting word vectors to linguistic constraints. In: Proceedings of the 2 016 Conference of the North American Chapter of the Association for Computat...

2016
[37]

arXiv preprint arXiv:2306.11397 (2023)

Nguyen, T., Yates, A.: Generative retrieval as dense re trieval. arXiv preprint arXiv:2306.11397 (2023)

2023 arXiv
[38]

In: Advances in Information Re- trieval: 44th European Conference on IR Research, ECIR 2022 , Stavanger, Norway, April 10–14, 2022, Proceedings, Part I, pp

Penha, G., Câmara, A., Hauﬀ, C.: Evaluating the robustn ess of retrieval pipelines with query variation generators. In: Advances in Information Re- trieval: 44th European Conference on IR Research, ECIR 2022 , Stavanger, Norway, April 10–14, 2022, Proceedings, Part I, pp. 397–...

2022
[39]

, De Cao, N., Thorne, J., Jernite, Y., Karpukhin, V., Maillard, J., Plachouras, V ., Rocktäschel, T., Riedel, S.: KILT: A benchmark for knowledge intensive la nguage tasks

Petroni, F., Piktus, A., Fan, A., Lewis, P., Yazdani, M. , De Cao, N., Thorne, J., Jernite, Y., Karpukhin, V., Maillard, J., Plachouras, V ., Rocktäschel, T., Riedel, S.: KILT: A benchmark for knowledge intensive la nguage tasks. In: Proceedings of the 2021 Conference of the N...

2021
[40]

In: SIGIR, pp

Ponte, J., Croft, W.B.: A language modeling approach to information re- trieval. In: SIGIR, pp. 275–281 (1998)

1998
[41]

In: Pr oceedings of the 17th Annual International ACM SIGIR Conference on Research and Devel- opment in Information Retrieval, p

Robertson, S., Walker, S.: Some simple eﬀective approx imations to the 2- poisson model for probabilistic weighted retrieval. In: Pr oceedings of the 17th Annual International ACM SIGIR Conference on Research and Devel- opment in Information Retrieval, p. 232–241 (1994)

1994
[42]

Foundations and Trends in Information Retr ieval 3(4), 333–389 (2009)

Robertson, S., Zaragoza, H., et al.: The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Information Retr ieval 3(4), 333–389 (2009)

2009
[43]

Communications of the ACM 18(11), 613–620 (1975)

Salton, G., Wong, A., Yang, C.S.: A vector space model fo r automatic in- dexing. Communications of the ACM 18(11), 613–620 (1975)

1975
[44]

In: Proceedings of the 45th Inter- national ACM SIGIR Conference on Research and Development i n Infor- mation Retrieval, p

Sidiropoulos, G., Kanoulas, E.: Analysing the robustn ess of dual encoders for dense retrieval against misspellings. In: Proceedings of the 45th Inter- national ACM SIGIR Conference on Research and Development i n Infor- mation Retrieval, p. 2132–2136 (2022)

2022
[45]

arXiv preprint arXiv:2202.06991 (2022)

Tay, Y., Tran, V.Q., Dehghani, M., Ni, J., Bahri, D., Meh ta, H., Qin, Z., Hui, K., Zhao, Z., Gupta, J., et al.: Transformer memory as a d iﬀerentiable search index. arXiv preprint arXiv:2202.06991 (2022)

2022 arXiv
[46]

In: Thirty-ﬁfth Conference on Neural Information P rocessing Sys- tems Datasets and Benchmarks Track (2021)

Thakur, N., Reimers, N., Rücklé, A., Srivastava, A., Gu revych, I.: BEIR: A heterogeneous benchmark for zero-shot evaluation of infor mation retrieval models. In: Thirty-ﬁfth Conference on Neural Information P rocessing Sys- tems Datasets and Benchmarks Track (2021)

2021
[47]

In: Proc eedings of the 2018 Conference of the North American Chapter of the Associa tion for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp

Thorne, J., Vlachos, A., Christodoulopoulos, C., Mitt al, A.: Fever: A large- scale dataset for fact extraction and veriﬁcation. In: Proc eedings of the 2018 Conference of the North American Chapter of the Associa tion for Computational Linguistics: Human Language Technologies...

2018
[48]

arXiv preprint arXiv:2206.02743 (2022)

Wang, Y., Hou, Y., Wang, H., Miao, Z., Wu, S., Sun, H., Che n, Q., Xia, Y., Chi, C., Zhao, G., et al.: A neural corpus indexer for documen t retrieval. arXiv preprint arXiv:2206.02743 (2022)

2022 arXiv
[49]

Wu, C., Zhang, R., Guo, J., Fan, Y., Cheng, X.: Are neural ranking models robust? ACM Transactions on Information Systems 41(2), 1–36 (2022)

2022
[50]

In: SIGIR 2024: 47th international ACM SIGIR Conference on Research and Dev elopment in Information Retrieval, pp

Wu, S., Wei, W., Zhang, M., Chen, Z., Ma, J., Ren, Z., de Ri jke, M., Ren, P.: Generative retrieval as multi-vector dense retrieval. In: SIGIR 2024: 47th international ACM SIGIR Conference on Research and Dev elopment in Information Retrieval, pp. 1828–1838, ACM (July 2024)

2024
[51]

arXiv preprint arXiv:2007.00808 (2021) 18 Y

Xiong, L., Xiong, C., Li, Y., Tang, K.F., Liu, J., Bennet t, P., Ahmed, J., Overwijk, A.: Approximate nearest neighbor negative contr astive learning for dense text retrieval. arXiv preprint arXiv:2007.00808 (2021) 18 Y. Liu et al

2021 arXiv
[52]

In: EMNLP, pp

Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salak hutdinov, R., Man- ning, C.D.: HotpotQA: A dataset for diverse, explainable mu lti-hop ques- tion answering. In: EMNLP, pp. 2369–2380, Association for C omputational Linguistics, Brussels, Belgium (Oct-Nov 2018)

2018
[53]

arXiv preprint arXiv:2210.15 212 (2022)

Yu, Y., Xiong, C., Sun, S., Zhang, C., Overwijk, A.: COCO -DR: Combating distribution shifts in zero-shot dense retrieval with cont rastive and distri- butionally robust learning. arXiv preprint arXiv:2210.15 212 (2022)

2022
[54]

Proceedings of the 44th Inter- national ACM SIGIR Conference on Research and Development i n Infor- mation Retrieval pp

Zhan, J., Mao, J., Liu, Y., Guo, J., Zhang, M., Ma, S.: Opt imizing dense retrieval model training with hard negatives. Proceedings of the 44th Inter- national ACM SIGIR Conference on Research and Development i n Infor- mation Retrieval pp. 1503–1512 (2021)

2021
[55]

International Journal of Machine Learnin g and Cybernetics 1, 43–52 (2010)

Zhang, Y., Jin, R., Zhou, Z.H.: Understanding bag-of-w ords model: A statis- tical framework. International Journal of Machine Learnin g and Cybernetics 1, 43–52 (2010)

2010
[56]

ACM Transactions on I nformation Systems 42(4), 1–60 (2024)

Zhao, W.X., Liu, J., Ren, R., Wen, J.R.: Dense text retri eval based on pretrained language models: A survey. ACM Transactions on I nformation Systems 42(4), 1–60 (2024)

2024
[57]

In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp

Zhuang, S., Zuccon, G.: Dealing with typos for BERT-bas ed passage re- trieval and ranking. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 2836–2842 (202 1)

2021

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Advances in Neural Information Processing Systems 35, 31668–31683 (2022)

Bevilacqua, M., Ottaviano, G., Lewis, P., Yih, S., Riede l, S., Petroni, F.: Autoregressive search engines: Generating substrings as d ocument identi- ﬁers. Advances in Neural Information Processing Systems 35, 31668–31683 (2022)

work page 2022

[2] [2]

In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Informatio n Retrieval, pp

Chen, J., Zhang, R., Guo, J., Fan, Y., Cheng, X.: Gere: Gen erative evidence retrieval for fact veriﬁcation. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Informatio n Retrieval, pp. 2184–2189 (2022)

work page 2022

[3] [3]

In: Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pp

Chen, J., Zhang, R., Guo, J., Liu, Y., Fan, Y., Cheng, X.: C orpusbrain: Pre- train a generative retrieval model for knowledge-intensiv e language tasks. In: Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pp. 191–200 (2022)

work page 2022

[4] [4]

, Cheng, X.: A uniﬁed generative retriever for knowledge-intensive lan guage tasks via prompt learning

Chen, J., Zhang, R., Guo, J., de Rijke, M., Liu, Y., Fan, Y. , Cheng, X.: A uniﬁed generative retriever for knowledge-intensive lan guage tasks via prompt learning. In: SIGIR, pp. 1448–1457 (2023)

work page 2023

[5] [5]

In: Proceedings of the 40th Inter- national ACM SIGIR Conference on Research and Development i n Infor- mation Retrieval, pp

Chen, R.C., Gallagher, L., Blanco, R., Culpepper, J.S.: Eﬃcient cost-aware cascade ranking in multi-stage retrieval. In: Proceedings of the 40th Inter- national ACM SIGIR Conference on Research and Development i n Infor- mation Retrieval, pp. 445–454 (2017)

work page 2017

[6] [6]

In: Proceedings of the Thirty- First Interna- tional Joint Conference on Artiﬁcial Intelligence, IJCAI, pp

Chen, X., Luo, J., He, B., Sun, L., Sun, Y.: Towards robust dense retrieval via local ranking alignment. In: Proceedings of the Thirty- First Interna- tional Joint Conference on Artiﬁcial Intelligence, IJCAI, pp. 1980–1986 (2022)

work page 2022

[7] [7]

In: International Conference on Learning Represe ntations (2020)

De Cao, N., Izacard, G., Riedel, S., Petroni, F.: Autoreg ressive entity re- trieval. In: International Conference on Learning Represe ntations (2020)

work page 2020

[8] [8]

In: Proceedings of NAACL-HLT, pp

Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: Pr e-training of deep bidirectional transformers for language understandi ng. In: Proceedings of NAACL-HLT, pp. 4171–4186 (2019)

work page 2019