Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Search results diversification in competitive search

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Diversity-based ranking functions give competitive search a stable equilibrium and reduce content mimicking.

desk verdict The paper contributes a valuable competition dataset and a timely question, but the central equilibrium theorem is not proven—the proof's key step fails for high beta, and the cost term never enters the formal model. read the letter →

arxiv 2501.14922 v1 pith:GPCOVHZ5 submitted 2025-01-24 cs.IR cs.GT

Tommy Mordo , Itamar Reinman , Moshe Tennenholtz , Oren Kurland This is my paper · ORCID
classification cs.IRcs.GT MSC 91A1091A2068P20
keywords competitivesearchresultsdiversificationrepeatedrankinggameminmaxregretequilibriummimickingthewinnerpublisherherdingmaximalmarginalrelevanceranking-incentivizedmanipulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether search engines can dampen a harmful side effect of ranking competition: publishers rewriting documents to copy whatever ranked highest before, which creates herds of near-identical content. It studies ranking functions that add results diversification, such as maximal marginal relevance (MMR), and proves that the resulting repeated ranking game has a minmax regret equilibrium in every round. In that equilibrium some publishers stop chasing the first position and instead aim for the second, which requires their documents to stay dissimilar from the top-ranked document. Ranking competitions with students show the mimic-the-winner strategy is used less and content diversity is higher when diversification is applied. The paper's message is that diversification does not just improve the ranked list; it changes publisher incentives in the long run.

What carries the argument

The central object is the class of ranking functions $\mathit{PRSP}(D)$ (Definition 2): a basic single-peak relevance function picks the top document $d^*$, and then any document within distance $\alpha$ of $d^*$ is ranked below every document at distance at least $\alpha$. This captures diversity-based reranking such as MMR with a strong enough diversity weight. The proof machinery is the minmax regret equilibrium, applied to a repeated game where players learn by shrinking the set of possible ranking functions; the key step is the construction mapping each player to the nearest document in $V^{(1)}_\ell$ or, failing that, $V^{(2)}_\ell$, the sets of documents that could still be ranked first or second.

What would settle it

A direct check would be to compute, for every query and round of the competitions, whether the MMR ranking actually satisfies the model's core condition: every document within distance $\alpha$ of the top document is ranked below every document at distance at least $\alpha$. If violations are common, or if lowering $\lambda$ below the value used in the experiments brings back the mimic-the-winner pattern, the central claim would be tested.

Watch

Extended reading notes

Core claim

The central claim is that in a competitive search setting where the ranking function penalizes documents too similar to the already-top-ranked document (the PRSP family, Definition 2), the repeated game among publishers has a minmax regret equilibrium in every round. The equilibrium construction shows that once a publisher cannot win, the regret-minimizing move is to position herself for the second rank rather than continue mimicking the winner, because a document too close to the winner is pushed down by the diversity penalty. Consequently the 'mimicking the winner' strategy, which previous work found to drive herding, is theoretically expected to diminish. The empirical ranking competitions using MMR with $\lambda=0.5$ support this: consecutive winners are less similar, inter-document similarities are lower, and the rank transitions differ from relevance-only rankings, consistent with players giving up on the top spot.

Load-bearing premise

The argument depends on the assumption that the diversity-based ranking used in practice, the maximal marginal relevance method with equal relevance and diversity weights, behaves like the theoretical model, in which any document too close to the top-ranked document is ranked below every sufficiently different document; if that bridge fails, the equilibrium theorem does not cover the experiments.

Editorial extensions

If this is right

  • Under diversity-based ranking, the repeated competition reaches a stable state in every round, so publishers do not oscillate indefinitely among rank positions.
  • Publishers who cannot realistically win will aim for the second rank, producing documents that are deliberately dissimilar from the top-ranked one.
  • Consecutive winner documents should become less similar over time under diversity-based ranking, unlike under relevance-only ranking where similarity grows.
  • Ranked lists produced under diversity-based ranking should show lower inter-document similarity, signaling reduced publisher herding at the corpus level.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to run the same competition with several MMR values of $\lambda$; the theory predicts that the herding reduction weakens as $\lambda$ drops below the threshold where MMR behaves like the PRSP model, which would directly test the modeling bridge.
  • The mechanism may apply beyond search: recommender systems that penalize similarity to previously surfaced items could shift content-creator strategies from imitation to differentiation, suggesting a design principle for performative recommendation.
  • The second-rank-seeking equilibrium implies that measuring the distance between each publisher's document and the current winner over rounds could detect the shift away from mimicking earlier than winner-to-winner similarity does.
  • If diversification truly changes publisher incentives, then diversification parameters could be tuned not only for immediate user satisfaction but also for long-run corpus health, a dimension current evaluation practice largely ignores.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper studies a repeated ranking game in which document authors (publishers) modify documents to improve their rank, under an undisclosed ranking function that incorporates search-results diversification. The theoretical model introduces the PRSP family of ranking functions, which penalize documents sufficiently similar to the top-ranked document, and assigns positive utility to the first and second ranks. The main theoretical claim (Theorem 1) is that a minmax regret equilibrium exists in every round, from which the authors derive Corollary 2 and Observation 1, predicting that some publishers eventually stop mimicking the winner and instead aim for the second rank, thereby mitigating herding. The empirical part reports ranking competitions with students, comparing a relevance-only dense retrieval ranker with an MMR-based diversity ranker, and finds lower similarity between consecutive winners and higher content diversity in the diversity condition. The authors release the dataset and code.

Significance. If the equilibrium theorem were valid, this would be the first game-theoretic analysis of diversification in competitive search and would offer a concrete mechanism for reducing publisher herding. The paper also contributes a new experimental setup: ranking competitions with dense retrieval and MMR, plus a public dataset and code. These resources are useful to the community. However, the central theorem is not established as written, and the link between the theoretical model and the experiments is incomplete, so the significance of the results is not realized in the current form.

major comments (4)
  1. [Section 3.2, proof of Theorem 1] The proof of Theorem 1 is invalid as written. The step 'In terms of regret, selecting a document from V_l^(1) is preferable to selecting from V_l^(2), since the utility for the first rank position is higher than that of the second' conflates best-case utility with worst-case regret. A document in V_l^(1) may be ranked third for some admissible PRSP functions, while a V_l^(2) document may be ranked second for all of them. Concretely, let D_A={0.49,0.9}, D_B={0.5}, D_C={0.0}, beta=0.9, and let the knowledge state contain two PRSP functions. f1 has basic peak at 0.5 with alpha=0.2 and r(0.5)>r(0.9)>r(0.0)>r(0.49); f4 has basic peak at 0.49 with alpha=0.2 and r(0.49)>r(0.5)>r(0.9)>r(0.0). For player A, choosing 0.49 gives utility 0 under f1 and 1 under f4, while choosing 0.9 gives utility beta under both; hence MR_A(0.49)=beta=0.9 and MR_A(0.9)=1-beta=0.1. Thus the proof's prescribed choice, the nearest V_l^(1) document, is not regret-minimizing. Since Corollary 2 and Observation 1 are derived from Theorem 1, the central theoretical claims are not established.
  2. [Section 3.1, Definitions 3-4 and 7-9] The modification cost C is never part of the model used in the equilibrium definition. Section 3.1 says utility is 'adjusted by subtracting the cost of changes, assuming C|D| < beta and eC > 0', but Definitions 3 and 4 define utility with no cost term, and Definitions 7-9 define regret and the minmax regret equilibrium without cost. Consequently, the proof's selection of d_i^l that minimizes |d_i^l - d_i^{l-1}| is an unformalized extra assumption rather than a consequence of the equilibrium definition. The expression 'eC > 0' is also not a meaningful bound as written; the magnitude of the cost relative to beta is never pinned down. Corollary 2's repeated appeals to 'minimizing the cost of modification' are therefore not supported by the formal definitions.
  3. [Definition 2 vs Section 4.1 (MMR)] The experimental ranking function is not shown to belong to the PRSP family analyzed in Theorem 1. Definition 2 requires the existence of a threshold alpha such that every document within distance alpha of the top-ranked document is ranked below every document at distance at least alpha. The MMR score in Eq. (1) with lambda=0.5, used in the Diversity competitions, does not in general produce such a threshold ranking; the penalty depends on the maximum similarity to all previously selected documents, not on a hard threshold around the top document, and its effect depends on the actual retrieval scores. The text only says MMR with a 'relatively high' lambda can serve as an example, while the experiments use lambda=0.5. Unless the authors prove that the specific MMR configuration is in PRSP, or argue that the qualitative predictions are robust outside PRSP, Theorem 1 does not apply to the reported competitions.
  4. [Section 4.2, Table 1 and Figures 1-2] The empirical results do not clearly distinguish the paper's strategy-changing claim from a mechanical effect of the ranking function. The paper itself explains the rank-drop pattern in Table 1 by saying that 'due to the MMR diversification algorithm in the D competitions d1's retrieval score is highly likely to be quite penalized; hence, the rank drop.' Thus the observed rank instability is not, by itself, evidence that publishers changed their behavior to aim for the second rank. Figures 1 and 2 show lower similarity between consecutive winners under diversity ranking, but no direct measurement is provided of whether players' modification choices shifted toward the second-ranked document; the findings are consistent with other mechanisms, including continued mimicking combined with the MMR penalty. The claim that large confidence intervals in Figure 3 reflect a 'transition from competing for the first rank position to competing for the second rank position' is post-hoc and not supported by a statistical or behavioral test.
minor comments (4)
  1. [Throughout] There are several spelling inconsistencies: 'Rafier' should be 'Raifer' in the Introduction, 'second highst' in footnote 6 should be 'second highest', and 'TD.IDF' in the caption of Figure 5 should be 'TF.IDF'.
  2. [Definition 2] The condition 'for which |d* - d_i| >= alpha' appears to be a typo; it should presumably be '|d* - d_j| >= alpha', since the threshold applies to the other documents d_j being compared with d_i.
  3. [Section 3.1] The sentence 'assuming C|D| < beta and eC > 0 for a change of distance e' is unclear: if C and e are positive, eC > 0 is trivially true, and the intended upper bound on the cost is not stated precisely.
  4. [Section 4.2.2] The comparisons in Figures 1 and 2 are presented without confidence intervals or significance tests, while significance tests are used elsewhere; adding them would clarify whether the reported differences are stable.

Circularity Check

2 steps flagged · score 6.0 of 10

Empirical 'amelioration' largely restates MMR's diversity penalty; the Theorem 1 proof assumes the regret ordering it must prove.

  1. self definitional [Definition 2; Section 3.2 (after Observation 1); Section 4.3 (Table 2)]
    "For any document d_i where 0 < |d^* - d_i| < α, d_i's retrieval score, r_p(d_i), is lower than r(d_j) for all documents d_j for which |d^* - d_i| ≥ α; α is a free parameter. ... When the diversity aspect (e.g., penalty as in Definition 2) is non-negligible, the set of documents expected to rank second differs significantly from those ranked first."

    PRSP is defined by the property that documents similar (within α) to the top-ranked document are ranked below all dissimilar documents. Consequently, the theoretical 'finding' that under diversity the second-ranked document differs from the first is an unpacking of Definition 2, not a consequence of regret equilibrium. The empirical test in Section 4.3 then measures inter-document similarity in ranked lists, the very quantity the MMR algorithm (Eq. 1) explicitly penalizes via −λ max sim(d,d'). Lower similarity in the D competitions is therefore guaranteed by the ranking function's construction; it cannot independently confirm the strategic claim that players 'give up' on first place.

  2. other [Section 3.2, Theorem 1 proof]
    "Since any document in V(1)_l or V(2)_l can potentially secure the first or second rank position, the highest regret for player i is for not publishing a document from V(1)_l or V(2)_l. In terms of regret, selecting a document from V(1)_l is preferable to selecting from V(2)_l, since the utility for the first rank position is higher than that of the second."

    Definitions 7-9 define regret as the worst-case loss relative to the best response over all r_p ∈ PRSP. The proof's line 'selecting a document from V(1)_l is preferable ... since the utility for the first rank position is higher' treats ordinal utility as if it determined maximal regret. This is exactly the proposition to be established: a V(1) document may achieve rank one under some PRSP functions and rank three under others, while a V(2) document can have a guaranteed rank two; then the V(1) document has larger maximal regret. The equilibrium conclusion is thus assumed rather than derived from the regret definitions.

full rationale

The paper's self-citations to Raifer et al. [25] and Goren et al. [13] are not counted as load-bearing circularity: those are prior empirical and theoretical results with overlapping authors, but this paper runs new competitions and does not invoke a uniqueness theorem to forbid alternatives. The central circularity is in the empirical support for the headline claim. Because the Diversity ranker is MMR/PRSP, whose defining feature is to demote documents similar to already-ranked documents, the observed reduction in similarity between top-ranked documents and across ranked lists is an immediate consequence of the ranking function's construction, not an independent observation about publishers' strategies. The game-theoretic existence theorem is also not established by the given proof: the decisive assertion that a V(1) document is regret-preferred to a V(2) document conflates utility maximization with worst-case regret minimization. That is a correctness gap as much as a circularity, but it is recorded above as an 'other' step because the conclusion is inserted as the proof's premise. Overall, the paper's claim that diversity-based ranking ameliorates herding is only partially supported by independent evidence; the empirical portion is substantially by construction, giving a circularity score of 6.

Assumptions & free parameters 4 free parameters · 7 assumptions · 0 invented entities

The theoretical contribution rests on a bespoke model of diversity ranking (PRSP), a rational-player knowledge-state model, and three free parameters (alpha, beta, C). The experimental bridge is further weakened because the MMR ranker used (lambda=0.5) is not shown to be an instance of PRSP.

free parameters (4)
  • alpha (diversity penalty threshold)
    Free parameter in Definition 2 defining PRSP: documents within alpha of the top-ranked document are penalized below all documents farther than alpha. Not fitted; the theory and experiments do not estimate it.
  • beta (second-rank utility)
    Free parameter in Definition 3; utility for second rank is beta less than 1. The equilibrium proof treats V1 as always preferable to V2, which fails for beta greater than 0.5. Not fitted.
  • C (modification cost per unit distance)
    Introduced in Section 3.1 with the condition C|D| less than beta and positive cost for any change; never formally included in the utility or regret definitions, though the proof and Corollary 2 rely on cost minimization.
  • lambda (MMR trade-off in experiments) = 0.5
    Set by hand for the Diversity competitions to give equal weight to relevance and diversity (Section 4.1, Ranking functions). Not derived from data.
assumptions (7)
  • domain assumption Basic ranking functions are single-peak (Definition 1).
    Invoked in Section 3.1 to model the effect of document modifications on retrieval score; used to define V_l^(1) and to argue PRSP functions are single-peak. Not justified for neural or learning-to-rank rankers.
  • ad hoc to paper There are no retrieval-score ties.
    Assumed in Section 3.1 'to simplify the mathematical analysis'; required for total orderings and for the definition of the top-ranked document.
  • ad hoc to paper PRSP family: any document within distance alpha of the winner is ranked below every document at distance at least alpha.
    Definition 2 is the paper's model of diversity-based ranking. It is a hard-threshold stylization; MMR with lambda=0.5 used in experiments is not shown to satisfy it.
  • domain assumption Only the two highest-ranked documents receive positive utility.
    Definition 3 gives utility 1 for first, beta for second, 0 otherwise; the WLOG claim is not proven and excludes competition for lower ranks.
  • domain assumption Players are rational, learn the knowledge state, and the knowledge state only shrinks.
    Assumed in Section 3.1 to justify the repeated-game analysis; no cognitive or empirical validation for human publishers.
  • domain assumption Minmax regret is the appropriate equilibrium concept.
    Adopted from Hyafil and Boutilier, following Raifer et al.; appropriate for incomplete information but not the only possible solution concept.
  • standard math Document sets D_i are finite and disjoint.
    Section 3.1 defines D_i as finite and D_i intersect D_j empty; needed for finite strategy spaces.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Search results diversification in competitive search." pith.science (2026). https://pith.science/paper/GPCOVHZ5

@misc{pith2026250114922,
  author       = {Pith},
  title        = {Pith review of: Search results diversification in competitive search},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GPCOVHZ5}},
  note         = {Machine review of arXiv:2501.14922}
}
read the original abstract

In Web retrieval, there are many cases of competition between authors of Web documents: their incentive is to have their documents highly ranked for queries of interest. As such, the Web is a prominent example of a competitive search setting. Past work on competitive search focused on ranking functions based solely on relevance estimation. We study ranking functions that integrate a results-diversification aspect. We show that the competitive search setting with diversity-based ranking has an equilibrium. Furthermore, we theoretically and empirically show that the phenomenon of authors mimicking content in documents highly ranked in the past, which was demonstrated in previous work, is mitigated when search results diversification is applied.

Figures

Figures reproduced from arXiv: 2501.14922 by the authors.

Figure 1
Figure 1. Average absolute difference of feature values of [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 3
Figure 3. The average (over queries) similarity (with confi [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. The average (over queries) mean inter-document [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. White Hat Search Engine Optimization using Large Language Models

    cs.IR 2025-02 conditional novelty 6.0 of 10

    LLM prompts that include past rankings produce document edits that improve retrieval ranking more than human students and a feature-based baseline, while keeping the text faithful.

Reference graph

Works this paper leans on

41 extracted references · 33 canonical work pages · cited by 1 Pith paper

  1. [1]

    Introducing Connect by CloudResearch: Advancing Online Participant Recruitment in the Digital Age | Request PDF

    2024. Introducing Connect by CloudResearch: Advancing Online Participant Recruitment in the Digital Age | Request PDF. https://doi.org/10.31234/osf.io/ ksgyr

    doi:10.31234/osf.io/ 2024

  2. [2]

    Connell, W

    James Allan, Margaret E. Connell, W. Bruce Croft, Fang-Fang Feng, David Fisher, and Xiaoyan Li. 2000. INQUERY and TREC-9. In Proceedings of TREC. 551–562

    work page 2000

  3. [3]

    1995.Repeated games with incomplete information

    Robert J Aumann, Michael Maschler, and Richard E Stearns. 1995.Repeated games with incomplete information. MIT press

    work page 1995

  4. [4]

    Banerjee. 1992. A simple model of herd behavior. The Quarterly Journal of Economics 107 (1992), 797–817

    work page 1992

  5. [5]

    Ran Ben-Basat, Moshe Tennenholtz, and Oren Kurland. 2017. A Game Theoretic Analysis of the Adversarial Retrieval Setting. J. Artif. Intell. Res. 60 (2017), 1127– 1164

    work page 2017

  6. [6]

    Bruce Croft, and Yanlei Diao

    Michael Bendersky, W. Bruce Croft, and Yanlei Diao. 2011. Quality-biased ranking of web documents. In Proceedings of WSDM. 95–104

    work page 2011

  7. [7]

    Bikhchandani, D

    S. Bikhchandani, D. Hirshleifer, and I. Welch. 1992. A theory of fads, fashion, custom and cultural change as information cascade. The Journal of Political Economy 100 (1992), 992–1026

    work page 1992

  8. [8]

    Carbonell and Jade Goldstein

    Jaime G. Carbonell and Jade Goldstein. 1998. The Use of MMR, Diversity-Based Reranking for Reordering Documents and Producing Summaries. In Proceedings of SIGIR. 335–336

    work page 1998

Show all 41 references
  1. [9]

    Carlos Castillo and Brian D. Davison. 2010. Adversarial Web Search. Foundations and Trends in Information Retrieval 4, 5 (2010), 377–486

    2010

  2. [10]

    Xuanang Chen, Ben He, Zheng Ye, Le Sun, and Yingfei Sun. 2023. Towards Imper- ceptible Document Manipulations against Neural Ranking Models. In Findings of the Association for Computational Linguistics: ACL 2023

    2023

  3. [11]

    Itay Eilat and Nir Rosenfeld. 2023. Performative Recommendation: Diver- sifying Content via Strategic Incentives. http://arxiv.org/abs/2302.04336 arXiv:2302.04336 [cs]

    2023 arXiv

  4. [12]

    Gregory Goren, Oren Kurland, Moshe Tennenholtz, and Fiana Raiber. 2020. Ranking-Incentivized Quality Preserving Content Modification. In Proceedings of SIGIR. Virtual Event China, 259–268

    2020

  5. [13]

    Gregory Goren, Oren Kurland, Moshe Tennenholtz, and Fiana Raiber. 2021. Driv- ing the Herd: Search Engines as Content Influencers. In Proceedings of CIKM . Virtual Event Queensland Australia, 586–595

    2021

  6. [14]

    Zoltán Gyöngyi and Hector Garcia-Molina. 2005. Web Spam Taxonomy. In Proceedings of AIRWeb 2005. 39–47

    2005

  7. [15]

    Nathanael Hyafil and Craig Boutilier. 2012. Regret Minimizing Equilibria and Mechanisms for Games with Strict Type Uncertainty.CoRR abs/1207.4147 (2012)

    2012 arXiv

  8. [16]

    Porter Jenkins, Jennifer Zhao, Heath Vinicombe, Anant Subramanian, Arun Prasad, Atillia Dobi, Eileen Li, and Yunsong Guo. 2020. Natural Language Anno- tations for Search Engine Optimization. In Proceedings of The Web Conference . 2856–2862

    2020

  9. [17]

    Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, and Geri Gay

  10. [18]

    Oren Kurland and Lillian Lee. 2005. PageRank without hyperlinks: Structural re-ranking using links induced by language models. In Proceedings of SIGIR . 306–313

    2005

  11. [19]

    Oren Kurland and Moshe Tennenholtz. 2022. Competitive Search. In Proceedings of SIGIR. 2838–2849

    2022

  12. [20]

    Tie-Yan Liu. 2011. Learning to Rank for Information Retrieval . Springer. I–XVII, 1–285 pages

    2011

  13. [21]

    Yu-An Liu, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Wei Chen, Yixing Fan, and Xueqi Cheng. 2023. Topic-oriented Adversarial Attacks against Black-box Neural Ranking Models. In Proceedings of SIGIR. 1700–1709

    2023

  14. [22]

    Yu-An Liu, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Yixing Fan, and Xueqi Cheng. 2024. Robust Neural Information Retrieval: An Adversarial and Out-of- distribution Perspective

    2024

  15. [23]

    Bhaskar Mitra and Nick Craswell. 2018. An Introduction to Neural Information Retrieval t. Foundations and Trends in Information Retrieval 13, 1 (2018), 1–126

    2018

  16. [24]

    Haya Nachimovsky, Moshe Tennenholtz, Fiana Raiber, and Oren Kurland. 2024. Ranking-Incentivized Document Manipulations for Multiple Queries. In Proceed- ings of ICTIR. 61–70

    2024

  17. [25]

    Nimrod Raifer, Fiana Raiber, Moshe Tennenholtz, and Oren Kurland. 2017. In- formation Retrieval Meets Game Theory: The Ranking Competition Between Documents’ Authors. In Proceedings of SIGIR. Shinjuku Tokyo Japan, 465–474

    2017

  18. [26]

    Nisarg Raval and Manisha Verma. 2020. One word at a time: adversarial attacks on retrieval models. CoRR abs/2008.02197 (2020)

    2020 arXiv

  19. [27]

    Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings EMNLP-IJCNLP. 3980–3990

    2019

  20. [28]

    Robertson

    Stephen E. Robertson. 1977. The Probability Ranking Principle in IR. Journal of Documentation (1977), 294–304. Reprinted in K. Sparck Jones and P. Willett (eds), Readings in Information Retrieval , pp. 281–286, 1997

    1977

  21. [29]

    Rodrygo L. T. Santos, Craig MacDonald, and Iadh Ounis. 2015. Search Result Diversification. Found. Trends Inf. Retr. 9, 1 (2015), 1–90

    2015

  22. [30]

    Smith and P

    L. Smith and P. Sorensen. 2000. Pathalogical outcomes of observational learning. Econometrica 68 (2000), 371–398

    2000

  23. [31]

    Rush, and Vitaly Shmatikov

    Congzheng Song, Alexander M. Rush, and Vitaly Shmatikov. 2020. Adversarial Semantic Collisions. CoRR abs/2011.04743 (2020)

    2020 arXiv

  24. [32]

    Junshuai Song, Jiangshan Zhang, Jifeng Zhu, Mengyun Tang, and Yong Yang

  25. [33]

    Ziv Vasilisky, Oren Kurland, Moshe Tennenholtz, and Fiana Raiber. 2023. Content- Based Relevance Estimation in Retrieval Settings with Ranking-Incentivized Document Manipulations. In Proceedings of ICTIR. 205–214

    2023

  26. [34]

    Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2024. Text Embeddings by Weakly-Supervised Contrastive Pre-training. http://arxiv.org/abs/2212.03533 arXiv:2212.03533 [cs]

    2024 arXiv

  27. [35]

    Yumeng Wang, Lijun Lyu, and Avishek Anand. 2022. BERT Rankers are Brittle: A Study using Adversarial Document Perturbations. In Proceedings of ICTIR . 115–120

    2022

  28. [36]

    Chen Wu, Ruqing Zhang, Jiafeng Guo, Wei Chen, Yixing Fan, Maarten de Rijke, and Xueqi Cheng. 2022. Certified Robustness to Word Substitution Ranking Attack for Neural Ranking Models. In Proceedings of CIKM. 2128–2137

    2022

  29. [37]

    Chen Wu, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Yixing Fan, and Xueqi Cheng. 2022. PRADA: Practical Black-Box Adversarial Attacks against Neural Ranking Models. arXiv:2204.01321

    2022 arXiv

  30. [38]

    Haolun Wu, Yansen Zhang, Chen Ma, Fuyuan Lyu, Bowei He, Bhaskar Mitra, and Xue Liu. 2024. Result Diversification in Search and Recommendation: A Survey. http://arxiv.org/abs/2212.14464 arXiv:2212.14464 [cs]

    2024 arXiv

  31. [39]

    Lafferty

    Chengxiang Zhai and John D. Lafferty. 2001. A Study of Smoothing Methods for Language Models Applied to Ad Hoc Information Retrieval. In Proceedings of SIGIR. 334–342

    2001

  32. [2005]

    In Proceed- ings of SIGIR

    Accurately interpreting clickthrough data as implicit feedback. In Proceed- ings of SIGIR. 154–161

  33. [2022]

    In Proceedings of RepL4NLP@ACL

    TRAttack: Text Rewriting Attack Against Text Retrieval. In Proceedings of RepL4NLP@ACL. 191–203

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.