Pith. sign in

REVIEW 4 major objections 5 minor 36 references

Language Bias in Information Retrieval: The Nature of the Beast and Mitigation Methods

T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Multilingual retrievers rank the same query differently depending on its language; this paper introduces a metric to measure the gap and a training loss that narrows it.

desk verdict Useful empirical package for MLIR fairness, but the load-bearing MRC metric is under-specified in a way that could change the main comparisons. read the letter →

arxiv 2509.06195 v1 pith:JP2Q5GWM submitted 2025-09-07 cs.IR cs.AIcs.CL

Jinrui Yang , Fan Jiang , Timothy Baldwin This is my paper · ORCID
classification cs.IRcs.AIcs.CL
keywords languagebiasmultilingualinformationretrievalrankingfairnessmeanrankcorrelationLaKDAlossMultiEuP-v2densepassageBM25
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Multilingual information retrieval should return equivalent rankings for queries that mean the same thing in different languages. This paper argues that current systems do not, and that the bias is intrinsic: for the same multilingual document collection, parallel queries in 24 European languages produce inconsistent top-k rankings, and lexical retrieval (BM25) is far more biased than neural dense retrieval. To make the problem measurable, the paper builds MultiEuP-v2 from European Parliament debate titles and speeches, with each test query available in all 24 languages, and defines MRC@k, the mean Spearman correlation between language-paired rankings. The paper then shows that adding the LaKDA loss, a KL divergence between score distributions of parallel queries over the document set, to DPR training improves fairness substantially, e.g. MRC@5 rises from 11.7 to 15.9 with XLM-R while MRR@100 also improves. The upshot is that cross-language ranking fairness can be measured, and trained for, directly.

What carries the argument

Three components carry the argument. (1) MRC@k: for each query, the mean Spearman rank correlation between the top-k ranking lists produced by a retriever for that query in one language and the same query in every other language; averaged over queries, it is the paper's fairness score. (2) MultiEuP-v2: a dataset that repurposes European Parliament debate titles as parallel queries in 24 languages and speeches in a debate as documents, using the original multilingual title tags as relevance judgments; this supplies the parallel-query structure that makes the fairness question measurable. (3) LaKDA loss: for a query and a randomly sampled parallel-language version, both encoders score all docu

What would settle it

Take MultiEuP-v2 and replace the original parallel queries with machine translations of the English queries into the other 23 languages, then recompute MRR@100 and MRC@5 for BM25, DPR, and DPR+LaKDA. If the BM25-versus-DPR fairness gap shrinks or the LaKDA gains disappear, the original-language title translations are carrying the fairness signal rather than the retrievers. A complementary check is to have bilingual annotators rate, for a sample of debates, whether all 24 title versions express the same information need; if a substantial share are judged non-equivalent, the MRC comparisons rest

Watch

Extended reading notes

Core claim

The paper's central claim is that language bias is a property of the retrieval method, not just the data. Using MultiEuP-v2, where each of the 100 test queries has an official parallel version in all 24 EU languages and the same document set is searched, the paper measures how much ranking lists diverge across languages. BM25 scores near zero on the MRC@5 fairness scale (0.6 on a scale from -100 to 100), while a DPR retriever with XLM-R scores 11.7, and the paper traces BM25's bias to lexical matching, which pulls documents from the query language to the top. The proposed remedy, LaKDA, aligns the softmax similarity distributions of a query and a randomly chosen parallel-language version of

Load-bearing premise

The benchmark treats the 24 official European Parliament versions of each debate title as semantically identical queries and the automatically attached debate tags as equal-quality relevance judgments in every language; if a title translation drifts in meaning, or if tags are noisier for low-resource languages, then the measured ranking disagreement is partly an artifact of the annotation rather than of the retrieval systems.

Editorial extensions

If this is right

  • Reporting MRR alone can hide large cross-language ranking divergence; MRC@k gives a second axis on which MLIR systems should be compared.
  • Lexical retrievers will keep exhibiting language bias by construction, since they can only match query-language words; fair multilingual retrieval needs dense semantic matching.
  • Low-resource fairness is not fixed by data volume alone: in MultiEuP-v2 Maltese has more documents than Estonian but still ranks lower, implicating pretraining resource imbalance.
  • Fairness gains are not a quality trade-off in these experiments: LaKDA improves MRC@5 and MRR@100 together on both mBERT and XLM-R.
  • Because same-family languages agree more, averaged fairness scores understate the problem for cross-family pairs; evaluation should be stratified by language pair and family.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extending the paper's logic, MRC could equally measure paraphrase invariance within one language, treating language as just one axis of surface-form variation in how an information need is expressed.
  • A natural variant of LaKDA would restrict the KL term to the top-k document scores, testing whether the fairness gain mainly reflects head-of-list agreement, which is the part users actually see.
  • The translated-query experiments suggest that silver-standard parallel data, mined or machine-generated for low-resource languages, could extend LaKDA training to any language pair even when official translations do not exist.
  • Since BM25's bias is diagnosed as lexical, a hybrid that combines BM25 scores with a LaKDA-trained dense ranker could plausibly reduce language bias in sparse-retrieval systems too.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies language bias in multilingual information retrieval (MLIR) from a query-level perspective: semantically parallel queries in different languages, run against the same multilingual document collection, should produce equivalent rankings. It introduces a metric (MRC@k), a dataset (MultiEuP-v2) built from European Parliament debate titles and speeches, and a mitigation loss (LaKDA) that aligns the score distributions of parallel queries over the document collection. Experiments with BM25 and DPR (mBERT, XLM-R) report that BM25 has substantially lower MRC@5 than DPR, that low-resource languages achieve lower fairness, and that LaKDA improves MRC@5 and usually MRR@100 over the vanilla DPR loss. The paper releases code and data.

Significance. If the claims hold, this is a useful empirical contribution: it provides a benchmark with genuinely multilingual parallel queries, a simple fairness metric, and a lightweight debiasing method that appears to improve both fairness and retrieval quality. The use of original (not machine-translated) queries and documents is a strength, as is the public release. The central weakness is that the main fairness metric, MRC@k, is not fully defined, and the paper's headline results all depend on it. The dataset construction also warrants scrutiny because relevance labels are derived from debate tags rather than explicit relevance judgments. These issues are fixable but need to be resolved before the reported numbers can be trusted.

major comments (4)
  1. [§2.1, Eqs. (2)–(4)] The definition of MRC@k is incomplete and partially ill-defined. First, Spearman's rank correlation is only defined for two rankings over the same set of items, but R(q(i,a),D) and R(q(i,b),D) are top-k lists that need not contain the same documents. The paper never states how non-overlapping lists are handled (intersection, union with tied ranks, position-based pairing, or some other scheme). Second, Eq. (3) writes RCi_(a) = 1/(L-1) * sum over 1≤a<b≤L RCi_(a,b), reusing 'a' both as the fixed language and as a summation index; this does not average over all L−1 other languages. Since MRC@5 is the sole fairness measure for all major claims (Tables 3–5, Figure 3), the exact computation must be specified and, if the code is released, cross-checked against the equations.
  2. [§3.1.2, Tables 2 and 3] The paper states 'All differences are significant at p < 0.0005' without reporting the number of independent runs, variance, or the statistical test used. With only 100 test queries per language, MRC@5 differences of a few points (e.g., 13.1 vs. 16.5 in Table 3) need confidence intervals to establish that they are not noise. The same applies to the percentage improvements attributed to LaKDA. Please report standard errors or confidence intervals over at least three seeds, and state the test used.
  3. [§2.3, §6 (MultiEuP-v2)] The benchmark relies on debate-tag relevance judgments: each speech is assumed to be relevant to the debate title serving as the query. This is a reasonable weak supervision source, but it may be noisy across languages—e.g., a title translation may have different alignment with the actual speeches—and it is not validated against human relevance judgments. This affects MRR/Recall comparisons, although not MRC directly. Additionally, the document counts for low-resource languages are very small (e.g., 80 for Irish, 88 for Estonian in Table 6), so the MRC@5 and MRR numbers for those languages are likely unstable. Please provide a per-language variance estimate or at least discuss this limitation.
  4. [§3.2.3, Tables 2 and 3] The claim that LaKDA 'improves language fairness substantially without sacrificing retrieval performance' is supported, but the mechanism deserves more scrutiny. LaKDA directly minimizes the KL divergence between softmax score distributions over documents, so increasing MRC@5 is by construction plausible. The more interesting result is that MRR@100 also improves in most settings. However, the paper does not analyze whether the improvement is due to better cross-lingual alignment or simply to a regularizing effect that happens to help both metrics. A controlled experiment varying α and reporting MRC@k and MRR@100 trade-offs is needed; Figure 4 only shows α sensitivity, not the actual trade-off curve.
minor comments (5)
  1. [§3.1.3 and §3.2.3] The text refers to 'Table 7' for MRC@5 results, but Table 7 in the appendix is Recall@100; the fairness results are in Table 3. Please correct these cross-references.
  2. [Throughout] Typos and formatting: 'traning' in §3.1.1, 'hyperprameter' in §3.2.3, 'V oorhees' in the references, and malformed subscripts in the DPR loss equations. A careful proofread is needed.
  3. [§2.2.2] The LaKDA loss description is missing the value of ε, and it is not stated whether the same document set is used for both queries in a pair (presumably yes, but please make it explicit).
  4. [Abstract and §1] The abstract claims 'current MLIR systems' exhibit intrinsic language bias, but the paper only evaluates BM25 and mDPR with two encoders. Please temper the language to reflect the specific systems studied.
  5. [Figure 3] The MRC@5 matrix is not fully described: it is unclear whether the displayed values are MRC@5 for each language pair or the average over all queries. Please add a caption that defines the plotted quantity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's empirical claims are supported by held-out evaluation and external data construction.

full rationale

The paper is an empirical study rather than a mathematical derivation. Its central claims—that BM25 exhibits higher language bias than DPR, and that the LaKDA loss improves MRC@5—are evaluated on a held-out test set from MultiEuP-v2, which is constructed from external European Parliament data rather than being defined in terms of the results. The LaKDA loss minimizes KL divergence between full softmax similarity distributions of parallel queries, whereas MRC@5 measures Spearman rank correlation over top-5 lists; these are related but not identical, so the fairness improvement is not a definitional identity. The dataset builds on the authors' earlier MultiEuP work, but that is a normal, externally available resource and is not used as an unverified premise. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported from self-citation, and no ansatz is smuggled in via citation. The main risk noted by a skeptical reader—the underspecified handling of non-overlapping top-k lists in the MRC metric—is a correctness/robustness concern rather than circularity. Overall, the derivation chain is self-contained and empirically grounded, so no significant circularity is present.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claims rest on the fairness definition (parallel queries should yield equivalent rankings), the correctness of the parallel translations and automatically derived relevance labels in MultiEuP-v2, an unspecified technical assumption about how Spearman correlation is applied to top-k lists with non-identical documents, and the standard cross-lingual embedding assumption for DPR. The only explicitly tuned free parameter is the LaKDA weight alpha.

free parameters (1)
  • alpha (LaKDA loss weight) = 0.5
    Weight of LaKDA loss in the joint objective (Eq. 1); set to 0.5 for comparability in main experiments and varied in a sensitivity analysis (Figure 4).
assumptions (4)
  • domain assumption Semantically parallel queries in different languages should yield equivalent ranking lists (fairness definition)
    Adopted as the paper's fairness premise in Section 2.1; not empirically justified, but a defensible normative choice.
  • domain assumption The 24 language versions of EP debate titles are semantically identical and the automatically attached debate tags are reliable relevance judgments
    Relied on to construct MultiEuP-v2 queries and relevance labels in Section 2.3; if translations or tags are noisy, the fairness measure is corrupted.
  • ad hoc to paper Spearman's rank correlation can be computed between top-k ranked lists of the same document collection even when the ranked sets differ
    The MRC definition in Section 2.1 does not specify how non-overlapping top-k documents are handled; a necessary but unstated assumption for the metric.
  • domain assumption mBERT and XLM-R produce a cross-lingual embedding space where dot product similarities are comparable across languages
    Standard assumption for DPR-based multilingual retrieval, invoked in Sections 2.2 and 6.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Language Bias in Information Retrieval: The Nature of the Beast and Mitigation Methods." pith.science (2026). https://pith.science/paper/JP2Q5GWM

@misc{pith2026250906195,
  author       = {Pith},
  title        = {Pith review of: Language Bias in Information Retrieval: The Nature of the Beast and Mitigation Methods},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JP2Q5GWM}},
  note         = {Machine review of arXiv:2509.06195}
}
read the original abstract

Language fairness in multilingual information retrieval (MLIR) systems is crucial for ensuring equitable access to information across diverse languages. This paper sheds light on the issue, based on the assumption that queries in different languages, but with identical semantics, should yield equivalent ranking lists when retrieving on the same multilingual documents. We evaluate the degree of fairness using both traditional retrieval methods, and a DPR neural ranker based on mBERT and XLM-R. Additionally, we introduce `LaKDA', a novel loss designed to mitigate language biases in neural MLIR approaches. Our analysis exposes intrinsic language biases in current MLIR technologies, with notable disparities across the retrieval methods, and the effectiveness of LaKDA in enhancing language fairness.

Figures

Figures reproduced from arXiv: 2509.06195 by the authors.

Figure 1
Figure 1. The case of language bias studied in this [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Our framework contains two parts: the IR component, and the parallel query component. For the IR part, [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The MRC@5 matrix among parallel queries. The x-axis and y-axis both represent query languages. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: MSE and LaKDA sensitivity plot. pretrained multilingual models. Our observations are as follows: mBERT Findings Compared to the vanilla set￾ting (DPR only): (1) incorporating either MSE or LaKDA enhances language fairness (MRC@5) with mBERT, but LaKDA is more effective…
Figure 5
Figure 5. Figure 5: Parallel query similarity over training. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: The correlation of query language with document language in top 100 ranking output. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 14 canonical work pages

  1. [1]

    Zahra Abbasiyantaeb and Saeedeh Momtazi. 2020. https://arxiv.org/abs/2002.06612 Text-based question answering from information retrieval and deep neural network perspectives: A survey . CoRR, abs/2002.06612

    work page Pith review arXiv 2020

  2. [2]

    Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, et al. 2016. Ms marco: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268

    arXiv 2016

  3. [3]

    Adam Berger, Rich Caruana, David Cohn, Dayne Freitag, and Vibhu Mittal. 2000. https://doi.org/10.1145/345508.345576 Bridging the lexical chasm: statistical approaches to answer-finding . In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '00, page 192–199, New York, NY, USA. Ass...

    work page arXiv 2000

  4. [4]

    Biega, Krishna P

    Asia J. Biega, Krishna P. Gummadi, and Gerhard Weikum. 2018. https://doi.org/10.1145/3209978.3210063 Equity of attention: Amortizing individual fairness in rankings . In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR ’18. ACM

    arXiv 2018

  5. [5]

    Luiz Henrique Bonifacio, Israel Campiotti, Roberto de Alencar Lotufo, and Rodrigo Frassetto Nogueira. 2021. https://arxiv.org/abs/2108.13897 mmarco: A multilingual version of MS MARCO passage ranking dataset . CoRR, abs/2108.13897

    arXiv 2021

  6. [6]

    Ilias Chalkidis, Manos Fergadiotis, and Ion Androutsopoulos. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.559 M ulti EURLEX - a multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfer . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6974--6996, Online a...

    doi:10.18653/v1/2021.emnlp-main.559 2021

  7. [7]

    Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. https://doi.org/10.18653/v1/P17-1171 Reading W ikipedia to answer open-domain questions . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1870--1879, Vancouver, Canada. Association for Computational Linguistics

    doi:10.18653/v1/p17-1171 2017

  8. [8]

    Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzm \'a n, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. https://doi.org/10.18653/v1/2020.acl-main.747 Unsupervised cross-lingual representation learning at scale . In Proceedings of the 58th Annual Meeting of the Association for Comp...

    doi:10.18653/v1/2020.acl-main.747 2020

Show all 36 references
  1. [9]

    Alexis Conneau and Guillaume Lample. 2019. Cross-lingual language model pretraining. In Advances in Neural Information Processing Systems

    2019

  2. [10]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North A merican Chapter of the Associat...

    2019 doi

  3. [11]

    Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Rich Zemel. 2011. https://arxiv.org/abs/1104.3913 Fairness through awareness . Preprint, arXiv:1104.3913

    2011 arXiv

  4. [12]

    Trevor Hastie, Robert Tibshirani, and Jerome Friedman. 2009. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Science & Business Media

    2009

  5. [13]

    Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, and Melvin Johnson. 2020. Xtreme: A massively multilingual multi-task benchmark for evaluating cross-lingual generalization. In Proceedings of the 37th International Conference on Machine Learning, pages ...

    2020

  6. [14]

    Zhiqi Huang, Hansi Zeng, Hamed Zamani, and James Allan. 2023. Soft prompt decoding for multilingual dense retrieval. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1208--1218

    2023

  7. [15]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.550 Dense passage retrieval for open-domain question answering . In Proceedings of the 2020 Conference on Empiric...

    2020 doi

  8. [16]

    Dawn Lawrie, Eugene Yang, Douglas W Oard, and James Mayfield. 2023. Neural approaches to multilingual information retrieval. In European Conference on Information Retrieval, pages 521--536. Springer

    2023

  9. [17]

    Jimmy Lin, Xueguang Ma, Sheng-Chieh Lin, Jheng-Hong Yang, Ronak Pradeep, and Rodrigo Nogueira. 2021. Pyserini: A Python toolkit for reproducible information retrieval research with sparse and dense representations. https://github.com/castorini/pyserini

    2021

  10. [18]

    Jing Miao, Charat Thongprayoon, Supawadee Suppadungsuk, Oscar Garcia Valencia, and Wisit Cheungpasitporn. 2024. https://doi.org/10.3390/medicina60030445 Integrating retrieval-augmented generation with large language models in nephrology: Advancing practical applications . Medi...

    2024 doi

  11. [19]

    John G. Oakes. 2010. https://academic.oup.com/ije/article/39/5/1151/806286 Commentary: Charles spearman and correlation: a commentary on ‘the proof and measurement of association between two things’ . International Journal of Epidemiology, 39(5):1151--1160

    2010

  12. [20]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pages 311--318. Association for Computational Linguistics

    2002

  13. [21]

    Zhang, Coleman Haley, Kenneth Steimel, Han Liu, and Lane Schwartz

    Hyunji Hayley Park, Katherine J. Zhang, Coleman Haley, Kenneth Steimel, Han Liu, and Lane Schwartz. 2021. https://doi.org/10.1162/tacl_a_00365 Morphology matters: A multilingual language modeling analysis . Transactions of the Association for Computational Linguistics, 9:261--276

    2021 doi

  14. [22]

    Radev, Hong Qi, Harris Wu, and Weiguo Fan

    Dragomir R. Radev, Hong Qi, Harris Wu, and Weiguo Fan. 2002. http://www.lrec-conf.org/proceedings/lrec2002/pdf/301.pdf Evaluating web-based question answering systems . In Proceedings of the Third International Conference on Language Resources and Evaluation ( LREC ' 02) , Las...

    2002

  15. [23]

    Robertson, Alan Mislove, and Christo Wilson

    Piotr Sapiezynski, Wesley Zeng, Ronald E. Robertson, Alan Mislove, and Christo Wilson. 2019. https://arxiv.org/abs/1901.10437 Quantifying the impact of user attention on fair group representation in ranked lists . Preprint, arXiv:1901.10437

    2019 arXiv

  16. [24]

    Charles Spearman. 1904. The proof and measurement of association between two things. The American Journal of Psychology, 15(1):72--101

    1904

  17. [25]

    Nandan Thakur, Nils Reimers, Andreas R \" u ckl \' e , Abhishek Srivastava, and Iryna Gurevych. 2021. https://arxiv.org/abs/2104.08663 BEIR: A heterogenous benchmark for zero-shot evaluation of information retrieval models . CoRR, abs/2104.08663

    2021 arXiv

  18. [26]

    Voorhees and Dawn M

    Ellen M. Voorhees and Dawn M. Tice. 2000. http://www.lrec-conf.org/proceedings/lrec2000/pdf/26.pdf The TREC -8 question answering track . In Proceedings of the Second International Conference on Language Resources and Evaluation ( LREC ' 00) , Athens, Greece. European Language...

    2000

  19. [27]

    Eugene Yang, Thomas Jänich, James Mayfield, and Dawn Lawrie. 2024. https://doi.org/10.1145/3626772.3657943 Language fairness in multilingual information retrieval . In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retriev...

    2024

  20. [28]

    Jinrui Yang, Timothy Baldwin, and Trevor Cohn. 2023. https://doi.org/10.18653/v1/2023.mrl-1.21 Multi- E u P : The multilingual E uropean parliament dataset for analysis of bias in information retrieval . In Proceedings of the 3rd Workshop on Multi-lingual Representation Learni...

    2023 doi

  21. [29]

    Peilin Yang, Hui Fang, and Jimmy Lin. 2017. https://doi.org/10.1145/3077136.3080721 Anserini: Enabling the use of lucene for information retrieval research . In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pag...

    2017

  22. [30]

    Meike Zehlike, Francesco Bonchi, Carlos Castillo, Sara Hajian, Mohamed Megahed, and Ricardo Baeza-Yates. 2017. https://doi.org/10.1145/3132847.3132938 Fa*ir: A fair top-k ranking algorithm . In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, ...

    2017

  23. [31]

    Meike Zehlike, Tom S\" u hr, Ricardo Baeza-Yates, Francesco Bonchi, Carlos Castillo, and Sara Hajian. 2022. https://doi.org/10.1016/j.ipm.2021.102707 Fair top-k ranking with multiple protected groups . Inf. Process. Manage., 59(1)

    2022

  24. [32]

    Xinyu Zhang, Xueguang Ma, Peng Shi, and Jimmy Lin. 2021. https://doi.org/10.18653/v1/2021.mrl-1.12 Mr. T y D i: A multi-lingual benchmark for dense retrieval . In Proceedings of the 1st Workshop on Multilingual Representation Learning, pages 127--137, Punta Cana, Dominican Rep...

    2021 doi

  25. [33]

    Xinyu Zhang, Nandan Thakur, Odunayo Ogundepo, Ehsan Kamalloo, David Alfonso-Hermelo, Xiaoguang Li, Qun Liu, Mehdi Rezagholizadeh, and Jimmy Lin. 2023. https://doi.org/10.1162/tacl_a_00595 MIRACL: A Multilingual Retrieval Dataset Covering 18 Diverse Languages . Transactions of ...

    2023 doi

  26. [34]

    Yutao Zhu, Huaying Yuan, Shuting Wang, Jiongnan Liu, Wenhan Liu, Chenlong Deng, Haonan Chen, Zhicheng Dou, and Ji-Rong Wen. 2024. https://arxiv.org/abs/2308.07107 Large language models for information retrieval: A survey . Preprint, arXiv:2308.07107

    2024

  27. [35]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  28. [36]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.