Pith. sign in

REVIEW 4 major objections 4 minor 63 references

Cross-Dialect Information Retrieval: Information Access in Low-Resource and High-Variance Languages

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Document translation into standard German is an effective way to reduce the dialect gap in retrieval, and WikiDIR—the first benchmark spanning seven German dialects—is built to test this.

desk verdict Useful German dialect retrieval resource with a solid translation result, but main-split labels are partly circular and one claim contradicts Table 3. read the letter →

arxiv 2412.12806 v2 pith:MJTXHWAL submitted 2024-12-17 cs.CL cs.IR

Robert Litschko , Oliver Kraus , Verena Blaschke , Barbara Plank This is my paper · ORCID
classification cs.CLcs.IR
keywords cross-dialectinformationretrievalGermandialectslow-resourcelexicalvariationdialectdictionariesdocumenttranslationWikipediabenchmarkdense
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Cross-dialect information retrieval—searching for content in one language while documents are written in a non-standard dialect—is nearly unexplored because dialects lack standardized orthography and training data. This paper introduces WikiDIR, the first retrieval benchmark built from seven German dialect Wikipedias, with queries in standard German and documents in dialect, plus manually annotated dictionaries of dialect spelling variants. Using WikiDIR, the paper argues that lexical retrieval methods such as BM25 (a standard keyword-matching retriever) are badly hurt by dialect variation, that zero-shot multilingual encoders transfer unevenly to these low-resource dialects, and that translating dialect documents into standard German is a practical and effective way to reduce the dialect gap. If correct, WikiDIR gives researchers a shared testbed for cross-dialect retrieval, and the translation result is an immediately usable recipe for making dialect content findable.

What carries the argument

The analytical engine is the pair of relevance assessments on the WikiDIR analysis split: one version (✗) labels as relevant only documents containing the exact query title, the other (✓) additionally labels documents containing any human-annotated dialect variant. The gap between the two measures the lexical dialect gap. The variants come from entity-centric dialect dictionaries built by extracting Wikipedia anchor-link mentions and filtering them through native-speaker annotation, yielding on average 2.5 variants per query entity across five dialects. The main test split follows a synthetic-label pipeline: documents are relevant only if they contain an exact phrasal BM25 match, with labels propagated through Wikipedia inter-language links that map dialect titles to German titles.

What would settle it

Take a random sample of queries from the WikiDIR test split, have native speakers label documents by topical relevance without seeing the lexical overlap, and re-rank the same systems under those labels; if BM25's strong baseline or the translation gains shrink, the synthetic-label pipeline is responsible.

Watch

Extended reading notes

Core claim

The paper's central claim is that the dialect gap—the orthographic and lexical distance between a standard-language query and a dialect document—is the main bottleneck in cross-dialect retrieval, and that translating documents into the standard language substantially closes it. On a held-out analysis split where queries have known dialect variants, every tested model loses ranking quality when documents containing those variants are counted as relevant, with BM25 and MonoBERT (a BERT-based cross-encoder re-ranker) suffering the largest drops. After using an LLM to translate documents to German, ranking quality rises by 0.15 to 0.29 nDCG@10 across five dialects, and the lexical retriever BM25 benefits the most. The paper also shows that a fine-tuned ColBERTv2 (a token-level late-interaction dense retriever) outperforms BM25 and zero-shot cross-lingual transfer, and that continual pretraining helps mainly for dialects with larger Wikipedias.

Load-bearing premise

The load-bearing premise is that exact phrasal containment of the query in a document, propagated through Wikipedia inter-language links, marks true relevance for the main test split; if that proxy is wrong, lexical methods like BM25 are favoured and every reported ranking shifts.

Editorial extensions

If this is right

  • Document translation to standard German raises nDCG@10 by 0.15 to 0.29 across the five analysed dialects, and the biggest gains go to BM25, making a simple keyword retriever competitive with neural models.
  • The dialect gap is systematic: on the analysis split, every tested model loses performance when relevant documents contain dialect spelling variants, so retrieval evaluation should report both ✗ and ✓ settings.
  • Zero-shot transfer from German is unreliable: MonoBERT drops well below BM25, while ColBERT roughly matches it, so dialect-specific fine-tuning is still needed for robust neural retrieval.
  • Dialect wikis are internally heterogeneous: BM25 ranking quality varies widely across subdialect groups, meaning a single dialect tag is too coarse a target for retrieval models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If relevance were judged semantically rather than lexically, the reported advantage of dense retrieval over BM25 might be larger, because the synthetic labels reward exact lexical hits by construction.
  • A natural extension of the translation recipe is to other dialect continua (Spanish, Arabic, Chinese) and to out-of-domain dialect text such as social media, where the LLM's Wikipedia pretraining advantage would be weaker.
  • The released dialect dictionaries could seed morphological analyzers or stemmers for these dialects, letting future systems do query expansion or normalization instead of full document translation.
  • The paper's own limitations—dictionary coverage depends on Wikipedia link structure, and Llama-3 was not fine-tuned for reranking—bound how far the translation gains generalize beyond Wikipedia-style text.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces WikiDIR, a cross-dialect retrieval dataset for seven German dialect Wikipedias, with queries in standard German and documents in dialects. Relevance labels are generated synthetically via exact-phrase BM25 matching and propagated through inter-language links; for five dialects the authors additionally build manually annotated dialect-variation dictionaries and a held-out analysis split. They benchmark BM25, MonoBERT, ColBERTv2, LLM reranking, continual-pretraining variants, and document translation with Llama-3, reporting nDCG@10. The main claims are that WikiDIR is the first German CDIR dataset, that fine-tuned ColBERT outperforms BM25, that zero-shot cross-lingual transfer with multilingual encoders does not transfer well, and that document translation reduces the dialect gap.

Significance. If the benchmark is valid, WikiDIR fills a real gap and is likely to become a standard resource for German cross-dialect retrieval. The paper has clear strengths: the dataset, dialect dictionaries, annotations, and source code are released; the dialect-variation dictionaries are built with human annotation and show high inter-annotator agreement; and the analysis split provides a useful way to quantify the effect of lexical variation. The headline comparative and translation claims are nevertheless weakened by the synthetic label-generation procedure and by an internal inconsistency in the zero-shot transfer claim, so the resource is more convincing than the current evaluation conclusions.

major comments (4)
  1. [Section 3.1] The main-split relevance labels are not human judgments: a document is labeled relevant only if it contains an exact phrasal match of the query in the first 200 tokens, and the labels are then propagated through inter-language links. Because BM25 is the same lexical matcher used to create these labels, the strong BM25 performance (0.737 average nDCG@10 in Table 3) is partly guaranteed by construction. Documents that mention the query entity under a valid alternative spelling are labeled non-relevant, so lexical models are never penalized for missing them. The paper cites the MIRACL critique (Zhang et al., 2023b) but retains the method for the headline benchmark. This needs to be addressed, for example by validating a sample of main-split labels with human judgments or by explicitly reframing WikiDIR as a benchmark for lexical containment rather than general cross-dialect relevance.
  2. [Section 5.1 / Abstract] The abstract states that zero-shot cross-lingual transfer with multilingual encoders does not transfer well, and Section 5.1 says that in zero-shot transfer 'both models perform worse.' Table 3, however, reports ColBERTZero-shot at 0.764 average nDCG@10, which is above BM25 (0.737) and close to ColBERTFine-tuned (0.785). This is a direct internal contradiction. The zero-shot claim should be restricted to MonoBERT or the text and table must be reconciled before the abstract's negative conclusion can be accepted.
  3. [Section 5.3 / Tables 6-7] The document-translation gains are measured against the same exact-match relevance labels used on the original dialect documents. Because the translated documents are in standard German, the percentage of documents containing an exact query match jumps from 35.2% to 77.1% on the ✗ assessments (Table 7). The reported gains of +0.15 to +0.29 nDCG@10 therefore partly reflect the label-generation rule rather than an improvement in semantic retrieval quality. The paper should report results on the variation-aware analysis split or on human judgments, or explicitly qualify the translation claim as applying only under the lexical-containment notion of relevance.
  4. [Section 5.1] No significance tests, confidence intervals, or variance estimates are reported for the nDCG@10 values in Table 3. Many pairwise differences (e.g., BM25 0.737 vs ColBERTZero-shot 0.764 vs ColBERTFine-tuned 0.785) are small relative to the per-dialect variation (e.g., 0.628 for ksh vs 0.890 for nds), so the model-ranking claims are not statistically supported. Bootstrap confidence intervals or paired significance tests across queries would make the main conclusions robust.
minor comments (4)
  1. [Section 1, RQ2] The text 'specializing models for dialects models' contains a duplicated word, and the author block contains 'V erena' with an errant space.
  2. [Table 6 caption] The caption says 'contains a the query'; it should read 'contains the query.'
  3. [Section 5.2] The 'dialect gap' is used informally throughout the analysis; a precise definition, for example the nDCG@10 difference between the ✗ and ✓ assessments, would improve reproducibility of the reported effect.
  4. [Section 7 / Abstract] The Limitations section appropriately states that the work focuses on lexical dialect variation, but the abstract and conclusion present the findings without this caveat; the claims should be explicitly scoped to lexical variation.

Circularity Check

2 steps flagged · score 6.0 of 10

Main-split relevance labels are generated by exact BM25 phrasal matches, so BM25's strong main-split score and part of the translation gains are by construction; the human-annotated analysis split provides partial independent grounding.

  1. self definitional [Section 3.1 (Relevance labels) and Section 5.1 (Main Results)]
    "we only score documents with BM25 (and thus label as relevant) if they contain an exact phrasal match. ... We find that BM25 already performs competitively, achieving an average nDCG@10 score of 0.737."

    The main-split relevance labels are generated by the same exact-phrasal-match rule that BM25 implements, so evaluating BM25 against those labels measures the label-generation rule rather than an independent notion of relevance. Documents that mention the query entity under a dialect variant are labeled non-relevant on the main split, so lexical methods are never penalized for missing them; when the human-annotated analysis split counts variants as relevant, BM25 drops (Table 4). The main-split ranking is therefore self-definitional with respect to lexical matching.

  2. self definitional [Section 5.3 (Document translation) and Appendix A (Table 7)]
    "We find that document translation yields large gains across all languages (see also Table 6 in Appendix A), with gains ranging from +0.15 to +0.29 nDCG@10. Unsurprisingly, BM25 benefits the most among our three models. This shows that document translation into a language with a standard orthography (German) is an effective way of reducing the dialectal orthographic variance."

    Because relevance is defined by exact phrasal lexical match, translating dialect documents into German mechanically raises the exact-match rate (Table 7: from 35.2% to 77.1% for the assessments without dialect variations). Part of the measured translation gain is therefore an arithmetic consequence of the label definition, not of semantic relevance. The human-annotated assessments with dialect variations also improve after translation, so the conclusion is not wholly circular, but the headline main-split evidence is inflated by the label rule.

full rationale

The paper's main-split relevance labels are synthetic: Section 3.1 states that documents are labeled relevant only if they contain an exact phrasal match of the query, using BM25 scores discretized by Jenks natural breaks, and the paper explicitly acknowledges the MIRACL critique (Zhang et al., 2023b) that such synthetic labels bias evaluation toward a lexical notion of relevance. Yet the main results (Table 3) are computed on this split, so the report that BM25 is competitive (0.737 nDCG@10) is in part guaranteed by the label-construction rule. Similarly, the document-translation gains in Figure 3 and Section 5.3 are partly mechanical, since translation increases exact-match rates (Table 7). However, the paper also contributes a human-annotated analysis split with native-speaker-checked dialect-variation dictionaries (Section 3.2), and on that split lexical methods do degrade when variants are counted as relevant (Table 4); this is independent evidence for the dialect-gap claim and prevents a higher score. No load-bearing self-citation chain is present: prior work by the authors is cited for background and baselines, not to forbid alternatives or import an unverified uniqueness result. Overall, the central benchmark comparisons partly reduce to the label-generation rule, while the human-annotated split preserves substantial independent content.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central evaluation depends on modeling assumptions inherited from Wikipedia-based dataset construction: inter-language link correctness, exact-phrasal containment as relevance, and representativeness of link-derived variation dictionaries. There are no fitted numerical parameters in the benchmark itself; the listed hyperparameters are standard model-training choices.

assumptions (4)
  • domain assumption Wikipedia inter-language links reliably connect dialect articles to their standard German counterparts.
    Used in Section 3.1 to replace dialect titles with German titles for cross-lingual relevance labels; noise in links would propagate to all evaluation scores.
  • domain assumption Article titles are meaningful retrieval queries and the first 200 tokens of an article are sufficient document representations.
    Section 3.1; this defines the benchmark and limits the task to short-title queries with truncated documents.
  • ad hoc to paper Relevance can be approximated by exact phrasal containment, with BM25 scores discretized by Jenks natural breaks providing valid graded labels.
    Section 3.1; the authors explicitly choose exact phrasal match to avoid partial-match false positives, but this aligns the label definition with lexical matching and inherits the CLIRMatrix critique that they themselves cite.
  • domain assumption Dialect variation dictionaries for five dialects are representative of true dialect variation.
    Section 3.2; dictionaries are built from Wikipedia backlinks, so variation outside the link structure is unmeasured, as the Limitations section concedes.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Cross-Dialect Information Retrieval: Information Access in Low-Resource and High-Variance Languages." pith.science (2026). https://pith.science/paper/MJTXHWAL

@misc{pith2026241212806,
  author       = {Pith},
  title        = {Pith review of: Cross-Dialect Information Retrieval: Information Access in Low-Resource and High-Variance Languages},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MJTXHWAL}},
  note         = {Machine review of arXiv:2412.12806}
}
read the original abstract

A large amount of local and culture-specific knowledge (e.g., people, traditions, food) can only be found in documents written in dialects. While there has been extensive research conducted on cross-lingual information retrieval (CLIR), the field of cross-dialect retrieval (CDIR) has received limited attention. Dialect retrieval poses unique challenges due to the limited availability of resources to train retrieval models and the high variability in non-standardized languages. We study these challenges on the example of German dialects and introduce the first German dialect retrieval dataset, dubbed WikiDIR, which consists of seven German dialects extracted from Wikipedia. Using WikiDIR, we demonstrate the weakness of lexical methods in dealing with high lexical variation in dialects. We further show that commonly used zero-shot cross-lingual transfer approach with multilingual encoders do not transfer well to extremely low-resource setups, motivating the need for resource-lean and dialect-specific retrieval models. We finally demonstrate that (document) translation is an effective way to reduce the dialect gap in CDIR.

Figures

Figures reproduced from arXiv: 2412.12806 by the authors.

Figure 1
Figure 1. Approximate locations of German regional [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Left: Dataset pipeline used to create WikiDIR. Our approach deviates from CLIRMatrix (Sun and Duh, 2020) in three aspects (highlighted in color). First, we only label documents as relevant if they contain an exact match of the query. Second, we use dialect dictionaries to also label documents as relevant if they contain any dialect variation of the query. Third, we use inter-language links to replace dialect queries… view at source ↗
Figure 3
Figure 3. CDIR results on the original training and [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

63 extracted references · 29 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Mofetoluwa Adeyemi, Akintunde Oladipo, Ronak Pradeep, and Jimmy Lin. 2024. https://aclanthology.org/2024.acl-short.59 Zero-shot cross-lingual reranking with large language models for low-resource languages . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 650--656, Bangkok, Thailan...

    work page 2024

  4. [4]

    Saied Alshahrani, Norah Alshahrani, and Jeanna Matthews. 2023. https://doi.org/10.18653/v1/2023.trustnlp-1.16 DEPTH +: An enhanced depth metric for W ikipedia corpora quality . In Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023), pages 175--189, Toronto, Canada. Association for Computational Linguistics

    doi:10.18653/v1/2023.trustnlp-1.16 2023

  5. [5]

    Akhil Arora, Robert West, and Martin Gerlach. 2024. Orphan articles: The dark matter of wikipedia. In Proceedings of the International AAAI Conference on Web and Social Media, volume 18, pages 100--112

    work page 2024

  6. [6]

    Ekaterina Artemova and Barbara Plank. 2023. https://aclanthology.org/2023.nodalida-1.39 Low-resource bilingual dialect lexicon induction with large language models . In Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa), pages 371--385, T \'o rshavn, Faroe Islands. University of Tartu Library

    2023

  7. [7]

    Mikel Artetxe, Vedanuj Goswami, Shruti Bhosale, Angela Fan, and Luke Zettlemoyer. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.399 Revisiting machine translation for cross-lingual classification . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6489--6499, Singapore. Association for Computational Linguistics

    doi:10.18653/v1/2023.emnlp-main.399 2023

  8. [8]

    Akari Asai, Jungo Kasai, Jonathan Clark, Kenton Lee, Eunsol Choi, and Hannaneh Hajishirzi. 2021. https://doi.org/10.18653/v1/2021.naacl-main.46 XOR QA : Cross-lingual open-retrieval question answering . In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5...

    doi:10.18653/v1/2021.naacl-main.46 2021

Show all 63 references
  1. [9]

    Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, et al. 2016. Ms marco: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268

    2016 arXiv

  2. [10]

    Stephen Barbour and Patrick Stevenson. 1998. Variation im Deutschen: Soziolinguistische Perspektiven. Walter de Gruyter

    1998

  3. [11]

    Adam Berger, Rich Caruana, David Cohn, Dayne Freitag, and Vibhu Mittal. 2000. Bridging the lexical chasm: statistical approaches to answer-finding. In Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, pages 192--199

    2000

  4. [12]

    Luiz Bonifacio, Vitor Jeronymo, Hugo Queiroz Abonizio, Israel Campiotti, Marzieh Fadaee, Roberto Lotufo, and Rodrigo Nogueira. 2021. mmarco: A multilingual version of the ms marco passage ranking dataset. arXiv preprint arXiv:2108.13897

    2021 arXiv

  5. [13]

    Martin Braschler. 2004. Clef 2003 -- overview of results. In Comparative Evaluation of Multilingual Information Access Systems, pages 44--63, Berlin, Heidelberg. Springer Berlin Heidelberg

    2004

  6. [14]

    Manuel Burghardt, Daniel Granvogl, and Christian Wolff. 2016. https://aclanthology.org/L16-1321 Creating a lexicon of B avarian dialect by means of F acebook language data and crowdsourcing . In Proceedings of the Tenth International Conference on Language Resources and Evalua...

    2016

  7. [15]

    Andreas Chari, Sean MacAvaney, and Iadh Ounis. 2023. On the effects of regional spelling conventions in retrieval models. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2220--2224

    2023

  8. [16]

    Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1):37--46

    1960

  9. [17]

    Marta R Costa-juss \`a , James Cross, Onur C elebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672

    2022 arXiv

  10. [18]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North A merican Chapter of the Associat...

    2019 doi

  11. [19]

    Stephan Elspaß and Robert Möller. 2003. https://www.atlas-alltagssprache.de/ Atlas zur deutschen Alltagssprache (AdA)

    2003

  12. [20]

    Fahim Faisal, Orevaoghene Ahia, Aarohi Srivastava, Kabir Ahuja, David Chiang, Yulia Tsvetkov, and Antonios Anastasopoulos. 2024. https://aclanthology.org/2024.acl-long.777 DIALECTBENCH : An NLP benchmark for dialects, varieties, and closely-related languages . In Proceedings o...

    2024

  13. [21]

    Jibril Frej, Didier Schwab, and Jean-Pierre Chevallet. 2020. https://aclanthology.org/2020.lrec-1.237 WIKIR : A python toolkit for building a large-scale W ikipedia-based E nglish information retrieval dataset . In Proceedings of the Twelfth Language Resources and Evaluation C...

    2020

  14. [22]

    Guti \'e rrez, Ekaterina Shutova, Patricia Lichtenstein, Gerard de Melo, and Luca Gilardi

    E.D. Guti \'e rrez, Ekaterina Shutova, Patricia Lichtenstein, Gerard de Melo, and Luca Gilardi. 2016. https://doi.org/10.1162/tacl_a_00082 Detecting cross-cultural differences using a multilingual topic model . Transactions of the Association for Computational Linguistics, 4:47--60

    2016 doi

  15. [23]

    Brent Hecht and Darren Gergle. 2010. The tower of babel meets web 2.0: user-generated content and its applications in a multilingual context. In Proceedings of the SIGCHI conference on human factors in computing systems, pages 291--300

    2010

  16. [24]

    Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, and Melvin Johnson. 2020. Xtreme: A massively multilingual multi-task benchmark for evaluating cross-lingual generalisation. In International Conference on Machine Learning, pages 4411--4421. PMLR

    2020

  17. [25]

    Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. https://doi.org/10.18653/v1/2020.acl-main.560 The state and fate of linguistic diversity and inclusion in the NLP world . In Proceedings of the 58th Annual Meeting of the Association for Co...

    2020 doi

  18. [26]

    Omar Khattab and Matei Zaharia. 2020. Colbert: Efficient and effective passage search via contextualized late interaction over bert. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pages 39--48

    2020

  19. [27]

    Oard, and Eugene Yang

    Dawn Lawrie, James Mayfield, Douglas W. Oard, and Eugene Yang. 2022. HC4 : A new suite of test collections for ad hoc CLIR . In Proceedings of the 44th European Conference on Information Retrieval (ECIR)

    2022

  20. [28]

    Wing Yan Li, Julie Weeds, and David Weir. 2022. https://aclanthology.org/2022.coling-1.96 M u S e CLIR : A multiple senses and cross-lingual information retrieval dataset . In Proceedings of the 29th International Conference on Computational Linguistics, pages 1128--1135, Gyeo...

    2022

  21. [29]

    Yaobo Liang, Nan Duan, Yeyun Gong, Ning Wu, Fenfei Guo, Weizhen Qi, Ming Gong, Linjun Shou, Daxin Jiang, Guihong Cao, Xiaodong Fan, Ruofei Zhang, Rahul Agrawal, Edward Cui, Sining Wei, Taroon Bharti, Ying Qiao, Jiun-Hung Chen, Winnie Wu, Shuguang Liu, Fan Yang, Daniel Campos, ...

    2020 doi

  22. [30]

    Bruce Croft, and Scott Miller

    Constantine Lignos, Daniel Cohen, Yen-Chieh Lien, Pratik Mehta, W. Bruce Croft, and Scott Miller. 2019. https://doi.org/10.18653/v1/D19-1353 The challenges of optimizing machine translation for low resource cross-language information retrieval . In Proceedings of the 2019 Conf...

    2019 doi

  23. [31]

    Jimmy Lin, Xueguang Ma, Sheng-Chieh Lin, Jheng-Hong Yang, Ronak Pradeep, and Rodrigo Nogueira. 2021. Pyserini: A python toolkit for reproducible information retrieval research with sparse and dense representations. In Proceedings of the 44th International ACM SIGIR Conference ...

    2021

  24. [32]

    Robert Litschko, Ekaterina Artemova, and Barbara Plank. 2023. https://doi.org/10.18653/v1/2023.findings-acl.193 Boosting zero-shot cross-lingual retrieval by training on artificially code-switched data . In Findings of the Association for Computational Linguistics: ACL 2023, p...

    2023 doi

  25. [33]

    Robert Litschko, Ivan Vuli \'c , Simone Paolo Ponzetto, and Goran Glava s . 2022. On cross-lingual retrieval with multilingual text encoders. Information Retrieval Journal, 25(2):149--183

    2022

  26. [34]

    Sean MacAvaney, Craig Macdonald, and Iadh Ounis. 2022. https://doi.org/10.1007/978-3-030-99739-7\_38 Streamlining evaluation with ir-measures . In Advances in Information Retrieval - 44th European Conference on IR Research, ECIR 2022, Stavanger, Norway, April 10-14, 2022, Proc...

    2022 doi

  27. [35]

    Sean MacAvaney, Luca Soldaini, and Nazli Goharian. 2020. Teaching a new dog old tricks: Resurrecting multilingual retrieval using zero-shot learning. In Advances in Information Retrieval: 42nd European Conference on IR Research, ECIR 2020, Lisbon, Portugal, April 14--17, 2020,...

    2020

  28. [36]

    Sean MacAvaney, Andrew Yates, Arman Cohan, and Nazli Goharian. 2019. Cedr: Contextualized embeddings for document ranking. In Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval, pages 1101--1104

    2019

  29. [37]

    Robert McMaster and Susanna McMaster. 2002. A history of twentieth-century american academic cartography. Cartography and Geographic Information Science, 29(3):305--321

    2002

  30. [38]

    Meta contributors . 2024. https://meta.wikimedia.org/w/index.php?title=Wikipedia_article_depth Wikipedia article depth

    2024

  31. [39]

    Alice Millour and Kar \"e n Fort. 2019. https://doi.org/10.26615/978-954-452-056-4_090 Unsupervised data augmentation for less-resourced languages with no standardized spelling . In Proceedings of the International Conference on Recent Advances in Natural Language Processing (...

    2019 doi

  32. [40]

    Rodrigo Nogueira and Kyunghyun Cho. 2019. Passage re-ranking with bert. arXiv preprint arXiv:1901.04085

    2019 arXiv

  33. [41]

    Rodrigo Nogueira, Wei Yang, Kyunghyun Cho, and Jimmy Lin. 2019. Multi-stage document ranking with bert. arXiv preprint arXiv:1910.14424

    2019 arXiv

  34. [42]

    Odunayo Ogundepo, Xinyu Zhang, Shuo Sun, Kevin Duh, and Jimmy Lin. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.597 A fri CLIRM atrix: Enabling cross-lingual information retrieval for A frican languages . In Proceedings of the 2022 Conference on Empirical Methods in Natur...

    2022 doi

  35. [43]

    Stephen Robertson, Hugo Zaragoza, et al. 2009. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends in Information Retrieval , 3(4):333--389

    2009

  36. [44]

    Uma Roy, Noah Constant, Rami Al-Rfou, Aditya Barua, Aaron Phillips, and Yinfei Yang. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.477 LAR e QA : Language-agnostic answer retrieval from a multilingual pool . In Proceedings of the 2020 Conference on Empirical Methods in Nat...

    2020 doi

  37. [45]

    Shadi Saleh and Pavel Pecina. 2020. https://doi.org/10.18653/v1/2020.acl-main.613 Document translation vs. query translation for cross-lingual information retrieval in the medical domain . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguisti...

    2020 doi

  38. [46]

    Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. 2022. https://doi.org/10.18653/v1/2022.naacl-main.272 C ol BERT v2: Effective and efficient retrieval via lightweight late interaction . In Proceedings of the 2022 Conference of the North Am...

    2022 doi

  39. [47]

    Shota Sasaki, Shuo Sun, Shigehiko Schamoni, Kevin Duh, and Kentaro Inui. 2018. https://doi.org/10.18653/v1/N18-2073 Cross-lingual learning-to-rank with shared representations . In Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computat...

    2018 doi

  40. [48]

    Shigehiko Schamoni, Felix Hieber, Artem Sokolov, and Stefan Riezler. 2014. https://doi.org/10.3115/v1/P14-2080 Learning translational and knowledge-based similarities from relevance rankings for cross-language retrieval . In Proceedings of the 52nd Annual Meeting of the Associ...

    2014 doi

  41. [49]

    Holger Schwenk, Vishrav Chaudhary, Shuo Sun, Hongyu Gong, and Francisco Guzm \'a n. 2021. https://doi.org/10.18653/v1/2021.eacl-main.115 W iki M atrix: Mining 135 M parallel sentences in 1620 language pairs from W ikipedia . In Proceedings of the 16th Conference of the Europea...

    2021 doi

  42. [50]

    L \"u tfi Kerem Senel, Benedikt Ebing, Konul Baghirova, Hinrich Schuetze, and Goran Glava s . 2024. https://aclanthology.org/2024.eacl-long.100 Karde s - NLU : Transfer to low-resource languages with the help of a high-resource cousin -- a benchmark and evaluation for T urkic ...

    2024

  43. [51]

    Peng Shi, He Bai, and Jimmy Lin. 2020. https://doi.org/10.18653/v1/2020.findings-emnlp.249 Cross-lingual training of neural models for document ranking . In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2768--2773, Online. Association for Computa...

    2020 doi

  44. [52]

    Shuo Sun and Kevin Duh. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.340 CLIRM atrix: A massively large collection of bilingual and multilingual datasets for cross-lingual information retrieval . In Proceedings of the 2020 Conference on Empirical Methods in Natural Langua...

    2020 doi

  45. [53]

    Weiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang, Pengjie Ren, Zhumin Chen, Dawei Yin, and Zhaochun Ren. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.923 Is C hat GPT good at search? investigating large language models as re-ranking agents . In Proceedings of the 2023 ...

    2023 doi

  46. [54]

    Kushal Tatariya, Artur Kulmizev, Wessel Poelman, Esther Ploeger, Marcel Bollmann, Johannes Bjerva, Jiaming Luo, Heather Lent, and Miryam de Lhoneux. 2024. https://arxiv.org/abs/2411.05527 How good is your Wikipedia ? Preprint, arXiv:2411.05527

    2024 arXiv

  47. [55]

    Francisco Valentini, Viviana Cotik, Dami \'a n Furman, Ivan Bercovich, Edgar Altszyler, and Juan Manuel P \'e rez. 2024. Messirve: A large-scale spanish information retrieval dataset. arXiv preprint arXiv:2409.05994

    2024

  48. [56]

    Jannis Vamvas, No \"e mi Aepli, and Rico Sennrich. 2024. https://aclanthology.org/2024.moomin-1.3 Modular adaptation of multilingual encoders to written S wiss G erman dialect . In Proceedings of the 1st Workshop on Modular and Open Multilingual NLP (MOOMIN 2024), pages 16--23...

    2024

  49. [57]

    Alastair G. H. Walker and Ommo Wilts. 2001. https://doi.org/10.1515/9783110946925.284 Die nordfriesischen Mundarten . In Horst Haider Munske, Nils Århammar, Volker F. Faltings, Jarich F. Hoekstra, Oebele Vries, Alastair G.H. Walker, and Ommo Wilts, editors, Handbook of Frisian...

    2001 doi

  50. [58]

    Peter Wiesinger. 1983. https://www.degruyter.com/document/doi/10.1515/9783110203332-003/html Die Einteilung der deutschen Dialekte . In Werner Besch, Ulrich Knoop, Wolfgang Putschke, and Herbert Ernst Wiegand, editors, Dialektologie: Ein Handbuch zur deutschen und allgemeinen ...

    1983 doi

  51. [59]

    Mike Zhang and Antonio Toral. 2019. https://doi.org/10.18653/v1/W19-5208 The effect of translationese in machine translation test sets . In Proceedings of the Fourth Conference on Machine Translation (Volume 1: Research Papers), pages 73--81, Florence, Italy. Association for C...

    2019 doi

  52. [60]

    Xinyu Zhang, Sebastian Hofst \"a tter, Patrick Lewis, Raphael Tang, and Jimmy Lin. 2023 a . Rank-without-gpt: Building gpt-independent listwise rerankers on open-source large language models. arXiv preprint arXiv:2312.02969

    2023 arXiv

  53. [61]

    Xinyu Zhang, Xueguang Ma, Peng Shi, and Jimmy Lin. 2021. https://doi.org/10.18653/v1/2021.mrl-1.12 Mr. T y D i: A multi-lingual benchmark for dense retrieval . In Proceedings of the 1st Workshop on Multilingual Representation Learning, pages 127--137, Punta Cana, Dominican Rep...

    2021 doi

  54. [62]

    Xinyu Zhang, Nandan Thakur, Odunayo Ogundepo, Ehsan Kamalloo, David Alfonso-Hermelo, Xiaoguang Li, Qun Liu, Mehdi Rezagholizadeh, and Jimmy Lin. 2023 b . https://doi.org/10.1162/tacl_a_00595 MIRACL : A multilingual retrieval dataset covering 18 diverse languages . Transactions...

    2023 doi

  55. [63]

    Wei Zhao, Goran Glava s , Maxime Peyrard, Yang Gao, Robert West, and Steffen Eger. 2020. https://doi.org/10.18653/v1/2020.acl-main.151 On the limitations of cross-lingual encoders as exposed by reference-free machine translation evaluation . In Proceedings of the 58th Annual M...

    2020 doi

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.