FactIR: A Real-World Zero-shot Open-Domain Retrieval Benchmark for Fact-Checking

Venktesh V; Vinay Setty

REVIEW 3 major objections 6 minor 29 references

FactIR: A Real-World Zero-shot Open-Domain Retrieval Benchmark for Fact-Checking

T0 review · 3 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A retrieval benchmark built from live fact-checking editor logs shows lexical and sparse retrievers beating most dense neural models in zero-shot open-domain evidence search.

desk verdict A genuinely new real-world fact-checking retrieval benchmark with a real validity flaw: the relevance labels only cover documents a production system happened to surface. read the letter →

arxiv 2502.06006 v1 pith:NFEUYCGR submitted 2025-02-09 cs.IR

Venktesh V , Vinay Setty This is my paper · ORCID

classification cs.IR

keywords fact-checkingretrievalbenchmarkopen-domainzero-shotevaluationevidencedensesparseproductionuserfeedback

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FactIR is a retrieval benchmark assembled from logs of the Factiverse fact-check editor in production use, with relevance labels supplied by professional fact-checkers and journalism-trained users. The paper's goal is to give the fact-checking community an open-domain benchmark that reflects authentic conditions rather than Wikipedia-bound synthetic claims, and to measure how well existing retrieval models work when dropped in without fine-tuning. Across 100 claims and 1,413 claim-evidence relevance judgments, lexical BM25 and learned sparse retrieval beat most dense neural retrievers, and the best dense model owes its advantage to a semantic-clustering-based training objective. An LLM-based re-ranker outperforms other re-rankers when placed on top of BM25 candidates. If the benchmark is accepted as representative, it implies that practical fact-checking systems need strong lexical baselines and that claims of dense-retriever superiority from other benchmarks do not transfer to this real-world setting.

What carries the argument

The benchmark itself is the key object: 100 organically generated claims, 1,413 claim-evidence relevance annotations from production users, and a retrieval corpus of 90,047 documents. The corpus is open-domain, drawn from web search results aggregated by the Factiverse editor's query generation and evidence filtering. The evaluation protocol is zero-shot retrieval with nDCG@k and Recall@k metrics, which is what makes the comparison informative for real-world deployment where fine-tuning is infeasible. The paper also singles out source stratification, a training objective inspired by the cluster hypothesis, as the mechanism explaining why snowflake-arctic-embed-s generalizes better than other dense encoders.

What would settle it

Re-annotate a random sample of FactIR's 1,413 claim-evidence pairs with fresh, independent annotators using the same relevance instructions; if agreement with the production labels falls well below the reported 88% correctness, or if relevant documents that the production query generator never surfaced are found in the 90,047-document corpus and are labeled relevant by the fresh annotators, the benchmark's retrieval estimates are biased by the production pipeline.

Watch

Extended reading notes

Core claim

The paper establishes FactIR as a real-world, open-domain retrieval benchmark and reports a zero-shot evaluation of state-of-the-art retrievers on it. The central result is that lexical and sparse retrieval methods are strong baselines: BM25 reaches nDCG@10 of 0.345 and Recall@100 of 0.779, while classical dense retrievers such as DPR, ANCE, tas-b, MPNet, and Contriever underperform BM25 on most metrics, with the strongest dense model being snowflake-arctic-embed-s (nDCG@10 of 0.420, a 21.74% improvement over BM25). The paper attributes this to domain shift from pre-training data and to complex, compositional real-world claims, and it hypothesizes that semantic-clustering source stratification in pre-training yields better generalization. In re-ranking, gte-multilingual-reranker-base gives the largest measured improvement over BM25, a 6.04% gain in nDCG@10.

Load-bearing premise

The relevance labels were produced by users of a working fact-checking editor and were only spot-checked by two researchers (88% agreement with the original judgments), so the benchmark's gold standard inherits whatever biases the production query generation and evidence filtering built into what users ever saw.

Editorial extensions

If this is right

A practical fact-checking retrieval stack should keep BM25 or a learned sparse retriever as a backbone, since dense models alone are not reliable zero-shot over open-domain claims.
State-of-the-art dense retrievers' gains on BEIR-style benchmarks do not automatically transfer to real fact-checking evidence, because domain shift and claim complexity change the retrieval problem.
Semantic-clustering-based source stratification is a promising direction for training dense retrievers that must handle topically diverse, hard-negative-rich fact-checking queries.
LLM-based cross-encoder re-rankers can improve over MS-MARCO-trained cross-encoders when re-ranking BM25 candidates, but the measured gains at top ranks are modest.
FactIR enables reproducible comparisons and can be extended with new retrievers and re-rankers through the released library.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because relevance labels came from a production editor's query-generation pipeline, the benchmark may under-represent evidence that a different query formulation would have surfaced; a useful stress test is to union candidates from several query-generation strategies and measure whether recall estimates shift.
An implicit consequence is that future fact-checking retrieval research should report lexical baselines alongside dense models, since a production-derived dataset can reverse the usual ordering seen on synthetic benchmarks.
The findings could transfer to other professional information-seeking settings where claims are long, compositional, and web evidence is heterogeneous, not only to fact-checking.
A testable extension is to fine-tune the best generalized dense retriever on FactIR and measure whether the benchmark's difficulty moves from retrieval to re-ranking, which would tell whether evidence filtering, not candidate generation, is the bottleneck.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The headline: FactIR is a genuinely new resource—a small but organic benchmark of 100 real claims with 1,413 relevance judgments over a 90k-document corpus, harvested from Factiverse production logs. That fills a real gap: most fact-checking retrieval is evaluated on Wikipedia (FEVER) or pre-verified evidence (AVeriTeC). The authors also run a reasonable zero-shot evaluation of lexical, sparse, dense, and reranking models, and they release code and data.

What's good: the claims are organically produced, the corpus is dedicated, and the annotations come from domain experts (journalists/fact-checkers) following clear relevance instructions. That is a step up from synthetic claims. The release is transparent about the production source (the authors' own company), and the evaluation includes standard baselines.

The load-bearing problem is the label pool. The relevance judgments only cover documents that the production retriever surfaced; there is no evidence of any judging outside that pool. The stress-test note has it right: Recall@100 is computed against a label set that was never sampled from the 90k corpus, so models retrieving different but relevant documents are penalized as if they missed them. The meta-analysis by two researchers only checks consistency of the existing labels—it cannot catch missing relevant documents. That means the model rankings in Table 3, especially Recall, likely reflect overlap with the production system as much as true retrieval quality. Also, this is a 4-page companion; 100 claims is small, and there is no significance testing, so the reported differences between models should be treated as tentative. The self-referential validation (authors from the same company, prior work cited) is a transparency concern, but not fatal; the real issue is the lack of independent validation.

Who is this for: anyone building fact-checking retrieval systems who wants a real-world, non-Wikipedia benchmark. Even with the pooling flaw, the dataset itself could be useful if treated as a partially-labeled resource rather than a complete relevance set.

Recommendation: I would send this to peer review with a request for major revision—either expand the pool (e.g., judge additional documents sampled from the corpus), or at minimum disclose the pooling limitation prominently and soften claims about Recall. Without that, the 'zero-shot retrieval benchmark' headline overstates what is measured. But the core idea is sound, and the resource deserves referee time.

Referee Report

3 major / 6 minor

Summary. The paper introduces FactIR, a retrieval benchmark for fact-checking constructed from Factiverse production fact-checking logs. The dataset consists of 100 organically generated claims, 1,413 claim--evidence relevance judgments, and a corpus of 90,047 documents. The authors evaluate lexical (BM25), sparse (SPLADE), dense (DPR, ANCE, Contriever, etc.), late-interaction (ColBERTv2), and re-ranking models in a zero-shot setting, reporting nDCG@k and Recall@k. They find that BM25 is a strong baseline, that a dense retriever with stratification-based pretraining (snowflake-arctic-embed-s) performs best among first-stage retrievers, and that an LLM-based re-ranker (gte-multilingual-reranker-base) performs best among re-rankers.

Significance. If the benchmark is valid, FactIR would fill a real gap: most existing fact-checking retrieval datasets are limited to Wikipedia, synthetic claims, or pre-verified evidence, whereas FactIR is derived from an operational fact-checking system with real user queries. The paper also ships code and data, and it evaluates a broad set of modern retrievers, which is useful for practitioners. However, the validity of the benchmark rests on the completeness and unbiasedness of the relevance labels, and the current evidence for that is insufficient. The production-log origin is a strength, but it also creates a selection-bias risk that the paper does not address; the meta-analysis in Section 3.1 validates only the labels that were assigned, not the unjudged documents that were never surfaced. The central benchmarking claim is defensible in principle, but the reported results should be treated as preliminary until the labeling bias and statistical reliability are addressed.

major comments (3)

[Section 3 and Section 3.1, Table 3] The relevance labels cover only documents that the Factiverse production pipeline surfaced to users. Section 3 states that the editor aggregates evidence from multiple search engines and applies query generation and evidence filtering, and that users provide feedback on those surfaced documents. The paper reports no pooling of candidate documents from the evaluated retrieval systems and no independent judgment of documents outside the production-retrieved pool. Consequently, any relevant document not retrieved by the production pipeline is scored as non-relevant in Table 3. This systematically penalizes models that retrieve different but relevant documents and favors models whose output overlaps with the production system. Recall@100 is especially affected, because it is computed over a label set that was never sampled from the 90,047-document corpus. The Section 3.1 meta-analysis (88.03% agreement, Cohen's kappa 0.946) checks only whether the existing judgments are internally consistent on the documents that were shown; it cannot detect missing relevant documents. The central claim that FactIR provides reliable zero-shot retrieval evaluation is therefore not yet supported. The authors should either add pooled relevance judgments over the union of candidate documents from all evaluated systems, or explicitly rescope the benchmark and all derived conclusions to the production-surfaced document set.
[Table 3 and Section 5] The headline comparisons are reported without any significance testing or confidence intervals. With only 100 claims and an average of 13.89 judgments per claim, the margins between BM25 (nDCG@5 0.288), Contriever (0.299), and snowflake-arctic-embed-s (0.367), or between the rerankers in the bottom half of Table 3, may not be reliable. The paper uses ▲ percentage marks to assert improvements and then draws conclusions such as 'snowflake-arctic-embed-s outperforms lexical sparse and other state-of-the-art dense retrieval models' and 'the LLM-based re-ranker gte-multilingual-reranker-base outperforms other cross-encoder models.' To support these claims, the paper should report bootstrap confidence intervals, paired significance tests (e.g., paired bootstrap or permutation tests per query), or both.
[Section 3.1] The corpus is described only by the total count of 90,047 documents. Since FactIR is proposed as an open-domain retrieval benchmark, the absence of a corpus construction protocol is a significant omission. The paper should describe how the documents were collected, whether the collection is the full production corpus or a sample, how duplicates and near-duplicates were handled, what languages and time ranges are covered, and how the claims relate temporally to the documents. Without this information, it is difficult to assess coverage, reproducibility, or the meaning of the open-domain claim.

minor comments (6)

[Table 3 caption] The caption says 'nDCG@10 across datasets' but the table reports nDCG@k and Recall@k for several values of k; the caption should be corrected.
[Table 3] There is a typo in the column header 'Recal@100'; it should read 'Recall@100'.
[Section 3.1] Table 2 is labeled 'Numerical taxonomy' but the paper does not explain how the categories Statistical, Temporal, and Comparative were assigned or whether they are mutually exclusive; a short definition would help.
[Header] The running header on the experimental pages reads 'Trovato et al.', which appears to be a template artifact from a different paper; this should be fixed.
[Section 5] The subsection heading 'Classical dense retrieval models fail on OOD data' uses the abbreviation 'OOD' without defining it; spell out 'out-of-distribution' at first use.
[Abstract and Section 6] The abstract and conclusion describe the benchmark as 'comprehensive', but the dataset contains 100 claims and 1,413 judgments; either add contextual comparison with other fact-checking retrieval benchmarks or soften the claim in light of the dataset size.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the benchmark's provenance is empirical and the retrieval scores are measurements against released judgments, not predictions derived from the benchmark's own inputs.

full rationale

The paper makes no first-principles derivation: it releases a benchmark assembled from production logs and human feedback, then reports zero-shot retrieval scores. The relevance judgments are defined by annotator feedback on documents surfaced by the Factiverse editor, so the label set may be incomplete; however, that is a benchmark-validity concern, not circularity. No fitted parameter is later called a prediction, no result is equivalent by construction to an input, and the self-citations ([17], [18], [19]) describe the production system rather than importing an unverified theorem or ansatz. The reported nDCG/Recall values are model-dependent measurements against a fixed released label set, so the ranking is not forced by the construction. The Section 3.1 meta-analysis checks internal consistency of existing labels only; it is weak support but not a circular derivation. Consequently no circular step meets the evidentiary bar of exhibiting a reduction by construction.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the quality and representativeness of the production-derived relevance labels and corpus, both of which are assumed rather than independently verified. The benchmark itself is a new artifact, not an invented physical entity, so no invented entities are listed.

assumptions (4)

domain assumption Relevance labels from Factiverse production user feedback are accurate and representative of real-world retrieval relevance.
Section 3 states labels come from professional fact-checkers using the product, but there is no external audit of the original feedback; a meta-analysis by two researchers in Section 3.1 confirms only 88% of existing assessments.
domain assumption The two-researcher meta-analysis is a sufficient validation of label quality.
Section 3.1: the researchers are not identified as independent of the authors, and they judge the correctness of existing labels rather than re-labeling from scratch.
domain assumption The 90,047-document corpus is a representative open-domain web corpus for fact-checking retrieval.
Section 3.1 lists corpus size but does not describe the document source mix, deduplication, or coverage, so representativeness is assumed.
domain assumption Standard IR metrics (nDCG and Recall) are appropriate for evaluating fact-checking evidence retrieval.
Section 4 selects these metrics, but does not justify them for the indirect-evidence scenarios the paper emphasizes.

how reviews work

0 comments

Cite this review

Pith. "Pith review of FactIR: A Real-World Zero-shot Open-Domain Retrieval Benchmark for Fact-Checking." pith.science (2026). https://pith.science/paper/NFEUYCGR

@misc{pith2026250206006,
  author       = {Pith},
  title        = {Pith review of: FactIR: A Real-World Zero-shot Open-Domain Retrieval Benchmark for Fact-Checking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NFEUYCGR}},
  note         = {Machine review of arXiv:2502.06006}
}

read the original abstract

The field of automated fact-checking increasingly depends on retrieving web-based evidence to determine the veracity of claims in real-world scenarios. A significant challenge in this process is not only retrieving relevant information, but also identifying evidence that can both support and refute complex claims. Traditional retrieval methods may return documents that directly address claims or lean toward supporting them, but often struggle with more complex claims requiring indirect reasoning. While some existing benchmarks and methods target retrieval for fact-checking, a comprehensive real-world open-domain benchmark has been lacking. In this paper, we present a real-world retrieval benchmark FactIR, derived from Factiverse production logs, enhanced with human annotations. We rigorously evaluate state-of-the-art retrieval models in a zero-shot setup on FactIR and offer insights for developing practical retrieval systems for fact-checking. Code and data are available at https://github.com/factiverse/factIR.

Figures

Figures reproduced from arXiv: 2502.06006 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 18 canonical work pages

[1]

Rami Aly, Zhijiang Guo, Michael Schlichtkrull, James Thorne, Andreas Vlachos, Christos Christodoulopoulos, Oana Cocarascu, and Arpit Mittal. 2021. FEVER- OUS: Fact Extraction and VERification Over Unstructured and Structured infor- mation. arXiv:2106.05707 [cs.CL]

arXiv 2021
[2]

Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. 2018. MS MARCO: A Human Generated MAchine Reading COmprehension Dataset. arXiv:1611.09268 [cs.CL]

arXiv 2018
[3]

Jifan Chen, Aniruddh Sriram, Eunsol Choi, and Greg Durrett. 2022. Gen- erating Literal and Implied Subquestions to Fact-check Complex Claims. arXiv:2205.06938 [cs.CL]

arXiv 2022
[4]

Angela Fan, Aleksandra Piktus, Fabio Petroni, Guillaume Wenzek, Marzieh Saeidi, Andreas Vlachos, Antoine Bordes, and Sebastian Riedel. 2020. Generating Fact Checking Briefs. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computat...

work page 2020
[5]

Anton Fogelberg and Jonas Nygren. 2023. Search Engine Evaluation

work page 2023
[6]

Thibault Formal, Carlos Lassance, Benjamin Piwowarski, and Stéphane Clinchant
[7]

Zhijiang Guo, Michael Schlichtkrull, and Andreas Vlachos. 2022. A survey on automated fact-checking. Transactions of the Association for Computational Linguistics 10 (2022), 178–206

2022
[8]

Prakhar Gupta, Chien-Sheng Wu, Wenhao Liu, and Caiming Xiong. 2022. DialFact: A Benchmark for Fact-Checking in Dialogue. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (Eds.). Association for Computational Linguistics, Dublin, Ire...

work page 2022

Show all 29 references

[9]

Sebastian Hofstätter, Sheng-Chieh Lin, Jheng-Hong Yang, Jimmy Lin, and Allan Hanbury. 2021. Efficiently teaching an effective dense retriever with balanced topic aware sampling. In Proceedings of the 44th International ACM SIGIR Confer- ence on Research and Development in Info...

2021
[10]

Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bo- janowski, Armand Joulin, and Edouard Grave. 2022. Unsupervised Dense Infor- mation Retrieval with Contrastive Learning. arXiv:2112.09118 [cs.IR]

2022 arXiv
[11]

Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense Passage Retrieval for Open- Domain Question Answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP...

2020
[12]

Omar Khattab and Matei Zaharia. 2020. ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT. arXiv:2004.12832 [cs.IR]

2020 arXiv
[13]

Luke Merrick. 2024. Embedding And Clustering Your Data Can Improve Con- trastive Pretraining. arXiv:2407.18887 [cs.LG]

2024 arXiv
[14]

Stephen Robertson and Hugo Zaragoza. 2009. The Probabilistic Relevance Frame- work: BM25 and Beyond. Foundations and Trends® in Information Retrieval 3, 4 (2009), 333–389

2009
[15]

Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. 2022. ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational ...

2022
[16]

Michael Schlichtkrull, Zhijiang Guo, and Andreas Vlachos. 2023. AVeriTeC: A dataset for real-world claim verification with evidence from the web. arXiv preprint arXiv:2305.13117 (2023)

2023 arXiv
[17]

Ritvik Setty and Vinay Setty. 2024. QuestGen: Effectiveness of Question Gener- ation Methods for Fact-Checking Applications. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management (Boise, ID, USA) (CIKM ’24). Association for Computing M...

2024
[18]

Vinay Setty. 2024. FactCheck Editor: Multilingual Text Editor with End-to-End fact-checking. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (Washington DC, USA) (SIGIR ’24). Association for Computing Machinery...

2024
[19]

Vinay Setty. 2024. Surprising Efficacy of Fine-Tuned Transformers for Fact- Checking over Larger Language Models. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (Washington DC, USA) (SIGIR ’24). Association fo...

2024
[20]

Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2020. Mpnet: Masked and permuted pre-training for language understanding. Advances in neural information processing systems 33 (2020), 16857–16867

2020
[21]

Aniruddh Sriram, Fangyuan Xu, Eunsol Choi, and Greg Durrett. 2024. Contrastive Learning to Improve Retrieval for Real-World Fact Checking. InProceedings of the Seventh Fact Extraction and VERification Workshop (FEVER), Michael Schlichtkrull, Yulong Chen, Chenxi Whitehouse, Zhe...

2024
[22]

Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks , ...

2021
[23]

James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal
[24]

Venktesh V, Abhijit Anand, Avishek Anand, and Vinay Setty. 2024. QuanTemp: A real-world open-domain benchmark for fact-checking numerical claims. In Proceedings of the 47th International ACM SIGIR Conference on Research and Devel- opment in Information Retrieval (Washington DC...

2024
[25]

Voorhees

Ellen M. Voorhees. 1985. The cluster hypothesis revisited. In Proceedings of the 8th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (Montreal, Quebec, Canada) (SIGIR ’85). Association for Computing Machinery, New York, NY, USA, 188–196

1985
[26]

Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul Bennett, Junaid Ahmed, and Arnold Overwijk. 2020. Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval. arXiv:2007.00808 [cs.IR]

2020 arXiv
[27]

Xin Zhang, Yanzhao Zhang, Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang, Pengjun Xie, Fei Huang, Meishan Zhang, Wenjie Li, and Min Zhang. 2024. mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval. In P...

2024
[2018]

In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)

FEVER: a Large-scale Dataset for Fact Extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). Association for Computational Linguisti...

2018
[2021]

SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.

[1] [1]

Rami Aly, Zhijiang Guo, Michael Schlichtkrull, James Thorne, Andreas Vlachos, Christos Christodoulopoulos, Oana Cocarascu, and Arpit Mittal. 2021. FEVER- OUS: Fact Extraction and VERification Over Unstructured and Structured infor- mation. arXiv:2106.05707 [cs.CL]

arXiv 2021

[2] [2]

Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. 2018. MS MARCO: A Human Generated MAchine Reading COmprehension Dataset. arXiv:1611.09268 [cs.CL]

arXiv 2018

[3] [3]

Jifan Chen, Aniruddh Sriram, Eunsol Choi, and Greg Durrett. 2022. Gen- erating Literal and Implied Subquestions to Fact-check Complex Claims. arXiv:2205.06938 [cs.CL]

arXiv 2022

[4] [4]

Angela Fan, Aleksandra Piktus, Fabio Petroni, Guillaume Wenzek, Marzieh Saeidi, Andreas Vlachos, Antoine Bordes, and Sebastian Riedel. 2020. Generating Fact Checking Briefs. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computat...

work page 2020

[5] [5]

Anton Fogelberg and Jonas Nygren. 2023. Search Engine Evaluation

work page 2023

[6] [6]

Thibault Formal, Carlos Lassance, Benjamin Piwowarski, and Stéphane Clinchant

[7] [7]

Zhijiang Guo, Michael Schlichtkrull, and Andreas Vlachos. 2022. A survey on automated fact-checking. Transactions of the Association for Computational Linguistics 10 (2022), 178–206

2022

[8] [8]

Prakhar Gupta, Chien-Sheng Wu, Wenhao Liu, and Caiming Xiong. 2022. DialFact: A Benchmark for Fact-Checking in Dialogue. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (Eds.). Association for Computational Linguistics, Dublin, Ire...

work page 2022