LLM rerankers can internally predict ranking quality via self-consistency of sampled outputs, matching SOTA external QPP while direct confidence is overconfident; supervised token-efficient methods improve calibration.
Overview of the TREC 2022 deep learning track
4 Pith papers cite this work. Polarity classification is still indexing.
4
Pith papers citing it
abstract
This is the fourth year of the TREC Deep Learning track. As in previous years, we leverage the MS MARCO datasets that made hundreds of thousands of human annotated training labels available for both passage and document ranking tasks. In addition, this year we also leverage both the refreshed passage and document collections that were released last year leading to a nearly $16$ times increase in the size of the passage collection and nearly four times increase in the document collection size. Unlike previous years, in 2022 we mainly focused on constructing a more complete test collection for the passage retrieval task, which has been the primary focus of the track. The document ranking task was kept as a secondary task, where document-level labels were inferred from the passage-level labels. Our analysis shows that similar to previous years, deep neural ranking models that employ large scale pretraining continued to outperform traditional retrieval methods. Due to the focusing our judging resources on passage judging, we are more confident in the quality of this year's queries and judgments, with respect to our ability to distinguish between runs and reuse the dataset in future. We also see some surprises in overall outcomes. Some top-performing runs did not do dense retrieval. Runs that did single-stage dense retrieval were not as competitive this year as they were last year.
citation-role summary
background 1
dataset 1
citation-polarity summary
fields
cs.IR 4years
2026 4representative citing papers
Transferring modern encoders to normalized (lowercased) vocabularies via geometric embedding initialization and activation calibration closes the performance gap in learned sparse retrieval, achieving 52.4 nDCG on BEIR.
Retrieved query variants from logs combined with LLM-augmented generation improve unsupervised QPP accuracy by up to 30% for neural rankers on TREC DL'19 and DL'20.
Entity-oriented rerankers show large gains in conditional evaluation but not open-world evaluation on Robust04; the paper attributes this to a coverage–discrimination tradeoff in the entity channel, not to model architecture.
citing papers explorer
-
Can LLM Rerankers Predict Their Own Ranking Performance?
LLM rerankers can internally predict ranking quality via self-consistency of sampled outputs, matching SOTA external QPP while direct confidence is overconfident; supervised token-efficient methods improve calibration.
-
Why Advanced Encoders Lag on Sparse Retrieval? The Answer and an Approach to Bridging Vocabulary Gaps
Transferring modern encoders to normalized (lowercased) vocabularies via geometric embedding initialization and activation calibration closes the performance gap in learned sparse retrieval, achieving 52.4 nDCG on BEIR.
-
RAQG-QPP: Query Performance Prediction with Retrieved Query Variants and Retrieval Augmented Query Generation
Retrieved query variants from logs combined with LLM-augmented generation improve unsupervised QPP accuracy by up to 30% for neural rankers on TREC DL'19 and DL'20.
-
Entities as Retrieval Signals: A Systematic Study of Coverage, Supervision, and Evaluation in Entity-Oriented Ranking
Entity-oriented rerankers show large gains in conditional evaluation but not open-world evaluation on Robust04; the paper attributes this to a coverage–discrimination tradeoff in the entity channel, not to model architecture.