REVIEW 6 cited by
Top-Down Partitioning for Efficient List-Wise Ranking
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large Language Models (LLMs) have significantly impacted many facets of natural language processing and information retrieval. Unlike previous encoder-based approaches, the enlarged context window of these generative models allows for ranking multiple documents at once, commonly called list-wise ranking. However, there are still limits to the number of documents that can be ranked in a single inference of the model, leading to the broad adoption of a sliding window approach to identify the k most relevant items in a ranked list. We argue that the sliding window approach is not well-suited for list-wise re-ranking because it (1) cannot be parallelized in its current form, (2) leads to redundant computational steps repeatedly re-scoring the best set of documents as it works its way up the initial ranking, and (3) prioritizes the lowest-ranked documents for scoring rather than the highest-ranked documents by taking a bottom-up approach. Motivated by these shortcomings and an initial study that shows list-wise rankers are biased towards relevant documents at the start of their context window, we propose a novel algorithm that partitions a ranking to depth k and processes documents top-down. Unlike sliding window approaches, our algorithm is inherently parallelizable due to the use of a pivot element, which can be compared to documents down to an arbitrary depth concurrently. In doing so, we reduce the number of expected inference calls by around 33% when ranking at depth 100 while matching the performance of prior approaches across multiple strong re-rankers.
Forward citations
Cited by 6 Pith papers
-
Likert or Not: LLM Absolute Relevance Judgments on Fine-Grained Ordinal Scales
Pointwise LLM scoring with an 11-point ordinal scale is statistically competitive with listwise ranking for 31 of 40 model-dataset combinations on NDCG@10.
-
Think When Needed: Model-Aware Reasoning Routing for LLM-based Ranking
A per-instance router trained to predict the utility gain of reasoning decides when an LLM should think before ranking, improving NDCG by up to 6.3% while cutting generation tokens by up to 75%.
-
JointRank: Rank Large Set with Single Pass
JointRank partitions candidates into overlapping blocks, ranks each block in parallel with an LLM, and reconstructs a global ranking by aggregating the resulting pairwise comparisons.
-
Precise Zero-Shot Pointwise Ranking with LLMs through Post-Aggregated Global Context Information
A summary-based anchor document enables contrastive pointwise scoring that, when averaged with ordinary pointwise scores, improves zero-shot LLM reranking.
-
Reinforcement Speculative Decoding for Fast Ranking
RSD uses reinforcement learning to train an agent that iteratively modifies an LLM's ranking under a fixed call budget, outperforming single-token and speculative-decoding baselines on IR and RS datasets.
-
Towards a Relevance Posterior in Neural Information Access
Fusing a cached query-independent document-quality prior with BM25 or re-rankers improves TREC DL nDCG, with largest gains for RankZephyr, under a prior–likelihood view of retrieval.
Discussion (0). Continue with ORCID to comment.