REVIEW 6 cited by
Large Language Models as Foundations for Next-Gen Dense Retrieval: A Comprehensive Empirical Assessment
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Pretrained language models like BERT and T5 serve as crucial backbone encoders for dense retrieval. However, these models often exhibit limited generalization capabilities and face challenges in improving in domain accuracy. Recent research has explored using large language models (LLMs) as retrievers, achieving SOTA performance across various tasks. Despite these advancements, the specific benefits of LLMs over traditional retrievers and the impact of different LLM configurations, such as parameter sizes, pretraining duration, and alignment processes on retrieval tasks remain unclear. In this work, we conduct a comprehensive empirical study on a wide range of retrieval tasks, including in domain accuracy, data efficiency, zero shot generalization, lengthy retrieval, instruction based retrieval, and multi task learning. We evaluate over 15 different backbone LLMs and non LLMs. Our findings reveal that larger models and extensive pretraining consistently enhance in domain accuracy and data efficiency. Additionally, larger models demonstrate significant potential in zero shot generalization, lengthy retrieval, instruction based retrieval, and multi task learning. These results underscore the advantages of LLMs as versatile and effective backbone encoders in dense retrieval, providing valuable insights for future research and development in this field.
Forward citations
Cited by 6 Pith papers
-
LLM-guided Hierarchical Search for End-to-end Reasoning Intensive Retrieval
An LLM directly traverses a hierarchical semantic index of a corpus, using calibrated path-relevance scores to retrieve documents for reasoning-intensive queries.
-
TongSearch-QR: Reinforced Query Reasoning for Retrieval
TongSearch-QR trains 1.5B and 7B models with GRPO and a frozen-embedding reward to rewrite queries, reaching 27.9 nDCG@10 on BRIGHT with BM25, above GPT-4o's 26.5.
-
O1 Embedder: Let Retrievers Think Before Action
A jointly trained retriever that first generates query thoughts and then encodes them improves accuracy on 12 retrieval benchmarks.
-
LUSIFER: Language Universal Space Integration for Enhanced Multilingual Embeddings with Large Language Models
A two-stage, English-only training method aligns XLM-R representations with Mistral-7B to produce multilingual embeddings, improving low-resource language scores by up to 22 points while losing about 10 points on English.
-
SEReDeEP: Hallucination Detection in Retrieval-Augmented Models via Semantic Entropy and Context-Parameter Fusion
SEReDeEP replaces ReDeEP's lexical scores with semantic entropy probe scores, but its claimed 3-10% accuracy gains are contradicted by its own tables and its probes are trained on the evaluation datasets.
-
LLMs are Also Effective Embedding Models: An In-depth Overview
A structured survey of using decoder-only LLMs as text embedding models, covering prompting, fine-tuning, data construction, benchmarks, and open problems.
Discussion (0). Continue with ORCID to comment.