Sparse autoencoders applied to frozen dense retrievers extract Zipfian latent vocabularies that support BM25 scoring and match or exceed the base model's performance on some tasks.
hub
Document expansion by query prediction.arXiv preprint arXiv:1904.08375
20 Pith papers cite this work, alongside 210 external citations. Polarity classification is still indexing.
20
Pith papers citing it
210
external citations · Pith
abstract
One technique to improve the retrieval effectiveness of a search engine is to expand documents with terms that are related or representative of the documents' content.From the perspective of a question answering system, this might comprise questions the document can potentially answer. Following this observation, we propose a simple method that predicts which queries will be issued for a given document and then expands it with those predictions with a vanilla sequence-to-sequence model, trained using datasets consisting of pairs of query and relevant documents. By combining our method with a highly-effective re-ranking component, we achieve the state of the art in two retrieval tasks. In a latency-critical regime, retrieval results alone (without re-ranking) approach the effectiveness of more computationally expensive neural re-rankers but are much faster.
hub tools
citation-role summary
background 2
citation-polarity summary
roles
background 2polarities
background 2representative citing papers
LLM adaptive exploration via runtime code execution outperforms static query generation for information extraction from heterogeneous BIM models on the new ifc-bench v2 benchmark.
Transferring modern encoders to normalized (lowercased) vocabularies via geometric embedding initialization and activation calibration closes the performance gap in learned sparse retrieval, achieving 52.4 nDCG on BEIR.
Permutation-invariant fine-tuning (PI-FT) randomizes field order and applies dropout during embedding model training to eliminate sensitivity to serialization order, reducing order-change penalty from 7.4 to 0.2 nDCG@10 on a generated multilingual DevDataBench while outperforming zero-shot baselines
SPECTRA generates reproducible synthetic IR corpora up to 60,000 documents with controllable distractors, long-tail vocabulary, and graded relevance labels via a single-process Python prototype.
INO is an index-time method that uses the production RAG agent to iteratively create, test with queries and paraphrases, reflect on failures, and revise factual nuggets until they are discoverable and used correctly.
SPLADE models produce wacky expansion terms whose prevalence rises with larger vocabularies and falls with stricter sparsity; these terms primarily aid in-domain retrieval rather than out-of-domain generalization.
SIRA compresses multi-round exploratory retrieval into one corpus-discriminative BM25 action via LLM document enrichment, query-time term prediction, and corpus-statistic filtering, reporting top average performance on ten BEIR benchmarks and strong results on BrowseComp-Wikipedia without relevance
Verbal-R3 uses a verbal reranker to generate analytic narratives that guide retrieval and reasoning in LLMs, achieving SOTA results on complex QA benchmarks.
Synthetically formalizing information needs into topics with descriptions and narratives improves LLM relevance assessor agreement with humans and reduces over-labeling of relevant documents on TREC Deep Learning and Robust04.
SelRoute routes queries to type-specific retrieval pipelines, achieving Recall@5 of 0.800 with a 109M model on LongMemEval_M and outperforming LLM-augmented baselines including a strong zero-ML lexical method.
RankFlow deploys four LLM roles in sequence to rewrite queries, generate pseudo-answers, summarize passages, and rerank candidates, outperforming prior methods on TREC-DL, BEIR, and NovelEval.
Empirical study of RLAIF for portable query generation finds reward shaping controls performance more than optimizer choice and a rule-based reward floor yields +0.147 quality gain.
EMBER learns to retain budgeted, source-backed evidence capsules so long-horizon agents recover answer-relevant facts without rereading the full history.
BEATS generates and validates e-commerce attribute taxonomies from category-only catalogs via multi-stage LLM pipelines refined by human feedback, yielding measurable gains in dense retrieval models.
TextClusterLab introduces an LLM-driven generator for synthetic text clustering datasets with tunable attributes and a suitability benchmark for evaluation.
Larger 100K vocabularies in SPLADE models, especially those initialized with ESPLADE pretraining, improve retrieval effectiveness after pruning compared to 32K baselines while keeping similar efficiency.
The paper introduces a seed-guided contrastive framework that uses LLMs to generate realistic synthetic queries and topicality labels for cold-start natural language search, outperforming no-seed and InPars baselines on realism metrics and producing harder evaluation sets.
Position paper proposing to extend the OSIRRC replicability infrastructure with two performance benchmark scenarios, backed by a case study on neural re-ranking model runtimes.
citing papers explorer
-
Latent Terms: Dense Retrievers Contain Trivially Extractable BM25-ready Zipfian Vocabularies
Sparse autoencoders applied to frozen dense retrievers extract Zipfian latent vocabularies that support BM25 scoring and match or exceed the base model's performance on some tasks.
-
BIM Information Extraction Through LLM-based Adaptive Exploration
LLM adaptive exploration via runtime code execution outperforms static query generation for information extraction from heterogeneous BIM models on the new ifc-bench v2 benchmark.
-
Why Advanced Encoders Lag on Sparse Retrieval? The Answer and an Approach to Bridging Vocabulary Gaps
Transferring modern encoders to normalized (lowercased) vocabularies via geometric embedding initialization and activation calibration closes the performance gap in learned sparse retrieval, achieving 52.4 nDCG on BEIR.
-
Field Order Should Not Matter: Permutation-Invariant Embedding Model Fine-Tuning for Structured Metadata Retrieval
Permutation-invariant fine-tuning (PI-FT) randomizes field order and applies dropout during embedding model training to eliminate sensitivity to serialization order, reducing order-change penalty from 7.4 to 0.2 nDCG@10 on a generated multilingual DevDataBench while outperforming zero-shot baselines
-
SPECTRA: Synthetic IR Test Collections with Relevance Oracles and Controlled Distractor Diagnostics
SPECTRA generates reproducible synthetic IR corpora up to 60,000 documents with controllable distractors, long-tail vocabulary, and graded relevance labels via a single-process Python prototype.
-
Iterate Until Retrieved: Factual Nugget Optimization for Discoverable Continual Corrections in Agentic RAG
INO is an index-time method that uses the production RAG agent to iteratively create, test with queries and paraphrases, reflect on failures, and revise factual nuggets until they are discoverable and used correctly.
-
Understanding Wacky Weights: A Dissection of SPLADE's Learned Term Importance
SPLADE models produce wacky expansion terms whose prevalence rises with larger vocabularies and falls with stricter sparsity; these terms primarily aid in-domain retrieval rather than out-of-domain generalization.
-
Superintelligent Retrieval Agent: The Next Frontier of Agentic Retrieval
SIRA compresses multi-round exploratory retrieval into one corpus-discriminative BM25 action via LLM document enrichment, query-time term prediction, and corpus-statistic filtering, reporting top average performance on ten BEIR benchmarks and strong results on BrowseComp-Wikipedia without relevance
-
Verbal-R3: Verbal Reranker as the Missing Bridge between Retrieval and Reasoning
Verbal-R3 uses a verbal reranker to generate analytic narratives that guide retrieval and reasoning in LLMs, achieving SOTA results on complex QA benchmarks.
-
Formalized Information Needs Improve Large-Language-Model Relevance Judgments
Synthetically formalizing information needs into topics with descriptions and narratives improves LLM relevance assessor agreement with humans and reduces over-labeling of relevant documents on TREC Deep Learning and Robust04.
-
SelRoute: Query-Type-Aware Routing for Long-Term Conversational Memory Retrieval
SelRoute routes queries to type-specific retrieval pipelines, achieving Recall@5 of 0.800 with a 109M model on LongMemEval_M and outperforming LLM-augmented baselines including a strong zero-ML lexical method.
-
RankFlow: A Multi-Role Collaborative Reranking Workflow Utilizing Large Language Models
RankFlow deploys four LLM roles in sequence to rewrite queries, generate pseudo-answers, summarize passages, and rerank candidates, outperforming prior methods on TREC-DL, BEIR, and NovelEval.
-
Designing Reward Signals for Portable Query Generation: A Case Study in Industrial Semantic Job Search
Empirical study of RLAIF for portable query generation finds reward shaping controls performance more than optimizer choice and a rule-based reward floor yields +0.147 quality gain.
-
EMBER: Efficient Memory via Budgeted Evidence Retention for Long-Horizon Agents
EMBER learns to retain budgeted, source-backed evidence capsules so long-horizon agents recover answer-relevant facts without rereading the full history.
-
BEATS: Bootstrapping E-commerce Attribute Taxonomies for Search through Iterative Human-AI Collaboration
BEATS generates and validates e-commerce attribute taxonomies from category-only catalogs via multi-stage LLM pipelines refined by human feedback, yielding measurable gains in dense retrieval models.
-
TextClusterLab: An Integrated Framework for Reliable Text Clustering Studies
TextClusterLab introduces an LLM-driven generator for synthetic text clustering datasets with tunable attributes and a suitability benchmark for evaluation.
-
The Role of Vocabularies in Learning Sparse Representations for Ranking
Larger 100K vocabularies in SPLADE models, especially those initialized with ESPLADE pretraining, improve retrieval effectiveness after pruning compared to 32K baselines while keeping similar efficiency.
-
Bridging the Cold-Start Gap: LLM-Powered Synthetic Data Generation for Natural Language Search at Airbnb
The paper introduces a seed-guided contrastive framework that uses LLMs to generate realistic synthetic queries and topicality labels for cold-start natural language search, outperforming no-seed and InPars baselines on realism metrics and producing harder evaluation sets.
-
Let's measure run time! Extending the IR replicability infrastructure to include performance aspects
Position paper proposing to extend the OSIRRC replicability infrastructure with two performance benchmark scenarios, backed by a case study on neural re-ranking model runtimes.
- RICE-PO: Turning Retrieval Interactions into Credit Signals for Reasoning Agents