Pith. sign in

REVIEW 21 cited by

Promptagator: Few-shot Dense Retrieval From 8 Examples

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2209.11755 v1 pith:7TLLTEKW submitted 2022-09-23 cs.CL cs.IR

Zhuyun Dai , Vincent Y. Zhao , Ji Ma , Yi Luan , Jianmo Ni , Jing Lu , Anton Bakalov , Kelvin Guu
show 2 more authors
Keith B. Hall Ming-Wei Chang
This is my paper · ORCID
classification cs.CLcs.IR
keywords examplesretrievaldatafew-shotpromptagatorquerytasktask-specific
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Much recent research on information retrieval has focused on how to transfer from one task (typically with abundant supervised data) to various other tasks where supervision is limited, with the implicit assumption that it is possible to generalize from one task to all the rest. However, this overlooks the fact that there are many diverse and unique retrieval tasks, each targeting different search intents, queries, and search domains. In this paper, we suggest to work on Few-shot Dense Retrieval, a setting where each task comes with a short description and a few examples. To amplify the power of a few examples, we propose Prompt-base Query Generation for Retriever (Promptagator), which leverages large language models (LLM) as a few-shot query generator, and creates task-specific retrievers based on the generated data. Powered by LLM's generalization ability, Promptagator makes it possible to create task-specific end-to-end retrievers solely based on a few examples {without} using Natural Questions or MS MARCO to train %question generators or dual encoders. Surprisingly, LLM prompting with no more than 8 examples allows dual encoders to outperform heavily engineered models trained on MS MARCO like ColBERT v2 by more than 1.2 nDCG on average on 11 retrieval sets. Further training standard-size re-rankers using the same generated data yields another 5.0 point nDCG improvement. Our studies determine that query generation can be far more effective than previously observed, especially when a small amount of task-specific knowledge is given.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 21 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Bekko Embedding: Parameter-Efficient Multilingual Retrieval with Ultra-Compact Encoders

    cs.IR 2026-07 conditional novelty 7.0 of 10

    Bekko a8m, with 7.7M active parameters, scores 56.2 on MMTEB Multilingual v2 Retrieval, beating mE5 models and BGE-M3, while a25m reaches 57.5, on par with gte-multilingual-base.

  2. Improving Rare Medication Recommendation with Counterfactual Data Augmentation and Large Language Models

    cs.IR 2026-07 conditional novelty 6.0 of 10

    GenRxR improves rare-medication recommendation using LLM-generated counterfactual data, instruction tuning, and sequential medication generation.

  3. Finding the Right Tables and Columns: A Benchmark and Corpus-Adaptive Embeddings for SQL Schema Retrieval

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Schema retrieval can be benchmarked as retrieval, and corpus-adaptive fine-tuning lifts a 305M embedder to 75.6 recall@10, rivaling 4–8B models.

  4. More Than Efficiency: Embedding Compression Improves Domain Adaptation in Dense Retrieval

    cs.IR 2026-01 conditional novelty 6.0 of 10

    Query-only PCA compression improves retrieval NDCG@10 in most tested model-dataset pairs at 90% retention, with gains largest in structured domains like SpartQA.

  5. Negative Matters: Multi-Granularity Hard-Negative Synthesis and Anchor-Token-Aware Pooling for Enhanced Text Embeddings

    cs.CL 2025-08 conditional novelty 6.0 of 10

    A new MTEB state-of-the-art for text embeddings is reported by combining multi-granularity LLM-generated hard negatives with curriculum training and an anchor-token-aware pooling method.

  6. Towards A Generalist Code Embedding Model Based On Massive Data Synthesis

    cs.IR 2025-05 conditional novelty 6.0 of 10

    CodeR, a 1.5B code embedding model trained on 2.9M synthetic retrieval triples through a three-stage curriculum, reports average NDCG@10 of 81.77 on CoIR and 72.8 on CodeRAG, above every baseline it lists.

  7. Incorporating Verification Standards for Security Requirements Generation from Functional Specifications

    cs.SE 2025-05 conditional novelty 6.0 of 10

    F2SRD retrieves relevant OWASP ASVS verification requirements for each functional requirement and uses them to prompt GPT-4, producing security requirements judged more inspired, diverse, and specific than two baselines.

  8. RbFT: Robust Fine-tuning for Retrieval-Augmented Generation against Retrieval Defects

    cs.CL 2025-01 conditional novelty 6.0 of 10

    Fine-tuning an LLM with defect detection and utility extraction tasks makes it more robust to noisy, irrelevant, and counterfactual documents in retrieval-augmented generation.

  9. Streaming Video Understanding and Multi-round Interaction with Memory-enhanced Knowledge

    cs.CV 2025-01 conditional novelty 6.0 of 10

    StreamChat uses hierarchical memory and three parallel threads to enable real-time multi-round video question answering, and StreamBench is a new benchmark for evaluating such streaming systems.

  10. Explainable LLM-driven Multi-dimensional Distillation for E-Commerce Relevance Learning

    cs.IR 2024-11 conditional novelty 6.0 of 10

    An explainable chain-of-thought LLM for relevance is distilled into small online models via score and reasoning supervision, improving e-commerce search ad relevance on Taobao.

  11. DR.EHR: Dense Retrieval for Electronic Health Record with Knowledge Injection and Synthetic Data

    cs.IR 2025-07 conditional novelty 5.0 of 10

    DR.EHR, a two-stage trained dense retriever using BIOS knowledge injection and Llama-generated synthetic data, achieves state-of-the-art results on the CliniQ EHR retrieval benchmark.

  12. Iterative Augmentation with Summarization Refinement (IASR) Evaluation for Unstructured Survey data Modeling and Analysis

    cs.CL 2025-07 reject novelty 5.0 of 10

    The paper evaluates four LLMs as text augmenters and reports GPT-3.5 Turbo as the best, and that combining augmentation with GPT topic labels increases BERTopic's discovered topics from 5 to 20 with zero overlap.

  13. Research on Graph-Retrieval Augmented Generation Based on Historical Text Knowledge Graphs

    cs.CL 2025-06 conditional novelty 5.0 of 10

    The authors construct a character-relationship dataset from the First Four Histories and show that a fine-tuned Qwen model plus GraphRAG improves relation extraction and question-answering for classical Chinese text.

  14. RankLLM: A Python Package for Reranking with LLMs

    cs.IR 2025-05 accept novelty 5.0 of 10

    RankLLM is an open-source Python package that modularly supports pointwise, pairwise, and listwise LLM rerankers, with integrated retrieval, evaluation, training, and response analysis, and reproduces results from Ran...

  15. QBD-RankedDataGen: Generating Custom Ranked Datasets for Improving Query-By-Document Search Using LLM-Reranking with Reduced Human Effort

    cs.IR 2025-05 conditional novelty 5.0 of 10

    LLM-generated rankings for query-by-document search do not improve BM25 tuning over default parameters unless validated by human ground truth.

  16. Can Generative LLMs Create Query Variants for Test Collections? An Exploratory Study

    cs.IR 2025-01 conditional novelty 5.0 of 10

    GPT-3.5 generated query variants from information-need descriptions that overlap substantially with human variants in retrieved relevant documents, up to 71.1% at pool depth 100.

  17. ASRank: Zero-Shot Re-Ranking with Answer Scent for Document Retrieval

    cs.CL 2025-01 conditional novelty 5.0 of 10

    ASRank re-ranks retrieved documents by scoring how well each document supports a zero-shot answer scent generated by a large LLM, beating UPR and RankGPT on several QA datasets.

  18. Arctic-Embed 2.0: Multilingual Retrieval Without Compromise

    cs.CL 2024-12 conditional novelty 5.0 of 10

    Arctic-Embed 2.0 delivers open multilingual embedding models with competitive MTEB-R and CLEF retrieval scores and strong 256-dimension MRL compression, plus new evidence on cross-lingual transfer.

  19. SyNeg: LLM-Driven Synthetic Hard-Negatives for Dense Retrieval

    cs.IR 2024-12 conditional novelty 4.0 of 10

    LLM-generated synthetic hard negatives, combined with retrieved negatives in a hybrid mix, improve dense retrieval accuracy on BEIR benchmarks.

  20. Linq-Embed-Mistral Technical Report

    cs.CL 2024-12 conditional novelty 4.0 of 10

    Linq-Embed-Mistral, a fine-tuned Mistral-7B embedding model, reaches an MTEB average of 68.2 and a retrieval score of 60.2, ranking first on retrieval as of May 29, 2024.

  21. A Survey on Retrieval And Structuring Augmented Generation with Large Language Models

    cs.CL 2025-09 conditional novelty 2.0 of 10

    The paper presents a comprehensive survey and taxonomy of RAS methods, covering retrieval, text structuring, and LLM integration.

Pith tools