Pith. sign in

REVIEW 8 cited by

Promptagator: Few-shot Dense Retrieval From 8 Examples

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2209.11755 v1 pith:7TLLTEKW submitted 2022-09-23 cs.CL cs.IR

Zhuyun Dai , Vincent Y. Zhao , Ji Ma , Yi Luan , Jianmo Ni , Jing Lu , Anton Bakalov , Kelvin Guu
show 2 more authors
Keith B. Hall Ming-Wei Chang
This is my paper · ORCID
classification cs.CLcs.IR
keywords examplesretrievaldatafew-shotpromptagatorquerytasktask-specific
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Much recent research on information retrieval has focused on how to transfer from one task (typically with abundant supervised data) to various other tasks where supervision is limited, with the implicit assumption that it is possible to generalize from one task to all the rest. However, this overlooks the fact that there are many diverse and unique retrieval tasks, each targeting different search intents, queries, and search domains. In this paper, we suggest to work on Few-shot Dense Retrieval, a setting where each task comes with a short description and a few examples. To amplify the power of a few examples, we propose Prompt-base Query Generation for Retriever (Promptagator), which leverages large language models (LLM) as a few-shot query generator, and creates task-specific retrievers based on the generated data. Powered by LLM's generalization ability, Promptagator makes it possible to create task-specific end-to-end retrievers solely based on a few examples {without} using Natural Questions or MS MARCO to train %question generators or dual encoders. Surprisingly, LLM prompting with no more than 8 examples allows dual encoders to outperform heavily engineered models trained on MS MARCO like ColBERT v2 by more than 1.2 nDCG on average on 11 retrieval sets. Further training standard-size re-rankers using the same generated data yields another 5.0 point nDCG improvement. Our studies determine that query generation can be far more effective than previously observed, especially when a small amount of task-specific knowledge is given.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Bekko Embedding: Parameter-Efficient Multilingual Retrieval with Ultra-Compact Encoders

    cs.IR 2026-07 conditional novelty 7.0 of 10

    Bekko a8m, with 7.7M active parameters, scores 56.2 on MMTEB Multilingual v2 Retrieval, beating mE5 models and BGE-M3, while a25m reaches 57.5, on par with gte-multilingual-base.

  2. Improving Rare Medication Recommendation with Counterfactual Data Augmentation and Large Language Models

    cs.IR 2026-07 conditional novelty 6.0 of 10

    GenRxR improves rare-medication recommendation using LLM-generated counterfactual data, instruction tuning, and sequential medication generation.

  3. Finding the Right Tables and Columns: A Benchmark and Corpus-Adaptive Embeddings for SQL Schema Retrieval

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Schema retrieval can be benchmarked as retrieval, and corpus-adaptive fine-tuning lifts a 305M embedder to 75.6 recall@10, rivaling 4–8B models.

  4. More Than Efficiency: Embedding Compression Improves Domain Adaptation in Dense Retrieval

    cs.IR 2026-01 conditional novelty 6.0 of 10

    Query-only PCA compression improves retrieval NDCG@10 in most tested model-dataset pairs at 90% retention, with gains largest in structured domains like SpartQA.

  5. Negative Matters: Multi-Granularity Hard-Negative Synthesis and Anchor-Token-Aware Pooling for Enhanced Text Embeddings

    cs.CL 2025-08 conditional novelty 6.0 of 10

    A new MTEB state-of-the-art for text embeddings is reported by combining multi-granularity LLM-generated hard negatives with curriculum training and an anchor-token-aware pooling method.

  6. DR.EHR: Dense Retrieval for Electronic Health Record with Knowledge Injection and Synthetic Data

    cs.IR 2025-07 conditional novelty 5.0 of 10

    DR.EHR, a two-stage trained dense retriever using BIOS knowledge injection and Llama-generated synthetic data, achieves state-of-the-art results on the CliniQ EHR retrieval benchmark.

  7. Iterative Augmentation with Summarization Refinement (IASR) Evaluation for Unstructured Survey data Modeling and Analysis

    cs.CL 2025-07 reject novelty 5.0 of 10

    The paper evaluates four LLMs as text augmenters and reports GPT-3.5 Turbo as the best, and that combining augmentation with GPT topic labels increases BERTopic's discovered topics from 5 to 20 with zero overlap.

  8. RankLLM: A Python Package for Reranking with LLMs

    cs.IR 2025-05 accept novelty 5.0 of 10

    RankLLM is an open-source Python package that modularly supports pointwise, pairwise, and listwise LLM rerankers, with integrated retrieval, evaluation, training, and response analysis, and reproduces results from Ran...

Pith tools