Pith. sign in

REVIEW 11 cited by

Dense X Retrieval: What Retrieval Granularity Should We Use?

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2312.06648 v3 pith:6J4EYKOM submitted 2023-12-11 cs.CL cs.AIcs.IR

Tong Chen , Hongwei Wang , Sihao Chen , Wenhao Yu , Kaixin Ma , Xinran Zhao , Hongming Zhang , Dong Yu This is my paper · ORCID
classification cs.CLcs.AIcs.IR
keywords retrievaldensetaskscorpusunitunitschoicedistinct
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Dense retrieval has become a prominent method to obtain relevant context or world knowledge in open-domain NLP tasks. When we use a learned dense retriever on a retrieval corpus at inference time, an often-overlooked design choice is the retrieval unit in which the corpus is indexed, e.g. document, passage, or sentence. We discover that the retrieval unit choice significantly impacts the performance of both retrieval and downstream tasks. Distinct from the typical approach of using passages or sentences, we introduce a novel retrieval unit, proposition, for dense retrieval. Propositions are defined as atomic expressions within text, each encapsulating a distinct factoid and presented in a concise, self-contained natural language format. We conduct an empirical comparison of different retrieval granularity. Our experiments reveal that indexing a corpus by fine-grained units such as propositions significantly outperforms passage-level units in retrieval tasks. Moreover, constructing prompts with fine-grained retrieved units for retrieval-augmented language models improves the performance of downstream QA tasks given a specific computation budget.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MoR: Better Handling Diverse Queries with a Mixture of Sparse, Dense, and Human Retrievers

    cs.IR 2025-06 conditional novelty 6.0 of 10

    A zero-shot mixture of sparse, dense, and simulated human retrievers, weighted by pre- and post-retrieval geometry signals, beats individual small retrievers and 7B LLM retrievers on four scientific retrieval benchmarks.

  2. AITEE -- Agentic Tutor for Electrical Engineering

    cs.CY 2025-05 conditional novelty 6.0 of 10

    AITEE combines YOLO circuit detection, graph-neural-network-based retrieval of lecture material, SPICE simulation, and Socratic prompting to help LLMs answer first-semester electrical engineering circuit questions mor...

  3. LLMs as Better Recommenders with Natural Language Collaborative Signals: A Self-Assessing Retrieval Approach

    cs.IR 2025-05 conditional novelty 6.0 of 10

    SCORE retrieves similar users' behaviors, uses an LLM self-assessment to rerank them, and injects the top ones in natural language into an LLM prompt, improving recommendation accuracy on two datasets.

  4. Streaming Video Understanding and Multi-round Interaction with Memory-enhanced Knowledge

    cs.CV 2025-01 conditional novelty 6.0 of 10

    StreamChat uses hierarchical memory and three parallel threads to enable real-time multi-round video question answering, and StreamBench is a new benchmark for evaluating such streaming systems.

  5. PIKE-RAG: sPecIalized KnowledgE and Rationale Augmented Generation

    cs.CL 2025-01 conditional novelty 6.0 of 10

    PIKE-RAG improves multihop and legal QA by atomizing chunks into LLM-generated question indexes and using knowledge-aware iterative task decomposition, outperforming tested RAG baselines.

  6. Passage Segmentation of Documents for Extractive Question Answering

    cs.CL 2025-01 conditional novelty 6.0 of 10

    LGMGC splits documents using language-model EOS probabilities and multi-granular sub-chunking, reporting improved retrieval and RAG question-answering scores over tested chunking baselines.

  7. SiReRAG: Indexing Similar and Related Information for Multihop Reasoning

    cs.CL 2024-12 conditional novelty 6.0 of 10

    SiReRAG indexes a corpus with both a similarity tree and an entity-based relatedness tree, improving average multihop QA F1 by about 1.9 points over prior RAG indexing methods.

  8. SlimRAG: Retrieval without Graphs via Entity-Aware Context Selection

    cs.IR 2025-06 conditional novelty 5.0 of 10

    SlimRAG shows that an entity-aware inverted index without graphs can match or beat graph-based RAG retrieval on HotpotQA while using far fewer index tokens.

  9. A Survey of LLM $\times$ DATA

    cs.DB 2025-05 conditional novelty 5.0 of 10

    A comprehensive survey of the bidirectional links between LLMs and data management, organized as DATA4LLM and LLM4DATA with a new 'IaaS' data-quality framework.

  10. Multiple Abstraction Level Retrieve Augment Generation

    cs.CL 2025-01 conditional novelty 4.0 of 10

    MAL-RAG retrieves document, section, paragraph, and multi-sentence chunks together and claims a 25.7% improvement in AI-judged answer correctness on glycoscience questions over single-level RAG.

  11. Top Ten Challenges Towards Agentic Neural Graph Databases

    cs.AI 2025-01 unverdicted novelty 3.0 of 10

    Agentic Neural Graph Databases are proposed as graph databases with autonomous query construction, neural query execution, and continuous learning, with ten open challenges listed.

Pith tools