REVIEW 5 cited by
Task-aware Retrieval with Instructions
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
We study the problem of retrieval with instructions, where users of a retrieval system explicitly describe their intent along with their queries. We aim to develop a general-purpose task-aware retrieval system using multi-task instruction tuning, which can follow human-written instructions to find the best documents for a given query. We introduce the first large-scale collection of approximately 40 retrieval datasets with instructions, BERRI, and present TART, a multi-task retrieval system trained on BERRI with instructions. TART shows strong capabilities to adapt to a new retrieval task via instructions and advances the state of the art on two zero-shot retrieval benchmarks, BEIR and LOTTE, outperforming models up to three times larger. We further introduce a new evaluation setup, X^2-Retrieval to better reflect real-world scenarios, where diverse domains and tasks are pooled and a system needs to find documents aligning users' intents. In this setup, TART significantly outperforms competitive baselines, further demonstrating the effectiveness of guiding retrieval with instructions.
Forward citations
Cited by 5 Pith papers
-
FitText: Evolving Agent Tool Ecologies via Memetic Retrieval
FitText embeds memetic evolutionary retrieval inside the agent's reasoning loop to iteratively refine pseudo-tool descriptions, raising retrieval rank from 8.81 to 2.78 on ToolRet and pass rate to 0.73 on StableToolBench.
-
SGIC: A Self-Guided Iterative Calibration Framework for RAG
SGIC feeds a model's own uncertainty scores back into its prompt for several calibration rounds and improves RAG accuracy on HotpotQA, NQ, and GSM8K.
-
Towards A Generalist Code Embedding Model Based On Massive Data Synthesis
CodeR, a 1.5B code embedding model trained on 2.9M synthetic retrieval triples through a three-stage curriculum, reports average NDCG@10 of 81.77 on CoIR and 72.8 on CodeRAG, above every baseline it lists.
-
mFollowIR: a Multilingual Benchmark for Instruction Following in Retrieval
The paper introduces mFollowIR, a multilingual instruction-following retrieval benchmark across Russian, Chinese, and Persian, and finds that English instruction-trained models transfer cross-lingually but struggle in...
-
Towards Better Instruction Following Retrieval Models
A new training corpus and embedding model improve instruction-following p-MRR by up to 9 points on FollowIR, MAIR, and Bright benchmarks.
Discussion (0). Continue with ORCID to comment.