HeadRank lifts preference optimization into attention space via entropy-regularized head selection and distribution regularizers to sharpen discriminability for efficient listwise reranking.
Open-source large language models are strong zero-shot query likelihood models for document ranking
7 Pith papers cite this work. Polarity classification is still indexing.
7
Pith papers citing it
abstract
In the field of information retrieval, Query Likelihood Models (QLMs) rank documents based on the probability of generating the query given the content of a document. Recently, advanced large language models (LLMs) have emerged as effective QLMs, showcasing promising ranking capabilities. This paper focuses on investigating the genuine zero-shot ranking effectiveness of recent LLMs, which are solely pre-trained on unstructured text data without supervised instruction fine-tuning. Our findings reveal the robust zero-shot ranking ability of such LLMs, highlighting that additional instruction fine-tuning may hinder effectiveness unless a question generation task is present in the fine-tuning dataset. Furthermore, we introduce a novel state-of-the-art ranking system that integrates LLM-based QLMs with a hybrid zero-shot retriever, demonstrating exceptional effectiveness in both zero-shot and few-shot scenarios. We make our codebase publicly available at https://github.com/ielab/llm-qlm.
citation-role summary
background 1
citation-polarity summary
roles
background 1polarities
background 1representative citing papers
LLM-written pipelines and LLM-generated labels are distilled into one small instruction-following model that performs classification and span extraction cheaply at corpus scale.
MM-R5, a 7B multimodal re-ranker trained with SFT and GRPO, achieves state-of-the-art page-level recall on MMDocIR by generating per-page reasoning chains.
Poisoning the knowledge base of a retrieval-augmented code generator with vulnerable snippets raises the vulnerability rate of the model's output, with the size of the rise depending on the retriever, the model, and the number of injected examples.
PaSa, a two-agent LLM system trained with session-level RL, reports substantially higher recall than existing academic search baselines on complex paper-finding queries.
LLM-dependent memory operations in agents and RAG consume orders of magnitude more energy than vector methods, and resource-constrained hardware pays higher energy for lower quality.
A survey of RAG paradigms, components, benchmarks, and challenges for improving LLMs on knowledge-intensive tasks.
citing papers explorer
-
HeadRank: Decoding-Free Passage Reranking via Preference-Aligned Attention Heads
HeadRank lifts preference optimization into attention space via entropy-regularized head selection and distribution regularizers to sharpen discriminability for efficient listwise reranking.
-
A Tale of LLMs and Induced Small Proxies: Scalable Small Language Models for Knowledge Mining
LLM-written pipelines and LLM-generated labels are distilled into one small instruction-following model that performs classification and span extraction cheaply at corpus scale.
-
MM-R5: MultiModal Reasoning-Enhanced ReRanker via Reinforcement Learning for Document Retrieval
MM-R5, a 7B multimodal re-ranker trained with SFT and GRPO, achieves state-of-the-art page-level recall on MMDocIR by generating per-page reasoning chains.
-
Exploring the Security Threats of Knowledge Base Poisoning in Retrieval-Augmented Code Generation
Poisoning the knowledge base of a retrieval-augmented code generator with vulnerable snippets raises the vulnerability rate of the model's output, with the size of the rise depending on the retriever, the model, and the number of injected examples.
-
PaSa: An LLM Agent for Comprehensive Academic Paper Search
PaSa, a two-agent LLM system trained with session-level RL, reports substantially higher recall than existing academic search baselines on complex paper-finding queries.
-
Addressing the sustainable AI trilemma: a case study on LLM agents and RAG
LLM-dependent memory operations in agents and RAG consume orders of magnitude more energy than vector methods, and resource-constrained hardware pays higher energy for lower quality.
-
Retrieval-Augmented Generation for Large Language Models: A Survey
A survey of RAG paradigms, components, benchmarks, and challenges for improving LLMs on knowledge-intensive tasks.