Re-ranking retrieval candidates via a cross-encoder trained on continuous perturbation-based attribution scores improves citation faithfulness and gold-answer alignment in legal QA over semantic similarity.
hub Canonical reference
[Online; accessed 2025-07-26]
Canonical reference. 78% of citing Pith papers cite this work as background.
28
Pith papers citing it
23
external citations · Pith
Background
78% of classified citations
abstract
Large Language Models (LLMs) have demonstrated remarkable zero-shot generalization across various language-related tasks, including search engines. However, existing work utilizes the generative ability of LLMs for Information Retrieval (IR) rather than direct passage ranking. The discrepancy between the pre-training objectives of LLMs and the ranking objective poses another challenge. In this paper, we first investigate generative LLMs such as ChatGPT and GPT-4 for relevance ranking in IR. Surprisingly, our experiments reveal that properly instructed LLMs can deliver competitive, even superior results to state-of-the-art supervised methods on popular IR benchmarks. Furthermore, to address concerns about data contamination of LLMs, we collect a new test set called NovelEval, based on the latest knowledge and aiming to verify the model's ability to rank unknown knowledge. Finally, to improve efficiency in real-world applications, we delve into the potential for distilling the ranking capabilities of ChatGPT into small specialized models using a permutation distillation scheme. Our evaluation results turn out that a distilled 440M model outperforms a 3B supervised model on the BEIR benchmark. The code to reproduce our results is available at www.github.com/sunnweiwei/RankGPT.
hub tools
citation-role summary
background 7
baseline 1
dataset 1
citation-polarity summary
representative citing papers
ResRank unifies retrieval and listwise reranking by compressing passages to one token each, using residual connections and cosine-similarity scoring, achieving competitive effectiveness on TREC DL and BEIR benchmarks with zero generated tokens.
Self-Correcting RAG formalizes retrieval as MMKP to maximize information density under token limits and uses NLI-guided MCTS to validate faithfulness, raising accuracy and cutting hallucinations on six multi-hop QA and fact-checking datasets.
MARVEL reaches 37.9 nDCG@10 on the MM-BRIGHT benchmark by combining LLM query expansion, a reasoning-enhanced dense retriever, and GPT-4o CoT reranking, beating prior multimodal encoders by 10.3 points.
A survey that defines Compound AI Systems, proposes a multi-dimensional taxonomy based on component roles and orchestration strategies, reviews four foundational paradigms, and identifies key challenges for future research.
Scideator enables facet-based scientific ideation through LLM-driven extraction, human-guided recombination, analogous retrieval, and facet-grounded novelty verification, showing significantly higher creativity support than a baseline LLM in a user study with CS researchers.
Agentic search over NASA EO-KG yields a 47k-pair benchmark where neural scoring plus LLM reranking raises MRR by over 5x then an additional 28%.
NovelQR recommends contextually novel yet semantically coherent quotations using a generative label agent for deep-meaning interpretation and a token-level novelty estimator for reranking, with human judges rating its outputs as more appropriate, novel, and engaging than baselines.
Introduces the LLM ORDER BY semantic operator with algorithmic improvements, a semantic-aware external merge sort, and a budget-aware optimizer that selects near-optimal access paths for LLM-based ordering.
RankZephyr is a new open-source LLM that closes the effectiveness gap with GPT-4 for zero-shot listwise reranking while showing robustness to input ordering and document count.
Statement-level ranking with an LLM-extracted, paraphrase-clustered Amazon benchmark shows popularity baselines often beat SOTA models in item-level personalized explanation ranking.
EviRank extracts three evidences from a single LLM forward pass, aggregates them with reliable opinion pooling and position-aware calibration, then uses the result to optimize rankings, claiming SOTA on recommendation and uncertainty quantification across three datasets.
LRanker combines K-means candidate aggregation with graph-partitioned ensemble of query embeddings to improve LLM ranking accuracy and scalability on massive candidate pools, reporting 3-30% gains on RBench tasks up to 6.8M candidates.
Ocean4Rec uses offline LLM to create OCEAN profiles for items and time-decayed user profiles for request-time numeric reranking, improving NDCG@20 by 7.6% and 61.5% over base+recency in offline VOD evaluations.
BeLink applies set-wise instruction-tuning to generative LLMs at the re-ranking stage of biomedical entity linking, reporting 3-24% accuracy gains and reduced inference time versus prior methods.
RRK compresses documents to multi-token embeddings for efficient listwise reranking, enabling an 8B model to achieve 3x-18x speedups over smaller models with comparable or better effectiveness.
AdaRankLLM shows adaptive listwise reranking outperforms fixed-depth retrieval for most LLMs by acting as a noise filter for weak models and an efficiency optimizer for strong ones, with lower context use.
BRIDGE reaches 29.7 nDCG@10 on MM-BRIGHT by RL-aligning multimodal queries to text and using a reasoning retriever, beating multimodal encoders and, when combined with Nomic-Vision, exceeding the best text-only retriever at 33.3.
Lightweight proxy models deliver over 100x cost and latency savings for semantic AI queries in databases with accuracy preserved or improved on benchmarks up to 10M rows.
Search-R3 trains LLMs to output search embeddings as a direct product of step-by-step reasoning via supervised pre-training and a specialized RL environment that avoids full corpus re-encoding.
A 4B-model cascade for Thai KOL matching reaches 94.1% P@5 on 11 queries, matching a frontier model, with pairwise SimPO training transferring end-to-end while pointwise SFT+GRPO does not.
Fine-tuned LLM acts as ancillary advertiser predictor in production ads RecSys, augmenting retrieval and ranking with measurable offline and online gains.
Multi-objective LTR combining clicks, VLM labels, and locale boosting improves relevance and local content visibility across five growth markets.
The paper introduces ClinQueryAgent, a conversational agent that converts natural language queries into database queries for population health management while keeping patient data secure, and reports its use by 128 staff across 15 NHS practices covering 148,319 patients.
citing papers explorer
-
Re-Ranking Through an Attribution Lens for Citation Quality in Legal QA
Re-ranking retrieval candidates via a cross-encoder trained on continuous perturbation-based attribution scores improves citation faithfulness and gold-answer alignment in legal QA over semantic similarity.
-
ResRank: Unifying Retrieval and Listwise Reranking via End-to-End Joint Training with Residual Passage Compression
ResRank unifies retrieval and listwise reranking by compressing passages to one token each, using residual connections and cosine-similarity scoring, achieving competitive effectiveness on TREC DL and BEIR benchmarks with zero generated tokens.
-
Self-Correcting RAG: Enhancing Faithfulness via MMKP Context Selection and NLI-Guided MCTS
Self-Correcting RAG formalizes retrieval as MMKP to maximize information density under token limits and uses NLI-guided MCTS to validate faithfulness, raising accuracy and cutting hallucinations on six multi-hop QA and fact-checking datasets.
-
MARVEL: Multimodal Adaptive Reasoning-intensiVe Expand-rerank and retrievaL
MARVEL reaches 37.9 nDCG@10 on the MM-BRIGHT benchmark by combining LLM query expansion, a reasoning-enhanced dense retriever, and GPT-4o CoT reranking, beating prior multimodal encoders by 10.3 points.
-
From Standalone LLMs to Integrated Intelligence: A Survey of Compound Al Systems
A survey that defines Compound AI Systems, proposes a multi-dimensional taxonomy based on component roles and orchestration strategies, reviews four foundational paradigms, and identifies key challenges for future research.
-
Human-LLM Compound System for Scientific Ideation through Facet Recombination and Novelty Evaluation
Scideator enables facet-based scientific ideation through LLM-driven extraction, human-guided recombination, analogous retrieval, and facet-grounded novelty verification, showing significantly higher creativity support than a baseline LLM in a user study with CS researchers.
-
Bringing Agentic Search to Earth Observation Data Discovery
Agentic search over NASA EO-KG yields a 47k-pair benchmark where neural scoring plus LLM reranking raises MRR by over 5x then an additional 28%.
-
What Makes an Ideal Quote? Recommending "Unexpected yet Rational" Quotations via Novelty
NovelQR recommends contextually novel yet semantically coherent quotations using a generative label agent for deep-meaning interpretation and a token-level novelty estimator for reranking, with human judges rating its outputs as more appropriate, novel, and engaging than baselines.
-
Access Paths for Efficient Ordering with Large Language Models
Introduces the LLM ORDER BY semantic operator with algorithmic improvements, a semantic-aware external merge sort, and a budget-aware optimizer that selects near-optimal access paths for LLM-based ordering.
-
RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!
RankZephyr is a new open-source LLM that closes the effectiveness gap with GPT-4 for zero-shot listwise reranking while showing robustness to input ordering and document count.
-
Rank, Don't Generate: Statement-level Ranking for Explainable Recommendation
Statement-level ranking with an LLM-extracted, paraphrase-clustered Amazon benchmark shows popularity baselines often beat SOTA models in item-level personalized explanation ranking.
-
EviRank: Evidence-Based Confidence Estimation for LLM-Based Ranking
EviRank extracts three evidences from a single LLM forward pass, aggregates them with reliable opinion pooling and position-aware calibration, then uses the result to optimize rankings, claiming SOTA on recommendation and uncertainty quantification across three datasets.
-
LRanker: LLM Ranker for Massive Candidates
LRanker combines K-means candidate aggregation with graph-partitioned ensemble of query embeddings to improve LLM ranking accuracy and scalability on massive candidate pools, reporting 3-30% gains on RBench tasks up to 6.8M candidates.
-
Ocean4Rec: Offline LLM-Derived OCEAN Profiles for Request-Time VOD Reranking
Ocean4Rec uses offline LLM to create OCEAN profiles for items and time-decayed user profiles for request-time numeric reranking, improving NDCG@20 by 7.6% and 61.5% over base+recency in offline VOD evaluations.
-
BeLink: Biomedical Entity Linking Meets Generative Re-Ranking
BeLink applies set-wise instruction-tuning to generative LLMs at the re-ranking stage of biomedical entity linking, reporting 3-24% accuracy gains and reduced inference time versus prior methods.
-
Efficient Listwise Reranking with Compressed Document Representations
RRK compresses documents to multi-token embeddings for efficient listwise reranking, enabling an 8B model to achieve 3x-18x speedups over smaller models with comparable or better effectiveness.
-
Rethinking the Necessity of Adaptive Retrieval-Augmented Generation through the Lens of Adaptive Listwise Ranking
AdaRankLLM shows adaptive listwise reranking outperforms fixed-depth retrieval for most LLMs by acting as a noise filter for weak models and an efficiency optimizer for strong ones, with lower context use.
-
BRIDGE: Multimodal-to-Text Retrieval via Reinforcement-Learned Query Alignment
BRIDGE reaches 29.7 nDCG@10 on MM-BRIGHT by RL-aligning multimodal queries to text and using a reasoning retriever, beating multimodal encoders and, when combined with Nomic-Vision, exceeding the best text-only retriever at 33.3.
-
100x Cost & Latency Reduction: Performance Analysis of AI Query Approximation using Lightweight Proxy Models
Lightweight proxy models deliver over 100x cost and latency savings for semantic AI queries in databases with accuracy preserved or improved on benchmarks up to 10M rows.
-
Search-R3: Unifying Reasoning and Embedding in Large Language Models
Search-R3 trains LLMs to output search embeddings as a direct product of step-by-step reasoning via supervised pre-training and a specialized RL environment that avoids full corpus re-encoding.
-
InfluMatch: Frontier-Quality KOL Search at 4B-Model Cost
A 4B-model cascade for Thai KOL matching reaches 94.1% P@5 on 11 queries, matching a frontier model, with pairwise SimPO training transferring end-to-end while pointwise SFT+GRPO does not.
-
Fine-Tuned LLM as a Complementary Predictor Improving Ads System
Fine-tuned LLM acts as ancillary advertiser predictor in production ads RecSys, augmenting retrieval and ranking with measurable offline and online gains.
-
Localization Boosting for Growth Markets: Mitigating Cross-Locale Behavioral Bias in Learning-to-Rank
Multi-objective LTR combining clicks, VLM labels, and locale boosting improves relevance and local content visibility across five growth markets.
-
ClinQueryAgent: A Conversational Agent for Population Health Management
The paper introduces ClinQueryAgent, a conversational agent that converts natural language queries into database queries for population health management while keeping patient data secure, and reports its use by 128 staff across 15 NHS practices covering 148,319 patients.
-
Multimodal and Multiscale Spatial-Temporal Semantic Search and Recommendation with AI Foundation Models
Multimodal framework using LLMs and VLMs with CAMERA fusion and ASTRA re-ranking outperforms text-only baselines on Local Environmental Observer Network dataset for spatiotemporal semantic search.
-
A Reproducibility Study of Metacognitive Retrieval-Augmented Generation
MetaRAG is only partially reproducible with lower absolute scores than originally reported, gains substantially from reranking, and shows greater robustness than SIM-RAG under extended retrieval features.
-
5ting at SemEval-2026 Task 8: Strong End-to-End Multi-Turn RAG via LLM-Based Reranking and Faithfulness Control
5ting achieves nDCG@5 of 0.4719 on Task A and harmonic score 0.5597 with RL_F 0.7692 on Task C for multi-turn RAG via standard dense retrieval plus LLM reranking and faithfulness constraints.
-
Reproducing Adaptive Reranking for Reasoning-Intensive IR
Reproducing GAR on BRIGHT shows it boosts reasoning-intensive retrieval effectiveness with low overhead when the reranker's signal quality is strong.