Pith. sign in

REVIEW 5 major objections 7 minor 1 cited by

Ranking Free RAG: Replacing Re-ranking with Selection in RAG for Sensitive Domains

T0 review · 5 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read METEORA replaces RAG re-ranking with rationale-driven selection, reporting higher recall and precision, 80% less evidence, and 4.4x stronger poison defense.

desk verdict A coherent and potentially useful RAG selection pipeline whose headline claims rest on unstated thresholds and an untested train/inference transfer; worth serious refereeing, but needs a real revision. read the letter →

arxiv 2505.16014 v5 pith:W2PYFC5Q submitted 2025-05-21 cs.CL

Yash Saxena , Ankur Padia , Mandar S Chaudhary , Kalpa Gunaratna , Srinivasan Parthasarathy , Manas Gaur This is my paper
classification cs.CL
keywords retrieval-augmentedgenerationrationale-drivenevidenceselectiondirectpreferenceoptimizationadversarialcorpuspoisoninginterpretableRAGadaptivetop-kremovallegalandfinancialquestionansweringlong-documentretrieval
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the standard retrieve-then-re-rank step in retrieval-augmented generation is the wrong place to spend effort when answers must be traceable and the knowledge base may be poisoned. It proposes METEORA, which first generates a short set of search-and-flag rationales for the query, uses those rationales to pick evidence chunks with a statistically determined cutoff instead of a fixed top-k, and then runs a verifier over the chosen chunks using the same rationales. Across six legal, financial, and academic benchmarks, the authors report that this selection approach beats re-ranking baselines by 13.41% in recall and 21.05% in precision, needs about 80% fewer evidence chunks to reach comparable recall, improves answer accuracy by 33.34%, and raises poisoned-content detection F1 from 0.10 to 0.44. The larger claim is that interpretability, robustness, and efficiency can be bought together rather than traded off, because one shared reasoning chain does selection, explanation, and verification.

What carries the argument

The load-bearing object is the query-specific rationale: a short natural-language set of search-and-flag instructions generated once per query. It does three jobs at once—it pairs with the evidence chunk it points to, it is pooled into a single embedding whose similarity scores across all chunks are cut at a z-score elbow or maximum-curvature point to determine the adaptive $k^*$, and it doubles as the flagging instruction set for the Verifier LLM, which removes chunks that contradict it, prior verified chunks, or known facts. The DPO preference signal makes the rationale trustworthy: rationales that select ground-truth evidence are preferred over those that do not, so the generator learns to emit search targets that actually locate answers, and the same target is then reused as the audit and defense mechanism.

What would settle it

Hold out a class of legal queries from DPO training and measure whether METEORA's recall advantage persists on those queries; if the advantage disappears when the correct evidence was never paired with the query during training, the paper's transfer assumption is falsified.

Watch

Extended reading notes

Core claim

The paper's central claim, stated on its own terms, is that re-ranking is not merely a component to improve but a stage to remove: a RAG system can generate explicit rationales for a query, let those rationales select evidence through cosine-similarity pairing and a statistically determined cutoff, and then verify the selected evidence against the same rationales, ending up with better retrieval, better answers, and stronger defense against corpus poisoning. The rationale generator is preference-tuned with DPO on automatically built pairs—rationales that select ground-truth evidence are preferred over rationales that select incorrect evidence—so the model learns document-navigation patterns. At inference the generator sees only the query; ECSE pools the rationales, ranks evidence by similarity to the pooled embedding, and chooses $k^*$ from z-score normalized drops (or maximum curvature if no single drop stands out), optionally expanding to neighboring chunks; the Verifier LLM then discards chunks that violate the rationales' instructions, contradict previously verified chunks, or contradict well-established facts with high confidence. Across six benchmarks the authors report that this selection scheme raises average recall to 0.93 (0.89 without expansion) against the best baseline's 0.82, cuts evidence volume by roughly 80%, improves answer accuracy by 33.34%, and raises poison-detection F1 from 0.10 to 0.44.

Load-bearing premise

Rationales are trained while the correct evidence is in context but generated at inference from the query alone, so the whole pipeline depends on evidence-navigation patterns transferring to queries whose correct evidence was never seen; if that transfer fails, selection and verification degrade together.

Editorial extensions

If this is right

  • A fixed top-k is replaceable by a per-query cutoff: ECSE's z-score elbow or maximum-curvature fallback chooses $k^*$ from the similarity distribution, so evidence budgets adapt to the query and document.
  • The same rationale can serve as the selection criterion, the human-facing explanation, and the verifier's flagging instructions, so adding a poison-check does not require a separate explanation pipeline.
  • Longer documents are where the advantage concentrates: recall gains over the best baseline grow from 5.31% on QASPER to 41.17% on MAUD as average document length rises to 351k tokens.
  • Smaller, better-grounded contexts improve downstream answers by 33.34% accuracy, so reducing evidence volume is not a recall sacrifice but a generation gain.
  • Because the pipeline avoids feeding all documents to a single LLM re-ranker, it sidesteps the context-length failure that limits LLM-based re-rankers on long contracts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the paper is right, then auditability comes for free with retrieval: a compliance officer could, in principle, ask not just which chunks were used but which search instruction found each chunk, since the evidence trail is stored as rationale-evidence pairs.
  • A two-condition ablation (verifier using the shared rationales vs. generic consistency instructions) would tell whether the reported 4.4x robustness is caused by rationale-sharing or by the verifier alone; the paper does not run that ablation.
  • Since $k^*$ varies per query, the architecture could be paired with a compute budget scheduler that routes simple queries to few chunks and reserves expansion for long, cross-referenced documents—an efficiency policy the paper leaves implicit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper proposes METEORA, a RAG framework that replaces re-ranking with rationale-driven evidence selection for sensitive domains. A Llama-3.1-8B-Instruct model is DPO-tuned to generate query rationales; the Evidence Chunk Selection Engine (ECSE) pairs each rationale with the most similar evidence chunk and uses a pooled-rationale embedding with statistical elbow detection to set an adaptive cutoff; a Verifier LLM then filters poisoned evidence using the same rationales. The evaluation covers six legal, financial, and academic datasets, reporting higher context-prioritization recall/precision, an 80% reduction in evidence volume, improved answer accuracy, and improved poisoning defense, together with an abstract-only human evaluation of interpretability.

Significance. If the central empirical claims were fully supported, METEORA would be a useful contribution: it connects interpretability, efficiency, and poisoning defense in one pipeline, avoids manual rationale annotation, evaluates on six datasets, and releases code. I also agree with the stress-test observation that the preference labeling is not circular: rationales are labeled by whether they select ground-truth evidence, an external supervision signal. However, as written, the paper leaves the load-bearing training-to-inference transfer untested, leaves a key threshold parameter unreported, and supports two headline numbers (33.34% accuracy improvement and the human evaluation) only through an abstract claim or a bare figure. These gaps must be closed before the empirical claims can be properly assessed.

major comments (5)
  1. [§2.1, Eq. (1); §4 DPO ablation; Appendix D] The central mechanism assumes that DPO training on πθ(r|q,e*) transfers to inference-time πθ(r|q), but no experiment or theory in the paper isolates this step. The DPO ablation in §4 compares METEORA (query-only rationales) with METEORA w/o DPO (base-LLM query-only rationales), not with evidence-conditioned rationales. The appendix proof (Eqs. (5), (12)) analyzes a joint policy πθ(e,r|q) that ECSE does not use, since ECSE selects by cosine similarity between rationales and chunks. Because the Verifier (§2.3) consumes the same rationales, a transfer failure degrades selection and adversarial filtering together, as the Design Limitations footnote acknowledges. Add a direct experiment with πθ(r|q,e*) at inference time, or an analysis showing that query-only rationales preserve the learned navigation behavior.
  2. [§2.2 and §A.5] The elbow detector selects k* at the first significant z-scored drop, with the fallback rule "if all zi ≤ τ". The threshold τ is never given a value, and no sensitivity analysis over τ is reported. Since τ controls the adaptive cutoff that replaces top-k, this is a free parameter in the central method; without reporting it or testing its sensitivity, the claim of eliminating arbitrary heuristics is not substantiated.
  3. [Abstract and §4, Figure 4] The 33.34% answer-accuracy improvement exists only as a claim tied to Figure 4, which has no numeric table, no per-dataset accuracy values, and no specification of which baseline or experimental condition it compares. Likewise, the human evaluation (3.64/5 confidence, 86% ground-truth agreement) appears only in the abstract, with no methods, rater pool, sample size, or annotation protocol anywhere in the manuscript. These two results directly support the "interpretability, efficiency, and robustness are synergistic" thesis and must be given full experimental descriptions.
  4. [Appendix D, Eqs. (5) and (12)] The theoretical section does not prove the deployed system's behavior. It derives convergence for πθ(e,r|q), a joint policy over evidence and rationales, while Eq. (1) in §2.1 trains πθ(r|q,e*) and ECSE selects evidence by cosine similarity rather than by sampling e from πθ. Consequently, the appendix cannot be cited as support for the query-only transfer claim; if it is intended as a proof, it needs to model the actual ECSE selection rule and the inference-time marginal πθ(r|q).
  5. [§2.2, Stage 1] The "Ranking Free" framing is overstated: the ECSE procedure explicitly sorts evidence by cosine similarity to the pooled rationale embedding ("we rank all evidence chunks by similarity to \bar r") before applying the elbow cutoff. The contribution may still be valuable as replacing a learned re-ranker with selection over rationale similarities, but the title and abstract should not claim that ranking itself has been eliminated.
minor comments (7)
  1. [Abstract and Table 4] The abstract's "increasing F1 from 0.10 to 0.44" is not directly supported by Table 4, where No Defense is 0.00 across all datasets and the Perplexity baseline averages about 0.10; state the comparison explicitly.
  2. [§4 and Abstract] The 21.05% precision claim comes from METEORA w/o Expansion; the abstract should make clear that the precision gain is achieved by the no-expansion variant.
  3. [Figure 4] Figure 4 has no y-axis label or numeric annotations, and the reported p<0.01 paired t-test is mentioned without details of the test procedure or the number of trials.
  4. [§4] The text contains a dangling cross-reference "for a better theoretical understanding, see §??"; this should be filled in or removed.
  5. [§A.1 and throughout] There are language and presentation issues: "explaninable" in §A.1, and "evidences" is used as a count noun throughout the paper.
  6. [Table 3] Table 3's computation is unclear: the main text says baseline k is matched to METEORA's average selection size for fair comparison, but the table reports how many times more evidence baselines need to reach comparable recall; describe the procedure used to produce the table.
  7. [Design Limitations footnote] The single-point-of-failure concern in the Design Limitations footnote is central enough to warrant a proper limitations subsection rather than a footnote.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported gains are evaluated on held-out evidence/QA benchmarks, and the theory appendix is a standard DPO restatement rather than a self-fulfilling fit.

full rationale

METEORA's central derivation chain is not circular. The DPO preference labels are defined by whether a generated rationale selects ground-truth evidence (Section 2.1), and the CP results are measured on a held-out 10% test split (Appendix A.4), so the recall and precision numbers are genuine generalization results rather than the training objective re-reported. The ECSE similarity-based selection and elbow cutoff (Section 2.2) contain no ground-truth information at inference time; the Verifier (Section 2.3) uses the same rationales, but the poisoning F1 of 0.44 versus 0.10 for perplexity is an externally measured outcome, not a construction. The main weakness is the untested train/inference transfer from pi(r|q,e*) to pi(r|q), explicitly acknowledged as an assumption in Section 2.1; this is a correctness and evaluation gap, not a definitional reduction. Appendix D's proof restates DPO's optimality guarantee, assuming preferences encode true relevance and concluding the policy tracks that reward, but the paper's empirical claims do not depend on that appendix. No load-bearing self-citation or uniqueness import is present.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The framework rests on similarity-based relevance, query-only inference transfer, correct annotations, rationales doubling as verification instructions, and a representative poisoning protocol. It introduces no new physical entities. Two free design choices (elbow threshold tau and rationale count) are unreported, which makes the no-heuristics claim weaker than stated.

free parameters (4)
  • Elbow detection z-score threshold tau = not reported
    Controls the fallback from first-order to second-order differences in ECSE; the claim of no manually defined thresholds depends on this hidden value.
  • Number of rationales generated per query = not reported; example prompt shows up to 10
    Changes the size of Ev and the pooled embedding; no ablation or stated value.
  • Verifier confidence threshold = >90%
    Manual design choice in Section 2.3; no calibration or sensitivity analysis.
  • DPO training hyperparameters = beta=0.05, epochs=3, LR=3e-5
    Reported in Table 5; standard hyperparameters but they influence rationale quality.
assumptions (6)
  • domain assumption Cosine similarity between rationale and evidence embeddings is a sufficient relevance signal for selecting correct evidence.
    Core of ECSE pairing and pooling in Section 2.2; no analysis of cases where similarity fails.
  • domain assumption Query-only rationales at inference transfer behavior learned when conditioning on ground-truth evidence at training.
    Section 2.1 trains pi_theta(r|q,e*) and infers pi_theta(r|q); the distribution shift is assumed to be harmless.
  • domain assumption Ground-truth evidence annotations in the six datasets are correct and comprehensive.
    Used to label DPO preference pairs and to compute CP precision and recall.
  • ad hoc to paper A rationale that works as a retrieval search strategy is also a valid flagging instruction for detecting poisoned evidence.
    Verifier in Section 2.3 reuses rationales as flagging instructions; there is no independent check that search strategies discriminate adversarial content.
  • standard math Standard DPO optimality result (Rafailov et al.) applies to the joint evidence-rationale policy.
    Appendix D restates DPO convergence; assumes Bradley-Terry model and the reference policy conditions.
  • domain assumption The Nazary et al. LLM-generated poisoning protocol is representative of real RAG corpus poisoning.
    Adversarial evaluation in Section 3 uses one non-adaptive attack type; adaptive attacks are declared out of scope.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Ranking Free RAG: Replacing Re-ranking with Selection in RAG for Sensitive Domains." pith.science (2026). https://pith.science/paper/W2PYFC5Q

@misc{pith2026250516014,
  author       = {Pith},
  title        = {Pith review of: Ranking Free RAG: Replacing Re-ranking with Selection in RAG for Sensitive Domains},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W2PYFC5Q}},
  note         = {Machine review of arXiv:2505.16014}
}
read the original abstract

Retrieval-Augmented Generation (RAG) systems deployed in sensitive domains must provide interpretable evidence selection and robust safeguards against data poisoning, yet current approaches rely on opaque similarity-based retrieval with arbitrary top-k cutoffs that offer no explanation for their selections and remain vulnerable to adversarial manipulation. METEORA replaces re-ranking with rationale-driven selection via three components: a DPO-tuned LLM that generates explicit retrieval rationales, an Evidence Chunk Selection Engine (ECSE) that uses those rationales with statistical elbow detection for adaptive cutoff determination, and a Verifier LLM that filters poisoned evidence using the same rationales. Across six datasets, METEORA achieves 13.41% higher recall, 21.05% higher precision (without expansion), an 80% reduction in evidence volume, a 33.34% improvement in answer accuracy, and a 4.4x improvement in adversarial robustness. Human evaluation confirms genuine interpretability (3.64/5 confidence; 86% ground-truth agreement), demonstrating that interpretability, efficiency, and robustness are synergistic rather than competing objectives. The code is available in the GitHub repository https://github.com/YashSaxena21/METEORA

Figures

Figures reproduced from arXiv: 2505.16014 by the authors.

Figure 1
Figure 1. METEORA achieves interpretable and robust evidence selection where existing approaches fail. On a challenging legal query from Merger Agreement Understanding Dataset with poisoned evidence, traditional re-rankers select contaminated content producing factually incorrect answers, while LLM-based re-rankers fail due to context length limitations. Only METEORA uses interpretable rationales to explain evidence selection… view at source ↗
Figure 2
Figure 2. Overview of our METEORA framework. We propose METEORA (Method for Interpretable rank-free evidence selec￾tion with Optimal Rationale), a frame￾work that solves the black box problem by replacing opaque re-ranking with ex￾plicit reasoning. As demonstrated in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Generated Rationale The resulting preference-tuned model π ∗ θ generates query-aligned rationales that serve dual purposes in METEORA: guiding evidence selection through the Evidence Chunk Selec￾tion Engine (subsection 2.2) and enabling adversarial detection through the Verifier LLM (subsection 2.3). Implementation de￾tails are provided in [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Response accuracy when retrieved evidence matches ground-truth, with variance bars indicating sta￾tistical significance (p < 0.01, paired t-test) across trials. Ablation Effect of DPO Training. DPO fine-tuning demonstrates its critical role in bridging the semantic gap…
Figure 5
Figure 5. Figure 5: Positive example demonstrating how METEORA links rationale-based evidence selec￾tion (interpretability) with rationale-grounded answer generation (explainability), resulting in a correct and traceable response to a legal question. 15 [PITH_FULL_IMAGE:figures/full_fig_…
Figure 6
Figure 6. Figure 6: Negative example showing how METEORA enables transparent error tracing. Al￾though the correct evidence was not selected, the rationale trail provides insight into why the incorrect evidences were selected, and how they influenced the erroneous answer. A.3 Prompts Used …

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Question Decomposition for Retrieval-Augmented Generation

    cs.CL 2025-07 conditional novelty 4.0 of 10

    Splitting multi-hop questions into subquestions and reranking the merged retrieval pool improves RAG evidence coverage and answer accuracy on MultiHop-RAG and HotpotQA.

Reference graph

Works this paper leans on

56 extracted references · 31 canonical work pages · cited by 1 Pith paper

  1. [1]

    Re2g: Retrieve, rerank, generate

    Michael Glass, Gaetano Rossiello, Md Faisal Mahbub Chowdhury, Ankita Rajaram Naik, Pengshan Cai, and Alfio Gliozzo. Re2g: Retrieve, rerank, generate. arXiv preprint arXiv:2207.06300, 2022

    arXiv 2022

  2. [2]

    Sentence-bert: Sentence embeddings using siamese bert-networks

    Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982--3992, 2019 a

    work page 2019

  3. [3]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4171--4186, 2019

    work page 2019

  4. [4]

    The trustworthiness of retrieval-augmented generation systems

    Qingyun Zhou et al. The trustworthiness of retrieval-augmented generation systems. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, 2024 a

    work page 2024

  5. [5]

    Domain-specific retrieval-augmented generation using expert-tuned llms

    Adam Barron et al. Domain-specific retrieval-augmented generation using expert-tuned llms. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, 2024 a

    work page 2024

  6. [6]

    Poisonedrag: Knowledge poisoning attacks to retrieval-augmented generation of large language models

    Wei Zou, Rui Geng, Bin Wang, and Jinyuan Jia. Poisonedrag: Knowledge poisoning attacks to retrieval-augmented generation of large language models. arXiv preprint arXiv:2402.07867, 2024

    arXiv 2024

  7. [7]

    Poison-rag: Adversarial data poisoning attacks on retrieval-augmented generation in recommender systems

    Fatemeh Nazary, Yashar Deldjoo, and Tommaso Di Noia. Poison-rag: Adversarial data poisoning attacks on retrieval-augmented generation in recommender systems. In Advances in Information Retrieval: 47th European Conference on Information Retrieval, ECIR 2025, pages 1--16. Springer, 2025

    work page 2025

  8. [8]

    Phantom: General trigger attacks on retrieval augmented language generation

    Harsh Chaudhari et al. Phantom: General trigger attacks on retrieval augmented language generation. arXiv preprint arXiv:2405.20485, 2024

    arXiv 2024

Show all 56 references
  1. [9]

    Adversarial threat vectors and risk mitigation for retrieval-augmented generation systems

    Enterprise Security Research. Adversarial threat vectors and risk mitigation for retrieval-augmented generation systems. Technical Report arXiv:2506.00281v1, arXiv, 2024 a . Recent reports indicate enterprise use exceeded 50\

    2024 arXiv

  2. [10]

    Rag data poisoning: Key concepts explained

    Promptfoo Research. Rag data poisoning: Key concepts explained. https://www.promptfoo.dev/blog/rag-poisoning/, 2024 b . Accessed November 2024

    2024

  3. [11]

    Rag poisoning in enterprise knowledge sources

    SplxAI. Rag poisoning in enterprise knowledge sources. https://splx.ai/blog/rag-poisoning-in-enterprise-knowledge-sources, 2024. Accessed October 2024

    2024

  4. [12]

    Rationale-guided retrieval augmented generation for medical question answering

    Jiwoong Sohn, Yein Park, Chanwoong Yoon, Sihyeon Park, Hyeon Hwang, Mujeen Sung, Hyunjae Kim, and Jaewoo Kang. Rationale-guided retrieval augmented generation for medical question answering. In Proceedings of the 2025 Conference of the North American Chapter of the Association...

    2025

  5. [13]

    Bridging relevance and reasoning: Rationale distillation in retrieval-augmented generation

    Pengyue Jia, Derong Xu, Xiaopeng Li, Zhaocheng Du, Xiangyang Li, Yichao Wang, Yuhao Wang, Qidong Liu, Maolin Wang, Huifeng Guo, Ruiming Tang, and Xiangyu Zhao. Bridging relevance and reasoning: Rationale distillation in retrieval-augmented generation. In Wanxiang Che, Joyce Na...

    2025

  6. [14]

    Rankrag: Unifying context ranking with retrieval-augmented generation in llms

    Yue Yu, Wei Ping, Zihan Liu, Boxin Wang, Jiaxuan You, Chao Zhang, Mohammad Shoeybi, and Bryan Catanzaro. Rankrag: Unifying context ranking with retrieval-augmented generation in llms. In Advances in Neural Information Processing Systems, volume 37, pages 121156--121184, 2024 a

    2024

  7. [15]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, volume 35, pages 24824--24837, 2022

    2022

  8. [16]

    Cotever: Chain of thought prompting annotation toolkit for explanation verification

    Seungone Kim, Se June Joo, Yul Jang, Hyungjoo Chae, and Jinyoung Yeo. Cotever: Chain of thought prompting annotation toolkit for explanation verification. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: System Dem...

    2023

  9. [17]

    annotator rationales

    Omar Zaidan, Jason Eisner, and Christine Piatko. Using "annotator rationales" to improve machine learning for text categorization. In Proceedings of the 2007 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,...

    2007

  10. [18]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In Advances in Neural Information Processing Systems, volume 36, pages 53728--53741, 2023 a

    2023

  11. [19]

    The impact of chunking strategies on domain-specific information retrieval in rag systems

    Maximilian Stäbler, Steffen Turnbull, Tobias Müller, Chris Langdon, Jorge Marx-Goméz, and Frank Köster. The impact of chunking strategies on domain-specific information retrieval in rag systems. In 2025 IEEE International Conference on Omni-layer Intelligent Systems (COINS), p...

    2025

  12. [20]

    Gulavani, Alexey Tumanov, and Ramachandran Ramjee

    Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav S. Gulavani, Alexey Tumanov, and Ramachandran Ramjee. Taming throughput-latency tradeoff in llm inference with sarathi-serve. In Proceedings of the 18th USENIX Conference on Operating Systems Desi...

    2024

  13. [21]

    Not what you've signed up for: Compromising LLM -integrated applications via indirect prompt injection

    Kilian Greshake, Luca Beurer-Kellner, Vasisht Duddu, Ross Anderson, and Ben Stock. Not what you've signed up for: Compromising LLM -integrated applications via indirect prompt injection. In arXiv preprint arXiv:2302.12173, 2023

    2023 arXiv

  14. [22]

    Benchmarking and defending against indirect prompt injection attacks

    Jieyi Yi, Ziyi Kou, Wenqi Fan, Qing Li, and Siheng Chen. Benchmarking and defending against indirect prompt injection attacks. In arXiv preprint arXiv:2312.14197, 2023

    2023 arXiv

  15. [23]

    Backdoored retrievers for prompt injection attacks on retrieval-augmented generation

    Carlos Clop, Daniel Russo, Matei Zaharia, and Ion Stoica. Backdoored retrievers for prompt injection attacks on retrieval-augmented generation. In arXiv preprint arXiv:2410.14479, 2024

    2024 arXiv

  16. [24]

    Gasliteing the retrieval: Poisoning knowledge dbs to game embedding-based search

    Mor Ben-Tov, Hila Gonen, and Omer Levy. Gasliteing the retrieval: Poisoning knowledge dbs to game embedding-based search. In OpenReview (preprint), 2025

    2025

  17. [25]

    Analyzing adversarial attacks on sequence-to-sequence relevance models

    Adam Parry et al. Analyzing adversarial attacks on sequence-to-sequence relevance models. In arXiv preprint arXiv:2406.XXXXX, 2024

    2024

  18. [26]

    Yuta Koreeda and Christopher D. Manning. Contractnli: A dataset for document-level natural language inference for contracts, 2021. URL https://arxiv.org/abs/2110.01799

    2021 arXiv

  19. [27]

    Question answering for privacy policies: Combining computational and legal perspectives, 2019

    Abhilasha Ravichander, Alan W Black, Shomir Wilson, Thomas Norton, and Norman Sadeh. Question answering for privacy policies: Combining computational and legal perspectives, 2019. URL https://arxiv.org/abs/1911.00841

    2019 arXiv

  20. [28]

    Cuad: An expert-annotated nlp dataset for legal contract review, 2021

    Dan Hendrycks, Collin Burns, Anya Chen, and Spencer Ball. Cuad: An expert-annotated nlp dataset for legal contract review, 2021. URL https://arxiv.org/abs/2103.06268

    2021 arXiv

  21. [29]

    Legalbench-rag: A benchmark for retrieval-augmented generation in the legal domain

    Nicholas Pipitone and Ghita Houir Alami. Legalbench-rag: A benchmark for retrieval-augmented generation in the legal domain. arXiv preprint arXiv:2408.10343, 2024

    2024 arXiv

  22. [30]

    Finqa: A dataset of numerical reasoning over financial data

    Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, et al. Finqa: A dataset of numerical reasoning over financial data. arXiv preprint arXiv:2109.00122, 2021

    2021 arXiv

  23. [31]

    A dataset of information-seeking questions and answers anchored in research papers

    Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A Smith, and Matt Gardner. A dataset of information-seeking questions and answers anchored in research papers. arXiv preprint arXiv:2105.03011, 2021

    2021 arXiv

  24. [32]

    cross-encoder/ms-marco- M ini L M - L 4-v2 · H ugging F ace --- huggingface.co

    Huggingface. cross-encoder/ms-marco- M ini L M - L 4-v2 · H ugging F ace --- huggingface.co. https://huggingface.co/cross-encoder/ms-marco-MiniLM-L4-v2. [Accessed 16-05-2025]

    2025

  25. [33]

    Unsupervised dense information retrieval with contrastive learning, 2022

    Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. Unsupervised dense information retrieval with contrastive learning, 2022. URL https://arxiv.org/abs/2112.09118

    2022 arXiv

  26. [34]

    Sentence-bert: Sentence embeddings using siamese bert-networks, 2019 b

    Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks, 2019 b . URL https://arxiv.org/abs/1908.10084

    2019 arXiv

  27. [35]

    S tern5497/sbert-legal-xlm-roberta-base · H ugging F ace --- huggingface.co

    Legal-huggingface. S tern5497/sbert-legal-xlm-roberta-base · H ugging F ace --- huggingface.co. https://huggingface.co/Stern5497/sbert-legal-xlm-roberta-base. [Accessed 16-05-2025]

    2025

  28. [36]

    Fine-tuning llama for multi-stage text retrieval

    Xueguang Ma, Liang Wang, Nan Yang, Furu Wei, and Jimmy Lin. Fine-tuning llama for multi-stage text retrieval. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '24, page 2421–2425, New York, NY, USA, 2024....

    2024

  29. [37]

    Promptriever: Instruction-trained retrievers can be prompted like language models, 2024

    Orion Weller, Benjamin Van Durme, Dawn Lawrie, Ashwin Paranjape, Yuhao Zhang, and Jack Hessel. Promptriever: Instruction-trained retrievers can be prompted like language models, 2024. URL https://arxiv.org/abs/2409.11136

    2024 arXiv

  30. [38]

    The trustworthiness of retrieval-augmented generation systems

    Qingyun Zhou et al. The trustworthiness of retrieval-augmented generation systems. ACL, 2024 b

    2024

  31. [39]

    Retrieval-augmented generation for knowledge-intensive nlp tasks, 2021

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive nlp tasks, 2021. URL https://arx...

    2021 arXiv

  32. [40]

    Lexrag: Retrieval-augmented generation for legal domain dialogue systems

    Tianyu Li et al. Lexrag: Retrieval-augmented generation for legal domain dialogue systems. arXiv preprint arXiv:2502.00001, 2025

    2025

  33. [41]

    Rationale-guided retrieval augmented generation for medical question answering

    Jiwoong Sohn, Yein Park, Chanwoong Yoon, Sihyeon Park, Hyeon Hwang, Mujeen Sung, Hyunjae Kim, and Jaewoo Kang. Rationale-guided retrieval augmented generation for medical question answering. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors, Proceedings of the 2025 Conferenc...

    2025

  34. [42]

    Assistive ai for augmenting human decision-making, 2024

    Natabara Máté Gyöngyössy, Bernát Török, Csilla Farkas, Laura Lucaj, Attila Menyhárd, Krisztina Menyhárd-Balázs, András Simonyi, Patrick van der Smagt, Zsolt Ződi, and András Lőrincz. Assistive ai for augmenting human decision-making, 2024. URL https://arxiv.org/abs/2410.14353

    2024 arXiv

  35. [43]

    Domain-specific retrieval-augmented generation using expert-tuned llms

    Adam Barron et al. Domain-specific retrieval-augmented generation using expert-tuned llms. ACL, 2024 b

    2024

  36. [44]

    Systematic knowledge injection in large language models for scientific qa

    Rohit Bhushan et al. Systematic knowledge injection in large language models for scientific qa. ICLR, 2025

    2025

  37. [45]

    The structural safety generalization problem

    Julius Broomfield, Tom Gibbs, George Ingebretsen, Ethan Kosak-Hine, Tia Nasir, Jason Zhang, Reihaneh Iranmanesh, Sara Pieri, Reihaneh Rabbany, and Kellin Pelrine. The structural safety generalization problem. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Tahe...

    2025 doi

  38. [46]

    Docent: Document-centric rag improves multi-hop qa

    Ori Yoran et al. Docent: Document-centric rag improves multi-hop qa. ACL, 2024

    2024

  39. [47]

    Self-rag: Learning to retrieve when answering questions

    Akari Asai et al. Self-rag: Learning to retrieve when answering questions. NeurIPS, 2023

    2023

  40. [48]

    A nearly instance optimal algorithm for top-k ranking under the multinomial logit model, 2017

    Xi Chen, Yuanzhi Li, and Jieming Mao. A nearly instance optimal algorithm for top-k ranking under the multinomial logit model, 2017. URL https://arxiv.org/abs/1707.08238

    2017 arXiv

  41. [49]

    Non-autoregressive generative models for reranking recommendation, 2025

    Yuxin Ren, Qiya Yang, Yichun Wu, Wei Xu, Yalong Wang, and Zhiqiang Zhang. Non-autoregressive generative models for reranking recommendation, 2025. URL https://arxiv.org/abs/2402.06871

    2025 arXiv

  42. [50]

    Rankrag: Unifying context ranking with retrieval-augmented generation in llms

    Yue Yu, Wei Ping, Zihan Liu, Boxin Wang, Jiaxuan You, Chao Zhang, Mohammad Shoeybi, and Bryan Catanzaro. Rankrag: Unifying context ranking with retrieval-augmented generation in llms. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors...

    2024

  43. [51]

    Mirage: Faithful attribution in retrieval-augmented generation

    Shaohan Qi et al. Mirage: Faithful attribution in retrieval-augmented generation. ICLR, 2024

    2024

  44. [52]

    Shifting from ranking to set selection for retrieval augmented generation

    Dahyun Lee, Yongrae Jo, Haeju Park, and Moontae Lee. Shifting from ranking to set selection for retrieval augmented generation. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Proceedings of the 63rd Annual Meeting of the Association fo...

    2025 doi

  45. [53]

    Badrag: Identifying vulnerabilities in retrieval augmented generation of large language models, 2024

    Jiaqi Xue, Mengxin Zheng, Yebowen Hu, Fei Liu, Xun Chen, and Qian Lou. Badrag: Identifying vulnerabilities in retrieval augmented generation of large language models, 2024. URL https://arxiv.org/abs/2406.00083

    2024 arXiv

  46. [54]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35: 0 2...

    2022

  47. [55]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. URL https://arxiv.org/abs/2402.03300

    2024 arXiv

  48. [56]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov et al. Direct preference optimization: Your language model is secretly a reward model. arXiv preprint arXiv:2305.18290, 2023 b

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.