pith. sign in

arxiv: 2605.18806 · v1 · pith:6DEETNCPnew · submitted 2026-05-11 · 💻 cs.IR

Towards FairRAG: Preventing Representational Harm in Retrieval-Augmented Generation by Enforcing Fair Exposure at Retrieval Time

Riddhi Tikoo This is my paper

Pith reviewed 2026-05-20 22:18 UTC · model grok-4.3

classification 💻 cs.IR
keywords retrieval-augmented generationrepresentational biasfair rankingexposure parityRAG systemsdemographic parityWikipedia dataset
0
0 comments X

The pith

A Representative Stochastic ranker achieves near-parity average exposure in RAG by treating initial relevance scores as already shaped by bias.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper examines how retrieval ranking in retrieval-augmented generation systems can introduce or reduce representational bias that then appears in the final LLM outputs. It evaluates two existing utility-focused rankers against two new exposure-aware methods on the TREC 2022 Fair Ranking Dataset of Wikipedia articles labeled as protected or non-protected. The Representative Stochastic ranker, which explicitly accounts for bias already present in the initial relevance scores, produces statistically significant near-parity in average exposure across groups. Demographic parity in the generated answers tracks the exposure parity closely for every ranking method tested. This indicates that retrieval is the key stage where fairness interventions can stop bias from reaching the generation step.

Core claim

By using a Representative Stochastic ranker that re-samples documents to enforce fair exposure while recognizing that relevance scores from the initial retrieval already embed representational bias, the system reaches statistically significant near-parity in average exposure. Across all ranking methods, the demographic parity observed in the LLM-generated answers closely matches the exposure parity achieved at retrieval, showing that representational bias in RAG pipelines originates in retrieval and propagates downstream.

What carries the argument

The Representative Stochastic ranker, a stochastic sampling method that reintroduces fairness by adjusting for bias already present in relevance scores rather than assuming those scores are neutral.

If this is right

  • Retrieval-stage ranking directly controls the level of representational bias that reaches the generation stage.
  • Generation demographic parity mirrors retrieval exposure parity under every ranking method examined.
  • Rankers that assume initial relevance scores are unbiased fail to reach exposure parity.
  • Intervening at retrieval prevents downstream bias from appearing in final answers.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same exposure-aware sampling could be tested on other retrieval corpora or with additional protected attributes beyond the binary split used here.
  • Replacing the scenario-based prompts with logs of actual user queries would show whether the parity results hold outside controlled prompts.
  • Combining the ranker with post-generation debiasing steps might produce even tighter parity if retrieval alone leaves residual bias.

Load-bearing premise

The TREC 2022 dataset annotations of Wikipedia articles as protected or non-protected accurately identify the groups whose exposure should be balanced, and the four scenario-based prompts represent typical real-world RAG use.

What would settle it

Re-running the Representative Stochastic ranker on the TREC 2022 dataset and finding that exposure parity is not statistically significant or that generation demographic parity no longer tracks exposure parity would falsify the central claim.

Figures

Figures reproduced from arXiv: 2605.18806 by Riddhi Tikoo.

Figure 1
Figure 1. Figure 1: Average Retrieval Exposure Disparity and [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Average Generation Demographic Parity and [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Descriptive statistics and independent t-test results for exposure disparity across ranking methods. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Descriptive statistics and independent t-test results for female exposure share across ranking methods. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
read the original abstract

As Large Language Model (LLM) integration has accelerated in high-stakes domains, model hallucination is a critical issue. Retrieval-augmented generation (RAG) is a technique for addressing hallucination; however, RAG's multi-component pipeline introduces vulnerabilities where biases can be introduced. This study considers two previously developed utility-focused ranking strategies (Standard and Stochastic) alongside two proposed exposure-aware approaches (Forced-Exposure and Representative Stochastic). Using the TREC 2022 Fair Ranking Dataset, which contains Wikipedia articles annotated as protected or non-protected, the LLM was asked to identify relevant articles with citations for four scenario-based Q&A prompts. The retrieval rankings and the generated outputs were evaluated for exposure bias and utility across all ranking methods. Overall, the Representative Stochastic ranker resulted in a statistically significant near-parity average exposure, acknowledging that relevance scores initially produced during retrieval are already shaped by representational bias, whereas the other rankers assume those scores are unbiased. Across all the methods of document ranking, generation demographic parity closely mirrored the exposure parity, reinforcing that representational bias in RAG systems is driven by retrieval and propagates to generation. These findings highlight that retrieval ranking is a critical point for mitigating downstream bias and propose a Representative Stochastic ranker that reintroduces fairness in RAG systems.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper proposes two new exposure-aware document ranking methods (Forced-Exposure and Representative Stochastic) for retrieval-augmented generation to reduce representational harm. Using the TREC 2022 Fair Ranking Dataset of Wikipedia articles labeled protected or non-protected, the authors compare these against Standard and Stochastic rankers across four scenario-based Q&A prompts. They report that the Representative Stochastic ranker produces statistically significant near-parity average exposure while acknowledging that initial relevance scores already embed bias; they further observe that generation demographic parity closely tracks retrieval exposure parity across all methods.

Significance. If the empirical results hold under scrutiny, the work provides concrete evidence that retrieval-time interventions can propagate fairness to the generation stage in RAG pipelines. The explicit recognition that relevance scores are already biased, the use of a public dataset, and the observation that generation parity mirrors exposure parity are useful contributions to the growing literature on fairness in retrieval-augmented systems.

major comments (3)
  1. The central claim of statistically significant near-parity exposure for the Representative Stochastic ranker is presented without error bars, exact p-values, sample sizes per prompt, or the precise statistical test used. This information is required to evaluate whether the reported near-parity is robust or sensitive to the small number of scenarios.
  2. The evaluation assumes that the TREC 2022 binary protected/non-protected annotations correctly identify the demographic groups whose exposure should be balanced to prevent representational harm in the four chosen Q&A scenarios. No validation or sensitivity analysis of this label-to-harm mapping is provided, which is load-bearing for interpreting the results as mitigation of actual harm rather than parity on an arbitrary grouping.
  3. Details of the LLM (model name, version, temperature, system prompt) and the exact four scenario-based Q&A prompts are not supplied. These omissions prevent reproduction and make it difficult to assess whether the observed mirroring of generation parity to exposure parity is an artifact of prompt construction or model choice.
minor comments (2)
  1. Notation for exposure and demographic parity metrics should be defined explicitly in a single location (e.g., §3 or §4) rather than introduced piecemeal.
  2. Figure captions and axis labels would benefit from stating the exact number of documents retrieved and the number of generations per prompt.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for their constructive and detailed feedback. The comments highlight important areas for strengthening the statistical rigor, transparency of assumptions, and reproducibility of the work. We respond to each major comment below and indicate the revisions that will be incorporated in the next version of the manuscript.

read point-by-point responses

  1. Referee: The central claim of statistically significant near-parity exposure for the Representative Stochastic ranker is presented without error bars, exact p-values, sample sizes per prompt, or the precise statistical test used. This information is required to evaluate whether the reported near-parity is robust or sensitive to the small number of scenarios.

    Authors: We agree that these statistical details are essential for evaluating the robustness of the reported results. The original manuscript did not include them. In the revised version, we will add error bars to all exposure plots, report the exact p-values, specify the sample sizes (four scenarios drawn from the TREC 2022 queries), and explicitly state that two-sample t-tests were used to assess differences in average exposure. These additions will allow readers to assess sensitivity to the limited number of scenarios. revision: yes

  2. Referee: The evaluation assumes that the TREC 2022 binary protected/non-protected annotations correctly identify the demographic groups whose exposure should be balanced to prevent representational harm in the four chosen Q&A scenarios. No validation or sensitivity analysis of this label-to-harm mapping is provided, which is load-bearing for interpreting the results as mitigation of actual harm rather than parity on an arbitrary grouping.

    Authors: This is a substantive point. The TREC 2022 annotations are used as the established benchmark for fair ranking evaluation, and our primary contribution is to demonstrate the propagation of exposure parity from retrieval to generation rather than to validate the harm mapping itself. We did not perform sensitivity analysis on alternative groupings. In the revision we will add an explicit limitations paragraph acknowledging reliance on the dataset's binary proxy and noting that different groupings could produce different numerical outcomes, while preserving the core observation that generation demographic parity tracks retrieval exposure across methods. revision: partial

  3. Referee: Details of the LLM (model name, version, temperature, system prompt) and the exact four scenario-based Q&A prompts are not supplied. These omissions prevent reproduction and make it difficult to assess whether the observed mirroring of generation parity to exposure parity is an artifact of prompt construction or model choice.

    Authors: We acknowledge the omission of these implementation details. In the revised manuscript we will add a new appendix containing the precise LLM configuration (model name, version, temperature, and full system prompt) together with the verbatim text of the four scenario-based Q&A prompts. This will enable full reproduction and permit independent evaluation of whether the observed mirroring effect depends on the specific prompts or model. revision: yes

Circularity Check

0 steps flagged

Empirical evaluation on external dataset with no circular derivation

full rationale

The paper presents an empirical study that applies four ranking methods (including two newly proposed exposure-aware ones) to the TREC 2022 Fair Ranking Dataset and measures exposure parity and downstream generation demographic parity across scenario-based prompts. No mathematical derivation, equation, or parameter-fitting step is described that reduces by construction to its own inputs or to a self-citation chain. All reported results are obtained by direct comparison against an external, independently annotated benchmark, satisfying the criteria for a self-contained, non-circular analysis.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the validity of the TREC dataset group annotations and the assumption that exposure measured at retrieval directly drives downstream generation parity.

axioms (1)
  • domain assumption Wikipedia articles in the TREC 2022 Fair Ranking Dataset can be reliably annotated as protected or non-protected groups for exposure measurement.
    This annotation underpins all exposure calculations and fairness comparisons.

pith-pipeline@v0.9.0 · 5760 in / 1143 out tokens · 39419 ms · 2026-05-20T22:18:50.976342+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

21 extracted references · 21 canonical work pages · 2 internal anchors

  1. [1]

    Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval , year =

    Sandeep Avula and Chia-Jung Lee and Rongting Zhang and Vanessa Murdock , title =. Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval , year =

    work page

  2. [2]

    ResearchGate , year =

    Ibrahim Al Azher and Hamed Alhoori , title =. ResearchGate , year =

    work page

  3. [3]

    Computing Research Repository , year =

    Lei Huang and Weiyu Yu and Wei Ma and Weizhi Zhong and Zhenyu Feng and Haonan Wang and Qiang Chen and Wenbo Peng and Xiaowei Feng and Bing Qin and Ting Liu , title =. Computing Research Repository , year =

    work page

  4. [4]

    Computing Research Repository , year =

    Yuelyu Ji and Hang Zhang and Yanshan Wang , title =. Computing Research Repository , year =

    work page

  5. [5]

    Computing Research Repository , year =

    Eun Kim and Fernando Diaz , title =. Computing Research Repository , year =

    work page

  6. [7]

    Computing Research Repository , year =

    Jiarui Li and Ye Yuan and Zehua Zhang , title =. Computing Research Repository , year =

    work page

  7. [8]

    Computing Research Repository , year =

    Orestis Loukas and Ho-Ryun Chung , title =. Computing Research Repository , year =

    work page

  8. [9]

    Reuters , year =

    Deepa Seetharaman , title =. Reuters , year =

    work page

  9. [10]

    Does RAG Introduce Unfairness in LLM s? Evaluating Fairness in Retrieval-Augmented Generation Systems

    Xuyang Wu and Shuowei Li and Hsin-Tai Wu and Zhiqiang Tao and Yi Fang. Does RAG Introduce Unfairness in LLM s? Evaluating Fairness in Retrieval-Augmented Generation Systems. Proceedings of the 31st International Conference on Computational Linguistics. 2025

    work page 2025

  10. [11]

    Vempala and Edwin Zhang , title =

    Adam Tauman Kalai and Ofir Nachum and Santosh S. Vempala and Edwin Zhang , title =. Computing Research Repository , year =

    work page

  11. [12]

    Sandeep Avula, Chia-Jung Lee, Rongting Zhang, and Vanessa Murdock. 2025. https://doi.org/10.1145/3726302.3730230 Measuring the fairness gap between retrieval and generation in rag systems using a cognitive complexity framework . In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association ...

    work page doi:10.1145/3726302.3730230 2025

  12. [13]

    Ibrahim Al Azher and Hamed Alhoori. 2025. A comprehensive survey of retrieval-augmented generation (rag) evaluation and benchmarks: Perspectives from information retrieval and llm. ResearchGate

    work page 2025

  13. [14]

    Lei Huang, Weiyu Yu, Wei Ma, Weizhi Zhong, Zhenyu Feng, Haonan Wang, Qiang Chen, Wenbo Peng, Xiaowei Feng, Bing Qin, and Ting Liu. 2024. https://arxiv.org/abs/2311.05232 A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions . Computing Research Repository

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  14. [15]

    Yuelyu Ji, Hang Zhang, and Yanshan Wang. 2025. https://arxiv.org/abs/2503.15454 Bias evaluation and mitigation in retrieval-augmented medical question-answering systems . Computing Research Repository

    work page arXiv 2025

  15. [16]

    Why Language Models Hallucinate

    Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala, and Edwin Zhang. 2025. https://arxiv.org/abs/2509.04664 Why language models hallucinate . Computing Research Repository

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  16. [17]

    Eun Kim and Fernando Diaz. 2025. https://arxiv.org/abs/2409.11598 Towards fair rag: On the impact of fair ranking in retrieval-augmented generation . Computing Research Repository

    work page arXiv 2025

  17. [18]

    Taeyoun Kim, Jacob Mitchell Springer, Aditi Raghunathan, and Maarten Sap. 2025. https://doi.org/10.18653/v1/2025.findings-acl.974 Mitigating bias in RAG : Controlling the embedder . In Findings of the Association for Computational Linguistics: ACL 2025, pages 18999--19024, Vienna, Austria. Association for Computational Linguistics

    work page doi:10.18653/v1/2025.findings-acl.974 2025

  18. [19]

    Jiarui Li, Ye Yuan, and Zehua Zhang. 2024. https://arxiv.org/abs/2403.10446 Enhancing llm factual accuracy with rag to counter hallucinations: A case study on domain-specific queries in private knowledge-bases . Computing Research Repository

    work page arXiv 2024

  19. [20]

    Orestis Loukas and Ho-Ryun Chung. 2023. https://arxiv.org/abs/2309.17347 Demographic parity: Mitigating biases in real-world data . Computing Research Repository

    work page arXiv 2023

  20. [21]

    Deepa Seetharaman. 2026. Openai seeks to increase global ai use in everyday life. Reuters

    work page 2026

  21. [22]

    Xuyang Wu, Shuowei Li, Hsin-Tai Wu, Zhiqiang Tao, and Yi Fang. 2025. https://aclanthology.org/2025.coling-main.669/ Does RAG introduce unfairness in LLM s? evaluating fairness in retrieval-augmented generation systems . In Proceedings of the 31st International Conference on Computational Linguistics, pages 10021--10036, Abu Dhabi, UAE. Association for Com...

    work page 2025