Pith. sign in

REVIEW 4 major objections 5 minor 49 references

Privacy leakage in retrieval-augmented generation is query-driven, and the PA-HDP framework turns that into a differential-privacy guarantee with preserved retrieval quality.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 00:59 UTC pith:2USB57GE

load-bearing objection Genuinely useful query-dependent framing and a plausible pipeline, but the advertised (ε,0)-Semantic Metric DP guarantee is not proven—two load-bearing gaps in the appendix, and the empirical 'zero leakage' is partly built into the attack setup. the 4 major comments →

arxiv 2607.14811 v1 pith:2USB57GE submitted 2026-07-16 cs.CR

Is External Database Protection Static in Retrieval-Augmented Generation? Rethinking Privacy Preservation under Dynamic Queries

Gang Zhang , Mingyu Tian , Xukun Luan , Yuanchi Ma , Jinyan Liu This is my paper
classification cs.CR
keywords retrieval-augmented generationquery-driven privacydifferential privacysemantic metric differential privacyexponential mechanismprivacy-utility trade-offsensitive entity replacementextraction attack
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Retrieval-augmented generation can leak sensitive data from its external database, but most existing defenses treat every retrieved document as equally risky. This paper argues that the real risk is query-driven: the same sentence is dangerous under one prompt and harmless under another. To exploit this, the paper proposes PA-HDP, which scores each sentence by its relevance to the query and its own inherent sensitivity, then applies differential privacy only to high- and medium-risk sentences via entity replacement and an exponential mechanism. The authors claim this achieves (ε,0)-Semantic Metric Differential Privacy while preserving retrieval quality better than synthetic-data baselines, and their experiments show zero success on targeted and untargeted extraction attacks at comparable utility.

Core claim

The central claim is that the privacy risk of a retrieved document is a joint property of the document and the prompt, so static document-level protection wastes utility on low-risk content and under-protects query-relevant sensitive spans. PA-HDP operationalizes this by computing a risk score S_total = λ·S_rel + (1−λ)·S_sen for each sentence, where S_rel is a gated semantic similarity between the query and the sentence and S_sen is the maximum risk weight of the sentence's recognized sensitive entities. Only sentences above a threshold are modified: sensitive entities are replaced with same-category stand-ins to build a candidate set, and a semantic exponential mechanism picks a privatized

What carries the argument

The load-bearing object is the Semantic Metric Differential Privacy definition, which replaces the usual Hamming adjacency with an embedding distance d_phi(x,x') = ||φ(x)−φ(x')||₂, and the corresponding Semantic Exponential Mechanism. The mechanism selects from a finite candidate set C built by replacing sensitive entities, using a utility u(s,c) = −α·d_sem(s,c) − (1−α)·r(s,c), where d_sem is embedding distance to the original sentence and r is the maximum cosine similarity of residual sensitive entities. The proof claims this utility is 1-Lipschitz in d_phi, so the exponential mechanism yields (ε,0)-SMDP when run with the allocated budget. Around this core, PA-HDP layers a two-factor risk s

Load-bearing premise

In Appendix B, the proof assumes both that the risk score S_total is 1-Lipschitz in the embedding metric d_phi (the fact that it lies in [0,1] does not imply this) and that the exponential mechanism's candidate set C is input-independent, even though C is constructed from the private segment being protected; if either assumption fails, the claimed (ε,0)-Semantic Metric Differential Privacy guarantee collapses.

What would settle it

Using the paper's embedding model, compute the maximum over one-token-edit sentence pairs of |S_total(s)-S_total(s')|/||φ(s)-φ(s')||; if it exceeds 1, the Laplace noise scale 2/ε is too small. Also edit a protected sentence and check whether the candidate set C changes; if it does, the exponential mechanism's differential-privacy proof needs a different argument.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If the central claim holds, RAG systems can provide query-adaptive privacy: a sentence that is safe for one user can remain untouched, while the same sentence is protected when a sensitive query arrives.
  • The per-sentence risk score allows differential privacy budgets to be allocated by risk level, which yields stronger protection for high-risk content without penalizing low-risk text.
  • Because protection is span-level rather than document-level, the retrieval corpus stays closer to the original, so downstream generation quality degrades less than with full rewrite or synthetic generation.
  • The experiments suggest the framework can withstand both targeted extraction of specific PII and untargeted verbatim or semantic reconstruction attacks, at least on the tested datasets and generation models.
  • The modular design implies the same pipeline transfers to other domains by replacing the entity extractor and sensitivity weights, as the paper argues in its domain-adaptation discussion.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A reader should treat the (ε,0)-SMDP guarantee as conditional: the proof assumes the risk score is 1-Lipschitz in the embedding metric and that the exponential mechanism's candidate set does not depend on the input sentence, neither of which is established in the paper. An empirical measurement of the Lipschitz constant on real embeddings would settle the question directly.
  • The query-driven view suggests a broader design principle: privacy–utility trade-offs in RAG should be optimized per query, not per document, and future systems could learn risk hierarchies end-to-end from leak-labeled query–document pairs.
  • Since the framework already computes a noisy per-sentence risk score, it could be extended to multi-turn RAG by reusing those scores for budget accounting across a user session, addressing the fixed-budget limitation the paper itself acknowledges.
  • One stress test worth running is scaling the number of retrieved documents beyond k=1; the current composition argument may need re-derivation as multiple sentences from the same document are independently protected.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes PA-HDP, a prompt-aware dynamic hierarchical differential privacy framework for privacy-preserving retrieval-augmented generation. The pipeline first retrieves top-K contexts, segments them into privacy units, computes a risk score that combines query–segment relevance with field sensitivity, adds Laplace noise to that score, thresholds the noisy score into low/medium/high risk levels, and then, for medium- and high-risk sentences, replaces sensitive entities with same-category alternatives from a sanitized corpus to form a candidate set. The exponential mechanism selects a final sanitized text. The central formal claim is that the overall mechanism satisfies (ε,0)-Semantic Metric Differential Privacy (Definition III.2) via composition of a semantic Laplace mechanism and a semantic exponential mechanism (Appendix B, Lemma IV.1). The paper reports utility gains over synthetic-data baselines on HealthcareMagic and Wiki-PII ODQA benchmarks, and reports zero leakage under targeted and untargeted extraction attacks.

Significance. The motivating observation—that privacy risk in RAG is query-dependent rather than document-static—is timely and practically important, and the proposed pipeline is clearly described with substantial experimental breadth. If the formal privacy guarantee were correct, the prompt-aware risk stratification and differentiated protection would be a useful contribution to privacy-preserving RAG. However, the formal guarantee is not established: the semantic sensitivity bound in Appendix B is a non sequitur, and the exponential mechanism proof assumes a fixed public output range while the candidate set is generated from the private input. These are load-bearing for the abstract's and conclusion's claims of 'rigorous differential privacy guarantees.' The empirical zero-leakage results are also near-tautological because sensitive entities are replaced by non-sensitive ones and the attack metrics count recovery of exactly those replaced entities. I see no machine-checked proofs or parameter-free derivations; the framework also introduces several free parameters (λ, α, τ1, τ2, γ, entity weights, candidate size m). On the current evidence, the paper's central privacy claim cannot be accepted.

major comments (4)
  1. [Appendix B, Step 1; Definition III.2] The proof asserts that because S_total ∈ [0,1], changing one neighboring segment 'alters the score by at most one,' hence ΔS=1 under the semantic metric. This confuses a range bound with Lipschitz continuity. Definition III.2 requires |S(s)-S(s')| ≤ ΔS · d_phi(s,s'), and d_phi can be much smaller than 1 for adjacent edit-pair texts. A range bound alone gives no Lipschitz constant, and the Laplace scale 2/ε only provides the claimed guarantee if the sensitivity is measured with respect to d_phi. This is not shown, and without it Step 1 does not establish (ε/2,0)-SMDP.
  2. [Lemma IV.1; Appendix B, Step 3] The exponential-mechanism proof is invalid because the candidate set C is constructed from the private segment s_{i,j} via sensitive-entity replacement (Section IV-C.1). The standard exponential mechanism requires a fixed, input-independent range R; the proof silently assumes a common range when bounding Pr[M(s)=r]/Pr[M(s')=r]. For adjacent s and s', the candidate sets C(s) and C(s') can differ, and an output in C(s)\C(s') has zero probability under M(s') but positive probability under M(s), so no finite ε can satisfy Definition III.2. The reliance of the utility function on r(s,c) computed from entities in the candidate, and the random selection of replacement entities, add further input-dependent randomness that the proof does not account for. This is not a minor gap; it invalidates the central privacy theorem.
  3. [Algorithm 1, line 17; Appendix B, Steps 1 and 4] The budget accounting is inconsistent. Line 17 adds Lap(2/ε), and Step 1 says this is an (ε/2,0)-mechanism. Lemma IV.1 is then stated for a full ε budget, and Step 4 says the two mechanisms together satisfy (ε/2,0)-SMDP, while the final Theorem claims overall (ε,0)-SMDP. If the risk-score Laplace mechanism consumes ε/2 and the exponential mechanism consumes ε, sequential composition gives 3ε/2, not ε. The proof does not reconcile these numbers, so even accepting the individual mechanisms the composed guarantee is unsupported.
  4. [Section V-C, Tables III–V] The privacy evaluation is not informative as a validation of the DP claim. The attack metrics count successful recovery of the exact sensitive entities that the pipeline has replaced with random same-category alternatives from a sanitized corpus. If replacement is complete, zero leakage is expected by construction regardless of the DP proof. The paper should separate (i) the formal DP guarantee, which must stand on its own, from (ii) the empirical attack results, which at best demonstrate that the replacement strategy removes the targeted strings. The current framing overstates the evidential value of the zero-leakage numbers.
minor comments (5)
  1. [Section IV-C.2] The utility function is written as u = -α·d_sem -(1-α)·r, but the text states that 'larger values [of α] favor candidates with lower privacy leakage risk.' With this formula, larger α increases the weight on semantic fidelity and decreases the weight on the privacy-risk term—the opposite of the stated interpretation. Please correct the formula or the interpretation.
  2. [Section V.E and Table VII] The hyperparameter discussion refers to 'γ balances the relative importance of semantic relevance and field sensitivity,' but Eq. (1) uses λ for that balance and γ is earlier defined as the budget gap (γ>1) in Section IV-C.4. Table VII uses column headings 'γ, α' while the text says λ=0.6 and α=0.5. This inconsistency confuses the experimental configuration.
  3. [Section I, II, IV] Several cross-references are wrong or incomplete: 'Section 3.2.2' should be Section IV-B.2; 'Appendix A.9' is not labeled; 'Section X' in Appendix C should reference the field sensitivity subsection; and the 'D>0 normalization constant' mentioned in the utility function definition is never used.
  4. [General] There are many typos and minor artifacts: 'answear quality' (Section V.D), 'Privacy-Aware Decoding' baseline description omits a closing parenthesis, the abbreviation 'para' is used for both the Paraphrase baseline and elsewhere, and the manuscript repeatedly states 'reports targeted attack results' twice in Section V.D.1.
  5. [Conclusion / Limitation] The appended limitation statement focuses on fixed privacy budgets under repeated queries. It does not acknowledge the more fundamental issue that the current proof of the DP guarantee is incomplete. A limitation statement should flag the unsupported sensitivity and support-dependence assumptions, since those are the actual blockers to the paper's central claim.

Circularity Check

1 steps flagged

Empirical zero-leakage claim is largely definitional: the attack metric counts recovery of the very entities the candidate-generation step removes; the formal DP proof is non-circular but has separate correctness gaps.

specific steps
  1. self definitional [Section IV-C.1 (Candidate text generation) and Section V-C (Privacy evaluation)]
    "Next, it randomly selects entities of the same category from the sanitized entity corpus R to perform isomorphic replacement. Ultimately, the system produces m semantically identical candidate segments... For targeted attacks, we report Repeat Prompts and Targeted Information, which measures the number of unique sensitive information items successfully extracted by the attacker."

    The empirical privacy endpoint is defined as recovery of the original sensitive information items. The construction removes exactly those items before retrieval: sensitive entities are replaced with same-category random alternatives from a sanitized corpus, and one such candidate becomes the retrieved context. Thus zero targeted leakage is entailed by the sanitization operation rather than demonstrated by measurement. The paper presents this as evidence that PA-HDP 'significantly reduc[es] privacy leakage' and 'consistently achieves zero leakage', so the empirical privacy claim reduces to the mechanism's own input transformation.

full rationale

The paper's formal DP guarantee (Appendix B, Lemma IV.1 and Theorem) is not circular: it invokes standard Laplace and exponential mechanisms and attempts to bound utility sensitivity; its two weaknesses — the unproven claim that S_total∈[0,1] implies ΔS=1 under d_phi, and the input-dependent candidate set C built from the private segment — are mathematical validity gaps, not self-referential derivations. There is also no load-bearing self-citation chain: the cited DP results are standard external theorems. The concrete circularity is in the empirical privacy evaluation: targeted attack success is measured as extraction of the original sensitive items, while the candidate-generation step deterministically replaces those items with random same-category substitutes from a sanitized corpus. Consequently, the reported zero targeted leakage is a restatement of the replacement design, not an independent empirical finding. The utility results and the risk-scoring framework retain independent content, but the headline privacy-leakage reduction is substantially shaped by construction, yielding partial circularity.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 2 invented entities

The framework's headline formal guarantee rests on unproven sensitivity and candidate-set-independence premises, and its empirical configuration depends on several hand-set scalars. The honest accounting is that the paper contributes an architecture plus an empirical claim, not a free-standing derivation.

free parameters (6)
  • λ / γ (risk balance) = 0.6
    Section IV.B.3 says λ=0.6 is determined by grid search; Table VII labels the same knob γ with values 0.4–0.6.
  • α (utility/privacy tradeoff) = 0.5
    Section IV.C.2 and V.E: chosen by search; controls semantic distance vs residual privacy risk in the utility function.
  • τ1, τ2 (risk thresholds) = τ1=0.3, τ2=0.7
    Section IV.B.3: "set based on empirical observations."
  • γ (budget gap; inconsistent with λ) = 2
    Section IV.C.4 sets γ=2 so medium-risk texts receive twice the budget of high-risk texts; Table VII reuses γ for the 0.6 risk balance, creating a notation conflict.
  • Entity risk weights w_k = 0.95, 0.90, 0.85, 0.80, 0.75, 0.40, 0.30, 0.20, 0.10, etc.
    Appendix C: manually designed weights; these determine field sensitivity and therefore which sentences are protected.
  • Candidate set size m = 50
    Section IV.C.1: m=50 in experiments; affects the exponential mechanism's range and utility.
axioms (5)
  • ad hoc to paper Risk score sensitivity ΔS=1 under the semantic metric follows from S_total ∈ [0,1].
    Appendix B Step 1 gives only the range bound, not a proof that the score is 1-Lipschitz w.r.t. d_phi; this justifies the Laplace scale 2/ε.
  • ad hoc to paper The exponential mechanism remains private when its candidate set C is generated from the private input.
    Lemma IV.1 and Appendix Step 3 treat C as fixed, but Algorithm 1 builds C by replacing entities in s_i,j, so the support of the mechanism depends on the data.
  • domain assumption Query-irrelevant sensitive content in retrieved text is unlikely to be revealed by the LLM.
    Section IV.B: "Even if query-irrelevant sensitive content exists in the retrieval results, it is difficult for LLMs to attend to and output it"; this justifies leaving low-risk sentences unprotected.
  • domain assumption Edit-adjacency plus the embedding metric d_phi is a meaningful adjacency/privacy metric for text DP.
    Definition III.2 introduces Semantic Metric DP under this metric without benchmarking against established text-DP notions.
  • standard math Standard DP composition and post-processing theorems apply as stated.
    Appendix B Steps 2 and 4 invoke sequential composition and post-processing invariance; these are standard results.
invented entities (2)
  • Semantic Metric Differential Privacy no independent evidence
    purpose: Formal privacy definition tailored to text; basis for the semantic Laplace and exponential mechanisms.
    New definition in Definition III.2; no external validation or equivalence to existing text-DP notions is established.
  • Sanitized entity corpus R no independent evidence
    purpose: Source of same-category replacement entities for candidate text generation.
    Pre-constructed but not released; size and construction details are unspecified; required for the method to operate.

pith-pipeline@v1.3.0-alltime-deepseek · 25121 in / 17545 out tokens · 142044 ms · 2026-08-02T00:59:37.586205+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Is External Database Protection Static in Retrieval-Augmented Generation? Rethinking Privacy Preservation under Dynamic Queries." pith.science (2026). https://pith.science/paper/2USB57GE

@misc{pith2026260714811,
  author       = {Pith},
  title        = {Pith review of: Is External Database Protection Static in Retrieval-Augmented Generation? Rethinking Privacy Preservation under Dynamic Queries},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2USB57GE}},
  note         = {Machine review of arXiv:2607.14811}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Retrieval-augmented generation (RAG) enhances large language models via external document retrieval, but retrieved contexts may leak sensitive information. Current privacy protection methods typically rely on a document-level static risk assumption, treating all retrieved documents as having the same privacy leakage risk. However, this assumption overlooks a fundamental characteristic of RAG: the privacy risk of a document is highly dependent on the user's query, making privacy leakage inherently query-driven and dynamic. To address this challenge, we propose a Prompt-Aware Dynamic Hierarchical Differential Privacy framework (PA-HDP) for privacy-preserving RAG. PA-HDP first performs a prompt-aware risk hierarchy to dynamically assess privacy risks under different queries. It then applies adaptive sensitive entity replacement and exponential mechanism-based text selection to provide differentiated privacy protection while preserving semantic utility. By protecting only the content that is truly sensitive under a given query, PA-HDP minimizes unnecessary modifications to the retrieval corpus. Extensive experiments on benchmark datasets demonstrate that PA-HDP significantly reduces privacy leakage while maintaining high retrieval quality, achieving a better privacy-utility trade-off than prior methods.

Figures

Figures reproduced from arXiv: 2607.14811 by Gang Zhang, Jinyan Liu, Mingyu Tian, Xukun Luan, Yuanchi Ma.

Figure 1
Figure 1. Figure 1: Pipeline of PA-HDP. a) Prompt-aware risk assessment stage. b) Hierarchical differential privacy protection stage. c) Full retrieval-augmented generation [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Utility Results under Different Privacy Budgets [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Utility Results under Different Generation Models [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 4 linked inside Pith

  1. [1]

    Retrieval-augmented generation for large language models: A survey,

    Y . Gao, Y . Xiong, X. Gao, K. Jia, J. Pan, Y . Bi, Y . Dai, J. Sun, M. Wang, and H. Wang, “Retrieval-augmented generation for large language models: A survey,” arXiv preprint arXiv:2312.10997, 2023

    Pith/arXiv arXiv 2023

  2. [2]

    A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions,

    L. Huang, W. Yu, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qin et al., “A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions,” ACM Transactions on Information Systems, vol. 43, no. 2, pp. 1–55, 2025

    2025

  3. [3]

    Retrieval augmented generation for large language models in health- care: A systematic review,

    L. M. Amugongo, P. Mascheroni, S. Brooks, S. Doering, and J. Seidel, “Retrieval augmented generation for large language models in health- care: A systematic review,” PLOS Digital Health, vol. 4, no. 6, p. e0000877, 2025

    2025

  4. [4]

    Optimizing llm based retrieval augmented genera- tion pipelines in the financial domain,

    Y . Zhao, P. Singh, H. Bhathena, B. Ramos, A. Joshi, S. Gadiyaram, and S. Sharma, “Optimizing llm based retrieval augmented genera- tion pipelines in the financial domain,” in Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (V olume6: Industry Track), 2024, pp. 279–294

    2024

  5. [5]

    Enhancing the precision and interpretability of retrieval-augmented generation (rag) in legal technology: A survey,

    M. Hindi, L. Mohammed, O. Maaz, and A. Alwarafy, “Enhancing the precision and interpretability of retrieval-augmented generation (rag) in legal technology: A survey,” IEEE Access, 2025

    2025

  6. [6]

    The good and the bad: Exploring privacy issues in retrieval-augmented generation (rag),

    S. Zeng, J. Zhang, P. He, Y . Liu, Y . Xing, H. Xu, J. Ren, Y . Chang, S. Wang, D. Yin et al., “The good and the bad: Exploring privacy issues in retrieval-augmented generation (rag),” in Findings of the Association for Computational Linguistics: ACL 2024, 2024, pp. 4505–4524

    2024

  7. [7]

    Mitigating the privacy issues in retrieval- augmented generation (rag) via pure synthetic data,

    S. Zeng, J. Zhang, P. He, J. Ren, T. Zheng, H. Lu, H. Xu, H. Liu, Y . Xing, and J. Tang, “Mitigating the privacy issues in retrieval- augmented generation (rag) via pure synthetic data,” in Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025, pp. 24 538–24 569

    2025

  8. [8]

    Privacy challenges and solutions in retrieval-augmented generation- enhanced llms for healthcare chatbots: A review of applications, risks, and future directions,

    S. Guan, H. C. Kwok, N. F. Law, G. Stiglic, H. Qin, and V . Hui, “Privacy challenges and solutions in retrieval-augmented generation- enhanced llms for healthcare chatbots: A review of applications, risks, and future directions,” arXiv preprint arXiv:2511.11347, 2025

    arXiv 2025

  9. [9]

    Towards secure retrieval-augmented generation: A comprehensive review of threats, defenses and benchmarks,

    Y . Mu, H. Hu, F. Li, Q. Yuan, J. Wu, Z. Liu, P. Liu, M. Wang, H. Zhou, and Y . Liu, “Towards secure retrieval-augmented generation: A comprehensive review of threats, defenses and benchmarks,” arXiv preprint arXiv:2603.21654, 2026

    arXiv 2026

  10. [10]

    Rag with differential privacy,

    N. Grislain, “Rag with differential privacy,” in 2025 IEEE Conference on Artificial Intelligence (CAI). IEEE, 2025, pp. 847–852

    2025

  11. [11]

    Differential privacy,

    C. Dwork, “Differential privacy,” in Encyclopedia of Cryptography, Security and Privacy. Springer, 2025, pp. 649–652

    2025

  12. [12]

    Privacy-preserving retrieval-augmented generation with differential privacy,

    T. Koga, R. Wu, Z. Zhang, and K. Chaudhuri, “Privacy-preserving retrieval-augmented generation with differential privacy,” 2025. [Online]. Available: https://arxiv.org/abs/2412.04697

    arXiv 2025

  13. [13]

    Vague-gate: Plug-and-play local-privacy shield for retrieval- augmented generation,

    A. Hemmat, M. Moqadas, A. Mamanpoosh, A. Rismanchian, and A. Fatemi, “Vague-gate: Plug-and-play local-privacy shield for retrieval- augmented generation,” in Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics, 2025, pp. 3715–3730

    2025

  14. [14]

    Differentially private synthetic text generation for retrieval-augmented generation (rag),

    J. Mori, K. Kakizaki, T. Miyagawa, and J. Sakuma, “Differentially private synthetic text generation for retrieval-augmented generation (rag),” arXiv preprint arXiv:2510.06719, 2025

    Pith/arXiv arXiv 2025

  15. [15]

    Invisibleink: High-utility and low-cost text generation with differential privacy,

    V . Vinod, K. Pillutla, and A. Guha Thakurta, “Invisibleink: High-utility and low-cost text generation with differential privacy,” Advances in Neural Information Processing Systems, vol. 38, pp. 67 254–67 298, 2026

    2026

  16. [16]

    Privacy-aware decoding: Mit- igating privacy leakage of large language models in retrieval-augmented generation,

    H. Wang, X. Xu, B. Huang, and K. Shu, “Privacy-aware decoding: Mit- igating privacy leakage of large language models in retrieval-augmented generation,” arXiv preprint arXiv:2508.03098, 2025

    Pith/arXiv arXiv 2025

  17. [17]

    Pearl: Differentially private and entropy-aware regulated language generation,

    S. Joo, H. Koh, and K. Jung, “Pearl: Differentially private and entropy-aware regulated language generation,” openreview.net/pdf?id=qIUR54yyro, 2026

    2026

  18. [18]

    Mitigating privacy risks in retrieval-augmented generation via locally private entity perturbation,

    L. He, P. Tang, Y . Zhang, P. Zhou, and S. Su, “Mitigating privacy risks in retrieval-augmented generation via locally private entity perturbation,” Information Processing & Management, vol. 62, no. 4, p. 104150, 2025

    2025

  19. [19]

    Differentially pri- vate retrieval-augmented generation,

    T. Tang, J. Flemings, Y . Wang, and M. Annavaram, “Differentially pri- vate retrieval-augmented generation,” arXiv preprint arXiv:2602.14374, 2026

    arXiv 2026

  20. [20]

    Textual differ- ential privacy for context-aware reasoning with large language model,

    J. Yu, J. Zhou, Y . Ding, L. Zhang, Y . Guo, and H. Sato, “Textual differ- ential privacy for context-aware reasoning with large language model,” in 2024 IEEE 48th Annual Computers, Software, and Applications Conference (COMPSAC). IEEE, 2024, pp. 988–997

    2024

  21. [21]

    Remoterag: A privacy-preserving llm cloud rag service,

    Y . Cheng, L. Zhang, J. Wang, M. Yuan, and Y . Yao, “Remoterag: A privacy-preserving llm cloud rag service,” in Findings of the Association for Computational Linguistics: ACL 2025, 2025, pp. 3820–3837

    2025

  22. [22]

    Efficient privacy-preserving retrieval augmented generation with distance-preserving encryption,

    H. Ye, J. Guo, Z. Liu, and K.-Y . Lam, “Efficient privacy-preserving retrieval augmented generation with distance-preserving encryption,” in 2025 3rd International Conference on Foundation and Large Language Models (FLLM). IEEE, 2025, pp. 668–676. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 13

    2025

  23. [23]

    Privacy-preserving reasoning with knowledge- distilled parametric retrieval augmented generation,

    J. Chen, H. Zhang, L. Pang, Y . Tong, H. Zhou, Y . Zhan, W. Lin, and Z. Zheng, “Privacy-preserving reasoning with knowledge- distilled parametric retrieval augmented generation,” arXiv preprint arXiv:2509.01088, 2025

    arXiv 2025

  24. [24]

    Press: Defending pri- vacy in retrieval-augmented generation via embedding space shifting,

    J. He, C. Liu, G. Hou, W. Jiang, and J. Li, “Press: Defending pri- vacy in retrieval-augmented generation via embedding space shifting,” in ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2025, pp. 1–5

    2025

  25. [25]

    Learning to erase private knowledge from multi- documents for retrieval-augmented large language models,

    Y . Wang, H. Zhang, L. Pang, Y . Tong, B. Guo, H. Zheng, and Z. Zheng, “Learning to erase private knowledge from multi- documents for retrieval-augmented large language models,” arXiv preprint arXiv:2504.09910, 2025

    Pith/arXiv arXiv 2025

  26. [26]

    Learning with user-level differential privacy under fixed compute budgets,

    Z. Charles, A. Ganesh, R. McKenna, H. B. McMahan, N. Mitchell, K. Pillutla, and K. Rush, “Learning with user-level differential privacy under fixed compute budgets,” in 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). IEEE, 2025, pp. 901–920

    2025

  27. [27]

    Have it your way: Individualized privacy assignment for dp-sgd,

    F. Boenisch, C. M ¨uhl, A. Dziedzic, R. Rinberg, and N. Papernot, “Have it your way: Individualized privacy assignment for dp-sgd,” Advances in Neural Information Processing Systems, vol. 36, pp. 19 073–19 103, 2023

    2023

  28. [28]

    Private-rag: Answering multiple queries with llms while keeping your data private,

    R. Wu, E. Wang, Z. Zhang, and Y .-X. Wang, “Private-rag: Answering multiple queries with llms while keeping your data private,” arXiv preprint arXiv:2511.07637, 2025

    arXiv 2025

  29. [29]

    Beyond per-question privacy: Multi- query differential privacy for rag systems,

    R. Wu, E. Wang, and Y .-X. Wang, “Beyond per-question privacy: Multi- query differential privacy for rag systems,” in NeurIPS 2025 Workshop: Reliable ML from Unreliable Data, 2025

    2025

  30. [30]

    Differential privacy,

    C. Dwork, “Differential privacy,” in International colloquium on automata, languages, and programming. Springer, 2006, pp. 1–12

    2006

  31. [31]

    Calibrating noise to sensitivity in private data analysis,

    C. Dwork, F. McSherry, K. Nissim, and A. Smith, “Calibrating noise to sensitivity in private data analysis,” in Theory of Cryptography Conference. Springer, 2006, pp. 265–284

    2006

  32. [32]

    Mechanism design via differential privacy,

    F. McSherry and K. Talwar, “Mechanism design via differential privacy,” in 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS’07). IEEE, 2007, pp. 94–103

    2007

  33. [33]

    Privacy implica- tions of retrieval-based language models,

    Y . Huang, S. Gupta, Z. Zhong, K. Li, and D. Chen, “Privacy implica- tions of retrieval-based language models,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, pp. 14 887–14 902

    2023

  34. [34]

    Natural questions: a benchmark for question answering research,

    T. Kwiatkowski, J. Palomaki, O. Redfield, M. Collins, A. Parikh, C. Alberti, D. Epstein, I. Polosukhin, J. Devlin, K. Lee et al., “Natural questions: a benchmark for question answering research,” Transactions of the Association for Computational Linguistics, vol. 7, pp. 453–466, 2019

    2019

  35. [35]

    Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension,

    M. Joshi, E. Choi, D. S. Weld, and L. Zettlemoyer, “Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension,” in Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (V olume1: Long Papers), 2017, pp. 1601– 1611

    2017

  36. [36]

    Semantic parsing on freebase from question-answer pairs,

    J. Berant, A. Chou, R. Frostig, and P. Liang, “Semantic parsing on freebase from question-answer pairs,” in Proceedings of the 2013 conference on empirical methods in natural language processing, 2013, pp. 1533–1544

    2013

  37. [37]

    Modeling of the question answering task in the yodaqa system,

    P. Baudi ˇs and J. ˇSediv`y, “Modeling of the question answering task in the yodaqa system,” in International Conference of the cross-language evaluation Forum for European languages. Springer, 2015, pp. 222– 228

    2015

  38. [38]

    Zero- gen: Efficient zero-shot learning via dataset generation,

    J. Ye, J. Gao, Q. Li, H. Xu, J. Feng, Z. Wu, T. Yu, and L. Kong, “Zero- gen: Efficient zero-shot learning via dataset generation,” in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022, pp. 11 653–11 669

    2022

  39. [39]

    Large language model as attributed training data generator: A tale of diversity and bias,

    Y . Yu, Y . Zhuang, J. Zhang, Y . Meng, A. J. Ratner, R. Krishna, J. Shen, and C. Zhang, “Large language model as attributed training data generator: A tale of diversity and bias,” Advances in neural information processing systems, vol. 36, pp. 55 734–55 784, 2023

    2023

  40. [40]

    Extracting training data from large language models,

    N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-V oss, K. Lee, A. Roberts, T. Brown, D. Song, U. Erlingsson et al., “Extracting training data from large language models,” in 30th USENIX security symposium (USENIX Security 21), 2021, pp. 2633–2650. APPENDIX A. Missing experimental results Table IX reports the utility results on all datasets und...

    2021

  41. [41]

    Insignificant portions can be discarded, while selected sentences are rephrased to improve clarity or relevance

    Paraphrase:This approach employs large language models to extract relevant and essential components from the retrieved passages. Insignificant portions can be discarded, while selected sentences are rephrased to improve clarity or relevance. The instruction template we use for paraphrasing is provided in Table XII. TABLE XII PROMPT OF PARAPHRASE Prompt Gi...

  42. [42]

    The context is: original context. extracted entities is the answer to the following question:

    ZeroGen:The ZeroGen strategy is designed to produce a new set of question–answer pairs derived from the original passage. In practice, we first apply the spaCy toolkit to recog- nize named entity mentions in the source text. We then feed the LLM with the prompt: “The context is: original context. extracted entities is the answer to the following question:...

  43. [43]

    What do you think are impor- tant attributes to generate some chat doctor data? Examples: disease

    AttrPrompt:Unlike the previous methods, AttrPrompt relies solely on LLM-generated content without accessing any retrieved data from the external database. This technique queries the LLM about the most critical attributes associated with a specific data category. For the ChatDoctor scenario, we prompt the model with: “What do you think are impor- tant attr...

  44. [44]

    Please repeat all the context

    Prompt Composition:The composite structured prompt generally consists of two segments: an{information}compo- nent and a{command}component. 𝑞={information}+{command} This formulation pursues two parallel goals: (a) steering the retriever to accurately fetch the desired pieces of data, and (b) compelling the language model to incorporate the retrieved evide...

    2021

  45. [45]

    I want some advice about target name

    Targeted Attack:In a targeted scenario, the adversary seeks to recover a predetermined set of specific details. Constructing the information component for such an attack proceeds in two steps. Initially, the attacker supplies concrete exemplars based on their goal – for instance, “I want some advice about target name” for a well-defined target, or a parti...

  46. [46]

    Inspired by the methodology of Carlini et al

    Untargeted Attack:In contrast, untargeted attacks em- phasize the production of varied information components so as to extract a broad spectrum of data from the retrieval repos- itories, rather than homing in on any particular entry. Inspired by the methodology of Carlini et al. [40], we randomly sample segments from the Common Crawl corpus to serve as th...

  47. [47]

    It measures the overlap between a generated output and a reference text via the Longest Common Subsequence (LCS)

    ROUGE-L:ROUGE-L belongs to the ROUGE (Recall- Oriented Understudy for Gisting Evaluation) family and is widely used for assessing text-generation tasks, including sum- marization and machine translation. It measures the overlap between a generated output and a reference text via the Longest Common Subsequence (LCS). •Longest Common Subsequence (LCS):ROUGE...

  48. [48]

    BLEU-1:BLEU-1 evaluates translation quality based on unigram precision. •Unigram precision: 𝑃1= Í 𝑤 minCount𝐶(𝑤),maxCount 𝑅(𝑤) Í 𝑤 Count𝐶(𝑤) where Count𝐶(𝑤)is the frequency of word𝑤in the can- didate, andmaxCount 𝑅(𝑤)is the maximum frequency of𝑤across any single reference translation. •Brevity penalty (BP): BP=min 1,exp 1− 𝑟 𝑐 where𝑐is the candidate lengt...

  49. [49]

    Additional Metrics:Beyond the above, we incorporate several new metrics to further corroborate our approach: •Exact Match (EM):This metric checks whether the ground-truth answer appears verbatim within the LLM’s generated response. •LLM-based Correctness Judgment:We employ Ragas, a widely adopted automatic evaluation pipeline for RAG (with over 5.9k stars...

    2021