REVIEW 3 major objections 4 minor 35 references
Privacy-Preserving RAG by Concealing Sensitive Information from External LLMs
T0 review · 3 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read The SEAG framework hides sensitive entities from external LLMs in RAG pipelines while preserving answer quality.
desk verdict Plausible privacy-preserving RAG wrapper that overclaims: its own entity-replacement accuracy leaves a quarter of sensitive entities exposed to the external generator. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The entity replacement table is the central mechanism. The locally deployed SEAG model examines the user query and the retrieved documents, identifies the sensitive entities, and emits a table mapping each original entity to a consistent alias, for example New Zealand to Australia and 4.2% to 5%. The same table is applied to every occurrence of each entity across the query and all documents, so the external generator sees coherent, semantically plausible text rather than inconsistent placeholders. After the generator answers, the table is applied in reverse to any aliases appearing in the response, restoring the original values before the answer reaches the user. This two-way table is what lets the framework preserve both privacy and utility.
What would settle it
Send the SEAG-replaced prompts from the evaluation dataset to a capable LLM with an instruction to infer the original entities, and measure how often it recovers the true values; if recovery succeeds on a nontrivial fraction, the framework's confidentiality claim fails.
Extended reading notes
Core claim
The central discovery is that a fine-tuned 3–4 billion parameter model can reliably locate sensitive entities in a RAG prompt, generate meaningful and consistent aliases, and restore the original values in the final answer, so the external generator never sees the real sensitive information. The paper's experiments use three locally fine-tuned models (Qwen-3, LLaMA-3.2, and Phi-4) and two external generators (GPT-5 and Claude-4 sonnet). On the Privacy metric, which checks whether the entities needed to answer the question were hidden, all models scored between 84.83% and 89.67%. On the User metric, which checks whether the final answer presented to the user is correct after alias restoration, all models scored above 80%. The paper argues this shows the framework succeeds at shielding sensitive content from the generator while keeping answer quality high.
Load-bearing premise
The privacy guarantee assumes the external generator cannot recover the original sensitive entities from the aliased prompt, but the paper never tests whether a capable LLM can infer the original values from context, alias patterns, or world knowledge.
Editorial extensions
If this is right
- Organizations can keep using third-party LLM generators in RAG systems without sending raw sensitive entities to the generator.
- A small local model of roughly 3–4 billion parameters is sufficient for the anonymization step, avoiding the cost of hosting a large generator locally.
- Answer quality remains high: all tested configurations exceeded 80% User accuracy, so users still receive correct answers after alias restoration.
- Privacy scores up to 89.67% indicate that most question-critical entities are hidden from the external generator, though full-document entity hiding is lower, around 75–78%.
Reading between the lines
- If the external generator can infer original entities from context or auxiliary knowledge, the alias replacement alone does not guarantee confidentiality; the paper does not test this adversarial de-anonymization.
- The framework could be extended to structured knowledge bases or knowledge graphs, where alias consistency across relations becomes even more critical.
- A natural testable extension is to evaluate SEAG against a generator explicitly prompted to recover the original entities, measuring how often aliases are reversible.
- The entity replacement table could be combined with differential privacy or synthetic data techniques to strengthen protection beyond what a single alias mapping provides.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes SEAG (Sensitive Entity Alias Generator), a privacy-preserving framework for RAG. A locally deployed fine-tuned model (Qwen-3, LLaMA-3.2, or Phi-4) identifies sensitive entities in the user query and retrieved documents, generates aliases for them, and constructs an entity replacement table. The query and documents are anonymized with this table before being sent to an external LLM generator (GPT-5 or Claude-4 sonnet); the generator's answer is then reverse-mapped to restore the original values for the user. The authors construct two datasets (one for fine-tuning, one for evaluation), fine-tune three 3-4B models with QLoRA, and report ORG, Privacy, and User metrics. Results show ORG scores near 99%, Privacy scores up to 89.67%, and User scores above 80% for all combinations. The paper also reports an entity-level evaluation in Table 2 showing total entity replacement accuracy of 77.83% (Qwen-3), 76.73% (LLaMA-3.2), and 74.91% (Phi-4).
Significance. If the central claim holds—that SEAG prevents external generators from accessing sensitive information while preserving answer quality—the framework would be practically valuable: organizations could use powerful third-party LLMs over private documents without exposing raw sensitive values, at the cost of deploying only a small local model. The paper provides a concrete pipeline, two constructed datasets, and a reproducibility-oriented release (datasets and code on GitHub). It also applies a consistent entity-alignment methodology and evaluates across multiple base models and two commercial generators. However, the significance is currently limited by a gap between the stated threat model and the measured behavior: the paper's own entity-level evaluation shows that roughly one in four sensitive entities is not replaced, which directly undermines the unqualified privacy claim. The significance will be substantially stronger if the authors address this gap and provide a direct measure of what the external generator actually receives.
major comments (3)
- [§4.2 vs. §5.4 (Table 2)] The threat model in Section 4.2 states a necessary condition: 'The fine-tuned model must correctly identify and replace all sensitive entities consistently. A missing entity may expose private information.' Yet Table 2 shows that total entity replacement accuracy is only 74.91%–77.83%, meaning about one in four sensitive entity occurrences in the documents remains in its original form and is forwarded to the external generator. The Privacy metric in Section 5.2 only checks the specific entity required to answer the user's question, so it does not detect leakage of other sensitive entities. Consequently, the evidence does not support the abstract's claim that SEAG 'prevents the external generator from accessing sensitive information.' The authors should either modify the claim to reflect probabilistic protection, or add a fail-closed mechanism (e.g., abstaining from sending prompts containing unreplaced entities) so that the stated condition is actually met.
- [§5.2 (Privacy metric)] The Privacy metric is defined operationally as the generator failing to produce the ground-truth answer (e.g., answering 'Mike' instead of 'John'). This conflates 'hiding' with 'answer failure': the generator might fail for reasons unrelated to replacement (e.g., ambiguity in the aliased prompt), or it might correctly infer the original entity from context even when the entity is replaced, which would be a privacy failure not captured by this metric. The paper does not directly inspect the anonymized prompts to verify that the original sensitive strings are absent. A direct leakage measurement—for example, checking the forwarded prompt for occurrences of original entities—is needed to substantiate the privacy claim.
- [§4.2 and §6 (threat model scope)] The threat model treats the external generator as a passive recipient of the altered prompt and assumes that the generator 'does not have direct access to the original sensitive information.' However, confidentiality also requires that the generator cannot reconstruct the original values from the aliases, context, or world knowledge. The paper provides no test of whether a capable LLM can reverse the replacement (e.g., inferring 'New Zealand' from 'Australia' or '4.2%' from '5%' in the running example). If such inference is possible, even perfect replacement would not protect confidentiality. The limitations section should discuss this, and ideally the authors should run an empirical reconstruction-attack evaluation.
minor comments (4)
- [§3.1] The text says 'There has been a strong emphasis on extracting all name entities, not just a few'—replace 'name entities' with 'named entities.'
- [§5.2] The phrase 'theORGmetric' lacks a space; it should read 'the ORG metric.'
- [Appendix A (Table A.3)] The replacement table lists entity types such as 'Figure 1.1' and 'Figure 2.2' as original entities, which appear to be figure references rather than sensitive entities; clarify whether these are part of the evaluated entity set or an artifact of the example.
- [§6 (Limitations)] The limitations section discusses document length and entity density but does not acknowledge the incomplete-replacement issue revealed by Table 2. Adding a discussion of the 25% leakage rate and its implications for the privacy guarantee would make the limitations more complete.
Circularity Check
No significant circularity: the SEAG results are empirical measurements, not derivations from the framework's own output; the main gap is an unmet completeness condition, which is a correctness issue, not a circular one.
full rationale
The paper contains no formal derivation chain whose conclusion is equivalent to its premises. The SEAG model is fine-tuned on a dataset generated by DeepSeek-V3 under explicit criteria (Section 3.1) and evaluated on a separately constructed 600-sample dataset (Section 3.2) with manual entity identification; the test labels are not used to fit any parameter, so this is a closed-loop evaluation rather than a circular reduction. The User metric (Section 5.2) does use the framework's own replacement table to restore aliases, but that is the intended mechanism of the system, and the Privacy and ORG metrics are defined independently; the numerical dependence User ≤ Privacy is a logical consequence of the metric definitions, not a fitted input renamed as a prediction. The only self-citations ([34], [35]) support the choice of DeepSeek-V3 for Arabic understanding and are not load-bearing for the central privacy claim. The paper itself states the central condition in Section 4.2 that 'the fine-tuned model must correctly identify and replace all sensitive entities consistently,' and Section 5.4 reports total accuracies of only 77.83%, 76.73%, and 74.91%, which contradicts that condition; however, an internal inconsistency between a stated assumption and measured performance is a correctness/falsification problem, not circularity. No step in the paper reduces by construction to its own input.
Assumptions & free parameters
free parameters (4)
- Fine-tuning hyperparameters (learning rate 5e-4, batch size 8, 5 epochs, QLoRA) =
LR 5e-4, batch size 8, 5 epochs, QLoRA (rank not stated)
- Training dataset size and domain split =
640 samples from 8 domains
- Evaluation dataset size and question design =
600 samples from 6 domains, 2 questions per document
- DeepSeek-V3 prompting criteria for gold replacement tables =
Criteria: exact syntactic match, consistency, and form preservation
assumptions (4)
- domain assumption The external generator cannot access the replacement table and cannot infer original sensitive values from the replaced prompt.
- domain assumption The fine-tuned SEAG model correctly identifies all sensitive entities and generates consistent, semantically preserving aliases.
- domain assumption The authors' manually constructed datasets provide an unbiased and representative measure of privacy-preserving RAG performance.
- domain assumption Entity substitution preserves enough of the original question's meaning for the external generator to produce the correct answer under the alias.
Cite this review
Pith. "Pith review of Privacy-Preserving RAG by Concealing Sensitive Information from External LLMs." pith.science (2026). https://pith.science/paper/7HO6F4PB
@misc{pith2026260812675,
author = {Pith},
title = {Pith review of: Privacy-Preserving RAG by Concealing Sensitive Information from External LLMs},
year = {2026},
howpublished = {\url{https://pith.science/paper/7HO6F4PB}},
note = {Machine review of arXiv:2608.12675}
}
read the original abstract
Retrieval-Augmented Generation (RAG) is widely used to improve the performance of Large Language Models (LLMs) in answering user queries. Existing privacy research on RAG has focused on preventing unauthorized users from accessing sensitive data. However, another important problem that is often overlooked in RAG privacy research is that external generators have access to the query and the retrieved documents, which may contain confidential information that could potentially be misused or accessed for unintended purposes. In this paper, we introduce the Sensitive Entity Alias Generator (SEAG), a privacy-preserving framework that empowers users to utilize powerful third-party generators without disclosing sensitive information. SEAG introduces a lightweight model that locates sensitive entities, generates corresponding aliases, and constructs an entity replacement table. The table is used to replace sensitive words in the user's query and in the retrieved documents before they are forwarded to an external generator. For this purpose, two datasets were constructed: one for fine-tuning SEAG models to generate entity replacement tables, and another for evaluating the entire SEAG framework. The experimental results demonstrate the success of the SEAG framework. As for the User metric, which measures the ability of the model to provide a correct response to the user while hiding sensitive information from the external generator, all SEAG models achieved over 80% accuracy. Additional analysis further evaluated the ability of SEAG models Qwen-3, LLaMA-3.2, and Phi-4 to hide all sensitive entities within given documents. The results show good performance with total accuracies of 77.83%, 76.73%, and 74.91%, respectively.
Figures
Reference graph
Works this paper leans on
-
[1]
S. Zeng, J. Zhang, P. He, Y . Xing, Y . Liu, H. Xu, J. Ren, S. Wang, D. Yin, Y . Chang, J. Tang, The good and the bad: Exploring privacy issues in retrieval-augmented generation (rag), in: Find- ings of the Association for Computational Lin- guistics: ACL 2024, Association for Computa- tional Linguistics, Bangkok, Thailand, 2024, pp. 4505–4524.doi:10.1865...
-
[2]
Y . Zhou, W. Zhang, J. Shao, Y . Liu, X. Li, J. Jin, H. Qian, Z. Liu, C. Li, J. C. Zhang, Z. Dou, P. S. Yu, J. Mao, Trustworthiness in retrieval- augmented generation systems: A survey, arXiv 9 preprint arXiv:2409.10102 (2024).doi:10.485 50/arXiv.2409.10102
- [3]
-
[4]
R. Nagpal, U. Usua, R. Palacios, A. Gupta, FairRAG: A privacy-preserving framework for fair financial decision-making, Applied Sciences 15 (15) (2025) 8282.doi:10.3390/app15158 282
-
[5]
T. B. Weerasekara, C. Chandeepa, O. S. Amara- suriya, C. Hettiarachchi, Privacy-preserving med- ical advising system on mobile devices: On- device phi anonymization, medical report re- trieval, and cloud-based rag, in: Proceedings of the ACM/IEEE International Conference on Con- nected Health: Applications, Systems and Engi- neering Technologies, Associat...
- [6]
-
[7]
OpenAI, GPT-5 system card, OpenAI, accessed: 12 August 2026 (Aug. 2025). URLhttps://openai.com/index/gpt-5-s ystem-card/
work page 2026
-
[8]
URLhttps://www.anthropic.com/news/c laude-4
Anthropic, Introducing Claude 4, Anthropic, ac- cessed: 12 August 2026 (May 2025). URLhttps://www.anthropic.com/news/c laude-4
work page 2026
Show all 35 references
-
[9]
N. G. Itoi, Be careful what you tell your AI chat- bot, Stanford Institute for Human-Centered Artifi- cial Intelligence, accessed: 12 August 2026 (Oct. 2025). URLhttps://hai.stanford.edu/news/be -careful-what-you-tell-your-ai-chatb ot
2026
-
[10]
Financial Times, Florida sues OpenAI and sam alt- man for “hurting” children, Financial Times, ac- cessed: 12 August 2026 (Jun. 2026). URLhttps://www.ft.com/content/7998d b0a-22d1-4b80-b861-2e8b2448e97f
2026
-
[11]
Pollina, A
E. Pollina, A. Armellini, Italy fines OpenAI over ChatGPT privacy rules breach, Reuters, accessed: 12 August 2026 (dec 2024). URLhttps://www.reuters.com/technolo gy/italy-fines-openai-15-million-eur os-over-privacy-rules-breach-2024-1 2-20/
2026
-
[12]
URLhttps://www.nvidia.com/en-us/data -center/h100/
NVIDIA Corporation, NVIDIA H100 GPU, NVIDIA, accessed: 12 August 2026. URLhttps://www.nvidia.com/en-us/data -center/h100/
2026
- [13]
-
[14]
URLhttps://github.com/meta-llama/lla ma-models/blob/main/models/llama3_2/MO DEL_CARD.md
Meta AI, Llama 3.2 model card, accessed: 12 Au- gust 2026 (2024). URLhttps://github.com/meta-llama/lla ma-models/blob/main/models/llama3_2/MO DEL_CARD.md
2024
-
[15]
Abouelenin, A
A. Abouelenin, A. Ashfaq, A. Atkinson, H. Awadalla, N. Bach, J. Bao, A. Benhaim, M. Cai, V . Chaudhary, C. Chen, et al., Phi-4- Mini technical report: Compact yet powerful multimodal language models via mixture-of- LoRAs, arXiv preprint arXiv:2503.01743 (2025). doi:10.48550/ar...
-
[16]
Dettmers, A
T. Dettmers, A. Pagnoni, A. Holtzman, L. Zettle- moyer, QLoRA: Efficient finetuning of quantized LLMs, in: Advances in Neural Information Pro- cessing Systems, V ol. 36, Curran Associates, Inc., 2023, pp. 10088–10115.doi:10.52202/07528 0-0441. URLhttps://proceedings.neurips.cc...
2023 doi
- [17]
-
[18]
Z. Qi, H. Zhang, E. P. Xing, S. M. Kakade, H. Lakkaraju, Follow my instruction and spill the 10 beans: Scalable data extraction from retrieval- augmented generation systems, in: The Thirteenth International Conference on Learning Representa- tions, International Conference on ...
2025
-
[19]
Y . Li, Z. Li, K. Zhang, R. Dan, S. Jiang, Y . Zhang, ChatDoctor: A medical chat model fine-tuned on a large language model meta-ai (LLaMA) using medical domain knowledge, GitHub repository, accessed: 12 August 2026 (2023). URLhttps://github.com/Kent0n-Li/Chat Doctor
2023
-
[20]
G. Wang, J. He, H. Li, M. Zhang, D. Feng, RAG- leaks: Difficulty-calibrated membership inference attacks on retrieval-augmented generation, Sci- ence China Information Sciences 68 (6) (2025) 160102.doi:10.1007/s11432-024-4441-4
2025 doi
-
[21]
M. Liu, S. Zhang, C. Long, Mask-based mem- bership inference attacks for retrieval-augmented generation, in: Proceedings of the ACM Web Con- ference 2025, WWW ’25, Association for Com- puting Machinery, Sydney, NSW, Australia, 2025, pp. 2894–2907.doi:10.1145/3696410.3714 771
2025
-
[22]
W. Zou, R. Geng, B. Wang, J. Jia, Poi- sonedRAG: Knowledge corruption attacks to Retrieval-Augmented generation of large language models, in: 34th USENIX Security Symposium (USENIX Security 25), USENIX Association, Seattle, W A, 2025, pp. 3827–3844. URLhttps://www.usenix.org/c...
2025
-
[23]
Chaudhari, G
H. Chaudhari, G. Severi, J. Abascal, A. Suri, M. Jagielski, C. A. Choquette-Choo, M. Nasr, C. Nita-Rotaru, A. Oprea, Phantom: General backdoor attacks on retrieval augmented language generation, ACM Transactions on AI Security and Privacy (Mar. 2026).doi:10.1145/3796729. URLht...
2026 doi
-
[24]
S. Zeng, J. Zhang, P. He, J. Ren, T. Zheng, H. Lu, H. Xu, H. Liu, Y . Xing, J. Tang, Mitigating the privacy issues in retrieval-augmented generation (RAG) via pure synthetic data, in: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Assoc...
2025 doi
-
[25]
Y . Yu, Y . Zhuang, J. Zhang, Y . Meng, A. J. Rat- ner, R. Krishna, J. Shen, C. Zhang, Large language model as attributed training data generator: A tale of diversity and bias, in: Advances in Neural Infor- mation Processing Systems, V ol. 36, Curran Asso- ciates, Inc., 2023.d...
2023 doi
-
[26]
T. Tang, J. Flemings, Y . Wang, M. Annavaram, Differentially private retrieval-augmented gen- eration, arXiv preprint arXiv:2602.14374 (Feb. 2026).doi:10.48550/arXiv.2602.14374. URLhttps://arxiv.org/abs/2602.14374
2026 doi
-
[27]
Grislain, RAG with differential privacy, in: 2025 IEEE Conference on Artificial Intelligence (CAI), IEEE, 2025, pp
N. Grislain, RAG with differential privacy, in: 2025 IEEE Conference on Artificial Intelligence (CAI), IEEE, 2025, pp. 847–852. URLhttps://ieeexplore.ieee.org/docu ment/11050672
2025
-
[28]
E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chen, LoRA: Low-rank adaptation of large language models, in: Inter- national Conference on Learning Representations, 2022. URLhttps://openreview.net/forum?id= nZeVKeeFYf9
2022
-
[29]
Pingua, A
B. Pingua, A. Sahoo, M. Kandpal, D. Murmu, J. Rautaray, R. K. Barik, M. J. Saikia, Medical LLMs: Fine-tuning vs. retrieval-augmented gen- eration, Bioengineering 12 (7) (2025) 687.doi: 10.3390/bioengineering12070687. URLhttps://doi.org/10.3390/bioengin eering12070687
2025 doi
-
[30]
Devine, ALoFTRAG: Automatic local fine tun- ing for retrieval augmented generation, arXiv preprint arXiv:2501.11929 (Jan
P. Devine, ALoFTRAG: Automatic local fine tun- ing for retrieval augmented generation, arXiv preprint arXiv:2501.11929 (Jan. 2025).doi:10 .48550/arXiv.2501.11929. URLhttps://arxiv.org/abs/2501.11929
-
[31]
Y . He, X. Zhu, D. Li, H. Wang, Enhancing large language models for specialized domains: A two- stage framework with parameter-sensitive LoRA 11 fine-tuning and chain-of-thought RAG, Electron- ics 14 (10) (2025) 1961.doi:10.3390/electr onics14101961. URLhttps://doi.org/10.3390...
2025 doi
-
[32]
S. B. Islam, M. A. Rahman, K. S. M. T. Hos- sain, E. Hoque, S. Joty, M. R. Parvez, Open- RAG: Enhanced retrieval augmented reasoning with open-source large language models, in: Find- ings of the Association for Computational Lin- guistics: EMNLP 2024, Association for Compu- ta...
2024 doi
- [33]
-
[34]
Almohaimeed, S
S. Almohaimeed, S. Almohaimeed, M. Jari, K. A. Alobaid, F. Alotaibi, AI text detectors and the mis- classification of slightly polished arabic text, Jour- nal of Big Data (Jun. 2026).doi:10.1186/s405 37-026-01492-8. URLhttps://doi.org/10.1186/s40537-0 26-01492-8
2026 doi
-
[35]
Almohaimeed, A
S. Almohaimeed, A. Alabduljabbar, M. Jari, M. Alkhowaiter, S. Almohaimeed, M. M. Al Rah- hal, Benchmarking retrieval augmented generation LLMs for arabic noise robustness, Frontiers in Big Data 9 (2026).doi:10.3389/fdata.2026.188 4673. URLhttps://doi.org/10.3389/fdata.20 26.18...
2026 doi
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.