pith. sign in

arxiv: 2606.26627 · v1 · pith:DZYLXMSMnew · submitted 2026-06-25 · 💻 cs.CR · cs.AI

Agents That Know Too Much: A Data-Centric Survey of Privacy in LLM Agents

Nada Lahjouji , Ashwin Gerard Colaco This is my paper

Pith reviewed 2026-06-26 04:26 UTC · model grok-4.3

classification 💻 cs.CR cs.AI
keywords LLM agentsprivacy risksinformation-flow controldata-centric surveycompositional leakagecross-session leakagebenchmarksgovernance mechanisms
0
0 comments X

The pith

Only information-flow control covers both compositional and cross-session inference leakage in LLM agents, while no benchmark evaluates agents across their data surfaces under one privacy policy.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper surveys privacy in LLM agents that query databases, search documents, call APIs, retain memory, and act across sessions. It organizes the scattered literature by the data surfaces an agent touches instead of by attack type. Two findings stand out: among available governance mechanisms, only information-flow control addresses the two least-protected risks of compositional leakage and cross-session inference leakage. No existing benchmark runs an agent across all its data surfaces while enforcing a single privacy policy. The survey maps risks, mechanisms, and benchmarks to highlight these gaps and open problems.

Core claim

The central claim is that a data-centric taxonomy of LLM agent data sources reveals that only information-flow control among governance mechanisms addresses both compositional and cross-session inference leakage—the two least-protected risks—while no benchmark drives an agent across its data surfaces under one unified privacy policy.

What carries the argument

The data-centric taxonomy that classifies an agent's data sources (databases, document collections, APIs, memory stores, and inter-agent messages) and maps the privacy risks and governance mechanisms each source creates.

If this is right

  • Future agent designs should incorporate information-flow control to close the identified leakage gaps.
  • Benchmarks must be developed that evaluate agents holistically across data surfaces under a single policy.
  • Research on retrieval-augmented generation, text-to-SQL, agent memory, and prompt injection can be unified under the data-source taxonomy.
  • Other governance mechanisms leave measurable gaps in protecting compositional and cross-session risks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Agent systems that maintain state across sessions may require explicit information-flow tracking to prevent unintended leakage over time.
  • Developers could test new benchmarks by simulating multi-step workflows that touch databases, memory, and external APIs in one run.
  • The taxonomy suggests that privacy policies for delegated agent actions should explicitly address how intermediate results propagate.

Load-bearing premise

The collected papers are representative enough of the broader field that the two recurring findings hold and the taxonomy does not omit major risk categories or defenses.

What would settle it

A review that identifies either a governance mechanism other than information-flow control that covers both compositional and cross-session leakage or an existing benchmark that tests an agent across multiple data surfaces under one privacy policy.

Figures

Figures reproduced from arXiv: 2606.26627 by Ashwin Gerard Colaco, Nada Lahjouji.

Figure 1
Figure 1. Figure 1: A data agent and the surfaces it operates over. The agent constructs queries to and receives results from heterogeneous [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Organization of the survey. We take a data-centric view of privacy in LLM agents, structured around the data an agent [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Privacy across a data agent’s execution. As the agent constructs a query, retrieves results, transforms and aggregates [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Worked case studies across the five application domains of Section 7, each a short agent conversation with the [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
read the original abstract

Large language model agents increasingly query databases, search document collections, call external APIs, remember past interactions, and act on a user's behalf. As they move from answering questions to operating over sensitive data, privacy becomes harder to enforce. An agent touches many data sources, runs multi-step workflows, keeps state across sessions, and acts with delegated permissions. Sensitive information can therefore leak not only through its final answer but through the queries it issues, the intermediate results it handles, the memory it writes, and the messages it exchanges with other agents. We survey the privacy of LLM agents from a data-centric view, organizing the field around the data an agent touches rather than by attack type, and we use data agent as shorthand for an LLM agent that works with data. Research on these risks is active but scattered across retrieval-augmented generation, text-to-SQL interfaces, agent memory, prompt injection, access control, and contextual privacy. This survey brings that work together: we taxonomize the data sources an agent touches, the privacy risks each source creates, and the governance mechanisms that address them; we map the benchmarks used to measure these risks and identify what is missing; and we set out the open problems. Two findings recur: among governance mechanisms only information-flow control covers both compositional and cross-session inference leakage, the two least-protected risks; and no benchmark drives an agent across its data surfaces under one privacy policy, the instrument the field most lacks. Our goal is a reference that situates the scattered literature and gives future work a common framing.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper is a data-centric survey of privacy risks for LLM agents that interact with databases, document collections, APIs, and memory across sessions. It taxonomizes data sources touched by agents, the inference leakage risks (compositional and cross-session) each creates, the governance mechanisms proposed in the literature (RAG, text-to-SQL, prompt injection, access control, contextual privacy), and the benchmarks used to evaluate them. The central claims are that only information-flow control simultaneously addresses both compositional and cross-session leakage and that no existing benchmark evaluates an agent across all its data surfaces under a single privacy policy.

Significance. If the survey's coverage is representative, the work supplies a unifying reference that consolidates scattered results across sub-fields and surfaces two concrete gaps (governance coverage and benchmark design) that future research can target directly.

major comments (2)
  1. [Abstract] Abstract: the two recurring findings are presented as results of the survey, yet the abstract supplies no search strategy, inclusion/exclusion criteria, databases queried, or count of papers examined. This directly affects the load-bearing claim that 'only information-flow control covers both compositional and cross-session inference leakage,' because the 'only' qualifier cannot be evaluated without evidence that the enumeration of mechanisms was exhaustive.
  2. [Governance Mechanisms section] The section mapping governance mechanisms to leakage types: the assertion that no mechanism except information-flow control addresses both leakage forms requires an explicit enumeration or table showing every reviewed mechanism (including any query-differential privacy or session-scoped capability systems) and the precise reason each fails one of the two risks; without such a table the claim rests on unverified completeness.
minor comments (1)
  1. [Abstract] The abstract uses 'data agent' as shorthand without an early formal definition; a one-sentence definition in the introduction would improve readability.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments, which correctly identify opportunities to strengthen the transparency of our survey methodology and the verifiability of our claims. We address each major comment below and will make the corresponding revisions.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the two recurring findings are presented as results of the survey, yet the abstract supplies no search strategy, inclusion/exclusion criteria, databases queried, or count of papers examined. This directly affects the load-bearing claim that 'only information-flow control covers both compositional and cross-session inference leakage,' because the 'only' qualifier cannot be evaluated without evidence that the enumeration of mechanisms was exhaustive.

    Authors: We agree that the abstract presents the two findings without accompanying details on survey scope or methodology, which limits the ability to assess the exhaustiveness underlying the 'only' claim. We will revise the abstract to include a short qualifier referencing the breadth of the review performed. In addition, we will add a dedicated 'Survey Methodology' subsection to the introduction that explicitly states the search strategy, inclusion/exclusion criteria, databases queried, and approximate count of papers examined. This will supply the missing context without lengthening the abstract beyond typical limits. revision: yes

  2. Referee: [Governance Mechanisms section] The section mapping governance mechanisms to leakage types: the assertion that no mechanism except information-flow control addresses both leakage forms requires an explicit enumeration or table showing every reviewed mechanism (including any query-differential privacy or session-scoped capability systems) and the precise reason each fails one of the two risks; without such a table the claim rests on unverified completeness.

    Authors: We accept that the current narrative presentation of the governance mechanisms section does not provide an explicit, checkable enumeration, leaving the completeness of the 'only information-flow control' claim difficult to verify. We will add a new summary table to this section that lists every reviewed mechanism, maps each to the leakage types it addresses (compositional and/or cross-session), and states the precise reason it fails to cover both risks where applicable. The table will incorporate the additional mechanisms mentioned by the referee, such as query-differential privacy and session-scoped capability systems. This change will make the claim directly verifiable from the manuscript. revision: yes

Circularity Check

0 steps flagged

Survey paper organizes external literature without self-referential derivations or fitted predictions.

full rationale

This is a literature survey that taxonomizes existing work across sub-fields (RAG, text-to-SQL, agent memory, etc.) and states two recurring observations drawn from the cited papers. No equations, parameters, or new derivations appear in the provided abstract or description. The central claims are presented as summaries of the collected literature rather than results obtained by construction from the paper's own inputs. No self-citation chains or ansatzes are invoked to justify the taxonomy itself. The paper is therefore self-contained against external benchmarks and receives the default non-circularity finding.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

As a survey the paper introduces no free parameters, mathematical axioms, or new invented entities; it relies entirely on the cited body of prior work.

pith-pipeline@v0.9.1-grok · 5811 in / 1108 out tokens · 32981 ms · 2026-06-26T04:26:12.497072+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

135 extracted references · 13 canonical work pages

  1. [1]

    Sepideh Abedini, Shubhankar Mohapatra, D. B. Emerson, Masoumeh Shafieine- jad, Jesse C. Cresswell, and Xi He. 2025. MaskSQL: Safeguarding Privacy for LLM- Based Text-to-SQL via Abstraction. arXiv:2509.23459 [cs.CR] arXiv:2509.23459

    arXiv 2025

  2. [2]

    Nguyen, Tomislav Medan, Jinali Shah, Moham- mad T

    Parker Addison, Minh-Tuan H. Nguyen, Tomislav Medan, Jinali Shah, Moham- mad T. Manzari, Brendan McElrone, Laksh Lalwani, Aboli More, Smita Sharma, Holger R. Roth, et al . 2024. C-FedRAG: A Confidential Federated Retrieval- Augmented Generation System. arXiv:2412.13163 [cs.CR] arXiv:2412.13163

    arXiv 2024

  3. [3]

    Rakesh Agrawal, Jerry Kiernan, Ramakrishnan Srikant, and Yirong Xu. 2002. Hippocratic Databases. InProceedings of the 28th International Conference on Very Large Data Bases (VLDB). 143–154

    2002

  4. [4]

    Meysam Alizadeh, Zeynab Samei, Daria Stetsenko, and Fabrizio Gilardi. 2025. Simple Prompt Injection Attacks Can Leak Personal Data Observed by LLM Agents During Task Execution. arXiv:2506.01055 [cs.CR] arXiv:2506.01055

    arXiv 2025

  5. [5]

    Faruk Alpay and Taylan Alpay. 2026. AgentSecBench: Measuring Prompt Injection, Privacy Leakage, and Tool-Use Integrity in LLM Agents. arXiv:2605.26269 [cs.CR] arXiv:2605.26269

    Pith/arXiv arXiv 2026

  6. [6]

    Orlando Amaral, Muhammad Ilyas Azeem, Sallam Abualhaija, and Lionel C. Briand. 2023. NLP-based Automated Compliance Checking of Data Processing Agreements against GDPR.IEEE Transactions on Software Engineering(2023). arXiv:2209.09722; doi:10.1109/TSE.2023.3288901

    work page doi:10.1109/tse.2023.3288901 2023

  7. [7]

    Maya Anderson, Guy Amit, and Abigail Goldsteen. 2025. Is My Data in Your Re- trieval Database? Membership Inference Attacks Against Retrieval Augmented Generation. InInternational Conference on Information Systems Security and Privacy (ICISSP). arXiv:2405.20446

    arXiv 2025

  8. [8]

    Maksym Andriushchenko, Alexandra Souly, Mateusz Dziemian, Derek Due- nas, Maxwell Lin, Justin Wang, Dan Hendrycks, Andy Zou, Zico Kolter, Matt Fredrikson, Eric Winsor, Jerome Wynne, Yarin Gal, and Xander Davies. 2025. AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents. InInter- national Conference on Learning Representations (ICLR). arXiv:2410.09024

    Pith/arXiv arXiv 2025

  9. [9]

    Noah Apthorpe, Yan Shvartzshnaider, Arunesh Mathur, Dillon Reisman, and Nick Feamster. 2018. Discovering Smart Home Internet of Things Privacy Norms Using Contextual Integrity.Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT)2, 2 (2018), 1–23. Article 59; doi:10.1145/3214262

    work page doi:10.1145/3214262 2018

  10. [10]

    Sadia Asif and Mohammad Mohammadi Amiri. 2026. Information- Theoretic Privacy Control for Sequential Multi-Agent LLM Systems. arXiv:2603.05520 [cs.MA] arXiv:2603.05520

    arXiv 2026

  11. [11]

    Eugene Bagdasarian, Ren Yi, Sahra Ghalebikesabi, Peter Kairouz, Marco Gruteser, Sewoong Oh, Borja Balle, and Daniel Ramage. 2024. AirGapAgent: Pro- tecting Privacy-Conscious Conversational Agents. InProceedings of the 2024 ACM SIGSAC Conference on Computer and Communications Security (CCS). arXiv:2405.05175

    arXiv 2024

  12. [12]

    Luca Beurer-Kellner, Beat Buesser, Ana-Maria Creţu, Edoardo Debenedetti, Daniel Dobos, Daniel Fabian, Marc Fischer, David Froelicher, Kathrin Grosse, Daniel Naeff, Ezinwanne Ozoani, Andrew Paverd, Florian Tramèr, and Václav Volhejn. 2025. Design Patterns for Securing LLM Agents against Prompt Injec- tions. arXiv:2506.08837 [cs.CR] arXiv:2506.08837

    arXiv 2025

  13. [13]

    Rohini Bhosale, Pankaj Chandre, Sushma Mehetre, Swati Powar, Shubhra Mathur, and Arun Ghandat. 2026. The Dark Side of Autonomous Intelligence: A Survey on Data Leakage and Privacy Failures in Agentic AI.Frontiers in Computer Science8 (2026), 1802727. doi:10.3389/fcomp.2026.1802727

    work page doi:10.3389/fcomp.2026.1802727 2026

  14. [14]

    Andreea-Elena Bodea, Stephen Meisenbacher, Alexandra Klymenko, and Florian Matthes. 2026. SoK: Privacy Risks and Mitigations in Retrieval-Augmented Generation Systems. InIEEE Conference on Secure and Trustworthy Machine Learning (SaTML). arXiv:2601.03979

    arXiv 2026

  15. [15]

    Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot

    Lucas Bourtoule, Varun Chandrasekaran, Christopher A. Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot

  16. [16]

    In: 2021 IEEE Symposium on Security and Privacy (2021)

    Machine Unlearning. InIEEE Symposium on Security and Privacy (S&P). doi:10.1109/SP40001.2021.00019 (SISA)

    work page doi:10.1109/sp40001.2021.00019 2021

  17. [17]

    Ji-Won Byun and Ninghui Li. 2008. Purpose Based Access Control for Privacy Protection in Relational Database Systems.The VLDB Journal17, 4 (2008), 603–619. doi:10.1007/s00778-006-0023-0

    work page doi:10.1007/s00778-006-0023-0 2008

  18. [18]

    Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Úlfar Erlingsson, Alina Oprea, and Colin Raffel. 2021. Extracting Training Data from Large Language Models. InUSENIX Security Symposium. arXiv:2012.07805

    arXiv 2021

  19. [19]

    Juan Carlos Carrillo, Jose Luis Martin-Navarro, Rongjun Ma, and Jose Such

  20. [20]

    doi:10.56553/popets-2026-0015

    Personal Data Flows and Privacy Policy Traceability in Third-party LLM Apps in the GPT Ecosystem.Proceedings on Privacy Enhancing Technologies (PoPETs)2026, 1 (2026), 273–295. doi:10.56553/popets-2026-0015

    work page doi:10.56553/popets-2026-0015 2026

  21. [21]

    Sizhe Chen, Julien Piet, Chawin Sitawarin, and David Wagner. 2025. StruQ: Defending Against Prompt Injection with Structured Queries. InUSENIX Security Symposium. arXiv:2402.06363

    arXiv 2025

  22. [22]

    Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, and Bo Li. 2024. Agent- Poison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases. In Advances in Neural Information Processing Systems (NeurIPS). arXiv:2407.12784

    arXiv 2024

  23. [23]

    Zhao Cheng, Diane Wan, Matthew Abueg, Sahra Ghalebikesabi, Ren Yi, Eu- gene Bagdasarian, Borja Balle, Stefan Mellem, and Shawn O’Banion. 2024. CI- Bench: Benchmarking Contextual Integrity of AI Assistants on Synthetic Data. arXiv:2409.13903 [cs.CL] arXiv:2409.13903

    arXiv 2024

  24. [24]

    Sahana Chennabasappa, Cyrus Nikolaidis, Daniel Song, David Molnar, Stephanie Ding, Shengye Wan, Spencer Whitman, et al . 2025. LlamaFire- wall: An Open Source Guardrail System for Building Secure AI Agents. arXiv:2505.03574 [cs.CR] arXiv:2505.03574

    arXiv 2025

  25. [25]

    Yujin Choi, Youngjoo Park, Junyoung Byun, Jaewook Lee, and Jinseong Park

  26. [26]

    InFindings of the Association for Computational Linguistics: EMNLP 2025

    Safeguarding Privacy of Retrieval Data against Membership Inference Attacks: Is This Query Too Close to Home?. InFindings of the Association for Computational Linguistics: EMNLP 2025. arXiv:2505.22061

    arXiv 2025

  27. [27]

    Manuel Costa, Boris Köpf, Aashish Kolluri, Andrew Paverd, Mark Russi- novich, Ahmed Salem, Shruti Tople, Lukas Wutschitz, and Santiago Zanella- Béguelin. 2025. Securing AI Agents with Information-Flow Control. arXiv:2505.23643 [cs.CR] arXiv:2505.23643 (Fides)

    Pith/arXiv arXiv 2025

  28. [28]

    Edoardo Debenedetti, Ilia Shumailov, Tianqi Fan, Jamie Hayes, Nicholas Carlini, Daniel Fabian, Christoph Kern, Chongyang Shi, Andreas Terzis, and Florian Tramèr. 2025. Defeating Prompt Injections by Design. arXiv:2503.18813 [cs.CR] arXiv:2503.18813 (CaMeL)

    Pith/arXiv arXiv 2025

  29. [29]

    Edoardo Debenedetti, Jie Zhang, Mislav Balunović, Luca Beurer-Kellner, Marc Fischer, and Florian Tramèr. 2024. AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents. InAdvances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track. arXiv:2406.13352

    Pith/arXiv arXiv 2024

  30. [30]

    Shen Dong, Shaochen Xu, Pengfei He, Yige Li, Jiliang Tang, Tianming Liu, Hui Liu, and Zhen Xiang. 2025. A Practical Memory Injection Attack against LLM Agents. arXiv:2503.03704 [cs.CR] arXiv:2503.03704 (MINJA)

    arXiv 2025

  31. [31]

    Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. 2006. Cali- brating Noise to Sensitivity in Private Data Analysis. InTheory of Cryptography Conference (TCC) (LNCS, Vol. 3876). Springer, 265–284. doi:10.1007/11681878_14

    work page doi:10.1007/11681878_14 2006

  32. [32]

    Faouzi El Yagoubi, Godwin Badu-Marfo, and Ranwa Al Mallah. 2026. AgentLeak: A Full-Stack Benchmark for Privacy Leakage in Multi-Agent LLM Systems. arXiv:2602.11510 [cs.CR] arXiv:2602.11510. 14

    arXiv 2026

  33. [33]

    Wei Fan, Haoran Li, Zheye Deng, Weiqi Wang, and Yangqiu Song. 2024. Gold- Coin: Grounding Large Language Models in Privacy Laws via Contextual In- tegrity Theory. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP). aclanthology.org/2024.emnlp-main.195

    2024

  34. [34]

    Habiba Farrukh, Nada Lahjouji, Sharad Mehrotra, Faisal Nawab, Julie Rousseau, Shantanu Sharma, Nalini Venkatasubramanian, and Roberto Yus. 2024. Pri- vacySphere: Privacy-Preserving Smart Spaces. In2024 IEEE 6th International Conference on Trust, Privacy and Security in Intelligent Systems, and Applications (TPS-ISA). 255–264

    2024

  35. [35]

    Yanjie Fu, Dongjie Wang, Wangyang Ying, et al. 2025. Autonomous Data Agents: A New Opportunity for Smart Data. arXiv:2509.18710 [cs.DB] arXiv:2509.18710

    arXiv 2025

  36. [36]

    Yuyou Gan, Yong Yang, Zhe Ma, Ping He, Rui Zeng, Yiming Wang, Qingming Li, Chunyi Zhou, Songze Li, Ting Wang, Yunjun Gao, Yingcai Wu, and Shouling Ji. 2024. Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agents. arXiv:2411.09523 [cs.CR] arXiv:2411.09523

    arXiv 2024

  37. [37]

    Ilyas, and Ashwin Machanavajjhala

    Chang Ge, Xi He, Ihab F. Ilyas, and Ashwin Machanavajjhala. 2019. APEx: Accuracy-Aware Differentially Private Data Exploration. InProceedings of the 2019 ACM SIGMOD International Conference on Management of Data (SIGMOD)

    2019

  38. [38]

    Sahra Ghalebikesabi, Eugene Bagdasaryan, Ren Yi, Itay Yona, Ilia Shumailov, Aneesh Pappu, Chongyang Shi, Laura Weidinger, Robert Stanforth, Leonard Berrada, Pushmeet Kohli, Po-Sen Huang, and Borja Balle. 2024. Operationalizing Contextual Integrity in Privacy-Conscious Assistants. arXiv:2408.02373 [cs.LG] arXiv:2408.02373

    arXiv 2024

  39. [39]

    Sameera Ghayyur, Debabrata Ghosh, Xi He, and Sharad Mehrotra. 2022. MIDE: Accuracy Aware Minimally Invasive Data Exploration for Decision Support. Proceedings of the VLDB Endowment15, 11 (2022), 2653–2665

    2022

  40. [40]

    Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. 2023. Not What You’ve Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security (AISec). arXiv:2302.12173

    Pith/arXiv arXiv 2023

  41. [41]

    Shanshan Han, Qifan Zhang, Yuhang Yao, Weizhao Jin, and Zhaozhuo Xu. 2024. LLM Multi-Agent Systems: Challenges and Open Problems. arXiv:2402.03578 [cs.MA] arXiv:2402.03578

    Pith/arXiv arXiv 2024

  42. [42]

    Feng He, Tianqing Zhu, Dayong Ye, Bo Liu, Wanlei Zhou, and Philip S. Yu. 2025. The Emerged Security and Privacy of LLM Agent: A Survey with Case Studies. Comput. Surveys(2025). arXiv:2407.19354; doi:10.1145/3773080

    work page doi:10.1145/3773080 2025

  43. [43]

    Xi He, Ashwin Machanavajjhala, and Bolin Ding. 2014. Blowfish Privacy: Tuning Privacy-Utility Trade-offs using Policies. InProceedings of the 2014 ACM SIGMOD International Conference on Management of Data (SIGMOD). 1447–1458. doi:10.1145/2588555.2588581

    work page doi:10.1145/2588555.2588581 2014

  44. [44]

    Keegan Hines, Gary Lopez, Matthew Hall, Federico Zarfati, Yonatan Zunger, and Emre Kiciman. 2024. Defending Against Indirect Prompt Injection Attacks With Spotlighting. arXiv:2403.14720 [cs.CR] arXiv:2403.14720

    Pith/arXiv arXiv 2024

  45. [45]

    Sirui Hong, Yizhang Lin, Bang Liu, Bangbang Liu, et al. 2024. Data Interpreter: An LLM Agent For Data Science. arXiv:2402.18679 [cs.AI] arXiv:2402.18679

    arXiv 2024

  46. [46]

    Zijin Hong, Zheng Yuan, Qinggang Zhang, Hao Chen, Junnan Dong, Feiran Huang, and Xiao Huang. 2024. Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL. arXiv:2406.08426 [cs.CL] arXiv:2406.08426

    arXiv 2024

  47. [47]

    Jie Huang, Hanyin Shao, and Kevin Chen-Chuan Chang. 2022. Are Large Pre- Trained Language Models Leaking Your Personal Information?. InFindings of the Association for Computational Linguistics: EMNLP 2022. arXiv:2205.12628

    arXiv 2022

  48. [48]

    Zimo Ji, Daoyuan Wu, Wenyuan Jiang, Pingchuan Ma, Zongjie Li, Yudong Gao, Shuai Wang, and Yingjiu Li. 2026. Taming Various Privilege Escala- tion in LLM-Based Agent Systems: A Mandatory Access Control Framework. arXiv:2601.11893 [cs.CR] arXiv:2601.11893

    arXiv 2026

  49. [49]

    Changyue Jiang, Xudong Pan, Geng Hong, Chenfu Bao, Yang Chen, and Min Yang. 2024. RAG-Thief: Scalable Extraction of Private Data from Retrieval-Augmented Generation Applications with Agent-based Attacks. arXiv:2411.14110 [cs.CR] arXiv:2411.14110

    arXiv 2024

  50. [50]

    Gurusha Juneja, Jayanth Naga Sai Pasupulati, Alon Albalak, Wenyue Hua, and William Yang Wang. 2025. MAGPIE: A Benchmark for Multi-Agent Contextual Privacy Evaluation. arXiv:2510.15186

    arXiv 2025

  51. [51]

    Maurits Kaptein, Vassilis-Javed Khan, and Andriy Podstavnychy. 2026. Run- time Governance for AI Agents: Policies on Paths. arXiv:2603.16586 [cs.AI] arXiv:2603.16586

    arXiv 2026

  52. [52]

    Daniel Kifer and Ashwin Machanavajjhala. 2014. Pufferfish: A Framework for Mathematical Privacy Definitions.ACM Transactions on Database Systems (TODS)39, 1 (2014). doi:10.1145/2514689

    work page doi:10.1145/2514689 2014

  53. [53]

    Ðorđe Klisura, Joseph Khoury, Ashish Kundu, Ram Krishnan, and Anthony Rios

  54. [54]

    arXiv:2510.07642 [cs.CR] arXiv:2510.07642

    Role-Conditioned Refusals: Evaluating Access Control Reasoning in Large Language Models. arXiv:2510.07642 [cs.CR] arXiv:2510.07642

    arXiv

  55. [55]

    Ðorđe Klisura and Anthony Rios. 2025. Unmasking Database Vulnerabilities: Zero-Knowledge Schema Inference Attacks in Text-to-SQL Systems. InFindings of the Association for Computational Linguistics: NAACL 2025. arXiv:2406.14545

    arXiv 2025

  56. [56]

    Tatsuki Koga, Ruihan Wu, Zhiyuan Zhang, and Kamalika Chaudhuri. 2024. Privacy-Preserving Retrieval-Augmented Generation with Differential Privacy. arXiv:2412.04697 [cs.LG] arXiv:2412.04697

    arXiv 2024

  57. [57]

    Nada Lahjouji, Sameera Ghayyur, Xi He, and Sharad Mehrotra. 2024. ProBE: Proportioning Privacy Budget for Complex Exploratory Decision Support. In Proceedings of the 2024 ACM SIGSAC Conference on Computer and Communica- tions Security (CCS). 1924–1938

    2024

  58. [58]

    Chingkwun Lam, Jiaxin Li, Lingfei Zhang, and Kuo Zhao. 2026. Governing Evolving Memory in LLM Agents: Risks, Mechanisms, and a Stability and Safety Governed Memory Framework. arXiv:2603.11768 [cs.AI] arXiv:2603.11768 (SSGM)

    Pith/arXiv arXiv 2026

  59. [59]

    Inan, Sahar Abdelnabi, Janardhan Kulkarni, Lukas Wutschitz, Reza Shokri, Christopher G

    Guangchen Lan, Huseyin A. Inan, Sahar Abdelnabi, Janardhan Kulkarni, Lukas Wutschitz, Reza Shokri, Christopher G. Brinton, and Robert Sim. 2025. Contex- tual Integrity in LLMs via Reasoning and Reinforcement Learning. InAdvances in Neural Information Processing Systems (NeurIPS). arXiv:2506.04245

    arXiv 2025

  60. [60]

    Fangyu Lei, Jixuan Chen, Yuxiao Ye, Ruisheng Cao, Caiming Xiong, Tao Yu, et al. 2025. Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows. InInternational Conference on Learning Representations (ICLR). arXiv:2411.07763

    arXiv 2025

  61. [61]

    Ido Levy, Ben Wiesel, Sami Marreed, Alon Oved, Avi Yaeli, Nir Mashkif, and Segev Shlomov. 2024. ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agents. arXiv:2410.06703 [cs.AI] arXiv:2410.06703

    Pith/arXiv arXiv 2024

  62. [62]

    Haoran Li, Wenbin Hu, Huihao Jing, Yulin Chen, Qi Hu, Sirui Han, Tianshu Chu, Peizhao Hu, and Yangqiu Song. 2025. PrivaCI-Bench: Evaluating Pri- vacy with Contextual Integrity and Legal Compliance. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL). aclanthology.org/2025.acl-long.518

    2025

  63. [63]

    Jingjie Li, Kaiwen Sun, Brittany Skye Huff, Anna Marie Bierley, Younghyun Kim, Florian Schaub, and Kassem Fawaz. 2023. It’s up to the Consumer to be Smart: Understanding the Security and Privacy Attitudes of Smart Home Users on Reddit. InProceedings of the 2023 IEEE Symposium on Security and Privacy (S&P). 2850–2866

    2023

  64. [64]

    Wenkai Li, Liwen Sun, Zhenxiang Guan, Xuhui Zhou, and Maarten Sap. 2025. 1-2-3 Check: Enhancing Contextual Privacy in LLM via Multi-Agent Reasoning. InFirst Workshop on LLM Security (LLMSEC). aclanthology.org/2025.llmsec-1.9

    2025

  65. [65]

    Xinfeng Li, Dong Huang, Jie Li, Hongyi Cai, Zhenhong Zhou, Wei Dong, Xi- aoFeng Wang, and Yang Liu. 2025. A Vision for Access Control in LLM-based Agent Systems. arXiv:2510.11108 [cs.CR] arXiv:2510.11108 (Agent Access Control)

    arXiv 2025

  66. [66]

    Xuechen Li, Florian Tramèr, Percy Liang, and Tatsunori Hashimoto. 2022. Large Language Models Can Be Strong Differentially Private Learners. InInternational Conference on Learning Representations (ICLR). arXiv:2110.05679

    arXiv 2022

  67. [67]

    Zhijun Li, Minghui Xu, Huayi Qi, Wenxuan Yu, Tingchuang Zhang, Qiao Zhang, GuangYong Shang, Zhen Ma, and Xiuzhen Cheng. 2026. PRAG: End-to-End Privacy-Preserving Retrieval-Augmented Generation. arXiv:2604.26525 [cs.CR] arXiv:2604.26525

    Pith/arXiv arXiv 2026

  68. [68]

    Katrina Ligett, Seth Neel, Aaron Roth, Bo Waggoner, and Zhiwei Steven Wu

  69. [69]

    Accuracy First: Selecting a Differential Privacy Level for Accuracy- Constrained ERM.Journal of Privacy and Confidentiality9, 2 (2017)

    2017

  70. [70]

    Meiyu Lin, Haichuan Zhang, Jiale Lao, Renyuan Li, Yuanchun Zhou, Carl Yang, Yang Cao, and Mingjie Tang. 2026. Are Your LLM-based Text-to-SQL Models Secure? Exploring SQL Injection via Backdoor Attacks. InProceedings of the ACM on Management of Data (SIGMOD). arXiv:2503.05445

    arXiv 2026

  71. [71]

    Yixi Lin, Jiangrong Wu, Yuhong Nan, Xueqiang Wang, Xinyuan Zhang, and Zibin Zheng. 2026. AgentRaft: Automated Detection of Data Over-Exposure in LLM Agents. arXiv:2603.07557 [cs.CR] arXiv:2603.07557

    arXiv 2026

  72. [72]

    Yuchen Ling, Shengcheng Yu, Zhenyu Chen, and Chunrong Fang. 2026. To- ward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation. arXiv:2606.10749 [cs.CR] arXiv:2606.10749

    Pith/arXiv arXiv 2026

  73. [73]

    Ruiheng Liu, Xiaobing Chen, Jinyu Zhang, Qiongwen Zhang, Yu Zhang, and Bailong Yang. 2026. SafeNLIDB: A Privacy-Preserving Safety Alignment Frame- work for LLM-based Natural Language Database Interfaces. InProceedings of the AAAI Conference on Artificial Intelligence (AAAI). arXiv:2511.06778 (2025 preprint)

    arXiv 2026

  74. [74]

    Varsh- ney, Mohit Bansal, Sanmi Koyejo, and Yang Liu

    Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Yuguang Yao, Chris Yuhao Liu, Xiaojun Xu, Hang Li, Kush R. Varsh- ney, Mohit Bansal, Sanmi Koyejo, and Yang Liu. 2025. Rethinking Machine Unlearning for Large Language Models.Nature Machine Intelligence(2025). arXiv:2402.08787

    arXiv 2025

  75. [75]

    Xinyu Liu, Shuyu Shen, Boyan Li, Peixian Ma, Guoliang Li, Nan Tang, and Yuyu Luo. 2025. A Survey of Text-to-SQL in the Era of LLMs: Where Are We, and Where Are We Going?IEEE Transactions on Knowledge and Data Engineering (TKDE)(2025). arXiv:2408.05109

    arXiv 2025

  76. [76]

    Elisabet Lobo-Vesga, Alejandro Russo, and Marco Gaboardi. 2020. A Program- ming Framework for Differential Privacy with Accuracy Concentration Bounds. InProceedings of the 2020 IEEE Symposium on Security and Privacy (S&P). 411– 428

    2020

  77. [77]

    Nils Lukas, Ahmed Salem, Robert Sim, Shruti Tople, Lukas Wutschitz, and Santiago Zanella-Béguelin. 2023. Analyzing Leakage of Personally Identifiable Information in Language Models. InIEEE Symposium on Security and Privacy (S&P). arXiv:2302.00539. 15

    arXiv 2023

  78. [78]

    Jiaqi Luo, Songyang Peng, Jiarun Dai, Zhile Chen, Zhuoxiang Shen, Geng Hong, Xudong Pan, Yuan Zhang, and Min Yang. 2026. AgentGuard: An Attribute-Based Access Control Framework for Tool-Use LLM-Based Agents. arXiv:2605.28071 [cs.CR] arXiv:2605.28071

    Pith/arXiv arXiv 2026

  79. [79]

    Yuyu Luo, Guoliang Li, Ju Fan, and Nan Tang. 2026. Data Agents: Levels, State of the Art, and Open Problems. InCompanion of the 2026 International Conference on Management of Data (SIGMOD Companion), Tutorial. arXiv:2602.04261; doi:10.1145/3788853.3801878

    work page doi:10.1145/3788853.3801878 2026

  80. [80]

    Lipton, and J

    Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary C. Lipton, and J. Zico Kolter. 2024. TOFU: A Task of Fictitious Unlearning for LLMs. InConference on Language Modeling (COLM). arXiv:2401.06121

    Pith/arXiv arXiv 2024

Showing first 80 references.