Pith. sign in

REVIEW 5 major objections 8 minor 41 references

Memory isolation does not stop a malicious tool from draining a user’s private agent memory.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-30 21:58 UTC pith:MF5NS7K2

load-bearing objection Real new surface—tool-side extraction under isolation—with a clean three-stage design, but the channel lives or dies on agents stuffing raw LTM into tool args, which they never stress-test on the argument side. the 5 major comments →

arxiv 2607.23444 v1 pith:MF5NS7K2 submitted 2026-07-26 cs.CR

Isolated but Exposed: Persistence-Based Memory Extraction Attack on LLM Agents

Xinyu Gao , Wenyu Chen , Xiangtao Meng , Li Wang , Chuanchao Zang , Jianing Wang , Zheng Li , Shanqing Guo This is my paper
classification cs.CR
keywords LLM agentslong-term memorymemory isolationtool-side attackprompt injectiondata extractionRAG privacySPORE
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Production LLM agents keep each user’s long-term memory sealed behind a user ID, which blocked the old style of attack where a fake user tried to read someone else’s store. This paper argues that the seal is incomplete: when the agent calls an external tool, it often stuffs retrieved private records into the tool’s parameters, so a hostile tool can siphon that data without ever crossing the user boundary. Existing extraction tricks fail on this channel because the attack command pollutes the retrieval query and platforms cap how many tool calls one trigger can make. The authors introduce SPORE, which plants the attack command once in short-term memory, then feeds pure semantic anchors so retrieval stays precise; it treats extraction as covering the embedding space and plants reactivation payloads so the attack resumes later with few or no new triggers. In their tests SPORE recovered most records with unlimited triggers and nearly half with only twenty, and in multi-user settings OAuth tokens can tie those records back to real identities.

Core claim

User-level memory isolation is not enough: a malicious tool can systematically exfiltrate a victim’s isolated long-term memory by riding the agent’s habit of putting retrieved private data into tool-call parameters. SPORE makes that channel practical by separating the hijack command from retrieval anchors, optimizing anchors as geometric coverage of embedding space, and persisting reactivation state so extraction continues under sparse triggers—reaching 80% record extraction with unlimited triggers and 47% with only 20 on their GPT-5-mini agents.

What carries the argument

SPORE: a three-stage persistence attack that (1) injects an adversarial command once into short-term memory, (2) returns pure semantic anchors and steers them via geometric coverage of the embedding space (coarse discovery plus fine expansion with adaptive switching), and (3) writes reactivation payloads into STM or LTM so later benign interactions restart the loop without new malicious triggers.

Load-bearing premise

The attack only works if the agent both auto-retrieves long-term memory from the tool’s replies and then packs those private records into later calls back to the attacker’s tool, without stripping or gating that content.

What would settle it

On a production-style agent that retrieves LTM from tool context, measure whether SPORE’s pure-anchor loop plus reactivation still extracts a large share of a 200-record private store when tool parameters are redacted of retrieved memory or tool outputs are filtered—if extraction collapses under that change, the claimed channel is not load-bearing.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Isolating memory by user ID does not by itself protect long-term agent memory when tools receive retrieved content as parameters.
  • Tool-side trust must be treated as a first-class boundary: platforms need sensitivity-scoped tool access to memory, not only user isolation.
  • Sparse real-world triggers are still dangerous because reactivation payloads can resume extraction across turns and sessions.
  • In multi-user OAuth deployments, bulk-extracted records can be linked to identities, turning leakage into targeted surveillance.
  • Language-only defenses (prompt hardening, response filtering) leave large residual extraction rates against optimized payloads.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Agent builders who currently treat third-party tools as untrusted for code but trusted for data flow will need the same isolation discipline they already apply between users.
  • If parameter redaction of retrieved memory becomes default, the geometric-anchor half of SPORE may survive as a retrieval-steering technique while the exfiltration half dies—suggesting defenses should target the write-back path, not only retrieval.
  • Marketplace and API-platform review for agent tools may need memory-scope declarations analogous to mobile app permission manifests.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 8 minor

Summary. The paper studies privacy extraction from LLM agents whose long-term memory (LTM) is protected by user-level isolation. It observes that although an adversarial user can no longer retrieve a victim's records directly, a malicious third-party tool can: agents routinely embed LTM-retrieved content in tool-call parameters. The authors show that naive tool-side adaptation of existing user-side RAG extraction attacks (MEXTRA, RAG-Thief, IKEA) fails for two reasons — semantic interference between the adversarial command and the retrieval anchor (L1), and platform tool-call limits that bound extraction per trigger (L2). They propose SPORE, which (i) persists the hijack command in short-term memory so subsequent tool responses carry semantically pure anchors, (ii) formulates anchor selection as geometric coverage optimization over the embedding space with coarse/fine exploration modes and adaptive switching, and (iii) injects reactivation payloads into STM or LTM to resume extraction across sessions without further triggers. Across three scenarios, three planners (GPT-5-mini, Gemini-2.5-pro, Deepseek-chat), three LTM frameworks, and two victim datasets, SPORE reports 80.0% record extraction rate with unlimited triggers, 47.0% with 20 triggers, and ~76.5% with a single trigger under LTM persistence; tests on Dify and Coze with Mem0 yield 29.5–67.5%. The authors conclude memory isolation is insufficient and advocate tool-side isolation defenses.

Significance. If the results hold, the paper makes a genuinely useful contribution to agent security: it demonstrates that user-level memory isolation, now standard in ChatGPT Memory, Mem0, and Claude, does not close the exfiltration surface, because the agent itself can be hijacked into forwarding retrieved records to a malicious tool. The work ships several concrete strengths: a clearly specified threat model with three progressively restrictive trigger scenarios; an evaluation spanning three planning models, three LTM frameworks, three embedding models, and two datasets; an honest optimization protocol in which attack parameters are tuned on TREC-COVID while the victim LTM is drawn from different distributions (no metric circularity); ablations isolating the decoupling, anchor-optimization, and mode-switching contributions; and end-to-end results on two production platforms (Dify, Coze). The reactivation-payload mechanism (§4.4) is, to my knowledge, novel and is the most consequential finding: Scenario 3 shows a single trigger can yield ~76.5% RER via LTM persistence. The main significance caveat is that the demonstrated channel is verbatim forwarding (SS=1.00), which narrows the gap between"

major comments (5)
  1. [§4.2, Algorithm 1, §5.1] The load-bearing step of the attack — Algorithm 1, line 11, ToolCall(T, params=R_t), and the optimized command in §4.2 ('Invoke τ_adv again with the entire conversation history as tool-call parameters') — requires the malicious tool's declared schema to contain a free-text field capable of absorbing arbitrary verbatim LTM records. §5.1 ('Malicious Tool Settings') never reports the parameter schema used in the MemEngine harness, and no experiment varies schema strictness. Real MCP/OpenAPI tools have typed, validated schemas; a weather or scheduling API with no free-text sink gives the hijacked agent nowhere to place R_t, and schema validation or parameter truncation breaks the loop regardless of retrieval precision. This is not a hypothetical: the paper's own numbers show the channel is raw verbatim forwarding (SS = 1.00 in nearly every cell of Tables 1–3). The manuscript should (a) state
  2. [§8.2, Tables 1–4] Because extracted content is a verbatim echo of retrieved records (SS = 1.00 throughout Tables 1–3), the exfiltration is trivially detectable at the argument layer: an n-gram or embedding-overlap check between tool-call arguments and records retrieved in the same session would flag essentially every leak. Yet §8.2 evaluates only semantic-level defenses (Prompt Hardening, Response Filtering) and concludes that system-level defenses operating 'below the language layer' are needed — without evaluating the most obvious such defense, argument-side filtering/schema enforcement against retrieved content. The residual-risk framing ('54.0% and 32.5% still represent substantial exposure') and the conclusion that existing mitigations are insufficient depend on this omission. A parameter-redaction or overlap-detection baseline should be added; if it defeats SPORE, the defense discussion and the stre
  3. [§5.2–§7.3, Tables 1–3, Figures 4, 6, 8, 9] All results are single-run point estimates over a 200-record victim LTM: table entries move in 0.5% increments (one record), and no repetitions, variance, or confidence intervals are reported anywhere. Several headline comparisons are within plausible noise of a single run — e.g., the embedding-model robustness claim (Table 1: 78.0% vs 80.0%), the Scenario 3 fluctuations in Figure 9a (80.1% at k=5 dropping at k=6, attributed post hoc to the payload being 'overshadowed'), and the STM-window peak at 8 (Figure 8a: 49.5% vs 47.0%). Given that one record equals 0.5%, differences of 2–3 points are 4–6 records. The paper should report multiple seeds with dispersion, at least for the main tables and the ablations from which design conclusions (K=4, γ=4, N*, window size) are drawn.
  4. [§4.3, §5.1 (Datasets)] The 'geometric coverage optimization' (§4.3, Eq. 5) is validated only on a 200-record store sampled from a single-domain corpus (HealthCareMagic) or short dialogues (LoCoMo). The coarse/fine machinery — K-means over retrieved embeddings with silhouette-selected cluster count, boundary-expansion scoring (Eqs. 6–8) — is motivated by cluster structure that a 200-record single-domain store exhibits almost by construction. Whether the adaptive switching (§4.3.4) still helps, and whether per-query yield degrades, at realistic memory scales (thousands of records accumulated over months, per the ChatGPT Memory/Mem0 deployments cited in §3.1) is untested. At minimum a scaling experiment (e.g., 1k–5k records) is needed to support the 'systematic' and 'large-scale' language in the abstract and §1.
  5. [§8.1.1, Table 5] The production-platform evaluation (§8.1.1, Table 5) is the strongest evidence for real-world relevance, but it is under-specified to the point of being unverifiable: the malicious tool's schema and implementation on Dify/Coze, the planning models used by each platform agent, the trigger-query distribution, and the number of trials are all unreported. Relatedly, there is no ethics or responsible-disclosure statement: the authors should confirm the agents were self-hosted instances with synthetic data (stated only obliquely: 'run in isolation with local evaluation datasets') and state whether the findings were disclosed to Dify/Coze/Mem0. For a security venue this is a required element, not a formality.
minor comments (8)
  1. [§5.1, Tables 1–3] Metric naming inconsistency: §5.1 defines 'Benign Response Rate (BRR)' but all tables (1–3, 6–8) label the column 'BER'. Also, baselines show BER = 0.0% in every cell, which is surprising and never explained — if a baseline's concatenated payload fails to hijack, why does the agent never produce a benign answer? Presumably the shared C_adv drives the loop into the tool-call limit and returns an execution error, but this should be stated explicitly since it bears on the stealth comparison.
  2. [Tables 1–2] Table 1, Embedding Model block, E5-base row: '18.0%%80.0%' contains a stray percent sign. Table 2, E5-base row under @5 shows RER 16.5% while @20 shows 34.5% for the same configuration family — fine, but the @5 Contriever SS shows '1.00 1.00 0.93 0.98' with inconsistent spacing/formatting throughout Table 2; please normalize.
  3. [§4.4] §4.4 (STM Persistence bullet): the sentence 'The payload is continuously refreshed during successful reactivations.' appears twice verbatim. Also in the same bullet, 'need user-trigger again' is ungrammatical.
  4. [§4.3, Definition 1] Definition 1 box (§4.3): 'Each retrieval operation formalize an explored ball' is garbled; the definition also uses d_k before introducing it ('where d_k is the k-th nearest record returned' should precede the radius formula r_k = ‖e(anchor_t) − e(d_k)‖).
  5. [§4.3.3 vs §5.1] Auxiliary-model naming is inconsistent: §4.3.3 uses 'GPT-4o-mini' for keyword extraction while §5.1 ('Malicious Tool Settings') says 'GPT-4-mini'. Please unify and state which model was actually used, since anchor quality depends on it.
  6. [§3.2.1, Figure 3] Figure 3 is central to the L1 motivation but is purely qualitative: no axes, no quantitative measure of 'embedding drift', and no description of how the t-SNE projection was computed (which anchors, which commands, how many). A small quantitative summary (e.g., mean pairwise cosine distance among anchor embeddings with vs without command) would make the interference claim falsifiable rather than illustrative.
  7. [§2.1, §4.2] §2.1 cites platform tool-call limits of '5–20 tool calls per user query' via documentation footnotes, but the attack's N is optimized per platform; the paper would benefit from a short table of measured limits on the platforms actually attacked (Dify, Coze, MemEngine defaults) to ground the Scenario 2/3 budgets.
  8. [§7.1, Figure 8b] The Scenario 3 claim that 'everyday user queries naturally surface' the LTM payload is evaluated with benign queries generated by GPT-4o-mini to be 'semantically related to the embedded topic' (§7.1). This is a somewhat favorable simulator; the low-similarity ablation (Figure 8b, 42.0%) partially addresses this, but using MS MARCO (as in Scenario 2) as an uncurated benign-query distribution for Scenario 3 would be a more convincing stress test.

Circularity Check

0 steps flagged

No significant circularity: empirical attack evaluation on held-out victim LTM, not a self-fulfilling derivation.

full rationale

SPORE is an empirical security paper. Its central claims are measured Record Extraction Rates (and related SS/BRR) under three trigger scenarios, not first-principles predictions. Command and reactivation payloads are optimized via LLM-Fuzzer on a reference LTM (TREC-COVID) that the paper explicitly states differs in distribution from the victim LTMs (HealthCareMagic-100k-en, LoCoMo); victim RER is then measured by actual tool-mediated exfiltration, so success is not true by construction of the fit. Geometric anchor selection uses observed retrieved embeddings to steer subsequent probes—an adaptive attack loop, not a tautological redefinition of the evaluation metric as the claim. Baselines share the same C_adv and differ only in anchor strategy, providing comparative measurement rather than circular self-validation. There is no uniqueness theorem, self-citation load-bearing premise, or renaming of a known closed-form result. Load-bearing threat-model assumptions (LTM auto-retrieval into tool parameters; free-text parameter sinks) are empirical premises about agent behavior, not circular reductions of outputs to inputs. Score 0 is appropriate.

Axiom & Free-Parameter Ledger

7 free parameters · 7 axioms · 3 invented entities

Central claims rest on standard agent/RAG architecture assumptions, a black-box malicious-tool capability model, and several hand-chosen attack hyperparameters optimized on a reference LTM—not on new physical entities. Invented constructs are attack artifacts (payloads, explored balls), not ontological discoveries.

free parameters (7)
  • extraction cycle length N = basic setting N=5 (range 3–7)
    Swept/fixed within platform tool-call limits (reported 3–7; basic setting 5); jointly optimized with C_adv on reference data.
  • mode-switch threshold γ = γ=4
    Consecutive failed-new-record iterations before switching coarse/fine mode; default chosen after small sweep.
  • inter-component KNN size K = K=4
    Neighbors used in Dinter for fine-grained expansion scoring.
  • fine-grained score weights w_intra, w_inter
    Combine normalized intra/inter distances into Sfine; paper states combination but does not derive unique values from first principles.
  • reactivation reward mix λ
    Balances benign-response quality vs reactivation/extraction success when fuzzing P_react.
  • top-k LTM retrieval depth = basic setting k=3
    Environment parameter varied 2–6; strongly affects RER surface.
  • STM window size = basic setting 6; peak ~8
    Affects Scenario 2 persistence; swept 5–10, peak reported at 8.
axioms (7)
  • domain assumption Agent loop is retrieve–plan–act–write with automatic LTM retrieval keyed by current observation (including tool responses), not only explicit user memory calls.
    Stated in §2.1 and threat model §3.1; required for anchors in tool outputs to steer LTM.
  • domain assumption LTM is user-isolated in production (OpenAI/Mem0/Anthropic-style IDs), so user-side shared-store extraction is impossible by construction.
    Appendix A survey; motivates tool-side surface as the residual channel.
  • domain assumption Planning LLM will, under injected STM instructions, issue tool calls that include retrieved conversation/memory content as parameters to the adversary tool.
    Core exfiltration channel in Fig. 1 and C_adv template §4.2; without it RER collapses.
  • domain assumption Platforms enforce small per-query tool-call budgets (cited ~5–20), creating L2.
    §2.1 citations to LangChain/CrewAI/Dify; motivates reactivation payloads.
  • domain assumption Adversary can distribute a tool (e.g., via third-party API marketplaces) that victims’ agents invoke, and can change server-side responses after integration.
    Adversary capabilities §3.1.
  • ad hoc to paper Cosine geometry on attacker-chosen embeddings is a useful proxy for unexplored victim memory mass even under embedding-model mismatch.
    Geometric coverage §4.3; supported empirically across MiniLM/Contriever/E5 but not proved.
  • standard math Standard dense retrieval / RAG and agent memory mechanisms (top-k embedding similarity, optional gating/summarization/reflection).
    Eq. (1) and MemEngine LTM variants; background, not novel math.
invented entities (3)
  • SPORE attack (C_adv + pure anchors + geometric coverage + P_react) no independent evidence
    purpose: Operationalize tool-side LTM extraction under isolation and trigger limits.
    Composite attack design introduced by the paper; evaluated empirically, not an external natural kind.
  • Explored ball B(e(anchor_t), r_k) and explored-region components C_j no independent evidence
    purpose: Approximate covered embedding volume to choose next anchors.
    Definition 1 §4.3 modeling device for coverage optimization.
  • Reactivation payload P_react (STM and LTM persistence modes) no independent evidence
    purpose: Resume extraction without fresh malicious triggers across queries/sessions.
    Attack artifact optimized via LLM-Fuzzer; effectiveness is experimental.

pith-pipeline@v1.2.0-grok45-kimik3 · 30863 in / 4303 out tokens · 75378 ms · 2026-07-30T21:58:47.455821+00:00 · methodology

0 comments
read the original abstract

LLM-based agents extend large language models with long-term memory (LTM) that persists privacy-sensitive user data across sessions. Production systems mitigate extraction risks through memory isolation, binding each user's LTM to a unique identifier. This defense has blocked known attacks on shared storage, fostering the assumption that isolated LTM is secure. We identify the tool interface as an overlooked attack surface. Agents routinely embed LTM-retrieved data in tool invocation parameters, enabling a malicious tool to exfiltrate private memory without violating user-level isolation. Naive adaptations of user-side extraction techniques fail because the adversarial command's semantics interfere with retrieval precision, and platform-imposed tool-call limits constrain the extraction budget per trigger. We present SPORE, the first extraction attack designed for this threat model. SPORE decouples the adversarial command from retrieval anchors by persisting the command in short-term memory and emitting semantically pure anchors in tool responses. The restored retrieval precision enables a geometric coverage optimization over the embedding space that systematically steers anchors toward unexplored memory regions. To sustain extraction beyond tool-call limits, SPORE persists reactivation payloads in memory that automatically resume the attack within and across sessions without additional user triggers. SPORE achieves an 80.0% record extraction rate with unlimited triggers and 47.0% with only 20 triggers. In multi-user deployments, attackers can link extracted records to user identities, enabling targeted surveillance. These results demonstrate that memory isolation alone is insufficient and call for reexamining tool-side trust boundaries in agent architectures.

Figures

Figures reproduced from arXiv: 2607.23444 by Chuanchao Zang, Jianing Wang, Li Wang, Shanqing Guo, Wenyu Chen, Xiangtao Meng, Xinyu Gao, Zheng Li.

Figure 1
Figure 1. Figure 1: Memory isolation effectively blocks data [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Illustration of Step 3— Planning. 2 Preliminaries and Related Works 2.1 LLM-Based Agent Workflow An LLM-based agent extends a base language model with memory and external tools, enabling persistent state management and interaction with external en￾vironments. The agent serves users through sessions, where each session corresponds to a single conversa￾tion thread (e.g., a chat window). When a user sub￾mits … view at source ↗
Figure 3
Figure 3. Figure 3: Semantic interference of command. returns crafted tool outputs that concatenate a se￾mantic anchor with an adversarial command: anchort ⊕ Cadv, where anchort is a semantic retrieval probe and Cadv is an adversarial command that manipulates the agent into leaking retrieved memory records. The anchor guides retrieval toward a target region of the victim’s LTM, while the command hijacks the agent into transmi… view at source ↗
Figure 4
Figure 4. Figure 4: Performance of Scenario 1 with different [PITH_FULL_IMAGE:figures/full_fig_p013_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Ablation study of anchor optimization al [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Performance of Scenario 1 with different [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗
Figure 8
Figure 8. Figure 8: Ablation of Scenario 2 and Scenario 3. improves, peaking at 49.5% RER@20 with window size of 8. 6.3 Ablation Studies We adopt the same experimental settings as Sec￾tion 5.3, with results shown below. Impact of Extraction Budgets. As shown in Fig￾ure 7b, SPORE’s performance first increases then de￾creases as extraction rounds increase, attributing to Scenario 2’s high sensitivity to context length. 7 Scenar… view at source ↗
Figure 9
Figure 9. Figure 9: Performance of Scenario 3 with different [PITH_FULL_IMAGE:figures/full_fig_p017_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Pareto tradeoff. C LLM-Fuzzer LLM-Fuzzer [35] is a black-box fuzzing framework for scalably assessing the jailbreak vulnerability of large language models. Inspired by coverage-guided fuzzers in software testing, it treats jailbreak templates as seeds and automatically evolves them to discover ef￾fective attacks. The core workflow is as follows: Seed Collection. Gather an initial pool of jailbreak templat… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

41 extracted references · 21 linked inside Pith

  1. [1]

    Conversational health agents: A personalized llm-powered agent framework.arXiv preprint arXiv:2310.02374, 2023

    Mahyar Abbasian, Iman Azimi, Amir M Rah- mani, and Ramesh C Jain. Conversational health agents: A personalized llm-powered agent framework.arXiv preprint arXiv:2310.02374, 2023

    Pith/arXiv arXiv 2023

  2. [2]

    Improving language models by retrieving from trillions of tokens

    Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Jean- Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al. Improving language models by retrieving from trillions of tokens. InInternational con- ference on machine learning, pages 2206–2240. PMLR, 2022

    2022

  3. [3]

    Yu, Qiang Yang, and Xing Xie

    Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xi- aoyuan Yi, Cunxiang Wang, Yidong Wang, Wei Ye, Yue Zhang, Yi Chang, Philip S. Yu, Qiang Yang, and Xing Xie. A survey on evaluation of large language models, 2023. URL:https://ar xiv.org/abs/2307.03109,arXiv:2307.03109

    Pith/arXiv arXiv 2023

  4. [4]

    Ms marco: Benchmarking ranking models in the large-data regime

    Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Jimmy Lin. Ms marco: Benchmarking ranking models in the large-data regime. InProceedings of the 44th international ACM SIGIR conference on research and devel- opment in information retrieval, pages 1566– 1576, 2021

    2021

  5. [5]

    Agentdojo: A dynamic en- vironment to evaluate prompt injection attacks and defenses for llm agents.Advances in Neu- ral Information Processing Systems, 37:82895– 82920, 2024

    Edoardo Debenedetti, Jie Zhang, Mislav Balunovic, Luca Beurer-Kellner, Marc Fischer, and Florian Tram` er. Agentdojo: A dynamic en- vironment to evaluate prompt injection attacks and defenses for llm agents.Advances in Neu- ral Information Processing Systems, 37:82895– 82920, 2024

    2024

  6. [6]

    More than you’ve asked for: A comprehensive analysis of novel prompt injec- tion threats to application-integrated large lan- guage models.arXiv preprint arXiv:2302.12173, 27, 2023

    Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. More than you’ve asked for: A comprehensive analysis of novel prompt injec- tion threats to application-integrated large lan- guage models.arXiv preprint arXiv:2302.12173, 27, 2023

    Pith/arXiv arXiv 2023

  7. [7]

    Not what you’ve signed up for: Compromising real-world llm-integrated appli- cations with indirect prompt injection

    Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. Not what you’ve signed up for: Compromising real-world llm-integrated appli- cations with indirect prompt injection. InPro- ceedings of the 16th ACM workshop on artificial intelligence and security, pages 79–90, 2023

    2023

  8. [8]

    [Online]

    Healthcaremagic-100k-en. [Online]. Available: https://huggingface.co/datasets/wangro ngsheng/HealthCareMagic-100k-en

  9. [9]

    Rag-thief: Scal- able extraction of private data from retrieval- augmented generation applications with agent- based attacks.arXiv preprint arXiv:2411.14110, 2024

    Changyue Jiang, Xudong Pan, Geng Hong, Chenfu Bao, and Min Yang. Rag-thief: Scal- able extraction of private data from retrieval- augmented generation applications with agent- based attacks.arXiv preprint arXiv:2411.14110, 2024

    Pith/arXiv arXiv 2024

  10. [10]

    PhD thesis, OW ASP, 2025

    Sotiropoulos John, Rosario Ron F Del, Kokuykin Evgeniy, Oakley Helen, Habler Idan, Underkoffler Kayla, Huang Ken, Stef- fensen Peter, Aralimatti Rakshith, Bitton Ron, et al.Owasp top 10 for llm apps & gen ai agentic security initiative. PhD thesis, OW ASP, 2025

    2025

  11. [11]

    Dense passage retrieval for open-domain question answering

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. InProceed- ings of the 2020 conference on empirical methods in natural language processing (EMNLP), pages 6769–6781, 2020

    2020

  12. [12]

    Retrieval- augmented generation for knowledge-intensive nlp tasks.Advances in neural information pro- cessing systems, 33:9459–9474, 2020

    Patrick Lewis, Ethan Perez, Aleksandra Pik- tus, Fabio Petroni, Vladimir Karpukhin, Na- man Goyal, Heinrich K¨ uttler, Mike Lewis, Wen- tau Yih, Tim Rockt¨ aschel, et al. Retrieval- augmented generation for knowledge-intensive nlp tasks.Advances in neural information pro- cessing systems, 33:9459–9474, 2020

    2020

  13. [14]

    Personal llm agents: Insights and survey about the ca- pability, efficiency and security, 2024

    Yuanchun Li, Hao Wen, Weijun Wang, Xiangyu Li, Yizhen Yuan, Guohong Liu, Jiacheng Liu, Wenxing Xu, Xiang Wang, Yi Sun, Rui Kong, Yile Wang, Hanfei Geng, Jian Luan, Xuefeng Jin, Zilong Ye, Guanjing Xiong, Fan Zhang, Xi- ang Li, Mengwei Xu, Zhijun Li, Peng Li, Yang Liu, Ya-Qin Zhang, and Yunxin Liu. Personal llm agents: Insights and survey about the ca- pab...

    Pith/arXiv arXiv 2024

  14. [15]

    Deepseek-v2: A strong, economical, and effi- cient mixture-of-experts language model.arXiv preprint arXiv:2405.04434, 2024

    Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, et al. Deepseek-v2: A strong, economical, and effi- cient mixture-of-experts language model.arXiv preprint arXiv:2405.04434, 2024

    Pith/arXiv arXiv 2024

  15. [16]

    Evaluating very long-term con- versational memory of llm agents, 2024

    Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. Evaluating very long-term con- versational memory of llm agents, 2024. URL: https://arxiv.org/abs/2402.17753,arXiv: 2402.17753

    Pith/arXiv arXiv 2024

  16. [17]

    Gpt-5-mini, 2023

    OpenAI. Gpt-5-mini, 2023. URL:https://pl atform.openai.com/docs/models

    2023

  17. [18]

    Generative agents: Interac- tive simulacra of human behavior

    Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interac- tive simulacra of human behavior. InProceedings of the 36th annual acm symposium on user inter- face software and technology, pages 1–22, 2023

    2023

  18. [19]

    Gorilla: Large lan- guage model connected with massive apis.Ad- vances in Neural Information Processing Sys- tems, 37:126544–126565, 2024

    Shishir G Patil, Tianjun Zhang, Xin Wang, and Joseph E Gonzalez. Gorilla: Large lan- guage model connected with massive apis.Ad- vances in Neural Information Processing Sys- tems, 37:126544–126565, 2024

    2024

  19. [20]

    Follow my instruction and spill the beans: Scalable data extraction from retrieval-augmented generation systems.arXiv preprint arXiv:2402.17840, 2024

    Zhenting Qi, Hanlin Zhang, Eric Xing, Sham Kakade, and Himabindu Lakkaraju. Follow my instruction and spill the beans: Scalable data extraction from retrieval-augmented generation systems.arXiv preprint arXiv:2402.17840, 2024

    Pith/arXiv arXiv 2024

  20. [21]

    Tool learning with foundation models.ACM Computing Surveys, 57(4):1–40, 2024

    Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Xu- anhe Zhou, Yufei Huang, Chaojun Xiao, et al. Tool learning with foundation models.ACM Computing Surveys, 57(4):1–40, 2024

    2024

  21. [22]

    In-context retrieval- augmented language models.Transactions of the Association for Computational Linguistics, 11:1316–1331, 2023

    Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton- Brown, and Yoav Shoham. In-context retrieval- augmented language models.Transactions of the Association for Computational Linguistics, 11:1316–1331, 2023

    2023

  22. [23]

    Replug: Retrieval-augmented black-box language mod- els

    Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Richard James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. Replug: Retrieval-augmented black-box language mod- els. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 8371–8...

    2024

  23. [24]

    Ehragent: Code empowers large language models for few-shot complex tabular reasoning on electronic health records.arXiv preprint arXiv:2401.07128, 2024

    Wenqi Shi, Ran Xu, Yuchen Zhuang, Yue Yu, Jieyu Zhang, Hang Wu, Yuanda Zhu, Joyce Ho, Carl Yang, and May D Wang. Ehragent: Code empowers large language models for few-shot complex tabular reasoning on electronic health records.arXiv preprint arXiv:2401.07128, 2024

    Pith/arXiv arXiv 2024

  24. [25]

    Gemini: a fam- ily of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Sori- cut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a fam- ily of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

    Pith/arXiv arXiv 2023

  25. [26]

    Beir: A heterogenous benchmark for zero-shot eval- uation of information retrieval models.arXiv preprint arXiv:2104.08663, 2021

    Nandan Thakur, Nils Reimers, Andreas R¨ uckl´ e, Abhishek Srivastava, and Iryna Gurevych. Beir: A heterogenous benchmark for zero-shot eval- uation of information retrieval models.arXiv preprint arXiv:2104.08663, 2021

    Pith/arXiv arXiv 2021

  26. [27]

    Adapted large language models can out- perform medical experts in clinical text sum- marization.Nature medicine, 30(4):1134–1142, 2024

    Dave Van Veen, Cara Van Uden, Louis Blanke- meier, Jean-Benoit Delbrouck, Asad Aali, Chris- 21 tian Bluethgen, Anuj Pareek, Malgorzata Po- lacin, Eduardo Pontes Reis, Anna Seehofnerov´ a, et al. Adapted large language models can out- perform medical experts in clinical text sum- marization.Nature medicine, 30(4):1134–1142, 2024

    2024

  27. [28]

    Enhancing large language model with self-controlled memory framework

    Bing Wang, Xinnian Liang, Jian Yang, Hui Huang, Shuangzhi Wu, Peihao Wu, Lu Lu, Zejun Ma, and Zhoujun Li. Enhancing large language model with self-controlled memory framework. arXiv preprint arXiv:2304.13343, 2023

    Pith/arXiv arXiv 2023

  28. [29]

    Unveil- ing privacy risks in llm agent memory.arXiv preprint arXiv:2502.13172, 2025

    Bo Wang, Weiyi He, Shenglai Zeng, Zhen Xiang, Yue Xing, Jiliang Tang, and Pengfei He. Unveil- ing privacy risks in llm agent memory.arXiv preprint arXiv:2502.13172, 2025

    Pith/arXiv arXiv 2025

  29. [30]

    A survey on large language model based au- tonomous agents.Frontiers of Computer Sci- ence, 18(6), March 2024

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Jirong Wen. A survey on large language model based au- tonomous agents.Frontiers of Computer Sci- ence, 18(6), March 2024. URL:http://dx .doi.org/10.1007/s11704- 024- 40231-1, doi:10.1007/s11704-024-40231-1

    doi:10.1007/s11704- 2024

  30. [31]

    Text embeddings by weakly-supervised contrastive pre-training

    Liang Wang, Nan Yang, Xiaolong Huang, Binx- ing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.03533, 2022

    Pith/arXiv arXiv 2022

  31. [32]

    Silent leaks: Implicit knowledge extraction attack on rag systems through benign queries, 2025

    Yuhao Wang, Wenjie Qu, Shengfang Zhai, Yanze Jiang, Zichen Liu, Yue Liu, Yinpeng Dong, and Jiaheng Zhang. Silent leaks: Implicit knowledge extraction attack on rag systems through benign queries, 2025. URL:https://arxiv.org/abs/ 2505.15420,arXiv:2505.15420

    arXiv 2025

  32. [33]

    Tradingagents: Multi-agents llm fi- nancial trading framework.arXiv preprint arXiv:2412.20138, 2024

    Yijia Xiao, Edward Sun, Di Luo, and Wei Wang. Tradingagents: Multi-agents llm fi- nancial trading framework.arXiv preprint arXiv:2412.20138, 2024

    Pith/arXiv arXiv 2024

  33. [34]

    Watch out for your agents! investigating backdoor threats to llm- based agents.Advances in Neural Information Processing Systems, 37:100938–100964, 2024

    Wenkai Yang, Xiaohan Bi, Yankai Lin, Sishuo Chen, Jie Zhou, and Xu Sun. Watch out for your agents! investigating backdoor threats to llm- based agents.Advances in Neural Information Processing Systems, 37:100938–100964, 2024

    2024

  34. [35]

    In33rd USENIX Security Symposium (USENIX Secu- rity 24), pages 4657–4674, 2024

    Jiahao Yu, Xingwei Lin, Zheng Yu, and Xinyu Xing.{LLM-Fuzzer}: Scaling assessment of large language model jailbreaks. In33rd USENIX Security Symposium (USENIX Secu- rity 24), pages 4657–4674, 2024

    2024

  35. [36]

    Finmem: A performance-enhanced llm trading agent with layered memory and character de- sign.arXiv preprint arXiv:2311.13743, 2024

    Yangyang Yu, Haohang Li, Zhi Chen, Yuechen Jiang, Yang Li, Denghui Zhang, Rong Liu, Jor- dan W Suchow, and Khaldoun Khoshnevisan. Finmem: A performance-enhanced llm trading agent with layered memory and character de- sign.arXiv preprint arXiv:2311.13743, 2024

    Pith/arXiv arXiv 2024

  36. [37]

    The good and the bad: Exploring privacy issues in retrieval- augmented generation (rag).arXiv preprint arXiv:2402.16893, 2024

    Shenglai Zeng, Jiankun Zhang, Pengfei He, Yue Xing, Yiding Liu, Han Xu, Jie Ren, Shuaiqiang Wang, Dawei Yin, Yi Chang, et al. The good and the bad: Exploring privacy issues in retrieval- augmented generation (rag).arXiv preprint arXiv:2402.16893, 2024

    Pith/arXiv arXiv 2024

  37. [38]

    Agent security bench (asb): Formalizing and benchmarking at- tacks and defenses in llm-based agents, 2025

    Hanrong Zhang, Jingyuan Huang, Kai Mei, Yifei Yao, Zhenting Wang, Chenlu Zhan, Hongwei Wang, and Yongfeng Zhang. Agent security bench (asb): Formalizing and benchmarking at- tacks and defenses in llm-based agents, 2025. URL:https://arxiv.org/abs/2410.02644, arXiv:2410.02644

    Pith/arXiv arXiv 2025

  38. [39]

    Towards ethical personal ai ap- plications: Practical considerations for ai assis- tants with long-term memory.arXiv preprint arXiv:2409.11192, 2024

    Xi Zhang, Srivatsan Setlur, Maarten Sap, and Yejin Choi. Towards ethical personal ai ap- plications: Practical considerations for ai assis- tants with long-term memory.arXiv preprint arXiv:2409.11192, 2024

    Pith/arXiv arXiv 2024

  39. [40]

    Memengine: A unified and modular library for developing ad- vanced memory of llm-based agents

    Zeyu Zhang, Quanyu Dai, Xu Chen, Rui Li, Zhongyang Li, and Zhenhua Dong. Memengine: A unified and modular library for developing ad- vanced memory of llm-based agents. InCompan- ion Proceedings of the ACM on Web Conference 2025, pages 821–824, 2025

    2025

  40. [41]

    A survey of large language models.arXiv preprint arXiv:2303.18223, 1(2), 2023

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, 22 et al. A survey of large language models.arXiv preprint arXiv:2303.18223, 1(2), 2023

    Pith/arXiv arXiv 2023

  41. [42]

    Universal and transferable adversarial attacks on aligned language models.arXiv preprint arXiv:2307.15043, 2023

    Andy Zou, Zifan Wang, Nicholas Carlini, Mi- lad Nasr, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models.arXiv preprint arXiv:2307.15043, 2023. 23 A Memory Isolation We survey the memory isolation strategies adopted by three representative production systems. Despite architectural differences, all...

    Pith/arXiv arXiv 2023