Pith. sign in

REVIEW 5 major objections 5 minor 2 cited by

Quantifying Conversation Drift in MCP via Latent Polytope

T0 review · 5 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SecMCP claims that adversarial content injected through MCP tools or data measurably shifts an LLM's internal activations, so a latent-polytope distance check catches hijacking, misleading, and exfiltration at a 0.98 average AUROC.

desk verdict A plausible extension of activation-based drift detection to MCP, with an unverifiable headline AUROC due to best-of-five layer selection and a latent polytope that is never actually constructed. read the letter →

arxiv 2508.06418 v1 pith:5SBHA5PB submitted 2025-08-08 cs.CL

Haoran Shi , Hongwei Yao , Shuo Shao , Shaopeng Jiao , Ziqi Peng , Zhan Qin , Cong Wang This is my paper
classification cs.CL
keywords ModelContextProtocolpromptinjectiondetectiontoolpoisoningconversationdriftlatentpolytopeactivationvectorsLLMsecuritydataexfiltration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The Model Context Protocol (MCP) lets LLM-powered agents pull real-time data and tools into a conversation, but that same open pipeline lets attacker-controlled text ride in through retrieved documents or tool descriptions. SecMCP's proposal is to stop trusting surface text and watch the model's internals: adversarial instructions, even when phrased innocuously, shift the trajectory of the conversation in the model's latent activation space. By collecting last-token activation vectors of previously legitimate queries as anchor points and measuring how far a new query's activation drifts from that anchor region—the "latent polytope"—SecMCP flags drift that exceeds a threshold and reports the drift magnitude as a quantification of hijacking, misleading, or data-exfiltration risk. Across Llama3-8B, Mistral-7B, and Vicuna-7B on FinQA, HotpotQA, and MS MARCO, the paper reports AUROC above 0.915 in all 27 risk–model–dataset cells, with an average of 0.98. The paper's own robustness and limitation sections bound the claim: synonym-replaced exfiltration prompts drop detection to about 0.86 AUROC, and the method assumes a stable query–response structure, leaving asynchronous multi-agent protocols, token-level attribution, and interpretability unaddressed.

What carries the argument

Latent polytope: the authorized region in activation space spanned by anchor points — the last-token activation vectors of previously legitimate queries, extracted at layers 0, 7, 15, 23, and 31 following the paper's cited method. For each incoming query SecMCP embeds its activation vectors, computes the squared Euclidean distance to every anchor per layer, and feeds the resulting distance profile to a decision tree that applies threshold $\tau$ to accept or reject. The polytope is what makes "conversation drift" measurable: deviation from the anchor region is the paper's quantitative proxy for adversarial manipulation, and the five layers act as a small ensemble of representational viewpoin

What would settle it

Run SecMCP's own protocol (1,000 anchors, layers 0/7/15/23/31, decision-tree threshold) on safe queries spanning many unrelated topics, with no attack present: if the false-positive rate on this diverse-but-benign traffic approaches the attack-detection rate, then the drift signal is topic novelty rather than adversarial intent. A cheaper check is already in the paper: synonym-replaced exfiltration prompts cut AUROC to about 0.86, so testing LLM-paraphrased exfiltration prompts would show whether the detector tracks semantic intent or surface wording.

Watch

Extended reading notes

Core claim

Central claim: the three risks — hijacking, misleading, and data exfiltration — are detectable and quantifiable before they take effect, because adversarial instructions, however benign in surface text, shift the conversation's trajectory in the model's latent representation space. SecMCP records last-token activation vectors of legitimate queries as anchor points spanning an authorized region, then computes for each incoming query the squared distance from its activation to all anchors, layer by layer; exceeding threshold $\tau$ rejects the query. The paper reports AUROC above 0.915 in every combination, averaging 0.98, with the weakest cell (misleading on MS MARCO, Llama3-8B) at 0.915.

Load-bearing premise

The load-bearing premise is that attacker-injected content reliably pushes the model's last-token activation at one of five chosen layers far enough from the benign anchor region that a single fixed distance threshold separates it from every safe query — and this premise is not tested against benign queries that are simply about unfamiliar topics, nor against attacks reworded beyond synonyms.

Editorial extensions

If this is right

  • Detection happens before the injected instruction acts: a per-query distance snapshot at inference time flags hijacking, misleading, and exfiltration attempts without needing attack signatures or a second LLM to judge responses.
  • The signal transfers across models and domains: every one of the 27 risk–model–dataset cells clears 0.915 AUROC, so the drift is not an artifact of one model's internals.
  • Drift is a continuous quantity, not just a verdict: the same deviation metric yields a per-turn seriousness score that operators could log and triage, which the binary AUROC evaluation under-uses.
  • Synonym-rewording resistance is uneven: hijacking and misleading cases stay near or above 0.94 AUROC under synonym replacement, while data-exfiltration detection falls to roughly 0.86, marking a concrete boundary of the claim.
  • No extra inference cost at generation time: the added work is an embedding projection and distance computation, so the defense can accompany normal MCP operation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The lowest cells all come from MS MARCO, the most topically diverse benchmark, which the paper attributes to breadth of topics. A direct stress test — benign queries deliberately spanning many unrelated topics with no attack present — would show whether the fixed threshold separates adversarial intent or just novelty; if the latter, the false-positive rate on diverse-but-safe traffic could approac
  • The synonym-replacement result for data exfiltration (0.989 to 0.862 on HotpotQA with Llama3-8B) hints that part of the exfiltration signal is surface wording rather than semantics. A natural next experiment is full paraphrase or translation of attack prompts; the prediction from this paper's numbers is that detection degrades further.
  • AUROC values are reported as single point estimates, with no seeds, no repeated runs, and no confidence intervals (the paper's own reproducibility checklist leaves randomness, run counts, and statistical tests unaddressed). The "0.98 average" should therefore be read as an approximate figure until variance is reported.
  • Because the deviation score is continuous, SecMCP could be turned into a per-turn "hijack meter" for agent logs — flagging not just whether a conversation drifted, but when and how far — which the paper's own token-level-attribution limitation leaves open.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes SecMCP, a framework for detecting and quantifying 'conversation drift' in Model Context Protocol (MCP) based agent systems. SecMCP extracts last-token activation vectors from LLMs, defines a set of benign anchor activations, and flags inputs whose activation distance to these anchors exceeds a threshold. The authors categorize threats as data exfiltration, misleading, and hijacking, and report AUROC values above 0.915 (average 0.98) across Llama3-8B, Mistral-7B, and Vicuna-7B on FinQA, HotpotQA, and MS MARCO. They also compare against three baselines, study robustness to synonym replacement, and ablate the number of anchor samples and visualization. The paper claims a novel 'latent polytope' methodology, but the actual method described is a distance-based threshold.

Significance. If supported, a lightweight activation-based detector for adversarial external knowledge in MCP would be a useful contribution, extending prior activation-based task-drift detection to security settings. The threat taxonomy and the idea of using last-token activations as a signal are reasonable and build on existing work (Abdelnabi et al., 2024). However, as presented, the evidence does not support the central claims: the 'latent polytope' is never constructed, Eq. (1) does not implement the described distance, the evaluation protocol is under-specified, the headline AUROC is likely inflated by best-of-five layer selection, and no confidence intervals or held-out splits are reported. The paper's own MS MARCO result and the synonym-replacement robustness table suggest the detector may be capturing topic or lexical differences rather than adversarial drift. The contribution could become publishable after a major rework of the methodology and evaluation, but the current manuscript is not a reliable basis for the claimed detection performance.

major comments (5)
  1. [Risk Matching / Eq. (1)] Eq. (1) writes the decision as the sum over anchors of ||E(Act(q_in,l,θ))||² - ||E(Act(q_anc_j,l,θ))||², with no vector subtraction and no absolute value. This is not the squared Euclidean distance described in the prose and can be negative. Additionally, the 'latent polytope authorized access region A' is never defined: anchor points do not by themselves form a polytope unless a convex hull or other geometric object is explicitly introduced. The method as described reduces to a distance-to-anchors threshold, so the central 'latent polytope' contribution is not actually implemented.
  2. [Hyper-parameters / Table 1] The paper states that 'computations are performed at layers 0, 7, 15, 23, and 31, with the best-performing result among them reported as the final outcome.' Selecting the best layer on the same test set and reporting only the best AUROC inflates expected performance. Table 1 reports no per-layer AUROC, no confidence intervals, and no standard deviations. The reproducibility checklist confirms 4.10 (number of runs) and 4.11 (variation/confidence) are not addressed. The average AUROC of 0.98 cannot be assessed without a proper layer-selection procedure on a validation split.
  3. [Experiment / Evaluation protocol] The evaluation protocol is under-specified. For Table 1, the paper does not state how many benign and malicious samples are used per dataset and per risk category, how the decision tree classifier in 'Risk Matching' is trained, or what train/test split (if any) is applied. The baseline comparison in the Effectiveness section uses a separate set of 3,000 malicious and 5,000 benign samples from FinQA only, but no analogous description is given for the main results. Without these details, the reported AUROC values are not reproducible and could reflect overfitting to the test set.
  4. [Effectiveness / MS MARCO discussion] The paper attributes lower AUROC on MS MARCO to 'broader topical diversity,' which acknowledges that the detector's signal may be topic-level difference rather than adversarial drift. The malicious samples are generated through different pipelines (ChatGPT-4.5 for data exfiltration, PoisonedRAG for misleading, HijackRAG for hijacking), so they may differ from the benign benchmark queries in domain, vocabulary, and style. To support the claim that the method detects adversarial latent shifts rather than out-of-distribution topic shifts, the authors should include topic-matched control groups, e.g., benign queries from different domains or paraphrased benign queries, and show that the distance metric separates attacks from these controls.
  5. [Robustness / Table 2] The robustness experiment shows a 0.116–0.127 drop in AUROC for data exfiltration under synonym replacement (e.g., Llama3-8B from 0.989 to 0.862). This is a substantial degradation, yet the conclusion states the method maintains 'robustness against adaptive threats.' A drop of this magnitude for a standard synonym perturbation should be discussed and contextualized; otherwise the robustness claim is overstated. It also reinforces the concern that the detector relies on surface lexical cues.
minor comments (5)
  1. [General / references] The text contains 'Table ??' placeholders (e.g., 'Table ?? presents the AUROC performance'), which should be resolved to the actual table number.
  2. [Related Works / references] The citation '(fak 2022)' is incomplete; the reference list entry 'GonzaloA/fake news' should be given a proper citation with the actual source and repository name.
  3. [Figures] Figure 3 and Figure 5 contain literal 'uni00AD' zero-width hyphen artifacts in the axis labels and legend (e.g., 'Llama3/uni00AD8B'), which should be cleaned in the final version.
  4. [Ablation Study / Figure 4] Figure 4 is a t-SNE scatter plot, but the text refers to it as a 'heatmap.' Please correct the terminology and, if possible, report the t-SNE perplexity and the number of points used.
  5. [Hyper-parameters] The threshold τ in Eq. (1) is never given a defau lt value; the decision-tree implementation presumably learns a threshold, but this should be stated explicitly.

Circularity Check

2 steps flagged · score 6.0 of 10

Headline AUROC partially reduces to test-set max-selection over five layers (no held-out split); the 'latent polytope' novelty is a relabeling of conventional anchor-distance detection.

  1. fitted input called prediction [Experiment section, 'Hyper-parameters' paragraph and 'Risk Matching' (Eq. 1); Reproducibility Checklist 4.10-4.11]
    "For the three large language models evaluated, computations are performed at layers 0, 7, 15, 23, and 31, with the best-performing result among them reported as the final outcome. In implementation, we utilize a decision tree classifier to systematically assign queries to categories based on the distance."

    For each scenario the reported AUROC is the maximum over five layer choices computed on the same evaluation set, and the decision-tree classifier that converts distance features into decisions is described with no train/validation split. The reproducibility checklist admits no run counts (4.10) and no variation/confidence information (4.11). Thus the headline 'average AUROC of 0.98' is not an out-of-sample prediction of a fixed detector: the fitted inputs (best layer, tree) are selected using the same labels on which the metric is then reported, so the reported value is, by construction, an optimistic selection statistic, statistically forced upward.

  2. renaming known result [Abstract/Introduction ('latent polytope space') vs 'Risk Matching' (Eq. 1)]
    "By modeling LLM activation vectors within a latent polytope space, SECMCP identifies anomalous shifts in conversational dynamics, enabling proactive detection of hijacking, misleading, and data exfiltration. ... This approach follows a conventional distance-based detection paradigm."

    The 'latent polytope' methodology is never defined: no polytope, vertices, faces, or convex hull appears anywhere; the only computation is Eq. 1's sum of squared distances between the query's last-token activation (or its embedding) and the anchor points. The paper itself says the stage 'follows a conventional distance-based detection paradigm,' and the activation-extraction recipe is explicitly inherited from the externally cited Abdelnabi et al. 2024. Therefore the claimed 'novel latent polytope-based methodology for quantifying conversation drift' reduces to a relabeling of a known anchor-distance/one-class detector; the novelty contribution is a rename presented as a new construction.

full rationale

The core detector is not definitionally circular: the paper states as a hypothesis that adversarial inputs shift activations away from benign anchors (Overview: 'Our core hypothesis is that these behavioral shifts are reflected in the latent representation space'), then tests it on malicious samples generated by external pipelines (ChatGPT-4.5 prompts, PoisonedRAG, HijackRAG, fake-news corpus) against benign samples from FinQA/HotpotQA/MS MARCO; AUROC would be near 0.5 if that premise were false. So the central separation claim has independent empirical content. Self-citations (Yao et al. 2025 ControlNET; Yao, Lou, and Qin 2024 Poisonprompt) appear only in the introduction to motivate the MCP threat model and are not load-bearing; the key methodological predecessor (last-token activation extraction) is the external Abdelnabi et al. 2024, no ansatz is smuggled via self-citation, and no uniqueness theorem or proofs are claimed (checklist 2.1: no theoretical contributions). Two genuine reductions remain. First, an evaluation-selection circularity: per-scenario AUROC is the best of five layers on the same evaluation data, the decision tree has no disclosed split, and checklist 4.10-4.11 admit no runs/variation, so the reported 0.98 average is partially a test-set-maximum statistic rather than a fixed-configuration prediction. Second, a naming circularity: the 'latent polytope' is never defined and the paper concedes its matching 'follows a conventional distance-based detection paradigm,' so the headline novelty reduces to a label for Eq. 1's anchor distances. The Limitations section further concedes the approach 'captures topic-level deviations effectively,' which supports a topic/lexical confounder reading but does not, by itself, create circularity. Overall: partial circularity in the reported metric and in the novelty framing, while the underlying activation-separability claim is empirically grounded.

Assumptions & free parameters 6 free parameters · 5 assumptions · 2 invented entities

The central claim depends on activation-separability and the MCP threat model. Several hyperparameters are unspecified, and the only invented construct with a strong name, the latent polytope, is not mathematically realized.

free parameters (6)
  • Acceptance threshold tau = not reported
    Equation (1) classifies inputs as malicious when the computed quantity exceeds tau, but tau is never reported or cross-validated, so AUROC depends on an unspecified operating point.
  • Layer selection (best of {0,7,15,23,31}) = best-performing layer per scenario
    The Hyper-parameters section reports the best AUROC among five layers, which is post-hoc selection and inflates the reported numbers.
  • Embedding model E = not specified
    Risk Matching relies on an embedding model E for the final distance computation, but no model name, weights, or dimension are given.
  • Decision tree classifier = not specified
    The paper says a decision tree assigns queries to categories based on the distance, but its training data, structure, and hyperparameters are omitted.
  • Number of anchor samples = 1000 default, 200 to 2000 ablated
    Anchor count is a tuned hyperparameter, and the ablation shows AUROC varies with it.
  • Top-k retrieval = 5
    The MCP server retrieval top-k is set to 5, which affects how much injected content reaches the model.
assumptions (5)
  • domain assumption Adversarial instructions activate distinct clusters of neurons in the latent space.
    Core hypothesis in the Methodology Overview; drives the entire detection method and is only illustrated with t-SNE on the same data.
  • domain assumption Semantic divergence between benign and malicious queries is captured by last-token activation vectors across selected layers.
    Inherited from Abdelnabi et al. 2024 and used as the design basis for Activation Collection.
  • domain assumption MCP threat model: adversarial servers can tamper with tool descriptions and adversarial data providers can alter retrieved documents.
    Stated in the Threat Model section; reasonable but limits generalization to weaker or stronger adversaries.
  • domain assumption Stable query-response structure with fixed conversation boundaries and speaker roles.
    The Limitations section states the method is not directly applicable to asynchronous multi-agent protocols such as A2A.
  • domain assumption MCP architecture and JSON-RPC lifecycle as described by Hou et al. 2025.
    The paper builds its system model on this cited characterization of MCP.
invented entities (2)
  • Latent polytope authorized access region A
    purpose: Frame the benign activation region and classify inputs outside it as malicious.
    The term 'latent polytope' appears in the title and abstract, but no polytope is ever constructed. Equation (1) is a scalar distance threshold, so the entity is rhetorical framing.
  • Conversation drift
    purpose: Quantified deviation of activation trajectories from benign anchors.
    Defined operationally as activation distance, but not validated against an external reference or a separate conversational-dynamics measurement.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Quantifying Conversation Drift in MCP via Latent Polytope." pith.science (2026). https://pith.science/paper/5SBHA5PB

@misc{pith2026250806418,
  author       = {Pith},
  title        = {Pith review of: Quantifying Conversation Drift in MCP via Latent Polytope},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5SBHA5PB}},
  note         = {Machine review of arXiv:2508.06418}
}
read the original abstract

The Model Context Protocol (MCP) enhances large language models (LLMs) by integrating external tools, enabling dynamic aggregation of real-time data to improve task execution. However, its non-isolated execution context introduces critical security and privacy risks. In particular, adversarially crafted content can induce tool poisoning or indirect prompt injection, leading to conversation hijacking, misinformation propagation, or data exfiltration. Existing defenses, such as rule-based filters or LLM-driven detection, remain inadequate due to their reliance on static signatures, computational inefficiency, and inability to quantify conversational hijacking. To address these limitations, we propose SecMCP, a secure framework that detects and quantifies conversation drift, deviations in latent space trajectories induced by adversarial external knowledge. By modeling LLM activation vectors within a latent polytope space, SecMCP identifies anomalous shifts in conversational dynamics, enabling proactive detection of hijacking, misleading, and data exfiltration. We evaluate SecMCP on three state-of-the-art LLMs (Llama3, Vicuna, Mistral) across benchmark datasets (MS MARCO, HotpotQA, FinQA), demonstrating robust detection with AUROC scores exceeding 0.915 while maintaining system usability. Our contributions include a systematic categorization of MCP security threats, a novel latent polytope-based methodology for quantifying conversation drift, and empirical validation of SecMCP's efficacy.

Figures

Figures reproduced from arXiv: 2508.06418 by the authors.

Figure 1
Figure 1. Overall architecture and workflow of the MCP-powered agent system. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Attacks during the operation of the MCP-powered agent system and the three associated security risks. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 2
Figure 2. Data Exfiltration We define data exfiltration as an adver￾sary’s attempt to manipulate prompts in order to bypass the LLM’s defense mechanisms and extract private information such as personally identifiable information (PII) from the model’s underlying database. Indirect Prompt Injection Attacks In an MCP host, the AI agent retrieves external knowledge from the MCP server’s data source to assist in addressing user q… view at source ↗
Figures from the paper (3 more)
Figure 3
Figure 3. Figure 3: Comparison of effectiveness with baseline meth [PITH_FULL_IMAGE:figures/full_fig_p006_3.png]
Figure 4
Figure 4. Figure 4: T-SNE visualizations of the activation deviation [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Effectiveness performance on three risks with different anchor samples quantity [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Rethinking MCP Security: A Large-Scale Study of Runtime MCP Servers and Security Scanner Reliability

    cs.CR 2026-07 conditional novelty 7.0 of 10

    MCP security scanners flag almost all runtime MCP servers as risky, yet manual and CVE validation show those alerts are inconsistent and often wrong.

  2. ShareLock: A Stealthy Multi-Tool Threshold Poisoning Attack Against MCP

    cs.CR 2026-06 unverdicted novelty 7.0 of 10

    ShareLock applies Shamir's threshold scheme to distribute poisoning payloads across multiple MCP tool descriptions, achieving information-theoretic secrecy and over 90% average attack success rate in multi-tool scenarios.

Reference graph

Works this paper leans on

31 extracted references · 8 canonical work pages · cited by 2 Pith papers

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    GonzaloA/fake\_news

    2022. GonzaloA/fake\_news

    work page 2022

  4. [4]

    Abdelnabi, S.; Fay, A.; Cherubin, G.; Salem, A.; Fritz, M.; and Paverd, A. 2024. Are you still on track!? Catching LLM Task Drift with Activations. arXiv preprint arXiv:2406.00799

    arXiv 2024

  5. [5]

    L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al

    Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F. L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al. 2023. GPT-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  6. [6]

    Bartoszcze, L.; Munshi, S.; Sukidi, B.; Yen, J.; Yang, Z.; Williams-King, D.; Le, L.; Asuzu, K.; and Maple, C. 2025. Representation Engineering for Large-Language Models: Survey and Research Challenges. arXiv preprint arXiv:2502.17601

    arXiv 2025

  7. [7]

    Bayat, R.; Rahimi-Kalahroudi, A.; Pezeshki, M.; Chandar, S.; and Vincent, P. 2025. Steering large language model activations in sparse spaces. arXiv preprint arXiv:2503.00177

    arXiv 2025

  8. [8]

    Chen, S.; Piet, J.; Sitawarin, C.; and Wagner, D. 2024. StruQ: Defending Against Prompt Injection with Structured Queries. arXiv:2402.06363

    arXiv 2024

Show all 31 references
  1. [9]

    Chen, Z.; Chen, W.; Smiley, C.; Shah, S.; Borova, I.; Langdon, D.; Moussa, R.; Beane, M.; Huang, T.-H.; Routledge, B.; and Wang, W. Y. 2021. F in QA : A Dataset of Numerical Reasoning over Financial Data. In Moens, M.-F.; Huang, X.; Specia, L.; and tau Yih, S. W., eds., Procee...

    2021

  2. [10]

    Fang, J.; Yao, Z.; Wang, R.; Ma, H.; Wang, X.; and Chua, T.-S. 2025. We Should Identify and Mitigate Third-Party Safety Risks in MCP-Powered Agent Systems. arXiv preprint arXiv:2506.13666

    2025 arXiv

  3. [11]

    He, X.; Xu, G.; Han, X.; Wang, Q.; Zhao, L.; Shen, C.; Lin, C.; Zhao, Z.; Li, Q.; Yang, L.; et al. 2025 a . Artificial intelligence security and privacy: a survey. Science China Information Sciences, 68(8): 1--90

    2025

  4. [12]

    He, Z.; Jin, M.; Shen, B.; Payani, A.; Zhang, Y.; and Du, M. 2025 b . SAE-SSV: Supervised Steering in Sparse Representation Spaces for Reliable Control of Language Models. arXiv preprint arXiv:2505.16188

    2025

  5. [13]

    Hou, X.; Zhao, Y.; Wang, S.; and Wang, H. 2025. Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directions. arXiv:2503.23278

    2025 arXiv

  6. [14]

    Jacob, D.; Alzahrani, H.; Hu, Z.; Alomair, B.; and Wagner, D. 2025. PromptShield: Deployable Detection for Prompt Injection Attacks. arXiv:2501.15145

    2025 arXiv

  7. [15]

    Liu, Y.; Deng, G.; Li, Y.; Wang, K.; Wang, Z.; Wang, X.; Zhang, T.; Liu, Y.; Wang, H.; Zheng, Y.; et al. 2023. Prompt Injection attack against LLM-integrated Applications. arXiv preprint arXiv:2306.05499

    2023 arXiv

  8. [16]

    Liu, Y.; Jia, Y.; Geng, R.; Jia, J.; and Gong, N. Z. 2024. Formalizing and Benchmarking Prompt Injection Attacks and Defenses. In 33rd USENIX Security Symposium (USENIX Security 24), 1831--1847

    2024

  9. [17]

    S.; and Habler, I

    Narajala, V. S.; and Habler, I. 2025. Enterprise-Grade Security for the Model Context Protocol (MCP): Frameworks and Mitigation Strategies. arXiv preprint arXiv:2504.08623

    2025 arXiv

  10. [18]

    S.; Huang, K.; and Habler, I

    Narajala, V. S.; Huang, K.; and Habler, I. 2025. Securing GenAI Multi-Agent Systems Against Tool Squatting: A Zero Trust Registry-Based Approach. arXiv:2504.19951

    2025 arXiv

  11. [19]

    Nguyen, T.; Rosenberg, M.; Song, X.; Gao, J.; Tiwary, S.; Majumder, R.; and Deng, L. 2017. MS MARCO : A Human-Generated MA chine Reading CO mprehension Dataset

    2017

  12. [20]

    Perez, F.; and Ribeiro, I. 2022. Ignore previous prompt: Attack techniques for language models. arXiv preprint arXiv:2211.09527

    2022 arXiv

  13. [21]

    Phute, M.; Helbling, A.; Hull, M.; Peng, S.; Szyller, S.; Cornelius, C.; and Chau, D. H. 2024. LLM Self Defense: By Self Examination, LLMs Know They Are Being Tricked. arXiv:2308.07308

    2024 arXiv

  14. [22]

    Radosevich, B.; and Halloran, J. 2025. MCP Safety Audit: LLMs with the Model Context Protocol Allow Major Security Exploits. arXiv:2504.03767

    2025 arXiv

  15. [23]

    Subramani, N.; Suresh, N.; and Peters, M. E. 2022. Extracting latent steering vectors from pretrained language models. arXiv preprint arXiv:2205.05124

    2022 arXiv

  16. [24]

    Suo, X. 2024. Signed-Prompt: A New Approach to Prevent Prompt Injection Attacks Against LLM-Integrated Applications. arXiv:2401.07612

    2024 arXiv

  17. [25]

    Yang, Z.; Qi, P.; Zhang, S.; Bengio, Y.; Cohen, W.; Salakhutdinov, R.; and Manning, C. D. 2018. H otpot QA : A Dataset for Diverse, Explainable Multi-hop Question Answering. In Riloff, E.; Chiang, D.; Hockenmaier, J.; and Tsujii, J., eds., Proceedings of the 2018 Conference on...

    2018

  18. [26]

    Yao, H.; Lou, J.; and Qin, Z. 2024. Poisonprompt: Backdoor attack on prompt-based large language models. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 7745--7749. IEEE

    2024

  19. [27]

    Yao, H.; Shi, H.; Chen, Y.; Jiang, Y.; Wang, C.; and Qin, Z. 2025. ControlNET: A firewall for rag-based LLM system. arXiv preprint arXiv:2504.09593

    2025 arXiv

  20. [28]

    Zhang, Y.; Li, Q.; Du, T.; Zhang, X.; Zhao, X.; Feng, Z.; and Yin, J. 2024. HijackRAG: Hijacking Attacks against Retrieval-Augmented Large Language Models. arXiv preprint arXiv:2410.22832

    2024 arXiv

  21. [29]

    Y.; Chen, S.; Wang, R.; McCall, M.; Titzer, B

    Zhong, P. Y.; Chen, S.; Wang, R.; McCall, M.; Titzer, B. L.; Miller, H.; and Gibbons, P. B. 2025. Rtbas: Defending llm agents against prompt injection and privacy leakage. arXiv preprint arXiv:2502.08966

    2025 arXiv

  22. [30]

    Z.; and Fredrikson, M

    Zou, A.; Wang, Z.; Kolter, J. Z.; and Fredrikson, M. 2024 a . Universal and transferable adversarial attacks on aligned language models, 2023. URL https://arxiv. org/abs/2307.15043, 19

    2024 arXiv

  23. [31]

    Zou, W.; Geng, R.; Wang, B.; and Jia, J. 2024 b . PoisonedRAG: Knowledge poisoning attacks to retrieval-augmented generation of large language models. arXiv preprint arXiv:2402.07867

    2024 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.