Pith Number

pith:C43N4527

pith:2026:C43N4527I7EBNA4U3P6FEMSGNT

not attested not anchored not stored refs pending

Content-Aware Attack Detection in LLM Agent Tool-Call Traffic: An Empirical Study of Features, Architectures, and Evaluation Protocols

Sultan Zavrak

MCPShield shows that content embeddings of tool arguments and responses are essential for detecting attacks on LLM agent traffic, lifting AUROC from 0.64 to above 0.89.

arxiv:2605.11053 v3 · 2026-05-11 · cs.CR · cs.AI · cs.LG

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{C43N4527I7EBNA4U3P6FEMSGNT}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

Content-level features are essential: metadata-only detection plateaus around an AUROC of 0.64 regardless of architecture, while content embeddings push the AUROC above 0.89.

C2weakest assumption

The attack examples in RAS-Eval and ATBench are representative of real-world threats and that SBERT embeddings capture generalizable attack signals rather than dataset-specific artifacts.

C3one line summary

MCPShield detects attacks on LLM agent tool-call traffic by encoding sessions as graphs enriched with SBERT content embeddings, achieving AUROC above 0.89 with content features versus 0.64 for metadata alone.

Formal links

1 machine-checked theorem link

Receipt and verification

First computed	2026-05-25T02:01:22.999408Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

1736de775f47c8168394dbfc5232466cc48c2b225a6cac2e82cdcf2700ae03c2

Aliases

arxiv: 2605.11053 · arxiv_version: 2605.11053v3 · doi: 10.48550/arxiv.2605.11053 · pith_short_12: C43N4527I7EB · pith_short_16: C43N4527I7EBNA4U · pith_short_8: C43N4527

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/C43N4527I7EBNA4U3P6FEMSGNT \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 1736de775f47c8168394dbfc5232466cc48c2b225a6cac2e82cdcf2700ae03c2

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "93766ad117e27044b395a3531afd639c203eda23c05c7c1bc349e4308001ddeb",
    "cross_cats_sorted": [
      "cs.AI",
      "cs.LG"
    ],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "cs.CR",
    "submitted_at": "2026-05-11T14:55:48Z",
    "title_canon_sha256": "56536732f9de1996a11d36b08864f8e9afceabd57fc31b3cd927a753d8126632"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2605.11053",
    "kind": "arxiv",
    "version": 3
  }
}