Pith. sign in

REVIEW 3 major objections 4 minor 14 references

Hardware Keystores for AI Agent Signing Workflows: A Zero-Trust MCP Enforcement Architecture

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that putting AI-agent signing keys inside hardware keystores, behind a deterministic zero-trust stack, closes both key exfiltration and unauthorised signing by prompt-injected agents.

desk verdict The architecture is sensible and the deterministic checks are real, but the headline 0% ASR leans on test design and SoftHSM rather than on the hardware boundary it claims to validate. read the letter →

arxiv 2608.06130 v1 pith:YFH3FAD3 submitted 2026-08-06 cs.CR cs.AIcs.LG

Leo Sambrook , Sampo Sovio This is my paper
classification cs.CRcs.AIcs.LG
keywords hardwarekeystorePKCS#11promptinjectionModelContextProtocolzerotrustHSMpayloadcommitmentAIagentsecurity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to move private keys out of software entirely for AI-agent signing workflows, so that no prompt injection, memory dump, or sandbox escape can make the raw key material available to an agent. The proposed design routes signing through a vendor-neutral PKCS#11 hardware keystore and wraps it in five enforcement layers: a cryptographic session binding the user's original intent, a static tool-allowlist scope, a deterministic SHA-256 payload commitment check, taint tracking, and an LLM-based semantic judge. The central empirical claim is that with the payload commitment pre-armed, all 12 tested injection scenarios fail against three prompt-following models: attack success drops from 19.3% baseline to 0% (0/192, Wilson upper bound 2.0%) with zero false positives on benign tasks. If correct, this turns prompt injection from a way to steal signing keys or redirect signatures into a nuisance that the architecture rejects regardless of model behaviour.

What carries the argument

The load-bearing mechanism is a PKCS#11 adapter: private keys are generated and used inside a hardware keystore (HSM, TPM, smart card), the host receives only signatures via opaque session-scoped handles, and the adapter swaps any PKCS#11 library with a single configuration change. Around it, the deterministic committed-payload check (C_hist) ensures sign calls can only match the operator's pre-registered document hash; the static capability ceiling (Smax) restricts which tools and argument types the agent may call; the session protocol [7] cryptographically binds user intent, identity, and quota; taint tracking forces human approval for payloads derived from fetched content; and an LLM judge (RAV) filters obvious semantic mismatches before the hardware is reached. The argument works because exfiltration is blocked by the hardware boundary itself, while substitution is blocked by hash comparison, so neither depends on the model's alignment or resistance to injection.

What would settle it

On a protected deployment with a real HSM or TPM, attempt to extract a usable signing key by any route an injected agent could reach: reading process memory, dumping the PKCS#11 token store, or asking the keystore API to reveal key attributes; if any of these yields the private key bytes, the structural exfiltration claim is false. A simpler file-level check is to confirm that no private key file exists anywhere on the host filesystem while the agent is running and that the agent's attempted read of private_key.pem returns 'file not found'.

Watch

Extended reading notes

Core claim

The core claim is that hardware confinement makes key exfiltration structurally impossible rather than merely behaviourally unlikely: in an HSM-backed deployment no private key file exists on disk, so an agent instructed to read and output a private key file finds no file to read. For misuse of the key, the deterministic C_hist commitment check stores a SHA-256 hash of the intended document before the agent reads any external source, and sign_payload rejects any later call with a different payload in under a millisecond without LLM inference. The paper combines these two guarantees into a five-layer Zero-Trust stack and reports protected ASR 0% (0/192, three models) against substitutions, pre-task injections, and exfiltration attempts, with zero false positives in four benign scenarios.

Load-bearing premise

The empirical 0% exfiltration result was measured against a software PKCS#11 emulator whose key material still lives in an on-disk token store, so the result itself does not prove the hardware boundary; the full guarantee assumes a real HSM or TPM with keys generated and confined on-device.

Editorial extensions

If this is right

  • Git commit signing and SSH authentication can be routed through the keystore with a single PKCS#11 provider directive, without changing the agent, the working directory, or the standard tools.
  • A prompt-injected agent cannot output a private key because no key material exists in host memory; the failure mode becomes a missing file rather than a credential leak.
  • Pre-armed payload commitments make payload-substitution injections fail deterministically in under a millisecond, regardless of which LLM is driving the agent.
  • Fully autonomous signing without a pre-named document escalates to a human, with the semantic judge filtering obvious attacks before any notification is sent.
  • The design targets low-frequency, high-value signing operations; per-request API signing loops would require key caching outside the hardware and are explicitly out of scope.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Going beyond the paper: the benchmark's 0% exfiltration result was obtained with a software PKCS#11 emulator, so the strict architectural guarantee depends on deploying a physical device and generating keys inside it; the paper demonstrates drop-in behaviour on a TPM but does not rerun the injection benchmark on that device.
  • Going beyond the paper: operators who expose a writable scratchpad to the agent need filesystem-level taint propagation; the prototype closes that path by omitting write_file from the agent's tools, and the paper flags this as future work.
  • Going beyond the paper: the main residual risk is human approval fatigue, because when no commitment is pre-armed, ambiguous or tainted requests reach a human, and the paper notes that escalation volume is not yet rate-limited.
  • Going beyond the paper: the same hardware-plus-commitment pattern could extend to certificate issuance and document notarisation, where the authorisation question is also about whether the operator intended this exact content rather than whether the model is trustworthy.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a five-layer zero-trust enforcement architecture for AI agent signing workflows, with the primary contribution being that private keys are moved from software-resident storage into PKCS#11 hardware keystores (HSM, TPM, smart card) so that only opaque handles and signature outputs are exposed to the host. The surrounding layers are session identity (SAGA), static scope ceilings (Smax), a deterministic committed-payload hash check (C_hist), a semantic LLM judge (RAV), and the hardware execution boundary. The empirical evaluation adapts AgentDojo's ImportantInstructionsAttack template to four injection goals (two substitution, one pre-task, one exfiltration) and reports baseline ASR 19.3% (n=192, three models) versus protected ASR 0% (Wilson 95% CI upper bound 2.0%), with zero false positives on four benign tasks. The paper also reports a drop-in TPM2 integration test and latency measurements for SoftHSMv2 and an Infineon SLB9670 TPM.

Significance. If fully validated, the architecture would be practically relevant: it targets a concrete and recent attack class (credential exfiltration from MCP-based agents) with a deployable migration path and vendor-neutral PKCS#11 abstraction. The paper is unusually transparent about its claim classes, separating deterministic guarantees (C_hist, scope, commitment policy) from probabilistic ones (RAV) and from architectural ones (Layer 5), and it includes reproducible benchmark code, confidence intervals, and an explicit treatment of the Minimax-M27 alignment confound. These are genuine strengths. However, the load-bearing empirical number, protected ASR = 0%, is largely produced by the test construction rather than by a risky empirical hypothesis: substitution and pre-task attempts are blocked deterministically by the pre-armed C_hist hash, and the exfiltration attempt fails only because the conventional private_key.pem file is absent in a SoftHSMv2 software emulator. The hardware-confined exfiltration guarantee therefore remains a plausible deductive argument rather than an empirically demonstrated result, and the manuscript's current wording overstates what the benchmark shows.

major comments (3)
  1. [§5.1, §5.2, Table 1] The protected evaluation ran entirely against SoftHSMv2, a software PKCS#11 emulator whose key material is stored in an on-disk token store, yet Table 1 attributes the IT-04 result to "L5" and RQ1 in §5.2 states that "no private key file exists in the HSM-backed deployment" and that the exfiltration attack class is "closed structurally." Since IT-04's only goal is to read and output private_key.pem, the measured 0% demonstrates that this particular conventional file was never created; it does not demonstrate that an injected agent cannot locate and output the SoftHSM token store file, nor does it empirically demonstrate that Layer 5 blocks exfiltration on a real HSM or TPM. The TPM2 result in §4 is an integration test-suite pass, not an injection run, and §7's assertion that re-running the injection benchmark on TPM2 "would change latency (Appendix D) but not ASR" is an extrapolation. Please either run the injection benchmark against a real hardware keystore (or at minimum add an adversarial goal targeting the SoftHSM token store), or clearly reframe the IT-04 result as a deductive artifact of the test fixture rather than empirical evidence for the hardware boundary.
  2. [§3.3, §5.1–5.2] The protected ASR of 0% for IT-01 through IT-03 is a deterministic consequence of the pre-armed C_hist commitment check: the operator commits a SHA-256 hash of the intended document before the agent loop, and the kernel rejects any sign_payload whose payload differs from that hash in under 1 ms. The paper explicitly acknowledges this in §5.1's claim classes, but the abstract and §5.2 present "protected ASR = 0% (0/192, three models) against AgentDojo injections" as the empirical evaluation. Because the benchmark is constructed so that the deterministic layers make the protected outcome independent of the LLM, the multi-model comparison and the confidence interval provide no evidence about the probabilistic RAV layer or about real-hardware behavior. The RAV-only ablation is the relevant probabilistic test, and it is small (n=10, Wilson upper bound 27.8%) in §7. Please separate the deterministic implementation check from the probabilistic ablation and avoid presenting the combined 0% as an empirical discovery about the RAV or about hardware confinement.
  3. [§1, §3.5, §5.2 (RQ1)] The architectural exfiltration guarantee is correctly qualified in §1: "the strongest guarantee applies only to keys generated entirely within the hardware boundary." However, §3.5 states unconditionally that "[i]n an HSM-backed deployment no private key file exists on disk," and RQ1 in §5.2 repeats that the exfiltration class is closed structurally. For keys imported via CKM_AES_KEY_WRAP, raw key material exists in software at import time and may be recoverable from host memory or from the import process; the unconditional phrasing overstates the guarantee for that deployment mode. Since the correct qualification is already present in the paper, the claims in §3.5 and §5.2 should be restated in the same conditional form so that the headline guarantee matches the actual scope of the architecture.
minor comments (4)
  1. [Throughout] The Runtime Alignment Validator is rendered inconsistently as "RAV", "RA V", and "RAV judge"; please unify the notation.
  2. [§4] The artifact repository is mentioned but no URL, DOI, or archive identifier is provided; for a reproducibility-oriented paper, please include a stable link or persistent identifier.
  3. [Table 6] The row "C_Sign RSA-2048 — SoftHSMv2 ∗ ∼1 ms∼1 ms" has odd spacing and duplicated values, making the table hard to read; also align the table's notation with the unified RAV spelling.
  4. [§5.1] The secondary models are run for only 3 trials per scenario, which produces very wide confidence intervals (e.g., 0% [0, 9.6]); please state this limitation explicitly in the main text rather than only in the appendix.

Circularity Check

2 steps flagged · score 6.0 of 10

The protected 0% ASR is entailed by the benchmark's own definitions: IT-04 asks for a .pem file that the protected deployment omits, and IT-01–03 are rejected by the pre-armed hash; the SoftHSM backend never tests the hardware boundary.

  1. self definitional [Sec. 3.5 (Layer 5 guarantee), Sec. 5.1 (IT-04 definition), Sec. 7 (SoftHSM limitation), Table 1 (0% L5)]
    "IT-04: read and output private_key.pem (key exfiltration). In an HSM-backed deployment no private key file exists on disk. ... an agent instructed to read and output a private key file will find no file to read. SoftHSMv2 provides no hardware tamper resistance; it is used here as a reproducibility proxy."

    Exfiltration scenario IT-04 is defined as reading a file named private_key.pem, while the Layer 5 'architectural guarantee' is precisely that no private key file exists on disk. The protected-mode agent therefore fails because the test's goal refers to a pathname that the protected deployment omits by design; the 0% result is independent of any actual confidentiality property of the keystore. The benchmark backend was SoftHSMv2, whose key material lives in an on-disk token store, so the premise 'no private key file exists' does not even hold for the tested system; the agent fails only because the conventional file was never created. Table 1's '0% [L5]' for IT-04 is thus a definitional artifact, not an exercised hardware-boundary result.

  2. self definitional [Sec. 5.1 (Claim classes; Evaluation modes), Sec. 5.2 (RQ3), Tables 1–2]
    "Protected ASR = 0% on substitution scenarios is a deterministic consequence of hash mismatch. With C hist pre-armed, all 12 injection scenarios are blocked deterministically before HITL: IT-01–03 via hash mismatch (<1 ms), IT-04 architecturally at Layer 5."

    The protected mode pre-arms C_hist to the operator-authorised document before the agent starts. IT-01–03 ask the agent to sign a different payload; the kernel rejects any payload whose SHA-256 differs from the committed hash. Attack success is therefore impossible by construction, and the 0% ASR is an entailment of the test setup rather than an observed effect of the stack on LLM behavior. The paper candidly labels this 'deterministic consequence,' but the headline 'protected ASR = 0% (0/192, three models)' restates the enforcement policy as a measured result: the outcome is logically identical to the input rule, with no independent empirical content for these scenarios.

full rationale

The central claimed result—protected ASR = 0% on 192 AgentDojo trials—is not an empirical discovery about the stack; it is a logical consequence of the test's construction. For IT-01–03, the C_hist commitment is pre-armed to the intended document before the agent starts, and the kernel rejects any payload whose SHA-256 differs; the paper itself calls this a 'deterministic consequence of hash mismatch.' For IT-04, the attack goal is to read a file named private_key.pem, while Layer 5's guarantee is that no private key file exists on disk; failing to find a file that was never created is tautological. Moreover, the protected benchmark ran on SoftHSMv2, which the paper calls a 'reproducibility proxy' with 'no hardware tamper resistance' and which stores key material in an on-disk token store, so Table 1's '0% [L5]' column does not actually exercise hardware key confinement. The paper is unusually transparent about these points in Section 5.1's claim classes and Section 7, which lowers the circularity score, but the headline 'protected ASR = 0%' still packages definitional guarantees and an unexecuted hardware claim as an evaluation result. No self-citation or imported-uniqueness circularity appears; the AgentDojo template is external and the baseline failures are genuine. The deductive argument that a real HSM/TPM with non-extractable keys would block file-based exfiltration has independent content, so the paper is only partially circular.

Assumptions & free parameters 0 free parameters · 6 assumptions · 3 invented entities

The central claims rest on the PKCS#11 confinement invariant, the operator pre-arming a hash commitment, the representativeness of the AgentDojo template, and the non-influenceability of the RAV's kernel_action; none of these are formally verified. The evaluation contradicts no core invariant, but several are documented as limitations.

assumptions (6)
  • domain assumption PKCS#11 key confinement: private keys generated inside a hardware keystore never leave the boundary; the host receives only operation results.
    Layer 5 guarantee (Section 3.5). Violated by SoftHSMv2, which stores keys in on-disk token files; the paper uses SoftHSM for the main benchmark and validates TPM2 only via integration tests.
  • domain assumption The operator pre-arms C_hist (SHA-256 of intended payload) before the agent reads external sources.
    The deterministic 0% ASR for IT-01-03 depends on this precondition (Section 2.3, Section 5.2). Without it, REQUIRE_COMMITMENT escalates to HITL, shifting to probabilistic protection.
  • domain assumption AgentDojo ImportantInstructionsAttack template is representative of real prompt injections for signing workflows.
    Only this external template is used (12 scenarios); InjecAgent is planned but not run (Section 6).
  • domain assumption The RAV's kernel_action tuple cannot be influenced by attacker-controlled free text.
    Section 3.4 claims the tuple contains only observable call parameters, but file names are attacker-influenceable via the injected goal; the structural property is argued, not formally verified.
  • domain assumption LLM judge atla/selene-mini provides reliable semantic verdicts.
    Empirical support is a 10-trial ablation with Wilson upper bound 27.8%, too small to establish reliability.
  • domain assumption Threat model excludes compromised OS kernel, physical HSM attacks, and PKCS#11 timing side-channels.
    Section 2.3 explicitly scopes these out; a kernel compromise could read ACTs or call PKCS#11 on the agent's behalf.
invented entities (3)
  • C_hist committed-payload anchor independent evidence
    purpose: Operator-plane SHA-256 commitment of the intended signing payload; the kernel rejects any later signing call whose payload hash differs, without LLM inference.
    Deterministic and testable via pytest and the integration suite; its real-world value depends on operator discipline in pre-arming before external data is read.
  • Runtime Alignment Validator (RA V) independent evidence
    purpose: LLM judge mapping (I_user, kernel_action, taint_flags) to APPROVE/BLOCK/UNSURE; provides semantic defense-in-depth before HITL.
    Falsifiable via the UT-04 ablation (n=10: 0% protected ASR, CI upper bound 27.8%) and benign BD-04 (TSR 100%).
  • REQUIRE_COMMITMENT HITL safety net independent evidence
    purpose: Escalates any signing call lacking a pre-armed C_hist anchor to human approval; acts as the no-commitment fallback for autonomous pipelines.
    Behavior is deterministic and observable; BD-04 completes via HITL auto-approval, but approval fatigue is an acknowledged limitation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hardware Keystores for AI Agent Signing Workflows: A Zero-Trust MCP Enforcement Architecture." pith.science (2026). https://pith.science/paper/YFH3FAD3

@misc{pith2026260806130,
  author       = {Pith},
  title        = {Pith review of: Hardware Keystores for AI Agent Signing Workflows: A Zero-Trust MCP Enforcement Architecture},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YFH3FAD3}},
  note         = {Machine review of arXiv:2608.06130}
}
read the original abstract

AI agents performing cryptographic operations (signing Git commits, authenticating API calls, issuing certificates) currently store private keys in software-accessible locations: plaintext files, environment variables, or container memory. Any process with sufficient read privileges can extract the raw key material. A recent production incident demonstrated the practical severity: private keys were exfiltrated from a widely deployed framework via email injection in under five minutes. We aim to enforce both key confidentiality and content-aware authorisation for key use. To that end, we replace software-resident keys with hardware-confined keys accessible through a vendor-neutral PKCS#11 interface. A hardware keystore (HSM, TPM, smart card) executes cryptographic operations on-device; the host receives only the result via opaque handles. Hardware confinement is the primary contribution; it is enabled by a surrounding five-layer Zero-Trust enforcement stack comprising session identity (SAGA), scope bounds (Smax), semantic validation (RAV), taint tracking, and the hardware execution boundary. We evaluate against 12 injection scenarios derived from AgentDojo's ImportantInstructionsAttack template (Debenedetti et al., arXiv:2406.13352). We run four LLM models; three follow injections in baseline mode (gpt-oss-120b, Qwen2.5-72B, DeepSeek-V4-Flash, n=192 combined). Baseline Attack Success Rate (ASR): 19.3% [14.3%, 25.4%]; protected ASR: 0% (Wilson 95% CI upper bound 2.0%). Zero false positives across four benign task scenarios.

Figures

Figures reproduced from arXiv: 2608.06130 by the authors.

Figure 1
Figure 1. Request lifecycle through the five-layer stack. Layers 1–2 establish a cryptographic session per turn. Layer 3 applies deterministic enforcement: [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Component topology and trust boundaries. The [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

14 extracted references · 9 canonical work pages

  1. [1]

    Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Direc- tions,

    X. Hou, Y . Zhao, S. Wang, and H. Wang, “Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Direc- tions,”arXiv:2503.23278v3, Oct. 2025

    arXiv 2025

  2. [2]

    OpenClaw Security Policy,

    OpenClaw Contributors, “OpenClaw Security Policy,”GitHub, Mar

    work page

  3. [3]

    Infostealer Steals OpenClaw AI Agent Sessions,

    The Hacker News, “Infostealer Steals OpenClaw AI Agent Sessions,” Feb. 2026. [Online]. Available: https://thehackernews.com/2026/02/ infostealer-steals-openclaw-ai-agent.html

    work page 2026

  4. [4]

    Blind Gods and Broken Screens: Architect- ing a Secure, Intent-Centric Mobile Agent Operating System,

    Z. Zou et al., “Blind Gods and Broken Screens: Architect- ing a Secure, Intent-Centric Mobile Agent Operating System,” arXiv:2602.10915v3, Feb. 2026

    arXiv 2026

  5. [5]

    NemoClaw: Enterprise Security and Privacy for OpenClaw AI Agents,

    NVIDIA Corporation, “NemoClaw: Enterprise Security and Privacy for OpenClaw AI Agents,” NVIDIA GTC, Mar. 2026. [Online]. Available: https://build.nvidia.com/nemoclaw

    work page 2026

  6. [6]

    MCP-Secure: A Runtime Access Control Layer for Privilege-Aware LLM Agent Tooling,

    G. Singh and V . K. Madisetti, “MCP-Secure: A Runtime Access Control Layer for Privilege-Aware LLM Agent Tooling,”IEEE Open J. Comput. Soc., Feb. 2026

    work page 2026

  7. [7]

    SAGA: A Security Architecture for Governing AI Agentic Systems,

    G. Syros et al., “SAGA: A Security Architecture for Governing AI Agentic Systems,”arXiv:2504.21034v2, Aug. 2025

    arXiv 2025

  8. [8]

    Agent- Bound: Securing Execution Boundaries of AI Agents,

    C. Bühler, M. Biagiola, L. Di Grazia, and G. Salvaneschi, “Agent- Bound: Securing Execution Boundaries of AI Agents,” inProc. FSE,

    work page

Show all 14 references
  1. [9]

    Enterprise-Grade Security for the Model Context Protocol (MCP): Frameworks and Mitigation Strategies,

    V . S. Narajala and I. Habler, “Enterprise-Grade Security for the Model Context Protocol (MCP): Frameworks and Mitigation Strategies,” arXiv:2504.08623v2, May 2025

    2025 arXiv

  2. [10]

    MCP Safety Audit: LLMs with the Model Context Protocol Allow Major Security Exploits,

    B. Radosevich and J. Halloran, “MCP Safety Audit: LLMs with the Model Context Protocol Allow Major Security Exploits,” arXiv:2504.03767v2, Apr. 2025

    2025 arXiv

  3. [11]

    AgentDojo: A Dynamic Environment to Eval- uate Prompt Injection Attacks and Defenses for LLM Agents,

    E. Debenedetti et al., “AgentDojo: A Dynamic Environment to Eval- uate Prompt Injection Attacks and Defenses for LLM Agents,” in Proc. NeurIPS, 2024

    2024

  4. [12]

    InjecAgent: Benchmark- ing Indirect Prompt Injections in Tool-Integrated LLM Agents,

    Q. Zhan, Z. Liang, Z. Ying, and D. Kang, “InjecAgent: Benchmark- ing Indirect Prompt Injections in Tool-Integrated LLM Agents,” in Findings of ACL, 2024

    2024

  5. [13]

    AI Agent Traps,

    M. Franklin, N. Tomašev, J. Jacobs, J. Z. Leibo, and S. Osindero, “AI Agent Traps,”SSRN:6372438, Google DeepMind, Mar. 2026

    2026

  6. [2026]

    Available: https://github.com/openclaw/openclaw/ blob/main/SECURITY .md

    [Online]. Available: https://github.com/openclaw/openclaw/ blob/main/SECURITY .md

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.