Roughly 1% of real resumes contain hidden prompt injections against LLM screeners, prevalence has risen over 1-2 years, and over 90% avoid explicit instructions.
hub Canonical reference
Prompt flow integrity to prevent privilege escalation in LLM agents
Canonical reference. 80% of citing Pith papers cite this work as background.
21
Pith papers citing it
Background
80% of classified citations
abstract
Large Language Models (LLMs) are combined with tools to create powerful LLM agents that provide a wide range of services. Unlike traditional software, LLM agent's behavior is determined at runtime by natural language prompts from either user or tool's data. This flexibility enables a new computing paradigm with unlimited capabilities and programmability, but also introduces new security risks, vulnerable to privilege escalation attacks. Moreover, user prompts are prone to be interpreted in an insecure way by LLM agents, creating non-deterministic behaviors that can be exploited by attackers. To address these security risks, we propose Prompt Flow Integrity (PFI), a system security-oriented solution to prevent privilege escalation in LLM agents. Analyzing the architectural characteristics of LLM agents, PFI features three mitigation techniques -- i.e., agent isolation, secure untrusted data processing, and privilege escalation guardrails. Our evaluation result shows that PFI effectively mitigates privilege escalation attacks while successfully preserving the utility of LLM agents.
hub tools
citation-role summary
background 4
method 1
citation-polarity summary
representative citing papers
TRUSTDESC prevents tool poisoning in LLM applications by automatically generating accurate tool descriptions from code via a three-stage pipeline of reachability analysis, description synthesis, and dynamic verification.
AutoDojo adaptively optimizes IPI attacks to bypass defenses, recovering substantial ASR on action-open tasks where static attacks fail.
The paper introduces Consent Integrity as the property that actions shown for approval must be rendered by a trusted mediator from the real boundary action over an unspoofable path and bound to execution, with uninspectable actions surfaced rather than silently approved.
CXI enforces that LLM agent tool calls execute only when protected sink fields, sink-interpreted effects, and the invocation event each carry authority bound to the same canonical action manifest.
SeClaw provides spec-driven synthesis of security tasks and an execution-based docker testbed for evaluating unsafe behaviors in autonomous LLM agents.
New benchmark Scammer4U finds 54-93% critical PII leakage from frontier web agents on scam sites versus 0% on benign twins, plus a 30-point gap between verbalized suspicion and actual submission.
AuthGraph aligns an execution provenance graph with a clean authorization graph to detect parameter-source deviations from user intent, reducing attack success rates to 1-2% on AgentDojo and AgentDyn while retaining most task utility.
BIV audits AI agent skills at scale, finding 80% deviate from declared behavior on 49,943 skills and achieving 0.946 F1 for malicious skill detection.
SkillScope detects over-privileged LLM agent skills with 94.53% F1 score via graph analysis and replay validation, finding 7,039 problematic skills in the wild and reducing violations by 88.56% while preserving task completion.
SSL representation disentangles skill scheduling, structure, and logic using an LLM normalizer, improving skill discovery MRR@50 from 0.649 to 0.729 and risk assessment macro F1 from 0.409 to 0.509 over text baselines.
Symbolic guardrails enforce about 74% of agent security and safety requirements on three benchmarks with mostly simple checks, improving safety without sacrificing utility.
An independent reproduction on AgentDojo with Qwen2.5-7B finds that the Progent out-of-band defense reduces mean attack success from 25.8% to 4.2% and holds against a hand-crafted adaptive attack at 2.6%.
PORTICO is a revocable capability reference monitor for coding agents that enforces task contracts via grant-invoke-closure lifecycles and rejects post-closure reuses while preserving task success.
ChainCaps uses monotonic capability attenuation via intersection of sink-specific budgets in a transparent proxy to reduce attack success on composed tool-using agents from 25-68% to 0-4.8% while keeping 96-100% benign task completion.
LLM agent security is reframed as an agent-human interaction issue, supported by a survey showing industry preference for human-centric mechanisms over academic favorites and proposing a new research agenda.
PIArena provides a unified evaluation platform for prompt injection attacks and defenses, featuring a new adaptive attack that reveals major weaknesses in existing protections.
Black-box optimization outperforms gradient-based methods for prompt injection on LLM agents, with success depending on attacker model strength and limited transfer from small to frontier models.
LLM agents enable universal interoperability by serving as automatic translators and adapters between proprietary digital services.
A synthesis of 247 papers on LLM agent security identifies prompt injection and tool hijacking as dominant threats, notes weakly compositional defenses, and argues for trust boundaries and realistic evaluations.
The Redpanda Agentic Data Plane uses out-of-band metadata channels to enforce data scoping, action constraints, and tamper-proof auditing on autonomous AI agents.
citing papers explorer
-
Don't Make Models Guess Security and Safety: Symbolic Guardrails for Domain-Specific AI Agents
Symbolic guardrails enforce about 74% of agent security and safety requirements on three benchmarks with mostly simple checks, improving safety without sacrificing utility.