Janus is a publicly available playground system and evaluation harness for testing user-involved permission management designs in AI agents, demonstrating benefits of user input and the need for context-sensitive approaches.
AI Agents May Always Fall for Prompt Injections
2 Pith papers cite this work. Polarity classification is still indexing.
abstract
Prompt injection is the most critical vulnerability in deployed AI agents. Despite recent progress, we show that the prevailing defense paradigm (data-instruction separation) both fails to detect attacks that operate through contextual manipulation and degrades contextually appropriate behavior. We then recast prompt injection via the lens of Contextual Integrity (CI), a privacy theory that judges information flow compliance with contextual norms. This explains types of attacks that current defenses attempt to patch and predict advanced ones future agents will face. We develop unique benign and attack scenarios that force an agent to violate the norms by (1) misrepresenting the flow, (2) manipulating norms, or (3) mixing multiple flows. This reframing suggests an impossibility result: an adversary can always construct a context under which a blocked flow appears legitimate, or a defender who tightens norms will block genuinely legitimate flows. Our findings suggest that current research addresses a shrinking fraction of future attack surfaces. Instead, through CI, we offer a principled framework for evaluating context-sensitive failures, and designing CI-aware alignment for the frontier autonomous agents.
fields
cs.AI 2years
2026 2verdicts
UNVERDICTED 2representative citing papers
Formalizes agentic surveillance, releases SurveilBench for testing AI reporting behaviors across corporate, education, and police scenarios, and develops three prompt-injection evasion techniques.
citing papers explorer
-
Janus: a Playground for User-Involved Agentic Permission Management
Janus is a publicly available playground system and evaluation harness for testing user-involved permission management designs in AI agents, demonstrating benefits of user input and the need for context-sensitive approaches.
-
AI Snitches Get Glitches: Towards Evading Agentic Surveillance
Formalizes agentic surveillance, releases SurveilBench for testing AI reporting behaviors across corporate, education, and police scenarios, and develops three prompt-injection evasion techniques.