REVIEW 20 cited by
Dissecting Adversarial Robustness of Multimodal LM Agents
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
As language models (LMs) are used to build autonomous agents in real environments, ensuring their adversarial robustness becomes a critical challenge. Unlike chatbots, agents are compound systems with multiple components taking actions, which existing LMs safety evaluations do not adequately address. To bridge this gap, we manually create 200 targeted adversarial tasks and evaluation scripts in a realistic threat model on top of VisualWebArena, a real environment for web agents. To systematically examine the robustness of agents, we propose the Agent Robustness Evaluation (ARE) framework. ARE views the agent as a graph showing the flow of intermediate outputs between components and decomposes robustness as the flow of adversarial information on the graph. We find that we can successfully break latest agents that use black-box frontier LMs, including those that perform reflection and tree search. With imperceptible perturbations to a single image (less than 5% of total web page pixels), an attacker can hijack these agents to execute targeted adversarial goals with success rates up to 67%. We also use ARE to rigorously evaluate how the robustness changes as new components are added. We find that inference-time compute that typically improves benign performance can open up new vulnerabilities and harm robustness. An attacker can compromise the evaluator used by the reflexion agent and the value function of the tree search agent, which increases the attack success relatively by 15% and 20%. Our data and code for attacks, defenses, and evaluation are at https://github.com/ChenWu98/agent-attack
Forward citations
Cited by 20 Pith papers
-
Agent Security Needs Redefinition through a Holistic Framework
Agent security should be redefined around four contextual authorization properties instead of the content of the action performed.
-
Measuring the Security of Mobile LLM Agents under Adversarial Prompts from Untrusted Third-Party Channels
Indirect prompt injection through ads, webviews, and notifications reliably diverts mobile LLM agents into leaking data and installing malware across eight evaluated agents.
-
Context manipulation attacks : Web agents are susceptible to corrupted memory
Web agents with protected prompts can still be hijacked by injecting malicious steps into their stored task plans, reaching up to 63% success on privacy leaks.
-
Adversarial Attacks on Robotic Vision Language Action Models
Text-based adversarial suffixes can make OpenVLA robot policies elicit chosen target actions with over 90% success on one-hot targets and persist across rollout steps.
-
RiOSWorld: Benchmarking the Risk of Multimodal Computer-Use Agents
Multimodal computer-use agents show risky intent in about 85% of tasks and complete risky actions in about 60%, measured on RiOSWorld, a new 492-task virtual-machine benchmark.
-
BadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimization
A two-stage, objective-decoupled training method embeds visual backdoors into VLA robot policies, achieving near-100% trigger-induced task failure with minimal clean-performance loss in simulation.
-
AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents
AgentVigil uses Monte Carlo Tree Search and coverage-guided scoring to automatically craft indirect prompt injection attacks that succeed 71% and 70% of the time on two agent benchmarks, about double the baseline.
-
Transferable Adversarial Attacks on Black-Box Vision-Language Models
Targeted, barely visible image perturbations transfer from open-source surrogate models to proprietary black-box VLLMs like GPT-4o, Claude, and Gemini, achieving high attack success on captioning, VQA, and receipt tex...
-
Manipulating Multimodal Agents via Cross-Modal Prompt Injection
A coordinated attack that embeds malicious cues in both visual and textual inputs can hijack black-box multimodal agents, outperforming single-modality prompt injection attacks.
-
MELON: Provable Defense Against Indirect Prompt Injection Attacks in AI Agents
MELON detects indirect prompt injection in LLM agents by re-executing each step with the user prompt replaced by a neutral task prompt and flagging attacks when the original and masked runs produce similar tool calls.
-
Adversarial Reasoning at Jailbreaking Time
A loss-guided 'reason, verify, search' loop with three LLM modules surpasses prior semantic jailbreak methods on several defended models.
-
LeakAgent: RL-based Red-teaming Agent for LLM Privacy Leakage
An RL-trained prompt generator outperforms rule-based and fuzzing baselines at extracting system prompts and training data from black-box LLMs.
-
Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Robotics
Adversarial patches optimized against OpenVLA raise manipulation failure rates from about 23% to 100% in LIBERO simulation and disrupt a physical robot arm in 43% of trials.
-
On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Models
A PRISMA-based survey of 85 papers shows agentic LLM security research is attack-heavy and perception-focused, leaving action-layer and code-execution risks understudied.
-
Invisible Injections: Exploiting Vision-Language Models Through Steganographic Prompt Embedding
Steganographic prompt injection is reported to covertly manipulate vision-language models with up to 31.8% success, but the evidence is not reproducible.
-
SAFEFLOW: A Principled Protocol for Trustworthy and Transactional Autonomous Agent Systems
SAFEFLOW wraps LLM/VLM agents in fine-grained information-flow control, verifier-gated trust adjustment, and transactional concurrency, and its authors report near-perfect safety on their own benchmark plus AgentHarm,...
-
A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents
The paper surveys security risks of LLM agents, organizes them into a five-level autonomy taxonomy, and proposes an untested CMDP-based architecture called R2A2.
-
From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystem
A structured survey of recent jailbreak attacks and defenses across LLMs, multimodal LLMs, and agents, with taxonomies for methods, datasets, metrics, and defenses.
-
On the Validity of Traditional Vulnerability Scoring Systems for Adversarial Attacks against LLMs
Applying CVSS, DREAD, OWASP, and SSVC to 56 adversarial LLM attacks via three LLM judges yields near-constant factor scores, which the authors take as evidence that these metrics cannot differentiate LLM attacks.
-
Agentic Web: Weaving the Next Web with AI Agents
A position paper defines the Agentic Web as the next web era and proposes a three-dimensional conceptual framework for understanding and building it.
Discussion (0). Continue with ORCID to comment.