REVIEW 14 cited by
Red Teaming the Mind of the Machine: A Systematic Evaluation of Prompt Injection and Jailbreak Vulnerabilities in LLMs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Red Teaming the Mind of the Machine: A Systematic Evaluation of Prompt Injection and Jailbreak Vulnerabilities in LLMs
read the original abstract
Large Language Models (LLMs) are increasingly integrated into consumer and enterprise applications. Despite their capabilities, they remain susceptible to adversarial attacks such as prompt injection and jailbreaks that override alignment safeguards. This paper provides a systematic investigation of jailbreak strategies against various state-of-the-art LLMs. We categorize over 1,400 adversarial prompts, analyze their success against GPT-4, Claude 2, Mistral 7B, and Vicuna, and examine their generalizability and construction logic. We further propose layered mitigation strategies and recommend a hybrid red-teaming and sandboxing approach for robust LLM security.
Forward citations
Cited by 14 Pith papers
-
OTora: A Unified Red Teaming Framework for Reasoning-Level Denial-of-Service in LLM Agents
OTora provides the first unified framework for reasoning-level denial-of-service attacks on LLM agents, achieving up to 10x more reasoning tokens and order-of-magnitude latency increases while preserving task accuracy...
-
ShareLock: A Stealthy Multi-Tool Threshold Poisoning Attack Against MCP
ShareLock applies Shamir's threshold scheme to distribute poisoning payloads across multiple MCP tool descriptions, achieving information-theoretic secrecy and over 90% average attack success rate in multi-tool scenarios.
-
How Reliable Are AI Attackers Against a Fixed Vulnerable Target? A 400-Run Empirical Study of LLM Penetration Testing Consistency
Empirical study of 400 LLM attack runs finds exploitation success rates of 25-85% across four models against a fixed multi-service honeypot, with model-distinctive failure modes and p<0.001 differences.
-
kNNGuard: Turning LLM Hidden Activations into a Training-Free Configurable Guardrail
Multi-layer Fisher-weighted kNN over frozen-LLM activations, fused with embedding kNN, yields competitive F1 guardrails from a 50-example bank with no fine-tuning and sub-10-second domain adaptation.
-
kNNGuard: Turning LLM Hidden Activations into a Training-Free Configurable Guardrail
kNNGuard classifies prompts using multi-layer kNN on LLM hidden activations from 50 examples, matching or exceeding fine-tuned guardrails in F1 while running 2.7x to 10x faster with no training required.
-
MemAudit: Post-hoc Auditing of Poisoned Agent Memory via Causal Attribution and Structural Anomaly Detection
MemAudit combines counterfactual causal influence scores with memory consistency graphs to identify poisoned records in LLM agent memory, reducing MINJA attack success from 70% to 0% in QA and 83.3% to 0% in reasoning tasks.
-
OTora: A Unified Red Teaming Framework for Reasoning-Level Denial-of-Service in LLM Agents
OTora induces reasoning-level denial-of-service in tool-using LLM agents: it triggers a visit to an attacker-controlled page whose optimized reasoning payload inflates latency by up to 10x while preserving task accuracy.
-
OTora: A Unified Red Teaming Framework for Reasoning-Level Denial-of-Service in LLM Agents
OTora is a two-stage framework that generates insertion-aware adversarial triggers and ICL-guided genetic payloads to induce reasoning-level denial-of-service in tool-augmented LLM agents across multiple backbones whi...
-
An AI Agent Execution Environment to Safeguard User Data
GAAP guarantees confidentiality of private user data for AI agents by enforcing user-specified permissions deterministically through persistent information flow tracking, without trusting the agent or requiring attack...
-
ANNIE: Be Careful of Your Robots
The authors build a safety-centered benchmark and attack method that induces vision-language-action robot policies to violate ISO-based safety rules in a majority of tested episodes.
-
Prompt Injection Detection is Regime-Dependent: A Deployment-Aware Evaluation with Interpretable Structural Signals
Prompt injection detection performance is highly regime-dependent with no single detector dominating across settings; transformer models perform best overall while structural signals offer modest gains in some regimes.
-
Automated Framework to Evaluate and Harden LLM System Instructions against Encoding Attacks
Refusal-aligned LLMs leak system instructions under encoding/serialization prompts at high rates, and one-shot CoT instruction reshaping substantially reduces that leakage without retraining.
-
Whispers of Wealth: Red-Teaming Google's Agent Payments Protocol via Prompt Injection
Red-teaming of the Agent Payments Protocol reveals vulnerabilities to direct and indirect prompt injection, with Branded Whisper and Vault Whisper attacks enabling product ranking manipulation and sensitive data extraction.
-
Beyond Context: Large Language Models' Failure to Grasp Users' Intent
LLMs fail to detect hidden harmful intent, allowing systematic bypass of safety mechanisms through framing techniques, with reasoning modes often worsening the issue.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.