REVIEW 5 cited by
ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Safety is critical to the usage of large language models (LLMs). Multiple techniques such as data filtering and supervised fine-tuning have been developed to strengthen LLM safety. However, currently known techniques presume that corpora used for safety alignment of LLMs are solely interpreted by semantics. This assumption, however, does not hold in real-world applications, which leads to severe vulnerabilities in LLMs. For example, users of forums often use ASCII art, a form of text-based art, to convey image information. In this paper, we propose a novel ASCII art-based jailbreak attack and introduce a comprehensive benchmark Vision-in-Text Challenge (ViTC) to evaluate the capabilities of LLMs in recognizing prompts that cannot be solely interpreted by semantics. We show that five SOTA LLMs (GPT-3.5, GPT-4, Gemini, Claude, and Llama2) struggle to recognize prompts provided in the form of ASCII art. Based on this observation, we develop the jailbreak attack ArtPrompt, which leverages the poor performance of LLMs in recognizing ASCII art to bypass safety measures and elicit undesired behaviors from LLMs. ArtPrompt only requires black-box access to the victim LLMs, making it a practical attack. We evaluate ArtPrompt on five SOTA LLMs, and show that ArtPrompt can effectively and efficiently induce undesired behaviors from all five LLMs. Our code is available at https://github.com/uw-nsl/ArtPrompt.
Forward citations
Cited by 5 Pith papers
-
Securing Multimodal AI through Internal Information Decomposition
A one-class detector using first-token distributional consistency between text-only, vision-only, and joint predictions reduces multimodal jailbreak attack success rates to below 15% with ~2.4% utility loss.
-
MetaBreak: Jailbreaking Online LLM Services via Special Token Manipulation
Special tokens that structure LLM conversations can be injected and swapped for lookalike words to bypass both built-in safety and external content filters.
-
Paper Summary Attack: Jailbreaking LLMs through LLM Safety Papers
Summaries of LLM safety papers, paired with a completion-style payload containing a harmful query, jailbreak aligned LLMs at high reported success rates and expose a defense paper versus attack paper bias.
-
Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation
Sparse autoencoder activation perturbation (SFPF) applied on top of existing jailbreak prompts raises attack success rate on Qwen3-32B, but with no defense evaluation and weak reproducibility.
-
Trojan Horse Prompting: Jailbreaking Conversational Multimodal Models by Forging Assistant Message
Trojan Horse Prompting injects malicious instructions into a fabricated assistant message in the API chat history, aiming to bypass Gemini's safety filters, but no quantitative evidence is provided.
Discussion (0). Sign in to comment.