Pith. sign in

REVIEW 37 cited by

GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2308.06463 v2 pith:Z5Y4SNF4 submitted 2023-08-12 cs.CL

Youliang Yuan , Wenxiang Jiao , Wenxuan Wang , Jen-tse Huang , Pinjia He , Shuming Shi , Zhaopeng Tu This is my paper · ORCID
classification cs.CL
keywords llmssafetyalignmentciphercipherchatciphershumanchat
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Safety lies at the core of the development of Large Language Models (LLMs). There is ample work on aligning LLMs with human ethics and preferences, including data filtering in pretraining, supervised fine-tuning, reinforcement learning from human feedback, and red teaming, etc. In this study, we discover that chat in cipher can bypass the safety alignment techniques of LLMs, which are mainly conducted in natural languages. We propose a novel framework CipherChat to systematically examine the generalizability of safety alignment to non-natural languages -- ciphers. CipherChat enables humans to chat with LLMs through cipher prompts topped with system role descriptions and few-shot enciphered demonstrations. We use CipherChat to assess state-of-the-art LLMs, including ChatGPT and GPT-4 for different representative human ciphers across 11 safety domains in both English and Chinese. Experimental results show that certain ciphers succeed almost 100% of the time to bypass the safety alignment of GPT-4 in several safety domains, demonstrating the necessity of developing safety alignment for non-natural languages. Notably, we identify that LLMs seem to have a ''secret cipher'', and propose a novel SelfCipher that uses only role play and several demonstrations in natural language to evoke this capability. SelfCipher surprisingly outperforms existing human ciphers in almost all cases. Our code and data will be released at https://github.com/RobustNLP/CipherChat.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 37 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. On Surjectivity of Neural Networks: Can you elicit any behavior from your model?

    cs.LG 2025-08 conditional novelty 7.0 of 10

    Pre-LayerNorm transformers and linear attention are almost always surjective, so any target output has an input that produces it in the continuous embedding space.

  2. Benign-to-Toxic Jailbreaking: Inducing Harmful Responses from Harmless Prompts

    cs.CV 2025-05 conditional novelty 7.0 of 10

    Adversarial images optimized to map harmless text prefixes to toxic tokens jailbreak vision-language models more effectively than continuing toxic text.

  3. Jailbreaking to Jailbreak

    cs.CL 2025-02 conditional novelty 7.0 of 10

    A transferable multi-turn jailbreak turns refusal-trained black-box LLMs into willing automated jailbreakers, with high attack success against other models and against themselves.

  4. MJ: Multi-turn LLM Jailbreaking via Decomposed Credit Assignment

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Turn-level group-relative credit assignment (immediate plus future return) trains multi-turn jailbreak attackers that reach about 98% ASR5@3, beating prior multi-turn methods by roughly 11 points.

  5. Mitigating Taint-Style Vulnerabilities in MCP Servers via Security-Aware Tool Descriptions

    cs.CR 2026-07 conditional novelty 6.0 of 10

    SPELLSMITH mitigates taint-style vulnerabilities in MCP servers by augmenting tool descriptions with security constraints and adding LLM self-reflection before tool invocation, reducing attack success rates to near zero.

  6. A Systematic Investigation of RL-Jailbreaking in LLMs

    cs.LG 2026-05 unverdicted novelty 6.0 of 10

    Systematic investigation reveals that dense rewards and extended episode lengths primarily drive the success of RL jailbreaking in LLMs.

  7. LCO: LLM-based Constraint Optimization for Safer Agentic LLMs in Real-world Tasks

    cs.CL 2026-04 conditional novelty 6.0 of 10

    LCO combines proactive LLM-generated safety constraints with evolutionary sampling to cut in-context reward hacking while preserving task performance on tweet and tool-use benchmarks.

  8. When Smiley Turns Hostile: Interpreting How Emojis Trigger LLMs' Toxicity

    cs.CL 2025-09 conditional novelty 6.0 of 10

    Emojis in harmful prompts bypass LLM safety more effectively than plain text, across 7 models and 5 languages, through a heterogeneous tokenization channel.

  9. PUZZLED: Jailbreaking LLMs through Word-Based Puzzles

    cs.AI 2025-08 unverdicted novelty 6.0 of 10

    PUZZLED masks harmful keywords as word-search, anagram, or crossword puzzles and achieves a reported 88.8% average attack success rate across five leading LLMs.

  10. Paper Summary Attack: Jailbreaking LLMs through LLM Safety Papers

    cs.CL 2025-07 conditional novelty 6.0 of 10

    Summaries of LLM safety papers, paired with a completion-style payload containing a harmful query, jailbreak aligned LLMs at high reported success rates and expose a defense paper versus attack paper bias.

  11. Cross-Modal Obfuscation for Jailbreak Attacks on Large Vision-Language Models

    cs.CL 2025-06 conditional novelty 6.0 of 10

    CAMO hides harmful instructions across text and image using masked keywords and math-puzzle clues, making several LVLMs answer banned queries while evading common safety filters.

  12. Lifelong Safety Alignment for Language Models

    cs.CR 2025-05 conditional novelty 6.0 of 10

    A co-evolutionary attacker-defender loop, warmed up by strategies extracted from jailbreak papers, reduces jailbreak success rate on a robust model from 73% to 7% in two iterations.

  13. Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models

    cs.CL 2025-01 conditional novelty 6.0 of 10

    Universal adversarial suffixes can shift any text's embedding toward a model's biased mean direction, breaking embedding-based LLM safety classifiers.

  14. LLMs are Vulnerable to Malicious Prompts Disguised as Scientific Language

    cs.CL 2025-01 conditional novelty 6.0 of 10

    Scientific-sounding persuasion, using real or fabricated research summaries, reliably increases stereotypical bias and toxicity in multiple commercial LLMs.

  15. DiffusionAttacker: Diffusion-Driven Prompt Manipulation for LLM Jailbreak

    cs.CL 2024-12 conditional novelty 6.0 of 10

    A diffusion-based prompt rewriter that pushes rewritten prompts toward harmless regions of a target model's hidden states achieves higher jailbreak success than existing suffix and template attacks.

  16. JailPO: A Novel Black-box Jailbreak Framework via Preference Optimization against Aligned LLMs

    cs.CR 2024-12 conditional novelty 6.0 of 10

    JailPO uses preference optimization to train attack models that generate covert jailbreak questions and templates, achieving high attack success on aligned LLMs with far fewer queries.

  17. Look Before You Leap: Enhancing Attention and Vigilance Regarding Harmful Content with GuidelineLLM

    cs.CL 2024-12 conditional novelty 6.0 of 10

    A fine-tuned guideline generator that prepends risk summaries to prompts cuts jailbreak attack success by about 34 percentage points on average across three chatbots without altering the target models.

  18. Does Safety Training of LLMs Generalize to Semantically Related Natural Prompts?

    cs.CL 2024-12 conditional novelty 6.0 of 10

    A new attack pipeline, ReG-QA, generates natural, semantically related questions from a toxic seed and jailbreaks aligned LLMs at rates up to 93% on GPT-3.5 and 82% on GPT-4.

  19. Reason Before You Retrieve: Agentic Planning for Multi-modal RAG

    cs.AI 2026-06 reject novelty 5.0 of 10

    MM-R2 claims SOTA multimodal RAG accuracy on InfoSeek and Encyclopedic VQA via intent grounding plus a 10-topic KnowledgeMap, but its teacher trajectories leak the gold Wikipedia page and omit the image.

  20. Harmful Prompt Laundering: Jailbreaking LLMs with Abductive Styles and Symbolic Encoding

    cs.AI 2025-09 conditional novelty 5.0 of 10

    HaPLa, a black-box jailbreak combining abductive framing and symbolic encoding, achieves over 95% ASR on GPT-series and over 70% on all six tested LLMs, and safety tuning against it significantly degrades helpfulness.

  21. Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation

    cs.CL 2025-08 reject novelty 5.0 of 10

    Sparse autoencoder activation perturbation (SFPF) applied on top of existing jailbreak prompts raises attack success rate on Qwen3-32B, but with no defense evaluation and weak reproducibility.

  22. SEALGuard: Safeguarding the Multilingual Conversations in Southeast Asian Languages for LLM Software Systems

    cs.CL 2025-07 conditional novelty 5.0 of 10

    A LoRA-adapted SeaLLM model detects unsafe and jailbreak prompts in nine Southeast Asian languages with 97% recall and 98% F1 on the authors' new SEALSBench benchmark, far above zero-shot LlamaGuard.

  23. We Should Identify and Mitigate Third-Party Safety Risks in MCP-Powered Agent Systems

    cs.LG 2025-06 conditional novelty 5.0 of 10

    MCP-powered LLM agents are vulnerable to prompt injection from third-party services, and simple detection or filtering defenses do not reliably stop these attacks.

  24. InfoFlood: Jailbreaking Large Language Models with Information Overload

    cs.CR 2025-06 conditional novelty 5.0 of 10

    InfoFlood claims near-perfect jailbreak success on four frontier LLMs by rewriting harmful queries into verbose academic prose with fake citations, past-tense framing, and ethical disclaimers, without adversarial suffixes.

  25. SafeTy Reasoning Elicitation Alignment for Multi-Turn Dialogues

    cs.CL 2025-05 conditional novelty 5.0 of 10

    STREAM fine-tunes a small reasoning model on human-labeled, reason-annotated multi-turn dialogues and uses it to warn target LLMs, cutting average attack success rates by roughly half while keeping benchmark scores close.

  26. Three Minds, One Legend: Jailbreak Large Reasoning Model with Adaptive Stacked Ciphers

    cs.CL 2025-05 conditional novelty 5.0 of 10

    A stacked-cipher jailbreak with adaptive code selection achieves 80.8% to 100% attack success on commercial large reasoning models.

  27. xJailbreak: Representation Space Guided Reinforcement Learning for Interpretable LLM Jailbreaking

    cs.CL 2025-01 conditional novelty 5.0 of 10

    xJailbreak uses a representation-space 'borderline' reward plus an intent-checking LLM judge in RL training to rewrite prompts for black-box LLM jailbreaking.

  28. Self-Instruct Few-Shot Jailbreaking: Decompose the Attack into Pattern and Behavior Learning

    cs.AI 2025-01 conditional novelty 5.0 of 10

    A few-shot jailbreak method that combines repeated special-token patterns with self-generated harmful demos to push sample-level attack success near 90% on several open-source LLMs.

  29. LLM360 K2: Building a 65B 360-Open-Source Large Language Model from Scratch

    cs.LG 2025-01 conditional novelty 5.0 of 10

    K2 Diamond is a fully open 65B-parameter LLM that reaches Llama 2 70B-level performance on standard benchmarks.

  30. Align is not Enough: Multimodal Universal Jailbreak Attack against Multimodal Large Language Models

    cs.CR 2025-06 conditional novelty 4.0 of 10

    An alternating image-text optimization produces a universal adversarial suffix and image that transfer across open multimodal LLMs more effectively than single-modality jailbreaks.

  31. Adversarial Preference Learning for Robust LLM Alignment

    cs.LG 2025-05 conditional novelty 4.0 of 10

    APL iteratively trains an attacker to generate adversarial prompt rewrites and a defender to resist them, using the defender's own preference probabilities as the attack signal.

  32. Safety Reasoning with Guidelines

    cs.LG 2025-02 conditional novelty 4.0 of 10

    Training LLMs to reason through explicit safety guidelines reduces out-of-distribution jailbreak success rates compared to standard refusal training.

  33. Large Language Model Adversarial Landscape Through the Lens of Attack Objectives

    cs.CR 2025-02 conditional novelty 4.0 of 10

    A survey that re-frames LLM adversarial attacks and defenses around four attacker objectives: privacy, integrity, availability, and misuse.

  34. LIAR: Leveraging Inference Time Alignment (Best-of-N) to Jailbreak LLMs in Seconds

    cs.CL 2024-12 reject novelty 4.0 of 10

    LIAR shows that best-of-N sampling of suffixes from a GPT-2 model jailbreaks several aligned LLMs with low-perplexity prompts and far faster time-to-attack than training-based attacks.

  35. The Resurgence of GCG Adversarial Attacks on Large Language Models

    cs.CL 2025-08 conditional novelty 3.0 of 10

    The paper reports that GCG attack success drops with model size, prefix-based metrics overestimate harm, and coding prompts are more vulnerable, but these findings rest on a small model set and a GPT-4o pipeline that ...

  36. The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs

    cs.CL 2025-06 conditional novelty 3.0 of 10

    A structured survey of LLM safety evaluation that proposes a why/what/where/how taxonomy and catalogs metrics, datasets, benchmarks, evaluators, and frameworks.

  37. From Hallucinations to Jailbreaks: Rethinking the Vulnerability of Large Foundation Models

    cs.CV 2025-05 reject novelty 3.0 of 10

    The paper argues hallucinations and jailbreaks share the same optimization dynamics and shows that defenses for one also reduce the other, but the theoretical support is largely circular.

Pith tools