Pith. sign in

REVIEW 18 cited by

JailGuard: A Universal Detection Framework for LLM Prompt-based Attacks

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2312.10766 v4 pith:6PR5XLRB submitted 2023-12-17 cs.CR

Xiaoyu Zhang , Cen Zhang , Tianlin Li , Yihao Huang , Xiaojun Jia , Ming Hu , Jie Zhang , Yang Liu
show 2 more authors
Shiqing Ma Chao Shen
This is my paper · ORCID
classification cs.CR
keywords attacksdetectionjailguardimageinputsprompt-basedsystemstext
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The systems and software powered by Large Language Models (LLMs) and Multi-Modal LLMs (MLLMs) have played a critical role in numerous scenarios. However, current LLM systems are vulnerable to prompt-based attacks, with jailbreaking attacks enabling the LLM system to generate harmful content, while hijacking attacks manipulate the LLM system to perform attacker-desired tasks, underscoring the necessity for detection tools. Unfortunately, existing detecting approaches are usually tailored to specific attacks, resulting in poor generalization in detecting various attacks across different modalities. To address it, we propose JailGuard, a universal detection framework deployed on top of LLM systems for prompt-based attacks across text and image modalities. JailGuard operates on the principle that attacks are inherently less robust than benign ones. Specifically, JailGuard mutates untrusted inputs to generate variants and leverages the discrepancy of the variants' responses on the target model to distinguish attack samples from benign samples. We implement 18 mutators for text and image inputs and design a mutator combination policy to further improve detection generalization. The evaluation on the dataset containing 15 known attack types suggests that JailGuard achieves the best detection accuracy of 86.14%/82.90% on text and image inputs, outperforming state-of-the-art methods by 11.81%-25.73% and 12.20%-21.40%.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 18 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Measuring the Wrong Thing: Internal Harmfulness Scores Anti-Rank Successful Jailbreaks

    cs.CL 2026-08 conditional novelty 7.0 of 10

    A harmfulness score that ranks harmful intent well can rank real jailbreak successes below failed attempts, reversing the ranking a safety filter depends on.

  2. Eyes-on-Me: Scalable RAG Poisoning through Transferable Attention-Steering Attractors

    cs.LG 2025-10 conditional novelty 7.0 of 10

    Eyes-on-Me makes RAG data poisoning reusable: a transferable attention-steering attractor is optimized once, then combined with different attack payloads at near-zero cost.

  3. Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency

    cs.CR 2025-01 conditional novelty 7.0 of 10

    Shuffling words and image patches in harmful prompts bypasses safety mechanisms of several commercial and open-source multimodal models, and a black-box search over shuffles raises attack success rates substantially.

  4. Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreaks

    cs.CV 2024-11 conditional novelty 7.0 of 10

    ASTRA reduces VLM jailbreak success by adaptively steering activations away from a harm direction learned via image attribution, with little utility loss and near-zero inference overhead.

  5. Safe responses matter: Output-aware safety guardrail mitigate over-refusal in MLLMs

    cs.LG 2026-06 conditional novelty 6.0 of 10

    A lightweight multi-instance contrastive classifier on MLLM hidden states predicts output harmfulness, matching input-side safety while sharply lowering over-refusal.

  6. SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defenses

    cs.CR 2025-10 conditional novelty 6.0 of 10

    A systemization of LLM jailbreak security that adds linked taxonomies, an evaluation platform, and JailbreakDB, while its main attack–defense comparison results remain deferred.

  7. Paper Summary Attack: Jailbreaking LLMs through LLM Safety Papers

    cs.CL 2025-07 conditional novelty 6.0 of 10

    Summaries of LLM safety papers, paired with a completion-style payload containing a harmful query, jailbreak aligned LLMs at high reported success rates and expose a defense paper versus attack paper bias.

  8. AMIA: Automatic Masking and Joint Intention Analysis Makes LVLMs Robust Jailbreak Defenders

    cs.CV 2025-05 conditional novelty 6.0 of 10

    AMIA is an inference-only jailbreak defense that masks text-irrelevant image patches and prompts single-pass intention analysis, lifting average defense success on LVLMs from 52.4% to 81.7%.

  9. Test-Time Immunization: A Universal Defense Framework Against Jailbreaks for (Multimodal) Large Language Models

    cs.CR 2025-05 conditional novelty 6.0 of 10

    TIM detects jailbreaks at test time using a gist-token classifier and adaptively fine-tunes the model with LoRA on detected harmful prompts, cutting attack success rates to near zero on tested benchmarks.

  10. Defending LVLMs Against Vision Attacks through Partial-Perception Supervision

    cs.CV 2024-12 conditional novelty 6.0 of 10

    DPS uses partial-image descriptions as supervision to prompt a vision-language model to correct itself, reducing attack success by about 76% across six datasets.

  11. Jailbreak Large Vision-Language Models Through Multi-Modal Linkage

    cs.CV 2024-11 conditional novelty 6.0 of 10

    Multi-modal image encryption plus a villain-game role-play prompt achieves near-perfect jailbreak success rates against GPT-4o and other vision-language models.

  12. Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation

    cs.CL 2025-08 reject novelty 5.0 of 10

    Sparse autoencoder activation perturbation (SFPF) applied on top of existing jailbreak prompts raises attack success rate on Qwen3-32B, but with no defense evaluation and weak reproducibility.

  13. Topological Signatures of Adversaries in Multimodal Alignments

    cs.LG 2025-01 conditional novelty 5.0 of 10

    Adversarial images induce monotonic changes in persistent-homology-based losses on CLIP/BLIP image-text alignments, and gradient features from these losses modestly improve MMD-based adversarial detection.

  14. Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignment

    cs.CR 2024-11 conditional novelty 5.0 of 10

    An inference-time alignment method using a safety reward model and controlled decoding that reduces jailbreak success rates in multimodal LLMs while preserving utility.

  15. Jailbreak Attacks and Defenses against Multimodal Generative Models: A Survey

    cs.CV 2024-11 conditional novelty 5.0 of 10

    A survey that taxonomizes multimodal jailbreak attacks and defenses into four lifecycle levels (input, encoder, generator, output) across Any-to-Text, Any-to-Vision, and Any-to-Any generative models.

  16. A Multimodal Automatic Redteaming Evaluation based on Atomic Jailbreak Strategy Decoupling and Combination

    cs.CR 2026-08 conditional novelty 4.0 of 10

    A new jailbreak framework, HACA, combines atomic text and image attack strategies selected by a cross-modal planner and generates attacks with LLMs and text-to-image models, reaching 95.48% average attack success acro...

  17. A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluations

    cs.CR 2025-02 conditional novelty 4.0 of 10

    A survey of LVLM safety that adds a lifecycle taxonomy and new benchmark results showing Janus-Pro-7B has weaker safety than several open-source LVLMs.

  18. When Data Manipulation Meets Attack Goals: An In-depth Survey of Attacks for VLMs

    cs.CV 2025-02 conditional novelty 3.0 of 10

    A survey that classifies VLM attacks by goal and data manipulation strategy, and reviews defenses and metrics.

Pith tools