{"work":{"id":"3b676de6-edef-4976-a8b5-082d4ff50867","openalex_id":null,"doi":null,"arxiv_id":"2310.04451","raw_key":null,"title":"AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models","authors":null,"authors_text":"Xiaogeng Liu, Nan Xu, Muhao Chen, Chaowei Xiao","year":2023,"venue":"cs.CL","abstract":"The aligned Large Language Models (LLMs) are powerful language understanding and decision-making tools that are created through extensive alignment with human feedback. However, these large models remain susceptible to jailbreak attacks, where adversaries manipulate prompts to elicit malicious outputs that should not be given by aligned LLMs. Investigating jailbreak prompts can lead us to delve into the limitations of LLMs and further guide us to secure them. Unfortunately, existing jailbreak techniques suffer from either (1) scalability issues, where attacks heavily rely on manual crafting of prompts, or (2) stealthiness problems, as attacks depend on token-based algorithms to generate prompts that are often semantically meaningless, making them susceptible to detection through basic perplexity testing. In light of these challenges, we intend to answer this question: Can we develop an approach that can automatically generate stealthy jailbreak prompts? In this paper, we introduce AutoDAN, a novel jailbreak attack against aligned LLMs. AutoDAN can automatically generate stealthy jailbreak prompts by the carefully designed hierarchical genetic algorithm. Extensive evaluations demonstrate that AutoDAN not only automates the process while preserving semantic meaningfulness, but also demonstrates superior attack strength in cross-model transferability, and cross-sample universality compared with the baseline. Moreover, we also compare AutoDAN with perplexity-based defense methods and show that AutoDAN can bypass them effectively.","external_url":"https://arxiv.org/abs/2310.04451","cited_by_count":null,"metadata_source":"pith","metadata_fetched_at":"2026-05-25T08:15:34.196581+00:00","pith_arxiv_id":"2310.04451","created_at":"2026-05-09T05:55:32.356779+00:00","updated_at":"2026-05-25T08:15:34.196581+00:00","title_quality_ok":true,"display_title":"AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models","render_title":"AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models"},"hub":{"state":{"work_id":"3b676de6-edef-4976-a8b5-082d4ff50867","tier":"hub","tier_reason":"10+ Pith inbound or 1,000+ external citations","pith_inbound_count":61,"external_cited_by_count":null,"distinct_field_count":8,"first_pith_cited_at":"2023-10-05T17:01:53+00:00","last_pith_cited_at":"2026-05-20T19:31:07+00:00","author_build_status":"not_needed","summary_status":"needed","contexts_status":"needed","graph_status":"needed","ask_index_status":"not_needed","reader_status":"not_needed","recognition_status":"not_needed","updated_at":"2026-06-05T13:48:59.612501+00:00","tier_text":"hub"},"tier":"hub","role_counts":[{"context_role":"background","n":18},{"context_role":"method","n":2},{"context_role":"baseline","n":1}],"polarity_counts":[{"context_polarity":"background","n":15},{"context_polarity":"support","n":2},{"context_polarity":"use_method","n":2},{"context_polarity":"baseline","n":1},{"context_polarity":"unclear","n":1}],"runs":{"context_extract":{"job_type":"context_extract","status":"succeeded","result":{"enqueued_papers":25},"error":null,"updated_at":"2026-05-14T18:30:16.042758+00:00"},"graph_features":{"job_type":"graph_features","status":"succeeded","result":{"co_cited":[{"title":"Universal and Transferable Adversarial Attacks on Aligned Language Models","work_id":"3322fa86-1768-4677-8425-dd326b45e078","shared_citers":23},{"title":"HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal","work_id":"b0b0303f-2444-4789-a979-8153624312ff","shared_citers":16},{"title":"Jailbreaking Black Box Large Language Models in Twenty Queries","work_id":"38678cda-6595-4ca3-916b-066c00cce063","shared_citers":11},{"title":"Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations","work_id":"93844332-869b-448c-a1be-35466150b1b2","shared_citers":11},{"title":"Llama 2: Open Foundation and Fine-Tuned Chat Models","work_id":"68a5177f-d644-44c1-bd4f-4e5278c22f5d","shared_citers":10},{"title":"Qwen3 Technical Report","work_id":"25a4e30c-1232-48e7-9925-02fa12ba7c9e","shared_citers":9},{"title":"The Llama 3 Herd of Models","work_id":"1549a635-88af-4ac1-acfe-51ae7bb53345","shared_citers":9},{"title":"GPT-4 Technical Report","work_id":"b928e041-6991-4c08-8c81-0359e4097c7b","shared_citers":7},{"title":"Ample GCG : Learning a universal and transferable generative model of adversarial suffixes for jailbreaking both open and closed LLMs","work_id":"cb7fb36d-4e30-499e-8c38-8a058f1b1a50","shared_citers":6},{"title":"Jailbroken: How Does LLM Safety Training Fail?","work_id":"a0b5df13-7a81-4b15-afb4-c715e6b400bc","shared_citers":6},{"title":"LLaMA: Open and Efficient Foundation Language Models","work_id":"c018fc23-6f3f-4035-9d02-28a2173b2b9d","shared_citers":6},{"title":"SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks","work_id":"f670dc77-bf11-46fc-af9d-b77215abd084","shared_citers":6},{"title":"Steering Language Models With Activation Engineering","work_id":"d525fe06-5560-4e97-86fc-7a0e551f5b17","shared_citers":6},{"title":"Autodan: inter- pretable gradient-based adversarial attacks on large lan- guage models.arXiv preprint arXiv:2310.15140","work_id":"cc605bcf-1f90-4b4e-bd3c-2c4a2207d263","shared_citers":5},{"title":"Baseline Defenses for Adversarial Attacks Against Aligned Language Models","work_id":"db5870ca-177b-4d1d-a08d-ee5ceab17fe3","shared_citers":5},{"title":"Constitutional classifiers: Defending against universal jailbreaks across thousands of hours of red teaming","work_id":"a2405e47-1ce4-4dfe-b7ce-5ae2a384f34b","shared_citers":5},{"title":"Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts","work_id":"230d395f-9220-4e6f-a52f-e593552b049b","shared_citers":5},{"title":"Ignore Previous Prompt: Attack Techniques For Language Models","work_id":"a7c5b6ec-3407-4330-96c8-3fc58e7d410b","shared_citers":5},{"title":"Jailbreak attacks and defenses against large language models: A survey","work_id":"0ee7fc45-ae61-432b-83ac-f1d93ccd88fb","shared_citers":5},{"title":"Mistral 7B","work_id":"eb5e1305-ad11-4875-ad8d-ad8b8f697599","shared_citers":5},{"title":"Red Teaming Language Models with Language Models","work_id":"d1274c54-508f-42f9-aeb3-91db13f3a622","shared_citers":5},{"title":"Training language models to follow instructions with human feedback","work_id":"52aff42f-4fa9-4fcf-bdb3-1459b9bebf65","shared_citers":5},{"title":"arXiv preprint arXiv:2410.05295 , year=","work_id":"256da404-2bb4-4b45-9068-be1a2d061a81","shared_citers":4},{"title":"A StrongREJECT for empty jailbreaks","work_id":"27281d18-31bd-4124-9fa7-4e61945ff9d1","shared_citers":4}],"time_series":[{"n":1,"year":2023},{"n":2,"year":2024},{"n":30,"year":2026}],"dependency_candidates":[]},"error":null,"updated_at":"2026-05-14T18:29:46.337601+00:00"},"identity_refresh":{"job_type":"identity_refresh","status":"succeeded","result":{"items":[{"title":"Qwen3 Technical Report","outcome":"unchanged","work_id":"25a4e30c-1232-48e7-9925-02fa12ba7c9e","resolver":"local_arxiv","confidence":0.98,"old_work_id":"25a4e30c-1232-48e7-9925-02fa12ba7c9e"}],"counts":{"fixed":0,"merged":0,"unchanged":1,"quarantined":0,"needs_external_resolution":0},"errors":[],"attempted":1},"error":null,"updated_at":"2026-05-14T18:30:03.618466+00:00"},"summary_claims":{"job_type":"summary_claims","status":"succeeded","result":{"title":"AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models","claims":[{"claim_text":"The aligned Large Language Models (LLMs) are powerful language understanding and decision-making tools that are created through extensive alignment with human feedback. However, these large models remain susceptible to jailbreak attacks, where adversaries manipulate prompts to elicit malicious outputs that should not be given by aligned LLMs. Investigating jailbreak prompts can lead us to delve into the limitations of LLMs and further guide us to secure them. Unfortunately, existing jailbreak techniques suffer from either (1) scalability issues, where attacks heavily rely on manual crafting of","claim_type":"abstract","evidence_strength":"source_metadata"}],"why_cited":"Pith tracks AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models because it crossed a citation-hub threshold.","role_counts":[]},"error":null,"updated_at":"2026-05-14T18:29:20.213828+00:00"}},"summary":{"title":"AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models","claims":[{"claim_text":"The aligned Large Language Models (LLMs) are powerful language understanding and decision-making tools that are created through extensive alignment with human feedback. However, these large models remain susceptible to jailbreak attacks, where adversaries manipulate prompts to elicit malicious outputs that should not be given by aligned LLMs. Investigating jailbreak prompts can lead us to delve into the limitations of LLMs and further guide us to secure them. Unfortunately, existing jailbreak techniques suffer from either (1) scalability issues, where attacks heavily rely on manual crafting of","claim_type":"abstract","evidence_strength":"source_metadata"}],"why_cited":"Pith tracks AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models because it crossed a citation-hub threshold.","role_counts":[]},"graph":{"co_cited":[{"title":"Universal and Transferable Adversarial Attacks on Aligned Language Models","work_id":"3322fa86-1768-4677-8425-dd326b45e078","shared_citers":23},{"title":"HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal","work_id":"b0b0303f-2444-4789-a979-8153624312ff","shared_citers":16},{"title":"Jailbreaking Black Box Large Language Models in Twenty Queries","work_id":"38678cda-6595-4ca3-916b-066c00cce063","shared_citers":11},{"title":"Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations","work_id":"93844332-869b-448c-a1be-35466150b1b2","shared_citers":11},{"title":"Llama 2: Open Foundation and Fine-Tuned Chat Models","work_id":"68a5177f-d644-44c1-bd4f-4e5278c22f5d","shared_citers":10},{"title":"Qwen3 Technical Report","work_id":"25a4e30c-1232-48e7-9925-02fa12ba7c9e","shared_citers":9},{"title":"The Llama 3 Herd of Models","work_id":"1549a635-88af-4ac1-acfe-51ae7bb53345","shared_citers":9},{"title":"GPT-4 Technical Report","work_id":"b928e041-6991-4c08-8c81-0359e4097c7b","shared_citers":7},{"title":"Ample GCG : Learning a universal and transferable generative model of adversarial suffixes for jailbreaking both open and closed LLMs","work_id":"cb7fb36d-4e30-499e-8c38-8a058f1b1a50","shared_citers":6},{"title":"Jailbroken: How Does LLM Safety Training Fail?","work_id":"a0b5df13-7a81-4b15-afb4-c715e6b400bc","shared_citers":6},{"title":"LLaMA: Open and Efficient Foundation Language Models","work_id":"c018fc23-6f3f-4035-9d02-28a2173b2b9d","shared_citers":6},{"title":"SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks","work_id":"f670dc77-bf11-46fc-af9d-b77215abd084","shared_citers":6},{"title":"Steering Language Models With Activation Engineering","work_id":"d525fe06-5560-4e97-86fc-7a0e551f5b17","shared_citers":6},{"title":"Autodan: inter- pretable gradient-based adversarial attacks on large lan- guage models.arXiv preprint arXiv:2310.15140","work_id":"cc605bcf-1f90-4b4e-bd3c-2c4a2207d263","shared_citers":5},{"title":"Baseline Defenses for Adversarial Attacks Against Aligned Language Models","work_id":"db5870ca-177b-4d1d-a08d-ee5ceab17fe3","shared_citers":5},{"title":"Constitutional classifiers: Defending against universal jailbreaks across thousands of hours of red teaming","work_id":"a2405e47-1ce4-4dfe-b7ce-5ae2a384f34b","shared_citers":5},{"title":"Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts","work_id":"230d395f-9220-4e6f-a52f-e593552b049b","shared_citers":5},{"title":"Ignore Previous Prompt: Attack Techniques For Language Models","work_id":"a7c5b6ec-3407-4330-96c8-3fc58e7d410b","shared_citers":5},{"title":"Jailbreak attacks and defenses against large language models: A survey","work_id":"0ee7fc45-ae61-432b-83ac-f1d93ccd88fb","shared_citers":5},{"title":"Mistral 7B","work_id":"eb5e1305-ad11-4875-ad8d-ad8b8f697599","shared_citers":5},{"title":"Red Teaming Language Models with Language Models","work_id":"d1274c54-508f-42f9-aeb3-91db13f3a622","shared_citers":5},{"title":"Training language models to follow instructions with human feedback","work_id":"52aff42f-4fa9-4fcf-bdb3-1459b9bebf65","shared_citers":5},{"title":"arXiv preprint arXiv:2410.05295 , year=","work_id":"256da404-2bb4-4b45-9068-be1a2d061a81","shared_citers":4},{"title":"A StrongREJECT for empty jailbreaks","work_id":"27281d18-31bd-4124-9fa7-4e61945ff9d1","shared_citers":4}],"time_series":[{"n":1,"year":2023},{"n":2,"year":2024},{"n":30,"year":2026}],"dependency_candidates":[]},"authors":[]}}