{"work":{"id":"d1274c54-508f-42f9-aeb3-91db13f3a622","openalex_id":"https://openalex.org/W4226404919","doi":"10.48550/arxiv.2202.03286","arxiv_id":"2202.03286","raw_key":null,"title":"Red Teaming Language Models with Language Models","authors":null,"authors_text":"Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides","year":2022,"venue":"cs.CL","abstract":"Language Models (LMs) often cannot be deployed because of their potential to harm users in hard-to-predict ways. Prior work identifies harmful behaviors before deployment by using human annotators to hand-write test cases. However, human annotation is expensive, limiting the number and diversity of test cases. In this work, we automatically find cases where a target LM behaves in a harmful way, by generating test cases (\"red teaming\") using another LM. We evaluate the target LM's replies to generated test questions using a classifier trained to detect offensive content, uncovering tens of thousands of offensive replies in a 280B parameter LM chatbot. We explore several methods, from zero-shot generation to reinforcement learning, for generating test cases with varying levels of diversity and difficulty. Furthermore, we use prompt engineering to control LM-generated test cases to uncover a variety of other harms, automatically finding groups of people that the chatbot discusses in offensive ways, personal and hospital phone numbers generated as the chatbot's own contact info, leakage of private training data in generated text, and harms that occur over the course of a conversation. Overall, LM-based red teaming is one promising tool (among many needed) for finding and fixing diverse, undesirable LM behaviors before impacting users.","external_url":"https://arxiv.org/abs/2202.03286","cited_by_count":18,"metadata_source":"pith","metadata_fetched_at":"2026-08-05T02:28:24.338817+00:00","pith_arxiv_id":"2202.03286","created_at":"2026-05-09T02:04:35.758614+00:00","updated_at":"2026-08-05T02:28:24.338817+00:00","title_quality_ok":true,"display_title":"Red Teaming Language Models with Language Models","render_title":"Red Teaming Language Models with Language Models"},"hub":{"state":{"work_id":"d1274c54-508f-42f9-aeb3-91db13f3a622","tier":"hub","tier_reason":"10+ Pith inbound or 1,000+ external citations","pith_inbound_count":94,"external_cited_by_count":18,"distinct_field_count":10,"first_pith_cited_at":"2022-04-12T15:02:38+00:00","last_pith_cited_at":"2026-07-08T21:03:27+00:00","author_build_status":"not_needed","summary_status":"needed","contexts_status":"needed","graph_status":"needed","ask_index_status":"not_needed","reader_status":"not_needed","recognition_status":"not_needed","updated_at":"2026-08-23T21:19:24.764865+00:00","tier_text":"hub"},"tier":"hub","role_counts":[{"context_role":"background","n":14},{"context_role":"baseline","n":1},{"context_role":"method","n":1}],"polarity_counts":[{"context_polarity":"background","n":10},{"context_polarity":"support","n":2},{"context_polarity":"unclear","n":2},{"context_polarity":"baseline","n":1},{"context_polarity":"use_method","n":1}],"runs":{"context_extract":{"job_type":"context_extract","status":"succeeded","result":{"enqueued_papers":25},"error":null,"updated_at":"2026-05-14T17:49:07.850901+00:00"},"graph_features":{"job_type":"graph_features","status":"succeeded","result":{"co_cited":[{"title":"Universal and Transferable Adversarial Attacks on Aligned Language Models","work_id":"3322fa86-1768-4677-8425-dd326b45e078","shared_citers":19},{"title":"Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned","work_id":"1aabd84d-3779-4ba9-ba2f-15ce264a9b1e","shared_citers":12},{"title":"Constitutional AI: Harmlessness from AI Feedback","work_id":"faaaa4e0-2676-4fac-a0b4-99aef10d2095","shared_citers":11},{"title":"Ethical and social risks of harm from Language Models","work_id":"b4ce1c45-ef69-445a-a872-dbb785b485e9","shared_citers":11},{"title":"Llama 2: Open Foundation and Fine-Tuned Chat Models","work_id":"68a5177f-d644-44c1-bd4f-4e5278c22f5d","shared_citers":11},{"title":"HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal","work_id":"b0b0303f-2444-4789-a979-8153624312ff","shared_citers":9},{"title":"LaMDA: Language Models for Dialog Applications","work_id":"1b66d0a5-f6ae-4332-8025-c662dc64b238","shared_citers":9},{"title":"On the Opportunities and Risks of Foundation Models","work_id":"a18039e9-928d-47c9-a836-32656a71bf71","shared_citers":9},{"title":"Scaling Language Models: Methods, Analysis & Insights from Training Gopher","work_id":"47ce8be9-e500-407d-af41-ac2d132215eb","shared_citers":8},{"title":"Scaling Laws for Neural Language Models","work_id":"b7dd8749-9c45-4977-ab9b-64478dce1ae8","shared_citers":8},{"title":"Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback","work_id":"a1f2574b-a899-4713-be60-c87ba332656c","shared_citers":8},{"title":"Training language models to follow instructions with human feedback","work_id":"52aff42f-4fa9-4fcf-bdb3-1459b9bebf65","shared_citers":8},{"title":"Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations","work_id":"93844332-869b-448c-a1be-35466150b1b2","shared_citers":7},{"title":"Training Verifiers to Solve Math Word Problems","work_id":"acab1aa8-b4d6-40e0-a3ee-25341701dca2","shared_citers":7},{"title":"Explaining and Harnessing Adversarial Examples","work_id":"2cedf8f6-7539-4c49-8136-f42a20487146","shared_citers":6},{"title":"GPT-4 Technical Report","work_id":"b928e041-6991-4c08-8c81-0359e4097c7b","shared_citers":6},{"title":"Jailbreaking Black Box Large Language Models in Twenty Queries","work_id":"38678cda-6595-4ca3-916b-066c00cce063","shared_citers":6},{"title":"Jailbroken: How does llm safety training fail?","work_id":"a0b5df13-7a81-4b15-afb4-c715e6b400bc","shared_citers":6},{"title":"Measuring Massive Multitask Language Understanding","work_id":"e87ec49a-544b-4ec8-8991-75298c64ff5e","shared_citers":6},{"title":"Training Compute-Optimal Large Language Models","work_id":"b2faf28d-86b7-429c-bc42-469458efc246","shared_citers":6},{"title":"TruthfulQA: Measuring How Models Mimic Human Falsehoods","work_id":"22e3b047-a6e8-4c4c-b62e-173b545a1a45","shared_citers":6},{"title":"AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models","work_id":"3b676de6-edef-4976-a8b5-082d4ff50867","shared_citers":5},{"title":"Evaluating Large Language Models Trained on Code","work_id":"042493e9-b26f-4b4e-bbde-382072ca9b08","shared_citers":5},{"title":"Holistic Evaluation of Language Models","work_id":"cc02a01e-7218-47dc-8e66-3333e7e4adec","shared_citers":5}],"time_series":[{"n":5,"year":2022},{"n":4,"year":2023},{"n":1,"year":2024},{"n":1,"year":2025},{"n":29,"year":2026}],"dependency_candidates":[]},"error":null,"updated_at":"2026-05-14T17:48:49.059927+00:00"},"identity_refresh":{"job_type":"identity_refresh","status":"succeeded","result":{"items":[{"title":"Qwen3 Technical Report","outcome":"unchanged","work_id":"25a4e30c-1232-48e7-9925-02fa12ba7c9e","resolver":"local_arxiv","confidence":0.98,"old_work_id":"25a4e30c-1232-48e7-9925-02fa12ba7c9e"}],"counts":{"fixed":0,"merged":0,"unchanged":1,"quarantined":0,"needs_external_resolution":0},"errors":[],"attempted":1},"error":null,"updated_at":"2026-05-14T17:49:23.704903+00:00"},"summary_claims":{"job_type":"summary_claims","status":"succeeded","result":{"title":"Red Teaming Language Models with Language Models","claims":[{"claim_text":"Language Models (LMs) often cannot be deployed because of their potential to harm users in hard-to-predict ways. Prior work identifies harmful behaviors before deployment by using human annotators to hand-write test cases. However, human annotation is expensive, limiting the number and diversity of test cases. In this work, we automatically find cases where a target LM behaves in a harmful way, by generating test cases (\"red teaming\") using another LM. We evaluate the target LM's replies to generated test questions using a classifier trained to detect offensive content, uncovering tens of thou","claim_type":"abstract","evidence_strength":"source_metadata"}],"why_cited":"Pith tracks Red Teaming Language Models with Language Models because it crossed a citation-hub threshold.","role_counts":[]},"error":null,"updated_at":"2026-05-14T17:48:45.517396+00:00"}},"summary":{"title":"Red Teaming Language Models with Language Models","claims":[{"claim_text":"Language Models (LMs) often cannot be deployed because of their potential to harm users in hard-to-predict ways. Prior work identifies harmful behaviors before deployment by using human annotators to hand-write test cases. However, human annotation is expensive, limiting the number and diversity of test cases. In this work, we automatically find cases where a target LM behaves in a harmful way, by generating test cases (\"red teaming\") using another LM. We evaluate the target LM's replies to generated test questions using a classifier trained to detect offensive content, uncovering tens of thou","claim_type":"abstract","evidence_strength":"source_metadata"}],"why_cited":"Pith tracks Red Teaming Language Models with Language Models because it crossed a citation-hub threshold.","role_counts":[]},"graph":{"co_cited":[{"title":"Universal and Transferable Adversarial Attacks on Aligned Language Models","work_id":"3322fa86-1768-4677-8425-dd326b45e078","shared_citers":19},{"title":"Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned","work_id":"1aabd84d-3779-4ba9-ba2f-15ce264a9b1e","shared_citers":12},{"title":"Constitutional AI: Harmlessness from AI Feedback","work_id":"faaaa4e0-2676-4fac-a0b4-99aef10d2095","shared_citers":11},{"title":"Ethical and social risks of harm from Language Models","work_id":"b4ce1c45-ef69-445a-a872-dbb785b485e9","shared_citers":11},{"title":"Llama 2: Open Foundation and Fine-Tuned Chat Models","work_id":"68a5177f-d644-44c1-bd4f-4e5278c22f5d","shared_citers":11},{"title":"HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal","work_id":"b0b0303f-2444-4789-a979-8153624312ff","shared_citers":9},{"title":"LaMDA: Language Models for Dialog Applications","work_id":"1b66d0a5-f6ae-4332-8025-c662dc64b238","shared_citers":9},{"title":"On the Opportunities and Risks of Foundation Models","work_id":"a18039e9-928d-47c9-a836-32656a71bf71","shared_citers":9},{"title":"Scaling Language Models: Methods, Analysis & Insights from Training Gopher","work_id":"47ce8be9-e500-407d-af41-ac2d132215eb","shared_citers":8},{"title":"Scaling Laws for Neural Language Models","work_id":"b7dd8749-9c45-4977-ab9b-64478dce1ae8","shared_citers":8},{"title":"Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback","work_id":"a1f2574b-a899-4713-be60-c87ba332656c","shared_citers":8},{"title":"Training language models to follow instructions with human feedback","work_id":"52aff42f-4fa9-4fcf-bdb3-1459b9bebf65","shared_citers":8},{"title":"Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations","work_id":"93844332-869b-448c-a1be-35466150b1b2","shared_citers":7},{"title":"Training Verifiers to Solve Math Word Problems","work_id":"acab1aa8-b4d6-40e0-a3ee-25341701dca2","shared_citers":7},{"title":"Explaining and Harnessing Adversarial Examples","work_id":"2cedf8f6-7539-4c49-8136-f42a20487146","shared_citers":6},{"title":"GPT-4 Technical Report","work_id":"b928e041-6991-4c08-8c81-0359e4097c7b","shared_citers":6},{"title":"Jailbreaking Black Box Large Language Models in Twenty Queries","work_id":"38678cda-6595-4ca3-916b-066c00cce063","shared_citers":6},{"title":"Jailbroken: How does llm safety training fail?","work_id":"a0b5df13-7a81-4b15-afb4-c715e6b400bc","shared_citers":6},{"title":"Measuring Massive Multitask Language Understanding","work_id":"e87ec49a-544b-4ec8-8991-75298c64ff5e","shared_citers":6},{"title":"Training Compute-Optimal Large Language Models","work_id":"b2faf28d-86b7-429c-bc42-469458efc246","shared_citers":6},{"title":"TruthfulQA: Measuring How Models Mimic Human Falsehoods","work_id":"22e3b047-a6e8-4c4c-b62e-173b545a1a45","shared_citers":6},{"title":"AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models","work_id":"3b676de6-edef-4976-a8b5-082d4ff50867","shared_citers":5},{"title":"Evaluating Large Language Models Trained on Code","work_id":"042493e9-b26f-4b4e-bbde-382072ca9b08","shared_citers":5},{"title":"Holistic Evaluation of Language Models","work_id":"cc02a01e-7218-47dc-8e66-3333e7e4adec","shared_citers":5}],"time_series":[{"n":5,"year":2022},{"n":4,"year":2023},{"n":1,"year":2024},{"n":1,"year":2025},{"n":29,"year":2026}],"dependency_candidates":[]},"authors":[]}}