Pith. sign in

REVIEW 19 cited by

MART: Improving LLM Safety with Multi-round Automatic Red-Teaming

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2311.07689 v1 pith:XWSBVZBK submitted 2023-11-13 cs.CL

Suyu Ge , Chunting Zhou , Rui Hou , Madian Khabsa , Yi-Chia Wang , Qifan Wang , Jiawei Han , Yuning Mao This is my paper · ORCID
classification cs.CL
keywords adversarialtargetred-teamingsafetyautomaticllmsmartprompt
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Red-teaming is a common practice for mitigating unsafe behaviors in Large Language Models (LLMs), which involves thoroughly assessing LLMs to identify potential flaws and addressing them with responsible and accurate responses. While effective, manual red-teaming is costly, and existing automatic red-teaming typically discovers safety risks without addressing them. In this paper, we propose a Multi-round Automatic Red-Teaming (MART) method, which incorporates both automatic adversarial prompt writing and safe response generation, significantly increasing red-teaming scalability and the safety of the target LLM. Specifically, an adversarial LLM and a target LLM interplay with each other in an iterative manner, where the adversarial LLM aims to generate challenging prompts that elicit unsafe responses from the target LLM, while the target LLM is fine-tuned with safety aligned data on these adversarial prompts. In each round, the adversarial LLM crafts better attacks on the updated target LLM, while the target LLM also improves itself through safety fine-tuning. On adversarial prompt benchmarks, the violation rate of an LLM with limited safety alignment reduces up to 84.7% after 4 rounds of MART, achieving comparable performance to LLMs with extensive adversarial prompt writing. Notably, model helpfulness on non-adversarial prompts remains stable throughout iterations, indicating the target LLM maintains strong performance on instruction following.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 19 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Paladin: Defending LLM-enabled Phishing Emails with a New Trigger-Tag Paradigm

    cs.CR 2025-09 conditional novelty 7.0 of 10

    A trigger-tag watermark embedded by fine-tuning lets modified LLMs mark their own phishing outputs for cheap detection.

  2. Jailbreaking to Jailbreak

    cs.CL 2025-02 conditional novelty 7.0 of 10

    A transferable multi-turn jailbreak turns refusal-trained black-box LLMs into willing automated jailbreakers, with high attack success against other models and against themselves.

  3. GPT-Red: Automated Red Teaming via Self-Play at Scale

    cs.CR 2026-07 conditional novelty 6.0 of 10

    A self-play-trained red-teaming agent, GPT-Red, discovers prompt injection attacks and is used to adversarially harden GPT-5.6, cutting attack success rates to near zero on several benchmarks.

  4. Paper Summary Attack: Jailbreaking LLMs through LLM Safety Papers

    cs.CL 2025-07 conditional novelty 6.0 of 10

    Summaries of LLM safety papers, paired with a completion-style payload containing a harmful query, jailbreak aligned LLMs at high reported success rates and expose a defense paper versus attack paper bias.

  5. MGC: A Compiler Framework Exploiting Compositional Blindness in Aligned LLMs for Malware Generation

    cs.CR 2025-07 conditional novelty 6.0 of 10

    MGC, a two-stage compiler framework, generates functional malware by decomposing malicious intents into benign-appearing MDIR components that strong aligned LLMs will implement, bypassing safety alignment.

  6. RedCoder: Automated Multi-Turn Red Teaming for Code LLMs

    cs.SE 2025-06 conditional novelty 6.0 of 10

    A multi-turn red-teaming agent trained on simulated attacker-defender conversations induces vulnerable code at higher rates than prior attack methods across several code LLMs.

  7. FORTRESS: Frontier Risk Evaluation for National Security and Public Safety

    cs.CY 2025-06 conditional novelty 6.0 of 10

    A new benchmark with instance-specific rubrics measures frontier LLMs' willingness to assist with national security and public safety threats, alongside a paired over-refusal test.

  8. Adversarial Reasoning at Jailbreaking Time

    cs.LG 2025-02 conditional novelty 6.0 of 10

    A loss-guided 'reason, verify, search' loop with three LLM modules surpasses prior semantic jailbreak methods on several defended models.

  9. Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models

    cs.CL 2025-01 conditional novelty 6.0 of 10

    Universal adversarial suffixes can shift any text's embedding toward a model's biased mean direction, breaking embedding-based LLM safety classifiers.

  10. Auto-RT: Automatic Jailbreak Strategy Exploration for Red-Teaming Large Language Models

    cs.CR 2025-01 conditional novelty 6.0 of 10

    Auto-RT uses early-terminated exploration plus reward shaping from progressively weakened copies of the target model to automatically discover jailbreak strategies, reporting up to 16.63% higher attack success than baselines.

  11. Diverse and Effective Red Teaming with Auto-generated Rewards and Multi-step Reinforcement Learning

    cs.LG 2024-12 conditional novelty 6.0 of 10

    An automated red-teaming method that uses LLM-generated per-goal rewards and multi-step RL with a style-diversity reward to produce diverse and effective attacks on language models.

  12. Private Yet Social: How LLM Chatbots Support and Challenge Eating Disorder Recovery

    cs.HC 2024-12 conditional novelty 6.0 of 10

    A 10-day field study found that an LLM chatbot supported eating disorder recovery through private storytelling, yet also produced unnoticed harmful responses such as praising weight loss and restriction.

  13. Look Before You Leap: Enhancing Attention and Vigilance Regarding Harmful Content with GuidelineLLM

    cs.CL 2024-12 conditional novelty 6.0 of 10

    A fine-tuned guideline generator that prepends risk summaries to prompts cuts jailbreak attack success by about 34 percentage points on average across three chatbots without altering the target models.

  14. Adversarial Stress Testing of Role-Playing Language Agents using Multi-Agent Evaluation

    cs.AI 2026-08 conditional novelty 5.0 of 10

    A multi-agent adversarial evaluation platform with six progressive attack strategies shows that role-playing LLMs degrade under sustained pressure, with automated judging correlating with human ratings.

  15. A Red Teaming Framework for Large Language Models: A Case Study on Faithfulness Evaluation

    cs.CL 2026-06 unverdicted novelty 5.0 of 10

    Introduces a multi-role red teaming framework using attacker and jury models that increases attack success rates by up to 7.9% on LLM faithfulness in question-answering tasks.

  16. SEALGuard: Safeguarding the Multilingual Conversations in Southeast Asian Languages for LLM Software Systems

    cs.CL 2025-07 conditional novelty 5.0 of 10

    A LoRA-adapted SeaLLM model detects unsafe and jailbreak prompts in nine Southeast Asian languages with 97% recall and 98% F1 on the authors' new SEALSBench benchmark, far above zero-shot LlamaGuard.

  17. Addressing Bias in LLMs: Strategies and Application to Fair AI-based Recruitment

    cs.AI 2025-06 conditional novelty 4.0 of 10

    A resume-scoring system built on BERT or RoBERTa reproduces artificially injected gender bias in training scores, and two gender-information removal methods reduce that bias on a synthetic recruitment dataset.

  18. Agentic Web: Weaving the Next Web with AI Agents

    cs.AI 2025-07 conditional novelty 3.0 of 10

    A position paper defines the Agentic Web as the next web era and proposes a three-dimensional conceptual framework for understanding and building it.

  19. The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs

    cs.CL 2025-06 conditional novelty 3.0 of 10

    A structured survey of LLM safety evaluation that proposes a why/what/where/how taxonomy and catalogs metrics, datasets, benchmarks, evaluators, and frameworks.

Pith tools