Pith. sign in

REVIEW 11 cited by

MART: Improving LLM Safety with Multi-round Automatic Red-Teaming

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2311.07689 v1 pith:XWSBVZBK submitted 2023-11-13 cs.CL

Suyu Ge , Chunting Zhou , Rui Hou , Madian Khabsa , Yi-Chia Wang , Qifan Wang , Jiawei Han , Yuning Mao This is my paper
classification cs.CL
keywords adversarialtargetred-teamingsafetyautomaticllmsmartprompt
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Red-teaming is a common practice for mitigating unsafe behaviors in Large Language Models (LLMs), which involves thoroughly assessing LLMs to identify potential flaws and addressing them with responsible and accurate responses. While effective, manual red-teaming is costly, and existing automatic red-teaming typically discovers safety risks without addressing them. In this paper, we propose a Multi-round Automatic Red-Teaming (MART) method, which incorporates both automatic adversarial prompt writing and safe response generation, significantly increasing red-teaming scalability and the safety of the target LLM. Specifically, an adversarial LLM and a target LLM interplay with each other in an iterative manner, where the adversarial LLM aims to generate challenging prompts that elicit unsafe responses from the target LLM, while the target LLM is fine-tuned with safety aligned data on these adversarial prompts. In each round, the adversarial LLM crafts better attacks on the updated target LLM, while the target LLM also improves itself through safety fine-tuning. On adversarial prompt benchmarks, the violation rate of an LLM with limited safety alignment reduces up to 84.7% after 4 rounds of MART, achieving comparable performance to LLMs with extensive adversarial prompt writing. Notably, model helpfulness on non-adversarial prompts remains stable throughout iterations, indicating the target LLM maintains strong performance on instruction following.

Discussion (0). Sign in to comment.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Paladin: Defending LLM-enabled Phishing Emails with a New Trigger-Tag Paradigm

    cs.CR 2025-09 conditional novelty 7.0 of 10

    A trigger-tag watermark embedded by fine-tuning lets modified LLMs mark their own phishing outputs for cheap detection.

  2. GPT-Red: Automated Red Teaming via Self-Play at Scale

    cs.CR 2026-07 conditional novelty 6.0 of 10

    A self-play-trained red-teaming agent, GPT-Red, discovers prompt injection attacks and is used to adversarially harden GPT-5.6, cutting attack success rates to near zero on several benchmarks.

  3. Paper Summary Attack: Jailbreaking LLMs through LLM Safety Papers

    cs.CL 2025-07 conditional novelty 6.0 of 10

    Summaries of LLM safety papers, paired with a completion-style payload containing a harmful query, jailbreak aligned LLMs at high reported success rates and expose a defense paper versus attack paper bias.

  4. MGC: A Compiler Framework Exploiting Compositional Blindness in Aligned LLMs for Malware Generation

    cs.CR 2025-07 conditional novelty 6.0 of 10

    MGC, a two-stage compiler framework, generates functional malware by decomposing malicious intents into benign-appearing MDIR components that strong aligned LLMs will implement, bypassing safety alignment.

  5. RedCoder: Automated Multi-Turn Red Teaming for Code LLMs

    cs.SE 2025-06 conditional novelty 6.0 of 10

    A multi-turn red-teaming agent trained on simulated attacker-defender conversations induces vulnerable code at higher rates than prior attack methods across several code LLMs.

  6. FORTRESS: Frontier Risk Evaluation for National Security and Public Safety

    cs.CY 2025-06 conditional novelty 6.0 of 10

    A new benchmark with instance-specific rubrics measures frontier LLMs' willingness to assist with national security and public safety threats, alongside a paired over-refusal test.

  7. Adversarial Stress Testing of Role-Playing Language Agents using Multi-Agent Evaluation

    cs.AI 2026-08 conditional novelty 5.0 of 10

    A multi-agent adversarial evaluation platform with six progressive attack strategies shows that role-playing LLMs degrade under sustained pressure, with automated judging correlating with human ratings.

  8. A Red Teaming Framework for Large Language Models: A Case Study on Faithfulness Evaluation

    cs.CL 2026-06 unverdicted novelty 5.0 of 10

    Introduces a multi-role red teaming framework using attacker and jury models that increases attack success rates by up to 7.9% on LLM faithfulness in question-answering tasks.

  9. SEALGuard: Safeguarding the Multilingual Conversations in Southeast Asian Languages for LLM Software Systems

    cs.CL 2025-07 conditional novelty 5.0 of 10

    A LoRA-adapted SeaLLM model detects unsafe and jailbreak prompts in nine Southeast Asian languages with 97% recall and 98% F1 on the authors' new SEALSBench benchmark, far above zero-shot LlamaGuard.

  10. Addressing Bias in LLMs: Strategies and Application to Fair AI-based Recruitment

    cs.AI 2025-06 conditional novelty 4.0 of 10

    A resume-scoring system built on BERT or RoBERTa reproduces artificially injected gender bias in training scores, and two gender-information removal methods reduce that bias on a synthetic recruitment dataset.

  11. Agentic Web: Weaving the Next Web with AI Agents

    cs.AI 2025-07 conditional novelty 3.0 of 10

    A position paper defines the Agentic Web as the next web era and proposes a three-dimensional conceptual framework for understanding and building it.

Pith tools