Pith. sign in

REVIEW 1 cited by

Ruby Teaming: Improving Quality Diversity Search with Memory for Automated Red Teaming

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2406.11654 v1 pith:ZKEHUJMK submitted 2024-06-17 cs.CL

Vernon Toh Yan Han , Rishabh Bhardwaj , Soujanya Poria This is my paper · ORCID
classification cs.CL
keywords teamingdiversityrubymemoryqualitydimensionindexrainbow
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We propose Ruby Teaming, a method that improves on Rainbow Teaming by including a memory cache as its third dimension. The memory dimension provides cues to the mutator to yield better-quality prompts, both in terms of attack success rate (ASR) and quality diversity. The prompt archive generated by Ruby Teaming has an ASR of 74%, which is 20% higher than the baseline. In terms of quality diversity, Ruby Teaming outperforms Rainbow Teaming by 6% and 3% on Shannon's Evenness Index (SEI) and Simpson's Diversity Index (SDI), respectively.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models

    cs.LG 2025-06 conditional novelty 6.0 of 10

    QDRT combines behavior-conditioned RL, multiple specialized attackers, and a MAP-Elites replay buffer to generate LLM attacks that are more toxic and cover more risk-category/style combinations.

Pith tools