REVIEW 3 cited by
Ruby Teaming: Improving Quality Diversity Search with Memory for Automated Red Teaming
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
We propose Ruby Teaming, a method that improves on Rainbow Teaming by including a memory cache as its third dimension. The memory dimension provides cues to the mutator to yield better-quality prompts, both in terms of attack success rate (ASR) and quality diversity. The prompt archive generated by Ruby Teaming has an ASR of 74%, which is 20% higher than the baseline. In terms of quality diversity, Ruby Teaming outperforms Rainbow Teaming by 6% and 3% on Shannon's Evenness Index (SEI) and Simpson's Diversity Index (SDI), respectively.
Forward citations
Cited by 3 Pith papers
-
Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models
QDRT combines behavior-conditioned RL, multiple specialized attackers, and a MAP-Elites replay buffer to generate LLM attacks that are more toxic and cover more risk-category/style combinations.
-
Towards Effective Discrimination Testing for Generative AI
Standard GenAI fairness tests can certify models as fair even when downstream interview decisions, red team rankings, multi-turn behavior, and user-modified image settings reveal systematic disparities.
-
Generating Attacks for LLMs with GFlowNets
The authors apply GFlowNet-based reinforcement learning to train LLMs that generate English and Turkish adversarial prompts, reporting improved red-teaming success rates over a prior English-only method.
Discussion (0). Continue with ORCID to comment.