REVIEW 31 cited by
AEGIS: Online Adaptive AI Content Safety Moderation with Ensemble of LLM Experts
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
As Large Language Models (LLMs) and generative AI become more widespread, the content safety risks associated with their use also increase. We find a notable deficiency in high-quality content safety datasets and benchmarks that comprehensively cover a wide range of critical safety areas. To address this, we define a broad content safety risk taxonomy, comprising 13 critical risk and 9 sparse risk categories. Additionally, we curate AEGISSAFETYDATASET, a new dataset of approximately 26, 000 human-LLM interaction instances, complete with human annotations adhering to the taxonomy. We plan to release this dataset to the community to further research and to help benchmark LLM models for safety. To demonstrate the effectiveness of the dataset, we instruction-tune multiple LLM-based safety models. We show that our models (named AEGISSAFETYEXPERTS), not only surpass or perform competitively with the state-of-the-art LLM-based safety models and general purpose LLMs, but also exhibit robustness across multiple jail-break attack categories. We also show how using AEGISSAFETYDATASET during the LLM alignment phase does not negatively impact the performance of the aligned models on MT Bench scores. Furthermore, we propose AEGIS, a novel application of a no-regret online adaptation framework with strong theoretical guarantees, to perform content moderation with an ensemble of LLM content safety experts in deployment
Forward citations
Cited by 31 Pith papers
-
Yesterday's Shield, Today's Spear: A Self-Evolving Safety Guardrail in Production
A production multi-agent pipeline lets a 1.7B LLM safety guardrail retrain itself on new jailbreak forms and harm categories within about a day, closing 14 of 15 new threat scenarios in two months.
-
Harm is not Universal: Community-Specific Toxicity Detection is Urgently Needed
SoTA T2I toxicity detectors miss ~35% of disability-community harms; zero-shot CTD fails below random, while ICL/VQA/LoRA improve but stay well below general TD performance.
-
When Are Reasoning-Based Guardrails Not Efficient? ResponseGuard: A Fast Vision-Language Guard for Real-Time Moderation
A label-only 2B guard outperforms a 3B reasoning-based vision-language guard on response-harmfulness detection at about 150x lower latency, with the remaining image gap attributed to frozen encoders.
-
A Dual-Hypothesis Reasoning Framework for LLM Guardrails
An 8B guardrail model trained with self-generated dual-hypothesis reasoning and a weighted multi-component loss matches or beats teacher-distilled reasoning guardrails on three safety benchmarks, with the largest gain...
-
When Words Are Safe But Actions Kill: Probing Physical Danger Beyond Text Safety in Hidden-State Risk Space
Content danger and physical danger form separable hidden-state signals in LLMs, and a single-layer logistic probe (PRISM) detects both at lower false-positive rates than LLM judges or text guardrails.
-
HyperSafe: Inference-Time Safety Recovery for Fine-Tuned Language Models
A hypernetwork maps layer-wise activation fingerprints of a fine-tuned LLM to a Safe Side Network that routes harmful prompts to refusal without editing model weights.
-
DT-Guard: Intent-Driven Reasoning-Active Training for Reasoning-Free LLM Safety Guardrail
A 4B LLM safety guardrail trained with reasoning supervision but deployed with reasoning-free inference outperforms 8B baselines on safety benchmarks.
-
Predict, Don't React: Value-Based Safety Forecasting for LLM Streaming
Forecasting expected future harmfulness from prefixes via Monte Carlo rollouts yields stronger streaming LLM moderation than boundary detection, without exact onset labels.
-
SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defenses
A systemization of LLM jailbreak security that adds linked taxonomies, an evaluation platform, and JailbreakDB, while its main attack–defense comparison results remain deferred.
-
YouthSafe: A Youth-Centric Safety Benchmark and Safeguard Model for Large Language Models
Introduces YAIR, a youth-GenAI risk benchmark, and YouthSafe, a fine-tuned classifier with AUPRC 0.94 on it, though it compares a trained model to untrained baselines.
-
Libra: Large Chinese-based Safeguard for AI Content
Libra-Guard, a Chinese-language AI content safety model, reaches 86.8% accuracy on the new 5,720-sample Libra-Test benchmark, beating open-source baselines and approaching GPT-4o and Claude-3.5-Sonnet.
-
PL-Guard: Benchmarking Language Model Safety for Polish
A small Polish BERT classifier proved more robust than larger fine-tuned LLMs at classifying safe versus unsafe Polish content, including under character-level adversarial perturbations.
-
Breaking the Cloak! Unveiling Chinese Cloaked Toxicity with Homophone Graph and Toxic Lexicon
C2TU combines a Chinese pronunciation graph, a toxic lexicon, and language-model probability checking to find and correct homophone-cloaked toxic words without any training.
-
ShieldVLM: Safeguarding the Multimodal Implicit Toxicity via Deliberative Reasoning with LVLMs
ShieldVLM detects multimodal implicit toxicity through deliberate cross-modal reasoning, outperforming existing moderation APIs and models on the new MMIT benchmark.
-
Unified Multi-Task Learning & Model Fusion for Efficient Language Model Guardrailing
Small discriminative classifiers trained on synthetic guardrail data, plus a bandit-based model merging search, beat GPT-4-level moderation systems on multiple safety benchmarks.
-
Advancing Embodied Agent Security: From Safety Benchmarks to Input Moderation
A masked-attention moderator isolates user instructions from functional prompts and detects malicious commands to embodied agents with 94.58% average accuracy.
-
Aegis2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrails
Aegis2.0 provides a commercially usable, human-annotated safety dataset with 24 risk categories, and models trained on it with parameter-efficient methods match WildGuard and beat Llama Guard 3.
-
Granite Guardian
Granite Guardian 2B and 8B are open-source LLM guardrails that detect harmful content, jailbreaks, and RAG hallucination risks, reporting AUC 0.871 on harm benchmarks and 0.854 on groundedness benchmarks.
-
HoloAegis: Frozen Representation, Topological Inference: Minimally Parametric Safety Manifolds for Zero-Shot LLM Guardrails
A frozen-encoder anchor-bank energy scorer for LLM safety achieves fast zero-shot moderation, but its own table shows the average accuracy is below baselines it compares against.
-
Operational Evidence Gaps for LLMs in Fraud Detection and Trust-and-Safety Workflows
A survey of 49 LLM fraud and trust-and-safety papers finds that fraud work reports almost no per-decision latency, cost, or calibration evidence, while moderation work reports more.
-
GAF-Guard: An Agentic Framework for Risk Management and Governance in Large Language Models
The paper presents GAF-Guard, a multi-agent governance framework that combines pre-deployment risk questionnaires, real-time drift and risk monitoring, and incident reporting for LLM applications.
-
GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoning
GuardReasoner-VL, a 3B/7B VLM guard model trained with reasoning SFT and online RL, reports large F1 gains over existing VLM guard models on 14 safety benchmarks.
-
Understanding and Mitigating Risks of Generative AI in Financial Services
Open-source AI guardrails miss most financial-services content risks that a new domain-specific taxonomy identifies, even when their prompts are expanded to cover the new categories.
-
Lightweight Safety Classification Using Pruned Language Models
Intermediate-layer features of small LLMs plus a penalized logistic regression classifier achieve high F1 scores on content safety and prompt injection classification with very few labeled examples, per the paper's ex...
-
The Dark Side of Trust: Authority Citation-Driven Jailbreak Attacks on Large Language Models
Fake authoritative citations matched to the type of harmful request can bypass safety alignment in several commercial and open LLMs.
-
Llama Guard 3 Vision: Safeguarding Human-AI Image Understanding Conversations
Llama Guard 3 Vision flags harmful multimodal prompts and responses across 13 hazard categories, reporting an F1 of 0.938 for response classification on an internal test set.
-
JavelinGuard: Low-Cost Transformer Architectures for LLM Security
A study of five small transformer classifier architectures for LLM jailbreak and prompt injection detection claims low-latency accuracy comparable to large models, led by the multi-task Raudra design.
-
A Red Teaming Roadmap Towards System-Level Safety
A position paper from Scale AI argues that red teaming research should prioritize product-level safety specifications, realistic attacker models, and system-level monitoring over abstract model-level harm benchmarks.
-
Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks
A survey that frames the core problem of LLM evaluation as 'evaluation generalization': finite test sets cannot scale with unbounded model capabilities, and proposes two transitions in evaluation design.
-
Agentic Web: Weaving the Next Web with AI Agents
A position paper defines the Agentic Web as the next web era and proposes a three-dimensional conceptual framework for understanding and building it.
-
The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs
A structured survey of LLM safety evaluation that proposes a why/what/where/how taxonomy and catalogs metrics, datasets, benchmarks, evaluators, and frameworks.
Discussion (0). Continue with ORCID to comment.