Fine-tuning updates frequently stale activation monitors for language model safety while quantization does not, with degradation predictable and repairable via label-free realignment.
hub
ShieldGemma: Generative AI Content Moderation Based on Gemma
53 Pith papers cite this work, alongside 2 external citations. Polarity classification is still indexing.
53
Pith papers citing it
2
external citations · Pith
abstract
We present ShieldGemma, a comprehensive suite of LLM-based safety content moderation models built upon Gemma2. These models provide robust, state-of-the-art predictions of safety risks across key harm types (sexually explicit, dangerous content, harassment, hate speech) in both user input and LLM-generated output. By evaluating on both public and internal benchmarks, we demonstrate superior performance compared to existing models, such as Llama Guard (+10.8\% AU-PRC on public benchmarks) and WildCard (+4.3\%). Additionally, we present a novel LLM-based data curation pipeline, adaptable to a variety of safety-related tasks and beyond. We have shown strong generalization performance for model trained mainly on synthetic data. By releasing ShieldGemma, we provide a valuable resource to the research community, advancing LLM safety and enabling the creation of more effective content moderation solutions for developers.
hub tools
citation-role summary
background 2
baseline 1
citation-polarity summary
representative citing papers
GrandGuard supplies the first taxonomy, 10k-example benchmark, and fine-tuned safeguards targeting contextual safety failures unique to older adults using chatbots.
SafePyramid is a three-level benchmark showing frontier LLMs identify all violated rules in only 54.0%, 35.3%, and 12.9% of cases on L0, L1, and L2 respectively, indicating in-context policy guardrailing remains difficult.
Relinking is a new compression-boundary attack on LLM agents where summarization of split benign fragments produces malicious instructions, shown via Relink tool at 86.9% success rate and mitigated by KBRA defense to 0%.
A reliable-to-expressive curriculum with dynamic rubrics trains a 12B safety judge to achieve 94%+ accuracy with only 0.76 cross-rubric variance on three different rubric prompts.
PreActBench is a new benchmark showing that LLMs struggle to predict unethical outcomes from partial action trajectories across five domains using the Prefix Foresight F1 metric.
ToxiAlert-Bench dataset and dual-head neural network detect toxic speech by distinguishing textual versus paralinguistic sources, reporting 21.1% Macro-F1 and 13% accuracy gains over baselines.
A 4B LLM safety guardrail trained with reasoning supervision but deployed with reasoning-free inference outperforms 8B baselines on safety benchmarks.
IndicGuard provides a culturally nuanced safety dataset for ten Indic languages and a fine-tuned Gemma-3-4B-IT model that outperforms CultureGuard on moderation tasks and generalizes to unseen low-resource languages.
RELIANCE is a new expert-annotated dataset of TikTok reproductive health content paired with LLM fact-checking evaluations showing 60% accuracy in sampled videos and a 15% gap between claim and full-content assessment.
SafeSteer restricts reverse KL penalty to safety tokens selected via activation steering, achieving strong safety on seven benchmarks with minimal degradation on five capability benchmarks using only 100 harmful samples and no general data.
SentGuard achieves 90.5% detection of unsafe cases within two sentences at 7.41% false positive rate by operating at sentence boundaries during LLM streaming generation.
TRACE introduces a trajectory-level compression method using a Compressor-Reader pair that improves safety detection accuracy by up to 12.6 percentage points on ASSEBench, Pre-Ex-Bench, and R-Judge while degrading less on longer contexts.
Introduces GuardZoo benchmark and RouteGuard router-expert system showing monolithic guardrails suffer task interference while specialized routing improves threat detection and generalization.
MedHarm benchmark shows aligned LLMs and guardrails can still produce unsafe responses on high-risk medical queries, indicating medical safety requires domain-specific testing.
A boundary-targeted MIA strategy recovers 19% of distress-flagged conversations from a safety classifier at 5% false-positive rate, 3.5 times better than prior methods.
Introduces MOOD benchmark for OOD LLM alignment failures and shows guard models plus Mahalanobis and perplexity OOD detectors improve recall from 39% to 45% with positive scaling.
A participatory red-teaming project in the Global South created the PLACES dataset of 26k T2I failure examples that reveal unique cultural and linguistic harms missed by existing safety frameworks.
The paper introduces a dynamical model that decomposes alignment updates in LLM fine-tuning into rebound and driving forces and predicts a rehearsal priming effect.
VerifyMAS improves failure attribution in LLM multi-agent systems via hypothesis verification on full trajectories, error taxonomy-based data construction, and fine-tuned verifier models, outperforming prior direct-prediction methods on Aegis-Bench and Who&When.
LPG compresses policy deliberation into 10 latent tokens to reach 84.5% safety accuracy and 11x speedup over explicit reasoning baselines on guardrail benchmarks.
Generative AI enables scalable, context-aware spear phishing by extracting profiles from public social media, producing emails that outperform real-world phishing samples in personalization and lower recipient suspicion.
GLiGuard is a compact schema-conditioned bidirectional encoder that matches 7B-27B guard models on safety benchmarks while delivering up to 16x higher throughput and 17x lower latency.
An ablation study of an RL-based jailbreaker finds the attack succeeds across tested open-weight models and safeguards, but its headline conclusion about dense rewards and long episodes is contradicted by its own data.
citing papers explorer
-
Do Activation Monitors Survive Model Updates? Benchmarking, Predicting, and Repairing Activation-Monitor Staleness
Fine-tuning updates frequently stale activation monitors for language model safety while quantization does not, with degradation predictable and repairable via label-free realignment.
-
GrandGuard: Taxonomy, Benchmark, and Safeguards for Elderly-Chatbot Interaction Safety
GrandGuard supplies the first taxonomy, 10k-example benchmark, and fine-tuned safeguards targeting contextual safety failures unique to older adults using chatbots.
-
SafePyramid: A Hierarchical Benchmark for In-context Policy Guardrailing
SafePyramid is a three-level benchmark showing frontier LLMs identify all violated rules in only 54.0%, 35.3%, and 12.9% of cases on L0, L1, and L2 respectively, indicating in-context policy guardrailing remains difficult.
-
Safe to Check, Unsafe to Use: Relinking at the Compression Boundary of LLM Agents
Relinking is a new compression-boundary attack on LLM agents where summarization of split benign fragments produces malicious instructions, shown via Relink tool at 86.9% success rate and mitigated by KBRA defense to 0%.
-
Reliable to Expressive: A Curriculum for Rubric-Following Safety Judges
A reliable-to-expressive curriculum with dynamic rubrics trains a 12B safety judge to achieve 94%+ accuracy with only 0.76 cross-rubric variance on three different rubric prompts.
-
PreAct-Bench: Benchmarking Predictive Monitoring in LLMs
PreActBench is a new benchmark showing that LLMs struggle to predict unethical outcomes from partial action trajectories across five domains using the Prefix Foresight F1 metric.
-
Beyond Content: A Comprehensive Speech Toxicity Dataset and Detection Framework Incorporating Paralinguistic Cues
ToxiAlert-Bench dataset and dual-head neural network detect toxic speech by distinguishing textual versus paralinguistic sources, reporting 21.1% Macro-F1 and 13% accuracy gains over baselines.
-
DT-Guard: Intent-Driven Reasoning-Active Training for Reasoning-Free LLM Safety Guardrail
A 4B LLM safety guardrail trained with reasoning supervision but deployed with reasoning-free inference outperforms 8B baselines on safety benchmarks.
-
IndicGuard: A Multilingual Safety Guard Model and Dataset for Indic Languages
IndicGuard provides a culturally nuanced safety dataset for ten Indic languages and a fine-tuned Gemma-3-4B-IT model that outperforms CultureGuard on moderation tasks and generalizes to unseen low-resource languages.
-
RELIANCE: Curating and Evaluating Reproductive Health Information on Social Media
RELIANCE is a new expert-annotated dataset of TikTok reproductive health content paired with LLM fact-checking evaluations showing 60% accuracy in sampled videos and a 15% gap between claim and full-content assessment.
-
SafeSteer: Localized On-Policy Distillation for Efficient Safety Alignment
SafeSteer restricts reverse KL penalty to safety tokens selected via activation steering, achieving strong safety on seven benchmarks with minimal degradation on five capability benchmarks using only 100 harmful samples and no general data.
-
SentGuard: Sentence-Level Streaming Guardrails for Large Language Models
SentGuard achieves 90.5% detection of unsafe cases within two sentences at 7.41% false positive rate by operating at sentence boundaries during LLM streaming generation.
-
TRACE: Trajectory Risk-Aware Compression for Long-Horizon Agent Safety
TRACE introduces a trajectory-level compression method using a Compressor-Reader pair that improves safety detection accuracy by up to 12.6 percentage points on ASSEBench, Pre-Ex-Bench, and R-Judge while degrading less on longer contexts.
-
Triaging Threats to Specialized Guardrails
Introduces GuardZoo benchmark and RouteGuard router-expert system showing monolithic guardrails suffer task interference while specialized routing improves threat detection and generalization.
-
When Medical Safety Alignment Fails: A Benchmark for Evaluating LLMs on High-Risk Medical Queries
MedHarm benchmark shows aligned LLMs and guardrails can still produce unsafe responses on high-risk medical queries, indicating medical safety requires domain-specific testing.
-
Boundary-targeted Membership Inference Attacks on Safety Classifiers
A boundary-targeted MIA strategy recovers 19% of distress-flagged conversations from a safety classifier at 5% false-positive rate, 3.5 times better than prior methods.
-
Benchmarking and Improving Monitors for Out-Of-Distribution Alignment Failure in LLMs
Introduces MOOD benchmark for OOD LLM alignment failures and shows guard models plus Mahalanobis and perplexity OOD detectors improve recall from 39% to 45% with positive scaling.
-
Going PLACES: Participatory Localized Red Teaming for Text-to-Image Safety in the Global South
A participatory red-teaming project in the Global South created the PLACES dataset of 26k T2I failure examples that reveal unique cultural and linguistic harms missed by existing safety frameworks.
-
Alignment Dynamics in LLM Fine-Tuning
The paper introduces a dynamical model that decomposes alignment updates in LLM fine-tuning into rebound and driving forces and predicts a rehearsal priming effect.
-
VerifyMAS: Hypothesis Verification for Failure Attribution in LLM Multi-Agent Systems
VerifyMAS improves failure attribution in LLM multi-agent systems via hypothesis verification on full trajectories, error taxonomy-based data construction, and fine-tuned verifier models, outperforming prior direct-prediction methods on Aegis-Bench and Who&When.
-
LPG: Balancing Efficiency and Policy Reasoning in Latent Policy Guardrails
LPG compresses policy deliberation into 10 latent tokens to reach 84.5% safety accuracy and 11x speedup over explicit reasoning baselines on guardrail benchmarks.
-
Context-Aware Spear Phishing: Generative AI-Enabled Attacks Against Individuals via Public Social Media Data
Generative AI enables scalable, context-aware spear phishing by extracting profiles from public social media, producing emails that outperform real-world phishing samples in personalization and lower recipient suspicion.
-
GLiGuard: Schema-Conditioned Classification for LLM Safeguard
GLiGuard is a compact schema-conditioned bidirectional encoder that matches 7B-27B guard models on safety benchmarks while delivering up to 16x higher throughput and 17x lower latency.
-
A Systematic Investigation of RL-Jailbreaking in LLMs
An ablation study of an RL-based jailbreaker finds the attack succeeds across tested open-weight models and safeguards, but its headline conclusion about dense rewards and long episodes is contradicted by its own data.
-
Revisiting JBShield: Breaking and Rebuilding Representation-Level Jailbreak Defenses
JBShield is vulnerable to adaptive JB-GCG attacks (up to 53% ASR) because jailbreak representations occupy a distinct region in refusal-direction space; the new RTV defense using Mahalanobis detection on multi-layer fingerprints reaches 0.99 AUROC and limits adaptive ASR to 7%.
-
Harmful Intent as a Geometrically Recoverable Feature of LLM Residual Streams
Harmful intent is linearly separable in LLM residual streams across 12 models and multiple architectures, reaching mean AUROC 0.982 while showing protocol-dependent directions and strong generalization to held-out harm benchmarks.
-
LLM Safety From Within: Detecting Harmful Content with Internal Representations
SIREN identifies safety neurons via linear probing on internal LLM layers and combines them with adaptive weighting to detect harm, outperforming prior guard models with 250x fewer parameters.
-
A Lightweight Explainable Guardrail for Prompt Safety
LEG is a compact model that jointly classifies unsafe prompts and explains its decisions using synthetic training data and a custom uncertainty-weighted loss.
-
Disentangled Safety Adapters Enable Efficient Guardrails and Flexible Inference-Time Alignment
Disentangled Safety Adapters decouple safety computations from task-optimized LLMs via lightweight adapters, yielding up to 53% better AUC on safety tasks and dynamic inference-time alignment with reduced performance trade-offs.
-
Peering Behind the Shield: Guardrail Identification in Large Language Models
AP-Test identifies deployed guardrails in LLMs via adversarial prompt testing and a match score metric, reporting perfect accuracy on four open-source guardrails.
-
HaloGuard 1.0: An Open Weights Constitutional Classifier for Multilingual AI Safety
HaloGuard 1.0-0.8B achieves the highest average F1 of 90.9 across seven prompt-safety benchmarks among evaluated open guard models while keeping FPR at 4.3 and FNR at 9.5, with a 4B variant reaching 92.1 F1.
-
Cognitive Firewall: A Proactive, Zero-Trust, Multi-Gate Framework for LLM Safety
Cognitive Firewall applies four gates (intent, zero-trust context, consistency, output risk) via an oversight model to cut jailbreak success to 2% or below on most tested sets while keeping over-refusal at 8%.
-
Defending Against Harmful Supervision Hidden in Benign Samples
The paper proposes Dual-Reference SFT (DR-SFT) to defend LLMs against harmful QA pairs embedded in benign training samples, where existing guardrails fail at the example level.
-
Do Safety Guardrails Need to Reason? LeanGuard: A Fast and Light Approach for Robust Moderation
A 395M label-only bidirectional encoder achieves 82.90 average F1 on moderation benchmarks without reasoning, matching larger reasoning decoders at ~100x lower inference cost and with better robustness properties.
-
Yuvion VL: A Multimodal Foundation Model for Adversarial Content and AI Safety
Yuvion VL is a multimodal LLM family using adversarial-aware data construction, three-stage training, and contrastive fine-tuning that claims industry-leading safety performance on new benchmarks while retaining general capabilities.
-
ConsisGuard: Aligning Safety Deliberation with Policy Enforcement in LLM Guardrails
ConsisGuard is a consistency-aware framework that applies Policy-to-Decision Trajectory Distillation and Functional Coupling Alignment to improve policy execution consistency in reasoning-based LLM guardrails on harmfulness detection tasks.
-
HARP: Measuring Harm Amplification in Multi-Agent LLM Systems
HARP defines and measures harm amplification as the ratio of global to local deviation in multi-agent LLM traces, instantiated in a seven-agent finance system to compare attacks and defenses.
-
Reflect-Guard: Enhancing LLM Safeguards against Adversarial Prompts via Logical Self-Reflection
Reflect-Guard fine-tunes Llama-Guard-3-8B with distilled self-reflections to raise F1 on WildGuardTest from 0.770 to 0.842 and cut JailbreakBench attack success from 10.3% to 1.8%.
-
CR4T: Rewrite-Based Guardrails for Adolescent LLM Safety
CR4T is a model-agnostic framework using lightweight risk detection and domain-conditioned rewriting to convert unsafe or refusal-style LLM responses into developmentally appropriate guidance for adolescents.
-
SafeLens: Deliberate and Efficient Video Guardrails with Fast-and-Slow Screening
SafeLens presents a fast-and-slow video guardrail framework that filters the SafeWatch dataset to 2.4% and adds Chain-of-Thought traces to achieve state-of-the-art moderation performance at reduced inference cost.
-
LiSA: Lifelong Safety Adaptation via Conservative Policy Induction
LiSA improves AI guardrails lifelong by inducing conservative policies from sparse noisy failure reports via structured memory, conflict-aware rules, and posterior lower-bound gating.
-
Rewriting the Response Path: Silent Tampering and Provider-Signed Defense in BYOK LLM Agents
A malicious BYOK relay can rewrite an LLM agent's execution-bearing response fields after safety alignment, achieving 73.5-99.1% attack success on agent benchmarks while bypassing model defenses.
-
Guardian-as-an-Advisor: Advancing Next-Generation Guardian Models for Trustworthy LLMs
Guardian-as-an-Advisor prepends risk labels and explanations from a guardian model to queries, improving LLM safety compliance and reducing over-refusal while adding minimal compute overhead.
-
DRAFT: Task Decoupled Latent Reasoning for Agent Safety
DRAFT decouples agent safety judgment into latent extraction and reasoning stages, raising average benchmark accuracy from 63.27% to 91.18%.
-
Bielik Guard: Efficient Polish Language Safety Classifiers for LLM Content Moderation
Bielik Guard delivers compact Polish safety classifiers with F1 scores near 0.79 and superior real-prompt precision over baselines.
-
Verifying Intent and Harm: A Unified Defense Against LLM-Generated Threats
A joint prompt-response verification framework using intent analysts, harm analysts, and a judge improves average F1 to 0.95 and cuts attack success rate to 4.1% across jailbreaks, prompt injection, phishing, cyber abuse, and harmful content.
-
Mod-Guide: An LLM-based Content Moderation Feedback System to Address Insensitive Speech toward Indigenous Ethnic and Religious Minority Communities
Mod-Guide uses RAG with a community co-created corpus to make LLM moderation responses more contextually accurate for insensitive speech toward Bangladesh's Hindu and Chakma minorities, with mixed-method evaluation showing differences by ethnic background.
-
GLiNER Guard: Unified Encoder Family for Production LLM Safety and Privacy
GLiNER Guard provides unified encoder variants for LLM safety and PII detection in a single pass, with high throughput on A100 hardware and a new PII-Bench benchmark.
-
One Jailbreak, Many Tongues: Learning Language-Insensitive Intention Representations for Multilingual Jailbreak Detection
MLJailDe achieves 98.5% F1 on multilingual jailbreak detection by combining back-translation data augmentation, supervised contrastive loss, and imbalance-aware classification on a DeBERTa backbone.
-
TWGuard: A Case Study of LLM Safety Guardrails for Localized Linguistic Contexts
TWGuard achieves +0.289 F1 improvement and 94.9% false-positive reduction for LLM safety guardrails in the Taiwan linguistic context compared to foundation models and baselines.