Pith. sign in

REVIEW 15 cited by

LlavaGuard: An Open VLM-based Framework for Safeguarding Vision Datasets and Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2406.05113 v3 pith:W3KFXE7K submitted 2024-06-07 cs.CV cs.AIcs.LG

Lukas Helff , Felix Friedrich , Manuel Brack , Kristian Kersting , Patrick Schramowski This is my paper · ORCID
classification cs.CVcs.AIcs.LG
keywords llavaguardsafetymodelsdatasetframeworkdatalarge-scaleopen
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

This paper introduces LlavaGuard, a suite of VLM-based vision safeguards that address the critical need for reliable guardrails in the era of large-scale data and models. To this end, we establish a novel open framework, describing a customizable safety taxonomy, data preprocessing, augmentation, and training setup. For teaching a VLM safeguard on safety, we further create a multimodal safety dataset with high-quality human expert annotations, where each image is labeled with a safety rating, category, and rationale. We also employ advanced augmentations to support context-specific assessments. The resulting LlavaGuard models, ranging from 0.5B to 7B, serve as a versatile tool for evaluating the safety compliance of visual content against flexible policies. In comprehensive experiments, LlavaGuard outperforms both state-of-the-art safeguards and VLMs in accuracy and in flexibly handling different policies. Additionally, we demonstrate LlavaGuard's performance in two real-world applications: large-scale dataset annotation and moderation of text-to-image models. We make our entire framework, including the dataset, model weights, and training code.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 15 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Hate in Plain Sight: On the Risks of Moderating AI-Generated Hateful Illusions

    cs.CR 2025-07 conditional novelty 7.0 of 10

    Hateful optical illusions generated with Stable Diffusion and ControlNet evade current moderation classifiers (best accuracy 0.245) and vision-language models (best accuracy 0.102), with simple image transformations s...

  2. MSTS: A Multimodal Safety Test Suite for Vision-Language Models

    cs.CL 2025-01 conditional novelty 7.0 of 10

    MSTS is a 400-prompt multimodal safety benchmark showing that open vision-language models give unsafe responses to up to 14% of prompts, and that image-plus-text inputs trigger more unsafe answers than text alone.

  3. Multimodal Reward Hacking in Reinforcement Learning

    cs.AI 2026-07 conditional novelty 6.5 of 10

    Imperfect multimodal RL rewards systematically create new failures (NRFR > RHR); scaling and answer-aware rewards help but do not eliminate hacking, and unreliable visual verifiers actively increase it.

  4. Old Tricks, New Models: How Simple Image Transformations Break Modern AI-based Content Moderation

    cs.AI 2026-07 conditional novelty 6.0 of 10

    Seven model-agnostic image transformations bypass OpenAI, Amazon, and Google image-moderation APIs, including under non-trivial perceptual-similarity constraints.

  5. Harm is not Universal: Community-Specific Toxicity Detection is Urgently Needed

    cs.CV 2026-07 conditional novelty 6.0 of 10

    SoTA T2I toxicity detectors miss ~35% of disability-community harms; zero-shot CTD fails below random, while ICL/VQA/LoRA improve but stay well below general TD performance.

  6. PolicyShiftGuard: Benchmarking and Improving Policy-Adaptive Image Guardrails

    cs.CV 2026-07 conditional novelty 6.0 of 10

    PolicyShiftGuard, trained with randomized policy SFT plus matched pass/block boundary pairs, reaches 76.9 Avg. F1 and 72.1 Avg. PSS on a new policy-shift image-guardrail benchmark where existing VLMs stay brittle.

  7. Multimodal LLMs as Customized Reward Models for Text-to-Image Generation

    cs.CV 2025-07 conditional novelty 6.0 of 10

    LLaVA-Reward extracts reward scores from the hidden states of a multimodal LLM with a skip-connection cross-attention head, and reports state-of-the-art text-to-image evaluation across alignment, fidelity, and safety.

  8. Bridging the Gap in Vision Language Models in Identifying Unsafe Concepts Across Modalities

    cs.CR 2025-07 conditional novelty 6.0 of 10

    Vision-language models consistently recognize unsafe content better from text than from images, and a simplified reinforcement learning fine-tune narrows that gap.

  9. T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generation

    cs.CL 2025-01 conditional novelty 6.0 of 10

    T2ISafety is a large annotated benchmark plus a fine-tuned MLLM evaluator (ImageGuard) for measuring toxicity, privacy, and fairness in text-to-image models.

  10. Automatic Hard Example Synthesis with Multi-Level Agentic Data Curation

    cs.AI 2026-07 conditional novelty 5.0 of 10

    Test-time retrieval of committee-disagreement-mined synthetic images cuts a safety classifier's false-negative rate on a hard HoliSafe subset from 41.2% to 24.5%.

  11. DREAM: Disentangling Risks to Enhance Safety Alignment in Multimodal Large Language Models

    cs.CL 2025-04 conditional novelty 5.0 of 10

    Multimodal risk disentanglement, where the model breaks down threats from images and text separately, improves MLLM safety at inference and during fine-tuning.

  12. AIN: The Arabic INclusive Large Multimodal Model

    cs.CV 2025-01 conditional novelty 5.0 of 10

    AIN, a 7B-parameter Arabic English multimodal model fine-tuned from Qwen2-VL on 3.6M samples, reports state-of-the-art Arabic scores including a 3.4-point average gain over GPT-4o on CAMEL-Bench.

  13. Understanding and Mitigating Toxicity in Image-Text Pretraining Datasets: A Case Study on LLaVA

    cs.CV 2025-05 conditional novelty 4.0 of 10

    A pipeline combining LlavaGuard, Command R+, and Toxic-BERT flags and removes 7,531 image-text pairs from the 558,000-pair LLaVA pretraining dataset.

  14. A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluations

    cs.CR 2025-02 conditional novelty 4.0 of 10

    A survey of LVLM safety that adds a lifecycle taxonomy and new benchmark results showing Janus-Pro-7B has weaker safety than several open-source LVLMs.

  15. Maya: An Instruction Finetuned Multilingual Multimodal Model

    cs.CV 2024-12 conditional novelty 4.0 of 10

    Maya, an 8B multilingual multimodal model built on Aya-23 and SigLIP, shows small gains over PALO-7B on a PALO-based benchmark after finetuning on PALO instruction data.

Pith tools