Pith. sign in

REVIEW 22 cited by

SafeBench: A Safety Evaluation Framework for Multimodal Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.18927 v1 pith:Z6WKJI5S submitted 2024-10-24 cs.CR

Zonghao Ying , Aishan Liu , Siyuan Liang , Lei Huang , Jinyang Guo , Wenbo Zhou , Xianglong Liu , Dacheng Tao This is my paper · ORCID
classification cs.CR
keywords safetymllmsevaluationharmfulframeworkmodelsquerybenchmarks
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Multimodal Large Language Models (MLLMs) are showing strong safety concerns (e.g., generating harmful outputs for users), which motivates the development of safety evaluation benchmarks. However, we observe that existing safety benchmarks for MLLMs show limitations in query quality and evaluation reliability limiting the detection of model safety implications as MLLMs continue to evolve. In this paper, we propose \toolns, a comprehensive framework designed for conducting safety evaluations of MLLMs. Our framework consists of a comprehensive harmful query dataset and an automated evaluation protocol that aims to address the above limitations, respectively. We first design an automatic safety dataset generation pipeline, where we employ a set of LLM judges to recognize and categorize the risk scenarios that are most harmful and diverse for MLLMs; based on the taxonomy, we further ask these judges to generate high-quality harmful queries accordingly resulting in 23 risk scenarios with 2,300 multi-modal harmful query pairs. During safety evaluation, we draw inspiration from the jury system in judicial proceedings and pioneer the jury deliberation evaluation protocol that adopts collaborative LLMs to evaluate whether target models exhibit specific harmful behaviors, providing a reliable and unbiased assessment of content security risks. In addition, our benchmark can also be extended to the audio modality showing high scalability and potential. Based on our framework, we conducted large-scale experiments on 15 widely-used open-source MLLMs and 6 commercial MLLMs (e.g., GPT-4o, Gemini), where we revealed widespread safety issues in existing MLLMs and instantiated several insights on MLLM safety performance such as image quality and parameter size.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 22 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MDIT-Bench: Evaluating the Dual-Implicit Toxicity in Large Multimodal Models

    cs.CL 2025-05 conditional novelty 7.0 of 10

    The authors release a large multimodal benchmark showing that current LMMs struggle to detect toxicity that emerges only from combining image and text, and that many-shot toxic demonstrations further reduce their accuracy.

  2. OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows

    cs.AI 2025-10 conditional novelty 6.0 of 10

    A hybrid rule-plus-VLM detector for mobile GUI agents, tested on a new 204-trajectory Android benchmark, reports 10-30% gains over baselines.

  3. Mask-GCG: Are All Tokens in Adversarial Suffixes Necessary for Jailbreak Attacks?

    cs.CL 2025-09 conditional novelty 6.0 of 10

    Mask-GCG uses learnable masks to prune a minority of low-impact tokens from GCG attack suffixes, slightly improving speed while showing most tokens are necessary.

  4. LLM Serving Optimization with Variable Prefill and Decode Lengths

    math.OC 2025-08 reject novelty 6.0 of 10

    The abstract claims a constant-factor approximation algorithm for LLM serving scheduling, but the full text is an unrelated safety-benchmark paper, leaving the claimed result without any derivation or experiments.

  5. Secure Tug-of-War (SecTOW): Iterative Defense-Attack Training with Reinforcement Learning for Multimodal Model Security

    cs.CR 2025-07 conditional novelty 6.0 of 10

    An iterative attacker-defender reinforcement learning method that makes a multimodal LLM refuse more jailbreak prompts without over-refusing ordinary queries.

  6. Hatevolution: What Static Benchmarks Don't Tell Us

    cs.CL 2025-06 conditional novelty 6.0 of 10

    Static hate speech benchmarks rank models differently from time-sensitive evaluations, with correlation coefficients near zero or negative, so high benchmark scores do not guarantee robustness to language change.

  7. USB: A Comprehensive and Unified Safety Evaluation Benchmark for Multimodal Large Language Models

    cs.CR 2025-05 conditional novelty 6.0 of 10

    USB-SafeBench is a unified MLLM safety benchmark with 61 risk categories, 4 modality combinations, and dual-language vulnerability and oversensitivity tests.

  8. Hierarchical Safety Realignment: Lightweight Restoration of Safety in Pruned Large Vision-Language Models

    cs.CL 2025-05 conditional novelty 6.0 of 10

    HSR restores safety in pruned vision-language models by selectively restoring safety-critical neurons inside the attention heads that matter most for safety.

  9. MORALISE: A Structured Benchmark for Moral Alignment in Visual Language Models

    cs.CV 2025-05 conditional novelty 6.0 of 10

    MORALISE is a new benchmark of 2,481 expert-annotated real image-text pairs spanning 13 moral topics, and 19 vision-language models score far worse on identifying the violated norm than on judging whether a violation ...

  10. Manipulating Multimodal Agents via Cross-Modal Prompt Injection

    cs.CV 2025-04 conditional novelty 6.0 of 10

    A coordinated attack that embeds malicious cues in both visual and textual inputs can hijack black-box multimodal agents, outperforming single-modality prompt injection attacks.

  11. Evidence Recomposition and Predictive Context Residualization for Visual Attribution in Multimodal Large Language Models

    cs.CV 2025-09 conditional novelty 5.0 of 10

    MSEA+ARC, a multi-scale and ranking-based residualization method, claims consistent F1-IoU gains over TAM for token-level MLLM visual attribution.

  12. A Novel Evaluation Benchmark for Medical LLMs: Illuminating Safety and Effectiveness in Clinical Domains

    cs.CL 2025-07 conditional novelty 5.0 of 10

    A new 2,069-item, 30-criterion benchmark of clinical scenarios shows six LLMs average 57.2%, with safety (54.7%) below effectiveness (62.3%) and a 13.3% drop in high-risk cases.

  13. Three Minds, One Legend: Jailbreak Large Reasoning Model with Adaptive Stacked Ciphers

    cs.CL 2025-05 conditional novelty 5.0 of 10

    A stacked-cipher jailbreak with adaptive code selection achieves 80.8% to 100% attack success on commercial large reasoning models.

  14. POISONCRAFT: Practical Poisoning of Retrieval-Augmented Generation for Large Language Models

    cs.CR 2025-05 conditional novelty 5.0 of 10

    POISONCRAFT injects adversarial documents into a RAG knowledge base that are likely to be retrieved for arbitrary user queries and that steer the language model into recommending a fake URL.

  15. o3-mini vs DeepSeek-R1: Which One is Safer?

    cs.SE 2025-01 conditional novelty 5.0 of 10

    DeepSeek-R1 (70B) produced unsafe responses to 11.98% of 1,260 unsafe test prompts, while OpenAI's o3-mini beta produced 1.19%, though the comparison is system-level due to API guardrails.

  16. Early External Safety Testing of OpenAI's o3-mini: Insights from the Pre-Deployment Evaluation

    cs.SE 2025-01 conditional novelty 5.0 of 10

    External testers generated 10,080 unsafe prompts against OpenAI's o3-mini beta, manually confirmed 87 unsafe behaviors, and found most protection came from an API-level policy filter rather than the model itself.

  17. PRJ: Perception-Retrieval-Judgement for Generated Images

    cs.CV 2025-06 reject novelty 4.0 of 10

    A new safety checker for AI-generated images, built from a vision-language model, retrieval-augmented knowledge lookup, and an LLM judge, reports higher detection rates and category-level toxicity scores than three ex...

  18. Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks

    cs.CL 2025-04 conditional novelty 4.0 of 10

    A survey that frames the core problem of LLM evaluation as 'evaluation generalization': finite test sets cannot scale with unbounded model capabilities, and proposes two transitions in evaluation design.

  19. A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluations

    cs.CR 2025-02 conditional novelty 4.0 of 10

    A survey of LVLM safety that adds a lifecycle taxonomy and new benchmark results showing Janus-Pro-7B has weaker safety than several open-source LVLMs.

  20. Universal Adversarial Attack on Aligned Multimodal LLMs

    cs.AI 2025-02 conditional novelty 4.0 of 10

    A single optimized image, trained through the vision and language modules, makes aligned multimodal LLMs produce dangerous responses across diverse prompts and some models.

  21. Pushing the Limits of Safety: A Technical Report on the ATLAS Challenge 2025

    cs.CR 2025-06 conditional novelty 3.0 of 10

    The ATLAS 2025 competition demonstrates that vision-language models remain highly vulnerable to flowchart-based and cross-modal jailbreak attacks, with top scores exceeding 93%.

  22. A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges

    cs.CV 2025-01 reject novelty 2.0 of 10

    A survey that catalogs large vision-language models, their alignment methods, benchmarks, and challenges, but is compromised by inconsistent counts and misclassified entries.

Pith tools