{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2025:P3RYV5VCGDVYSZWQHTFYJSWTAC","short_pith_number":"pith:P3RYV5VC","schema_version":"1.0","canonical_sha256":"7ee38af6a230eb8966d03ccb84cad300b4e45ef792f46c1e2699334331d27b7d","source":{"kind":"arxiv","id":"2501.10057","version":1},"attestation_state":"computed","paper":{"title":"MSTS: A Multimodal Safety Test Suite for Vision-Language Models","license":"http://creativecommons.org/licenses/by/4.0/","headline":"","cross_cats":[],"primary_cat":"cs.CL","authors_text":"Alicia Parrish, Anastassia Shaitarova, Andrea Zugarini, Bertie Vidgen, Chiara Di Bonaventura, Dirk Hovy, Donya Rooein, Felix Friedrich, Flor Miriam Plaza-del-Arco, Gaia El Khoury Geagea, Giuseppe Attanasio, Janis Goldzycher, Jieun Han, Paloma Jereti\\v{c}, Patrick Schramowski, Paul R\\\"ottger, Richard Willats, Rishabh Bhardwaj, Roman Eng, Seogyeong Jeong, Sujata Goswami, Xudong Shen","submitted_at":"2025-01-17T09:22:35Z","abstract_excerpt":"Vision-language models (VLMs), which process image and text inputs, are increasingly integrated into chat assistants and other consumer AI applications. Without proper safeguards, however, VLMs may give harmful advice (e.g. how to self-harm) or encourage unsafe behaviours (e.g. to consume drugs). Despite these clear hazards, little work so far has evaluated VLM safety and the novel risks created by multimodal inputs. To address this gap, we introduce MSTS, a Multimodal Safety Test Suite for VLMs. MSTS comprises 400 test prompts across 40 fine-grained hazard categories. Each test prompt consist"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":false,"formal_links_present":false},"canonical_record":{"source":{"id":"2501.10057","kind":"arxiv","version":1},"metadata":{"license":"http://creativecommons.org/licenses/by/4.0/","primary_cat":"cs.CL","submitted_at":"2025-01-17T09:22:35Z","cross_cats_sorted":[],"title_canon_sha256":"99316fa4d1c9ab668b9aeae86a03c7ea9456b48a19adbe5e17d413ce80f2b077","abstract_canon_sha256":"ac2e4f36340adcf275e51582f8b019057537c9bf5fddbeb31439d8fa0456ac9f"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-07-05T10:02:15.989161Z","signature_b64":"Tk3s/i2nC4lx1hr4ZA+Wm9S2Ftb7SyuiXCOkeT2pt7jyXVsFvdRcVKLJpRpqk2MnAYlUQ4znJ+eDxH+knqR5BA==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"7ee38af6a230eb8966d03ccb84cad300b4e45ef792f46c1e2699334331d27b7d","last_reissued_at":"2026-07-05T10:02:15.988707Z","signature_status":"signed_v1","first_computed_at":"2026-07-05T10:02:15.988707Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"MSTS: A Multimodal Safety Test Suite for Vision-Language Models","license":"http://creativecommons.org/licenses/by/4.0/","headline":"","cross_cats":[],"primary_cat":"cs.CL","authors_text":"Alicia Parrish, Anastassia Shaitarova, Andrea Zugarini, Bertie Vidgen, Chiara Di Bonaventura, Dirk Hovy, Donya Rooein, Felix Friedrich, Flor Miriam Plaza-del-Arco, Gaia El Khoury Geagea, Giuseppe Attanasio, Janis Goldzycher, Jieun Han, Paloma Jereti\\v{c}, Patrick Schramowski, Paul R\\\"ottger, Richard Willats, Rishabh Bhardwaj, Roman Eng, Seogyeong Jeong, Sujata Goswami, Xudong Shen","submitted_at":"2025-01-17T09:22:35Z","abstract_excerpt":"Vision-language models (VLMs), which process image and text inputs, are increasingly integrated into chat assistants and other consumer AI applications. Without proper safeguards, however, VLMs may give harmful advice (e.g. how to self-harm) or encourage unsafe behaviours (e.g. to consume drugs). Despite these clear hazards, little work so far has evaluated VLM safety and the novel risks created by multimodal inputs. To address this gap, we introduce MSTS, a Multimodal Safety Test Suite for VLMs. MSTS comprises 400 test prompts across 40 fine-grained hazard categories. Each test prompt consist"},"claims":{"count":0,"items":[],"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"source":{"id":"2501.10057","kind":"arxiv","version":1},"verdict":{"id":null,"model_set":{},"created_at":null,"strongest_claim":"","one_line_summary":"","pipeline_version":null,"weakest_assumption":"","pith_extraction_headline":""},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2501.10057/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2501.10057","created_at":"2026-07-05T10:02:15.988763+00:00"},{"alias_kind":"arxiv_version","alias_value":"2501.10057v1","created_at":"2026-07-05T10:02:15.988763+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2501.10057","created_at":"2026-07-05T10:02:15.988763+00:00"},{"alias_kind":"pith_short_12","alias_value":"P3RYV5VCGDVY","created_at":"2026-07-05T10:02:15.988763+00:00"},{"alias_kind":"pith_short_16","alias_value":"P3RYV5VCGDVYSZWQ","created_at":"2026-07-05T10:02:15.988763+00:00"},{"alias_kind":"pith_short_8","alias_value":"P3RYV5VC","created_at":"2026-07-05T10:02:15.988763+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":4,"internal_anchor_count":1,"sample":[{"citing_arxiv_id":"2607.06196","citing_title":"Pluralis v0.1: Towards a Multicultural, Multimodal, Multilingual Benchmark for AI Risk and Reliability","ref_index":125,"is_internal_anchor":true},{"citing_arxiv_id":"2606.26968","citing_title":"RedVox: Safety and Fairness Gaps in Speech Models Across Languages","ref_index":147,"is_internal_anchor":false},{"citing_arxiv_id":"2605.28137","citing_title":"No Safe Dose: How Training Data Drives Unsafe Image Generation","ref_index":50,"is_internal_anchor":false},{"citing_arxiv_id":"2605.07760","citing_title":"RuleSafe-VL: Evaluating Rule-Conditioned Decision Reasoning in Vision-Language Content Moderation","ref_index":8,"is_internal_anchor":false}]},"formal_canon":{"evidence_count":0,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/P3RYV5VCGDVYSZWQHTFYJSWTAC","json":"https://pith.science/pith/P3RYV5VCGDVYSZWQHTFYJSWTAC.json","graph_json":"https://pith.science/api/pith-number/P3RYV5VCGDVYSZWQHTFYJSWTAC/graph.json","events_json":"https://pith.science/api/pith-number/P3RYV5VCGDVYSZWQHTFYJSWTAC/events.json","paper":"https://pith.science/paper/P3RYV5VC"},"agent_actions":{"view_html":"https://pith.science/pith/P3RYV5VCGDVYSZWQHTFYJSWTAC","download_json":"https://pith.science/pith/P3RYV5VCGDVYSZWQHTFYJSWTAC.json","view_paper":"https://pith.science/paper/P3RYV5VC","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2501.10057&json=true","fetch_graph":"https://pith.science/api/pith-number/P3RYV5VCGDVYSZWQHTFYJSWTAC/graph.json","fetch_events":"https://pith.science/api/pith-number/P3RYV5VCGDVYSZWQHTFYJSWTAC/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/P3RYV5VCGDVYSZWQHTFYJSWTAC/action/timestamp_anchor","attest_storage":"https://pith.science/pith/P3RYV5VCGDVYSZWQHTFYJSWTAC/action/storage_attestation","attest_author":"https://pith.science/pith/P3RYV5VCGDVYSZWQHTFYJSWTAC/action/author_attestation","sign_citation":"https://pith.science/pith/P3RYV5VCGDVYSZWQHTFYJSWTAC/action/citation_signature","submit_replication":"https://pith.science/pith/P3RYV5VCGDVYSZWQHTFYJSWTAC/action/replication_record"}},"created_at":"2026-07-05T10:02:15.988763+00:00","updated_at":"2026-07-05T10:02:15.988763+00:00"}