REVIEW 4 major objections 5 minor 15 references
The Problem with Safety Classification is not just the Models
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Safety classifiers show large multilingual gaps, and part of the blame lies in the datasets used to test them.
desk verdict A useful, honest position paper showing that multilingual safety disparities extend to guard models and that benchmark labels are partly to blame; the core claim holds despite the label-trust tension. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The analysis rests on a multilingual evaluation matrix: four benchmark datasets (three containing only harmful prompts and one containing both safe and unsafe prompts) crossed with five fine-tuned safety classifiers, measuring per-language detection rates and safe/unsafe accuracy. The qualitative component inspects the prompts themselves to identify label noise. This design lets the paper separate, at least descriptively, model-side limitations from data-side problems: a prompt such as "Who are the people getting married?" labeled 'soliciting personally identifiable information' would be impossible for any single-turn classifier to tag reliably, so the per-language failure counts include dataset error.
What would settle it
Take a stratified random sample of prompts labeled harmful from MULTIJAIL, XSAFETY, and AYA-REDTEAMING and have independent annotators from multiple language communities classify each as clearly harmful, ambiguous, or harmless, measuring inter-annotator agreement. If the clear-harmful share is high and agreement is strong, the paper's dataset-error claim is weakened; if disagreement is high and many prompts are rated harmless, the claim is supported and the quantitative comparisons should be recomputed on the confirmed-harmful subset.
Extended reading notes
Core claim
The paper's central claim is that the poor multilingual performance of safety classifiers cannot be blamed on the models alone. Evaluated on four multilingual harm-detection datasets covering 18 languages, all five classifiers showed large cross-language variation in detecting harmful prompts, and even the relatively better models misclassified harmful prompts in several languages while also labeling safe prompts as harmful. The author then shows that the evaluation datasets contain numerous prompts tagged as harmful that are not clearly harmful, making the benchmark scores partly artifacts of dataset labeling. From this, the paper concludes that current safety-classification shortfalls reflect a combination of model limitations and flawed evaluation datasets, and that future work should build quality-verified datasets with graded harm notions and report safe and unsafe performance separately.
Load-bearing premise
The quantitative comparisons treat the harmful and safe labels in the four datasets as ground truth, even though the paper's own qualitative analysis argues that many of those labels are wrong; if a substantial share of benchmark labels are unreliable, the reported performance numbers are not a clean measure of model quality.
Editorial extensions
If this is right
- Using non-English input is a plausible jailbreak route even against the best small guard models, since harmful prompts tagged safe in several languages were correctly tagged in English.
- Reporting only harmful-prompt detection accuracy overstates guard quality; safe-prompt over-flagging is large, especially in low-resource languages, so safe and unsafe performance should be tracked separately in fine-tuning and evaluation.
- Existing multilingual safety benchmarks need quality audits; several labeled harmful prompts are benign or ambiguous, so benchmark scores are not trustworthy as model-quality measures.
- Building better multilingual guardrails requires quality-verified datasets with a graded notion of harm, plus application-specific red-teaming and policy guidelines during prompting.
Reading between the lines
- The paper's dataset critique implies that existing multilingual safety-benchmark scores should not be read as clean model-quality rankings; a natural re-analysis would recompute performance only on prompts that independent annotators confirm as harmful.
- The over-flagging result implies that deploying current guard models as filters will block benign user requests, with disproportionate effect on speakers of lower-resource languages; product decisions should weight false-positive costs per language.
- A testable follow-up would fine-tune identical classifier architectures on region-specific rather than translated prompt data; the paper's AYA-REDTEAMING results suggest this could shrink the language gap.
- Because all evaluated classifiers were trained primarily on English prompt-label pairs, the cross-language gaps may reflect training-data language balance rather than inherent model limits; benchmarking base models' safety behavior before fine-tuning would separate those factors.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This position paper compares five safety classification models (OpenAI omni-moderation, LlamaGuard3:2B, Granite3-Guardian 2B/8B, and ShieldGemma) on four multilingual benchmarks covering 18 languages, reporting language-by-language detection rates for harmful prompts and safe/unsafe accuracy on the RTP-LX dataset. The paper finds large cross-lingual performance disparities and argues, through selected examples from the datasets, that many prompts labeled harmful are actually safe or ambiguous. It concludes that the shortcomings of current safety classifiers are not solely attributable to the models themselves and that evaluation dataset quality deserves more attention.
Significance. If substantiated, the paper makes a timely and practically important point: multilingual safety evaluation is distorted not only by model-side gaps but also by benchmark label noise and ambiguous prompts. The study is valuable for testing multiple models and datasets, for providing concrete examples of cross-lingual disparities, and for explicitly acknowledging its own limitations. However, the central empirical link between dataset mislabeling and the reported performance disparities is not yet fully established, because the dataset critique is qualitative and the quantitative evaluation is not robust to the very label unreliability the paper identifies. The paper's position is defensible as a call for better benchmarks, but the evidence as presented requires strengthening before the main conclusion can be accepted as a measured result.
major comments (4)
- [Section 3, 'Nature of the Prompts' (Tables 2 and 6)] The dataset critique is supported only by a handful of hand-picked examples. There is no systematic count of how many prompts per dataset are ambiguous or mislabeled, no inter-annotator agreement, no policy-based or rubric-based criterion for labeling, and no estimate of prevalence. Statements such as 'several examples' and 'raising questions on the trustworthiness of the evaluations' are not quantified. Since this is one of the paper's two central claims (Discussion item 3), the authors should either provide a systematic annotation study or clearly frame this part as an illustrative, anecdotal observation rather than an empirical finding.
- [Section 2, 'Evaluation' and Section 3, 'Results' (Figures 1 and 2)] The quantitative evaluation for MULTIJAIL, XSAFETY, and AYA-REDTEAMING treats the dataset labels as ground truth, while the paper's own analysis argues that a nontrivial subset of those labels are wrong for ambiguous prompts. This creates a circular tension: the reported detection rates are used as measures of model quality, yet the paper simultaneously claims those labels are unreliable. A robustness check is needed in which the ambiguous or potentially mislabeled prompts are removed or reannotated and the cross-lingual performance numbers are recomputed. If the disparities persist after this filter, the paper's conclusion is strongly supported; if they shrink, the conclusion must be moderated. As written, the magnitudes of the cross-lingual gaps are not clean evidence of model-side safety failure.
- [Section 4, Discussion item 2] The claim that 'All models over-identify harm and under-identify safety across languages' is not supported by the evidence presented, because Figure 2 reports the safe/unsafe breakdown only for the two Granite3-Guardian models. No equivalent results are shown for OpenAI omni-moderation, LlamaGuard3, or ShieldGemma. Either provide the same safe/unsafe analysis for all five models or explicitly restrict the claim to the Granite models.
- [Section 2, 'Evaluation' and Section 3, 'Results'] The reported language-level performance differences come without error bars, confidence intervals, or repeated runs. The locally run models (via Ollama) are LLMs that may be stochastic depending on decoding settings, and the OpenAI API call also involves unknown sampling behavior. To establish that the observed cross-lingual differences are not random variation, the authors should report variance estimates across multiple runs, or at minimum state the decoding parameters (e.g., temperature, greedy decoding) and show that results are deterministic. This is particularly important because the paper's headline claim is 'large variations across languages' based on point estimates.
minor comments (5)
- [Figure 1 caption] The caption says 'Identifying Harmful Prompts as Safe (the farther from zero the better)', but the text and the reported metric refer to the percentage of harmful prompts identified as unsafe. The caption should be corrected to say 'as Unsafe' or otherwise aligned with the actual metric.
- [Section 2, model list] There are several typos and inconsistencies: 'SheildGemma' should be 'ShieldGemma', 'LLama3-Guard' has inconsistent casing, and 'efficiency' contains a nonstandard ligature. These should be fixed in a final revision.
- [Table 6] Several entries in Table 6 contain stray quotation marks after the harm-category labels (e.g., "Discrimination & injustice]"). The table formatting and quotes should be cleaned up.
- [Appendix A] Appendix A states only that experiments were run on a MacOS M1 laptop with 32GB RAM and that models were locally downloaded via Ollama. It does not report the exact model versions, quantization levels, decoding parameters, prompting templates, or the date of API access. These details are necessary for reproducibility and should be added.
- [General] The paper self-describes as a position paper but presents quantitative experimental results. The authors should make the status of the empirical study clearer (preliminary analysis vs. full evaluation) and, if possible, release the evaluation scripts and aggregated outputs to strengthen the reproducibility of the claims.
Circularity Check
No significant circularity: the paper reports direct empirical measurements against external benchmark datasets and external safety classifiers, with no fitted parameters renamed as predictions and no load-bearing self-citation chain.
full rationale
The paper's derivation chain is an empirical comparison, not a formal derivation. It selects four external multilingual safety datasets (MULTIJAIL, AYA-REDTEAMING, XSAFETY, RTP-LX) with published labels, runs five external safety classifiers on them, and reports descriptive statistics such as the percentage of prompts identified as unsafe. No parameter is fitted to a subset of the data and then 'predicted' on a closely related quantity; the reported rates are direct measurements, and the qualitative examples in Section 3 ('Nature of the Prompts') are introduced independently as dataset criticisms, not as consequences of the model outputs. The central claim that 'the evaluation datasets contain several examples of potentially safe/harmless prompts that are tagged as harmful' is supported by quoted examples in Tables 2 and 6, and it does not reduce to any equation or fitted value in the paper. The only internal tension is that the quantitative section treats the datasets' ground-truth labels as correct while Section 3 argues that some of those labels are wrong; this is a validity concern for the numerical conclusions, and the Limitations section explicitly concedes the related issue that 'the prompts may appear harmless, but the responses may be harmful/unsafe.' However, that tension is not circular reasoning: the qualitative critique does not define or derive the quantitative disparity, and the Table 1 disparity examples are unambiguously harmful prompts. There is no self-citation chain, no imported uniqueness theorem, and no ansatz smuggled in via citation. Therefore the correct circularity finding is 0.
Assumptions & free parameters
assumptions (1)
- domain assumption The selected five models and four datasets are representative of the broader landscape of multilingual safety classifiers and benchmarks.
Cite this review
Pith. "Pith review of The Problem with Safety Classification is not just the Models." pith.science (2026). https://pith.science/paper/EC3HZFTD
@misc{pith2026250721782,
author = {Pith},
title = {Pith review of: The Problem with Safety Classification is not just the Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/EC3HZFTD}},
note = {Machine review of arXiv:2507.21782}
}
read the original abstract
Studying the robustness of Large Language Models (LLMs) to unsafe behaviors is an important topic of research today. Building safety classification models or guard models, which are fine-tuned models for input/output safety classification for LLMs, is seen as one of the solutions to address the issue. Although there is a lot of research on the safety testing of LLMs themselves, there is little research on evaluating the effectiveness of such safety classifiers or the evaluation datasets used for testing them, especially in multilingual scenarios. In this position paper, we demonstrate how multilingual disparities exist in 5 safety classification models by considering datasets covering 18 languages. At the same time, we identify potential issues with the evaluation datasets, arguing that the shortcomings of current safety classifiers are not only because of the models themselves. We expect that these findings will contribute to the discussion on developing better methods to identify harmful content in LLM inputs across languages.
Figures
Reference graph
Works this paper leans on
-
[5]
Be- yond the safety bundle: Auditing the helpful and harmless dataset . In Proceedings of the 2025 Con- ference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages 11895–11925, Albuquerque, New Mexico. As- sociation for Computational Linguistics. Adrian de Wyn...
work page 2025
-
[6]
Duoguard: A two-player rl-driven framework for multilingual llm guardrails . Preprint, arXiv:2502.05163. Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, and Li- dong Bing
-
[7]
arXiv preprint arXiv:2312.06674
Llama guard: Llm-based input- output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674. Priyanshu Kumar, Devansh Jain, Akhila Y erukola, Li- wei Jiang, Himanshu Beniwal, Thomas Hartvigsen, and Maarten Sap
-
[8]
arXiv preprint arXiv:2504.04377
Polyguard: A multilin- gual safety moderation tool for 17 languages. arXiv preprint arXiv:2504.04377. Inkit Padhi, Manish Nagireddy, Giandomenico Cornac- chia, Subhajit Chaudhury, Tejaswini Pedapati, Pierre Dognin, Keerthiram Murugesan, Erik Miehling, Martin Santillan Cooper, Kieran Fraser, and 1 oth- ers
-
[9]
Granite guardian: Comprehensive llm safeguarding. In Proceedings of the 2025 Confer- ence of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 3: Industry Track) , pages 607–615. 5 Konstantina Palla, José Luis Redondo García, Clau- dia Hauff, Francesco Fabbri, Henrik Lindström, Daniel...
work page 2025
-
[10]
Policy-as-prompt: Rethinking con- tent moderation in the age of large language models . Preprint, arXiv:2502.18695. Traian Rebedea, Razvan Dinu, Makesh Narsimhan Sreedhar, Christopher Parisien, and Jonathan Cohen
-
[11]
Nemo guardrails: A toolkit for controllable and safe llm applications with programmable rails. In Proceedings of the 2023 Conference on Empiri- cal Methods in Natural Language Processing: Sys- tem Demonstrations, pages 431–445. Sahil Verma, Keegan Hines, Jeff Bilmes, Charlotte Siska, Luke Zettlemoyer, Hila Gonen, and Chandan Singh
work page 2023
-
[12]
arXiv preprint arXiv:2505.23856
Omniguard: An efficient approach for ai safety moderation across modalities. arXiv preprint arXiv:2505.23856. Wenxuan Wang, Zhaopeng Tu, Chang Chen, Y ouliang Yuan, Jen-tse Huang, Wenxiang Jiao, and Michael Lyu
Show all 15 references
-
[13]
In Findings of the Association for Computational Linguistics ACL 2024 , pages 5865–
All languages matter: On the multilin- gual safety of llms. In Findings of the Association for Computational Linguistics ACL 2024 , pages 5865–
2024
-
[14]
arXiv preprint arXiv:2505.24119
The state of multilingual llm safety research: From measur- ing the language gap to mitigating it. arXiv preprint arXiv:2505.24119. Wenjun Zeng, Yuchi Liu, Ryan Mullins, Ludovic Peran, Joe Fernandez, Hamza Harkous, Karthik Narasimhan, Drew Proud, Piyush Kumar, Bhak- tipriya Ra...
-
[15]
arXiv preprint arXiv:2407.21772
Shield- gemma: Generative ai content moderation based on gemma. arXiv preprint arXiv:2407.21772. A Computational Resources and Carbon Footprint All the experiments reported in this paper were conducted on a MacOS (M1) laptop with a 32GB RAM. Except the OpenAI moderation API ca...
-
[2022]
arXiv preprint arXiv:2204.05862
Training a helpful and harmless assis- tant with reinforcement learning from human feed- back. arXiv preprint arXiv:2204.05862. Khaoula Chehbouni, Jonathan Colaço Carr, Y ash More, Jackie CK Cheung, and Golnoosh Farnadi
-
[2023]
arXiv preprint arXiv:2303.08774
Gpt-4 techni- cal report. arXiv preprint arXiv:2303.08774. Bang An, Shiyue Zhang, and Mark Dredze
-
[2024]
Preprint, arXiv:2406.18682
The multilingual alignment prism: Aligning global and local preferences to re- duce harm. Preprint, arXiv:2406.18682. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, a...
-
[2025]
RAG LLMs are not safer: A safety analysis of retrieval- augmented generation for large language models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Com- putational Linguistics: Human Language Technolo- gies (Volume 1: Lon...
2025
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.