M-ALERT, a 75k-prompt multilingual safety benchmark, shows that LLM safety varies substantially across five languages and across risk categories, with no model reaching the 99% safe threshold in every language.
Amplifying Limitations, Harms and Risks of Large Language Models
1 Pith paper cite this work. Polarity classification is still indexing.
abstract
We present this article as a small gesture in an attempt to counter what appears to be exponentially growing hype around Artificial Intelligence (AI) and its capabilities, and the distraction provided by the associated talk of science-fiction scenarios that might arise if AI should become sentient and super-intelligent. It may also help those outside of the field to become more informed about some of the limitations of AI technology. In the current context of popular discourse AI defaults to mean foundation and large language models (LLMs) such as those used to create ChatGPT. This in itself is a misrepresentation of the diversity, depth and volume of research, researchers, and technology that truly represents the field of AI. AI being a field of research that has existed in software artefacts since at least the 1950's. We set out to highlight a number of limitations of LLMs, and in so doing highlight that harms have already arisen and will continue to arise due to these limitations. Along the way we also highlight some of the associated risks for individuals and organisations in using this technology.
fields
cs.CL 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
LLMs Lost in Translation: M-ALERT uncovers Cross-Linguistic Safety Inconsistencies
M-ALERT, a 75k-prompt multilingual safety benchmark, shows that LLM safety varies substantially across five languages and across risk categories, with no model reaching the 99% safe threshold in every language.