The language gap in AI models creates safety disparities across languages, and closing it requires funding multilingual datasets, transparency, and research.
NusaCrowd: Open Source Initiative for Indonesian NLP Resources
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
We present NusaCrowd, a collaborative initiative to collect and unify existing resources for Indonesian languages, including opening access to previously non-public resources. Through this initiative, we have brought together 137 datasets and 118 standardized data loaders. The quality of the datasets has been assessed manually and automatically, and their value is demonstrated through multiple experiments. NusaCrowd's data collection enables the creation of the first zero-shot benchmarks for natural language understanding and generation in Indonesian and the local languages of Indonesia. Furthermore, NusaCrowd brings the creation of the first multilingual automatic speech recognition benchmark in Indonesian and the local languages of Indonesia. Our work strives to advance natural language processing (NLP) research for languages that are under-represented despite being widely spoken.
fields
cs.AI 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
The Multilingual Divide and Its Impact on Global AI Safety
The language gap in AI models creates safety disparities across languages, and closing it requires funding multilingual datasets, transparency, and research.