Nepali-language BERT, RoBERTa, and GPT-2 models trained on a new 27.5 GB corpus score 95.60 on Nep-gLUE, beating the prior best by about two points.
Large-Scale Contextualised Language Modelling for Norwegian
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
We present the ongoing NorLM initiative to support the creation and use of very large contextualised language models for Norwegian (and in principle other Nordic languages), including a ready-to-use software environment, as well as an experience report for data preparation and training. This paper introduces the first large-scale monolingual language models for Norwegian, based on both the ELMo and BERT frameworks. In addition to detailing the training process, we present contrastive benchmark results on a suite of NLP tasks for Norwegian. For additional background and access to the data, models, and software, please see http://norlm.nlpl.eu
fields
cs.CL 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Development of Pre-Trained Transformer-based Models for the Nepali Language
Nepali-language BERT, RoBERTa, and GPT-2 models trained on a new 27.5 GB corpus score 95.60 on Nep-gLUE, beating the prior best by about two points.