REVIEW 8 cited by
ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Pre-trained language models (LMs) are currently integral to many natural language processing systems. Although multilingual LMs were also introduced to serve many languages, these have limitations such as being costly at inference time and the size and diversity of non-English data involved in their pre-training. We remedy these issues for a collection of diverse Arabic varieties by introducing two powerful deep bidirectional transformer-based models, ARBERT and MARBERT. To evaluate our models, we also introduce ARLUE, a new benchmark for multi-dialectal Arabic language understanding evaluation. ARLUE is built using 42 datasets targeting six different task clusters, allowing us to offer a series of standardized experiments under rich conditions. When fine-tuned on ARLUE, our models collectively achieve new state-of-the-art results across the majority of tasks (37 out of 48 classification tasks, on the 42 datasets). Our best model acquires the highest ARLUE score (77.40) across all six task clusters, outperforming all other models including XLM-R Large (~ 3.4 x larger size). Our models are publicly available at https://github.com/UBC-NLP/marbert and ARLUE will be released through the same repository.
Forward citations
Cited by 8 Pith papers
-
Mawqif-XT: An Arabic Benchmark Dataset for Cross-Target Stance Detection
A new Arabic benchmark extends Mawqif with three targets and reports baselines for cross-target stance detection.
-
Fann or Flop: A Multigenre, Multiera Benchmark for Arabic Poetry Understanding in LLMs
The new Fann or Flop benchmark measures LLM comprehension of Arabic poetry through expert-written verse explanations and shows current LLMs perform poorly on interpretive depth.
-
Towards the Development of Balanced Synthetic Data for Correcting Grammatical Errors in Arabic: An Approach Based on Error Tagging Model and Synthetic Data Generating Model
A tag-conditioned Arabic synthetic data pipeline is claimed to set a new GEC state of the art, but the reported 79.36% F1 is the F0.5 score from the paper's own table.
-
On The Origin of Cultural Biases in Language Models: From Pre-training Data to Linguistic Phenomena
Arab cultural entities that double as everyday Arabic words are harder for language models to recognize, especially when tokenized as single tokens.
-
CVPD at QIAS 2025 Shared Task: An Efficient Encoder-Based Approach for Islamic Inheritance Reasoning
An encoder-based relevance-scoring system achieves 69.87% accuracy on Islamic inheritance multiple-choice questions, below Gemini's 87.60% but with far smaller compute.
-
Enhanced Arabic Text Retrieval with Attentive Relevance Scoring
An Arabic dense retriever using a trainable attentive scoring module instead of dot-product similarity reports improved top-k passage retrieval on ArabicaQA.
-
The Role of Orthographic Consistency in Multilingual Embedding Models for Text Classification in Arabic-Script Languages
Language-specific RoBERTa models for four Arabic-script languages beat multilingual baselines on news classification, though the claimed orthographic-consistency mechanism is not demonstrated.
-
GATE: General Arabic Text Embedding for Enhanced Semantic Textual Similarity with Matryoshka Representation Learning and Hybrid Loss Training
GATE's Arabic-Triplet-Matryoshka-V2 reports the highest average scores on the MTEB Arabic STS17/STS22/STS22-v2 tasks among the models compared in the paper.
Discussion (0). Continue with ORCID to comment.