The reviewed record of science sign in
Pith

arxiv: 2407.15390 · v1 · pith:RE4PAEWT · submitted 2024-07-22 · cs.CL · cs.AI

ALLaM: Large Language Models for Arabic and English

M Saiful Bari , Yazeed Alnumay , Norah A. Alzahrani , Nouf M. Alotaibi , Hisham A. Alyahya , Sultan AlRashed , Faisal A. Mirza , Shaykhah Z. Alsubaie
show 17 more authors
Hassan A. Alahmed Ghadah Alabduljabbar Raghad Alkhathran Yousef Almushayqih Raneem Alnajim Salman Alsubaihi Maryam Al Mansour Majed Alrubaian Ali Alammari Zaki Alawami Abdulmohsen Al-Thubaity Ahmed Abdelali Jeril Kuriakose Abdalghani Abujabal Nora Al-Twairesh Areeb Alowisheq Haidar Khan
This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:RE4PAEWTrecord.jsonopen to challenge →

classification cs.CL cs.AI
keywords arabiclanguagemodelsalignmentallamenglishlargemodel
0
0 comments X
read the original abstract

We present ALLaM: Arabic Large Language Model, a series of large language models to support the ecosystem of Arabic Language Technologies (ALT). ALLaM is carefully trained considering the values of language alignment and knowledge transfer at scale. Our autoregressive decoder-only architecture models demonstrate how second-language acquisition via vocabulary expansion and pretraining on a mixture of Arabic and English text can steer a model towards a new language (Arabic) without any catastrophic forgetting in the original language (English). Furthermore, we highlight the effectiveness of using parallel/translated data to aid the process of knowledge alignment between languages. Finally, we show that extensive alignment with human preferences can significantly enhance the performance of a language model compared to models of a larger scale with lower quality alignment. ALLaM achieves state-of-the-art performance in various Arabic benchmarks, including MMLU Arabic, ACVA, and Arabic Exams. Our aligned models improve both in Arabic and English from their base aligned models.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. HalluScore: Large Language Model Hallucination Question Answering Benchmark

    cs.CL 2026-05 unverdicted novelty 7.0

    HalluScore is a curated Arabic QA dataset with 827 questions, ground-truth evidence, and human annotations used to measure hallucination rates across 17 LLMs.

  2. LQM: Linguistically Motivated Multidimensional Quality Metrics for Machine Translation

    cs.CL 2026-04 unverdicted novelty 7.0

    LQM introduces a six-level linguistically motivated error taxonomy for MT evaluation and applies it via expert annotation to LLM outputs on a new 3,850-sentence multi-dialect Arabic corpus.

  3. LLM-Based Financial Sentiment Analysis in Arabic: Evidence from Saudi Markets

    cs.CL 2026-05 unverdicted novelty 5.0

    Introduces a multi-stage Arabic financial sentiment pipeline that produces an 84K-sample corpus for company-level analysis tied to Saudi stock market behavior.

  4. Noise Steering for Controlled Text Generation: Improving Diversity and Reading-Level Fidelity in Arabic Educational Story Generation

    cs.CL 2026-04 unverdicted novelty 5.0

    Residual-stream noise injection raises narrative diversity in Arabic educational stories while preserving reading-grade level, outperforming high-temperature sampling across five 7-9B models.

  5. A Red Teaming Framework for Large Language Models: A Case Study on Faithfulness Evaluation

    cs.CL 2026-06 unverdicted novelty 4.0

    Introduces a multi-role red teaming framework using attacker and jury models that increases attack success rates by up to 7.9% on LLM faithfulness in question-answering tasks.