Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

Arabic-Nougat: Fine-Tuning Vision Transformers for Arabic OCR and Markdown Extraction

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Arabic-Nougat claims that fine-tuning the Nougat document-parsing transformer with an Arabic tokenizer and a synthetic paired dataset converts Arabic book pages to Markdown with 98.84% structure accuracy and 6.62% character error on the…

desk verdict Useful open-sourced Arabic OCR resource, but the SOTA claim rests on a same-distribution test set and an undefined metric; the artifacts, not the numbers, are the contribution. read the letter →

arxiv 2411.17835 v1 pith:JTWXCF4U submitted 2024-11-19 cs.CL cs.AIcs.CV

Mohamed Rashad This is my paper · ORCID
classification cs.CLcs.AIcs.CV
keywords ArabicOCRMarkdownextractiondocumentparsingvisiontransformerencoder-decoderAranizertokenizersyntheticdatasetbooks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a family of OCR models fine-tuned from the Nougat document-parsing architecture can turn images of Arabic book pages into structured Markdown text with high formatting accuracy. On a held-out set of 160 synthetic Arabic book pages, the largest model reports 98.84% Markdown Structure Accuracy and a 6.62% Character Error Rate, far outperforming the original Nougat models, which score below 1% BLEU on Arabic. The paper's case rests on three pieces working together: an Arabic-specific tokenizer (Aranizer-PBE-86k) that compresses Arabic text, a synthetic dataset (arabic-img2md) of 13.7k page-Markdown pairs scraped from publisher HTML, and reduced-precision training with Flash Attention 2 to allow longer sequences. A sympathetic reader would care because reliable Arabic document-to-Markdown conversion is a missing building block for digitizing Arabic literature and for generating training data.

What carries the argument

The apparatus that carries the argument is the Nougat-style encoder-decoder: a Donut vision encoder downsamples an 896×672 page image to a 588-token visual sequence, and an MBART decoder autoregressively emits Markdown. Onto this, the paper adds the Aranizer-PBE-86k tokenizer, whose 86k vocabulary packs roughly four base tokens into one Arabic token and extends the practical decoder context to 32k tokens, plus a synthetic pipeline that scrapes publisher HTML, renders pages to PDFs, and extracts Markdown to create 13.7k aligned image-text pairs.

What would settle it

Run arabic-large-nougat on a set of scanned Arabic books that were not produced by the synthetic pipeline, covering old print, varied fonts, tables, and marginal noise, and compare CER and structure accuracy against the reported 6.62% and 98.84%. If character error rises severalfold or structure accuracy falls well below 98%, the central claim of state-of-the-art transfer is not supported.

Watch

Extended reading notes

Core claim

The central claim, on the paper's own terms, is that fine-tuning a visual document-parsing transformer on Arabic-specific data moves it from near-total failure on Arabic (BLEU 0.0037 and 0.0094 for the base models) to a usable OCR engine: the best model, arabic-large-nougat, reaches 0.6771 BLEU, 0.0662 CER, 0.1916 WER, and 0.9884 structure accuracy on the reported test set. The authors attribute the gain less to the vision backbone than to the Arabic tokenizer and the synthetic paired corpus, and they release the corpus, the trained models, and a 1.1-billion-token Arabic book corpus extracted by the best model as evidence that the approach scales.

Load-bearing premise

The accuracy numbers assume that a held-out set of pages generated by the same synthetic HTML-to-PDF-to-Markdown pipeline as the training data is representative of real Arabic book pages.

Editorial extensions

If this is right

  • With arabic-large-nougat, a page of Arabic prose can be converted to Markdown at roughly 1-in-15 character error and better than 98% structure accuracy, which is enough to make bulk digitization of standard modern Arabic books feasible.
  • The 13.7k-pair arabic-img2md dataset and the 1.1-billion-token arabic-books corpus are open resources that let other teams fine-tune or build on the approach without collecting new paired data.
  • Because the Aranizer tokenizer compresses Arabic roughly 4:1 relative to the base Nougat tokenizer, sequence-length limits that constrained Latin-script models are less binding for Arabic, and the model can process longer pages.
  • For practical use, the paper recommends arabic-base-nougat for general pages and arabic-large-nougat for lengthy or complex layouts, with a repetition penalty above 1 to control the repetition the large models exhibit.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported test numbers come from the same synthetic generation pipeline as the training data, so a real-book benchmark with scanned older print, marginal noise, and varied typesetting is the natural next test; if transfer is weak, the 98.8% figure is a pipeline-similarity score rather than a deployment number.
  • The 1.1-billion-token corpus was extracted by the model itself, so OCR errors and hallucinated Markdown artifacts may be baked into downstream NLP training; filtering or re-scoring this corpus could change how useful it is.
  • The tokenizer's compression suggests the same fine-tuning recipe might lift other Arabic document tasks, such as layout-aware translation, table extraction, or handwritten text recognition, but nothing in the paper verifies that transfer yet.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The manuscript introduces Arabic-Nougat, a suite of three fine-tuned variants of Meta's Nougat model (arabic-small-nougat, arabic-base-nougat, arabic-large-nougat) for Arabic OCR and Markdown extraction. The models are trained on a synthetic dataset, arabic-img2md, of 13.7k Arabic book page / Markdown pairs, using the Aranizer-PBE-86k tokenizer, bfloat16 precision, and Flash Attention 2. The paper reports BLEU, CER, WER, and a custom Markdown Structure Accuracy (MSA) on a test set of 160 Arabic book pages, claiming that arabic-large-nougat achieves the highest MSA and lowest CER, and releases a 1.1-billion-token Arabic corpus (arabic-books) extracted with the best model.

Significance. If the reported results held on real Arabic documents, this would be a useful and timely contribution: Arabic book OCR with structured Markdown output is underserved, and the open-sourced models, tokenizer, and corpus would be concrete assets. The use of Aranizer-PBE-86k to extend effective context length and the bfloat16/Flash Attention 2 training choices are sensible engineering contributions. However, the central 'state-of-the-art' claim is not yet supported. The evaluation set is drawn from the same synthetic generation pipeline that produced the training data, the primary MSA metric is not defined, the baseline set contains only non-Arabic Meta Nougat models, and the paper's own Section 5 numbers contradict Table 1. These are load-bearing issues that require additional experiments and a corrected, externally validated evaluation before the headline claim can be accepted.

major comments (4)
  1. [§4.1 / §3.3 / §6] The test set of 160 'unseen' pages is sampled from arabic-img2md, the same synthetic pipeline (scraping Hindawi HTML, converting to PDFs, extracting Markdown) that generated the training data. Reported BLEU, CER, WER, and Structure Accuracy are therefore same-distribution estimates, not evidence of generalization to scanned Arabic books or other domains. Section 6 itself acknowledges that the dataset 'may not generalize well to other domains.' The central claim requires an independent evaluation on real, scanned Arabic book pages, ideally with genre/layout diversity, and a comparison of error patterns beyond aggregate scores.
  2. [§4.2 / Table 1 / §5] The paper contains an internal numerical contradiction. Table 1 reports arabic-large-nougat with Structure Accuracy 0.9884 and CER 0.0662, while Section 5 states that arabic-large-nougat achieves the highest Markdown Structure Accuracy (94.7%) and lowest CER (6.1%). These numbers are not consistent. The authors must reconcile the two sets of values and state which corresponds to the released model checkpoints.
  3. [§4.3] Markdown Structure Accuracy (MSA) is described only as 'a custom metric' that evaluates structural elements such as headers and lists. No formula, matching algorithm, or implementation is given. Since MSA is the headline metric supporting the state-of-the-art claim, the paper must specify exactly how structure tokens are extracted, how predicted and reference Markdown are aligned, and how partial credit is assigned, and should release the evaluation script.
  4. [§4.2 / Abstract] The 'state-of-the-art' claim is not supported by the chosen baselines. Meta's Nougat Small and Nougat Base were trained on English scientific documents and are not Arabic-capable OCR systems; their near-zero BLEU and CER values above 1.0 only show that a non-Arabic model fails on Arabic text. A SOTA claim requires comparison with state-of-the-art Arabic OCR or document-parsing systems, for example fine-tuned TrOCR, PaddleOCR, Tesseract with Arabic language data, or recent multimodal document-understanding models trained or adapted for Arabic.
minor comments (6)
  1. [§3.5 / Introduction] The effective context length is presented as both '8192 tokens' (Section 3.5) and '32k tokens' (Introduction and Section 3.5). The relationship between raw decoder length and the tokenizer-compression-based 'equivalent' length should be stated explicitly to avoid confusion.
  2. [§4.4] Token Efficiency Ratio (TER) is defined as 'the ratio of tokens produced by the tokenizer to ground truth tokens,' but the paper does not state whether lower or higher values are better. As reported, TER 1.05 for arabic-large-nougat and 1.25 for arabic-small-nougat appears to be lower-is-better; please clarify.
  3. [§4.2 / Table 1, Section 5] The manuscript mixes decimal fractions (e.g., 0.0662) and percentages (e.g., 6.1%) for the same metric types. Please choose one convention and apply it consistently in the table and text.
  4. [References] References [24] and [25] both list arXiv:2409.03420; the second citation appears to be a different paper and the arXiv identifier should be corrected.
  5. [Figure 1] The text inside Figure 1 appears garbled ('قد م# BOS ة ق\n\n'), which makes the figure hard to interpret. A clean example of an input page and its tokenized output would be more informative.
  6. [§2.3] BCE-Arabic-v1 is mentioned without a reference. Please provide a citation or remove the mention if it is not used elsewhere.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found; the in-distribution test set is an external-validity limitation, not a circularity.

full rationale

The paper's claimed derivation chain is empirical rather than analytic: models are trained on arabic-img2md and evaluated on 160 unseen pages from the same synthetic pipeline (Sections 3.3 and 4.1). This is a standard held-out evaluation within one data distribution, not a definitional equivalence or a fitted parameter renamed as a prediction. The ground-truth Markdown is generated from the same Hindawi HTML-to-PDF pipeline, so the test measures fit to that pipeline; the paper itself concedes in Section 6 that the dataset 'may not generalize well to other domains,' which is a generalization limitation, not circularity. The custom Structure Accuracy metric has no published formula, and Section 5's numbers (94.7% MSA, 6.1% CER) conflict with Table 1 (98.84% MSA, 6.62% CER), but these are reproducibility/reporting defects rather than circular reasoning. Self-citations to the released arabic-img2md and arabic-books artifacts are descriptive references to contributed resources, not load-bearing external justifications for the central performance claim. No step in the paper reduces, by its own equations or by self-citation, to its inputs.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central result rests on the fidelity of a synthetic data generation pipeline and on an undefined custom metric, plus several unstated training hyperparameters. No new theoretical entities are introduced.

free parameters (4)
  • learning_rate = 1e-4
    Set by hand; no sweep or sensitivity analysis is reported.
  • max_epochs = 100
    Training budget chosen without a reported justification beyond EarlyStopping.
  • repetition_penalty = >1
    Recommended to mitigate repetition, but no exact value or ablation is reported.
  • test_set_size = 160 pages
    Evaluation subset chosen without power analysis or stratification; no error bars are reported.
assumptions (3)
  • domain assumption Synthetic pages rendered from Hindawi HTML into PDFs, with Markdown extracted from the HTML, adequately represent real Arabic book pages.
    Stated in Section 3.3 and used in Section 4.1 to define train and test sets.
  • domain assumption The custom Structure Accuracy metric is a valid measure of Markdown formatting correctness.
    The metric is named in Section 4.3 but its formula is not given, so the 98.84% claim cannot be audited.
  • domain assumption A random holdout from arabic-img2md is representative of the target deployment distribution.
    Section 4.1 relies on this to interpret the 160-page test set as predictive of real-world OCR quality.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Arabic-Nougat: Fine-Tuning Vision Transformers for Arabic OCR and Markdown Extraction." pith.science (2026). https://pith.science/paper/JTWXCF4U

@misc{pith2026241117835,
  author       = {Pith},
  title        = {Pith review of: Arabic-Nougat: Fine-Tuning Vision Transformers for Arabic OCR and Markdown Extraction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JTWXCF4U}},
  note         = {Machine review of arXiv:2411.17835}
}
read the original abstract

We present Arabic-Nougat, a suite of OCR models for converting Arabic book pages into structured Markdown text. Based on Meta's Nougat architecture, Arabic-Nougat includes three specialized models: arabic-small-nougat, arabic-base-nougat, and arabic-large-nougat. These models are fine-tuned on a synthetic dataset, arabic-img2md, comprising 13.7k pairs of Arabic book pages and their Markdown representations. Key contributions include the Aranizer-PBE-86k tokenizer, designed for efficient tokenization, and the use of torch.bfloat16 precision with Flash Attention 2 for optimized training and inference. Our models achieve state-of-the-art performance, with arabic-large-nougat delivering the highest Markdown Structure Accuracy and the lowest Character Error Rate. Additionally, we release a large-scale dataset containing 1.1 billion Arabic tokens extracted from over 8,500 books using our best-performing model, providing a valuable resource for Arabic OCR research. All models, datasets, and code are open-sourced and available at https://github.com/MohamedAliRashad/arabic-nougat.

Figures

Figures reproduced from arXiv: 2411.17835 by the authors.

Figure 1
Figure 1. Overview of the Arabic-Nougat architecture, illustrating the integration of the Donut Vision Encoder with an auto-regressive MBART decoder for Arabic OCR and Markdown extraction. The diagram highlights key components such as image encoding from an Arabic book page and the overall decoding process. 3 Methodology 3.1 Model Architecture The Arabic-Nougat suite builds on Meta’s Nougat architecture, using Donut vision en… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SARD: A Large-Scale Synthetic Arabic OCR Dataset for Book-Style Text Recognition

    cs.CV 2025-05 conditional novelty 6.0 of 10

    SARD is a new synthetic dataset of 843,622 Arabic book pages with ten fonts, plus baseline OCR benchmarks showing large differences between modern vision-language models and traditional engines.

  2. A-SEA3L-QA: A Fully Automated Self-Evolving, Adversarial Workflow for Arabic Long-Context Question-Answer Generation

    cs.CL 2025-09 reject novelty 5.0 of 10

    An AI-driven, self-refining loop generates multi-page Arabic QA pairs and a new benchmark, but the claimed gains over static pipelines are not demonstrated.

Reference graph

Works this paper leans on

24 extracted references · 14 canonical work pages · cited by 2 Pith papers

  1. [1]

    Nougat: Neural Optical Understanding for Academic Documents,

    Meta AI, “Nougat: Neural Optical Understanding for Academic Documents,” 2023. https://arxiv. org/abs/2308.13418

    arXiv 2023

  2. [2]

    Multilingual Denois- ing Pre-training for Neural Machine Translation,

    Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer, “Multilingual Denois- ing Pre-training for Neural Machine Translation,” arXiv preprint arXiv:2001.08210 , 2020. https: //arxiv.org/abs/2001.08210

    arXiv 2001

  3. [3]

    FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning,

    Tri Dao, “FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning,” arXiv preprint arXiv:2307.08691 , 2023. https: //arxiv.org/abs/2307.08691

    arXiv 2023

  4. [4]

    LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking,

    Xu, Yiheng, et al., “LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 3, 2022, pp. 11158–11166

    work page 2022

  5. [5]

    Donut: Document Understand- ing Transformer without OCR,

    Kim, Jaemin, et al., “Donut: Document Understand- ing Transformer without OCR,” Advances in Neural Information Processing Systems, 2021

    work page 2021

  6. [6]

    DS-YOLOv5: Deformable Single Shot YOLO for Document Parsing,

    Wang, Xin, et al., “DS-YOLOv5: Deformable Single Shot YOLO for Document Parsing,” ICDAR Work- shop on Document Analysis, 2023

    work page 2023

  7. [7]

    https://www

    Hindawi Publishing Corporation. https://www. hindawi.org/

    work page

  8. [8]

    Khatt: An Open Arabic Hand- written Text Database,

    Fakhraddin, V ., et al., “Khatt: An Open Arabic Hand- written Text Database,” International Conference on Frontiers in Handwriting Recognition (ICFHR), 2012, pp. 19–22

    work page 2012

Show all 24 references
  1. [9]

    PubLayNet: Largest Dataset Ever for Document Layout Analysis,

    Zhong, Xinyu, et al., “PubLayNet: Largest Dataset Ever for Document Layout Analysis,” Document In- telligence Workshop at NeurIPS, 2019

    2019

  2. [10]

    VisionLAN: Visual Alignment Network for Scene Text Recognition,

    Wang, Yi, et al., “VisionLAN: Visual Alignment Network for Scene Text Recognition,”Pattern Recog- nition, vol. 120, 2021

    2021

  3. [11]

    TrOCR: Transformer-based Optical Character Recognition with Pre-trained Mod- els,

    Li, Minghao, et al., “TrOCR: Transformer-based Optical Character Recognition with Pre-trained Mod- els,” arXiv preprint arXiv:2109.10282, 2022

    2022 arXiv

  4. [12]

    riotu-lab/Aranizer-PBE-86k · Hug- ging Face,

    riotu-lab, “riotu-lab/Aranizer-PBE-86k · Hug- ging Face,” https://huggingface.co/ riotu-lab/Aranizer-PBE-86k

  5. [13]

    MohamedRashad/arabic- img2md · Hugging Face,

    Mohamed Rashad, “MohamedRashad/arabic- img2md · Hugging Face,” https: //huggingface.co/datasets/ MohamedRashad/arabic-img2md

  6. [14]

    MohamedRashad/arabic-books · Hugging Face,

    Mohamed Rashad, “MohamedRashad/arabic-books · Hugging Face,” https://huggingface. co/datasets/MohamedRashad/ arabic-books

  7. [15]

    LayoutLM: Pre-training of Text and Layout for Document Image Understanding,

    Xu, Yiheng, et al., “LayoutLM: Pre-training of Text and Layout for Document Image Understanding,” Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020, pp. 1192–1200

    2020

  8. [16]

    BERT- grid: Contextualized Embedding for 2D Document Representation and Understanding,

    Timo I. Denk and Christian Reisswig, “BERT- grid: Contextualized Embedding for 2D Document Representation and Understanding,” arXiv preprint arXiv:1909.04948, 2019

    1909 arXiv

  9. [17]

    Mathematical Formula Detection in Document Im- ages: A New Dataset and a New Approach,

    Hu, Kai, Zhuoyao Zhong, Lei Sun, and Qiang Huo, “Mathematical Formula Detection in Document Im- ages: A New Dataset and a New Approach,” Pattern Recognition, vol. 148, 2024, p. 110212

    2024

  10. [18]

    OmniParser: A Unified Framework for Text Spotting, Key Information Ex- traction and Table Recognition,

    Wan, Jianqiang, et al., “OmniParser: A Unified Framework for Text Spotting, Key Information Ex- traction and Table Recognition,”Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, 2024, pp. 15641–15653

    2024

  11. [19]

    General OCR Theory: To- wards OCR-2.0 via a Unified End-to-End Model,

    Wei, Haoran, et al., “General OCR Theory: To- wards OCR-2.0 via a Unified End-to-End Model,” arXiv preprint arXiv:2409.01704, 2024

    2024 arXiv

  12. [20]

    Focus Any- where for Fine-Grained Multi-Page Document Under- standing,

    Chenglong Liu, Haoran Wei, Jinyue Chen, Lingyu Kong, Zheng Ge, Zining Zhu, Liang Zhao, Jianjian Sun, Chunrui Han, and Xiangyu Zhang, “Focus Any- where for Fine-Grained Multi-Page Document Under- standing,” arXiv preprint arXiv:2405.14295, 2024

    2024 arXiv

  13. [21]

    Ureader: Universal OCR- Free Visually-Situated Language Understanding with Multimodal Large Language Model,

    Jiabo Ye, Anwen Hu, Haiyang Xu, Qinghao Ye, Ming Yan, Guohai Xu, Chenliang Li, Junfeng Tian, Qi Qian, Ji Zhang, et al., “Ureader: Universal OCR- Free Visually-Situated Language Understanding with Multimodal Large Language Model,” arXiv preprint arXiv:2310.05126, 2023

    arXiv

  14. [22]

    MPLUG-DocOwl 1.5: Unified Struc- ture Learning for OCR-Free Document Understand- ing,

    Anwen Hu, Haiyang Xu, Jiabo Ye, Ming Yan, Liang Zhang, Bo Zhang, Chen Li, Ji Zhang, Qin Jin, Fei Huang, et al., “MPLUG-DocOwl 1.5: Unified Struc- ture Learning for OCR-Free Document Understand- ing,” arXiv preprint arXiv:2403.12895, 2024

    2024 arXiv

  15. [23]

    MPLUG-PaperOwl: Scientific Dia- gram Analysis with the Multimodal Large Language Model,

    Anwen Hu, Yaya Shi, Haiyang Xu, Jiabo Ye, Qing- hao Ye, Ming Yan, Chenliang Li, Qi Qian, Ji Zhang, and Fei Huang, “MPLUG-PaperOwl: Scientific Dia- gram Analysis with the Multimodal Large Language Model,” ACM Multimedia 2024, 2024

    2024

  16. [25]

    Vary: Scaling Up the Vision V ocabulary for Large Vision-Language Models,

    Haoran Wei, Lingyu Kong, Jinyue Chen, Liang Zhao, Zheng Ge, Jinrong Yang, Jianjian Sun, Chun- rui Han, and Xiangyu Zhang, “Vary: Scaling Up the Vision V ocabulary for Large Vision-Language Models,” arXiv preprint arXiv:2409.03420, 2024

    2024 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.