Pith. sign in

REVIEW 3 major objections 6 minor 137 references

MonkeyOCRv2: A Visual-Text Foundation Model for Document AI

T0 review · 3 major / 6 minor · reviewed 2026-07-14 · grok-4.5

Pith's one-line read A document-native vision encoder, trained with text generation plus pixel reconstruction, transfers across OCR, parsing, and understanding better than natural-image foundations.

desk verdict Solid document-native encoder with clean multi-task transfer; the MDPBench SOTA headline is real but not cleanly encoder-attributable. read the letter →

arxiv 2607.11562 v1 pith:NDLC6ZRV submitted 2026-07-13 cs.CV

classification cs.CV
keywords documentAIvisualpretrainingOCRparsingunderstandingimagereconstructionmultilingualdocumentsvisionfoundationmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Mainstream vision encoders learn object and scene semantics from natural photos, so they miss the stroke-level detail that document images depend on. This paper argues that a dedicated visual foundation for documents is possible: pretrain an encoder on a large multilingual document corpus so that it both reads text and reconstructs pixels. The generation objective ties features to words; the reconstruction objective keeps character strokes, glyphs, and layout that pure language supervision would throw away. Swapping that encoder into existing systems improves text recognition, formula recognition, detection, tampering localization, and overlapping-text segmentation. Kept frozen and paired with a small language model, it also drives competitive multilingual document parsing and stronger document understanding than CLIP-, DINO-, or SAM-style backbones under matched training. The practical claim is that document intelligence can rest on its own visual pretraining rather than borrowed natural-image features.

What carries the argument

Dual-objective document pretraining on MonkeyDoc v2 (113M images, 17 languages): image-to-text generation aligns visual tokens with textual content, while pixel-level reconstruction (MSE, optionally edge- and distance-aware) forces the encoder to retain strokes, glyphs, and layout that text supervision alone can discard.

What would settle it

Under fully matched data, optimization, and decoding, freeze MonkeyOCRv2 and a strong natural-image encoder of similar size, train the same lightweight document parser, and check whether the document-native encoder still wins on photographed non-Latin pages of MDPBench and on scrambled-text recognition at low resolution; a clear loss would falsify the claim that the dual objective, not scale or task setup, is doing the work.

Watch

Extended reading notes

Core claim

Document-oriented pretraining that jointly optimizes image-to-text generation and pixel-level reconstruction produces transferable character-level visual representations: as a backbone swap it improves five document analysis tasks, and as a frozen encoder with a lightweight language model it yields a 0.7B parser that reaches open-source state of the art on multilingual MDPBench while also outperforming CLIP, DINO, and SAM counterparts on eight document-understanding benchmarks under identical training.

Load-bearing premise

The large training labels from multi-expert agreement and automatic layout filters are clean enough that measured gains reflect better visual features rather than shared errors between those labels and the evaluation stack.

Editorial extensions

If this is right

  • Document systems can replace ImageNet, CLIP, DINO, or SAM backbones with a compact document encoder and expect gains without rewriting the rest of the pipeline.
  • A frozen ~0.1B document vision encoder plus a ~0.6B language model is enough for competitive multilingual parsing, so large general VLMs are not required for that task class.
  • Pixel reconstruction should reduce reliance on language priors when text is scrambled, low-resolution, or deliberately conflicted with linguistic expectations.
  • Future document foundation work can treat text strokes and layout as first-class visual targets rather than side effects of semantic alignment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same dual recipe may help other dense-symbol domains (sheet music, circuit diagrams, engineering drawings) where global semantic encoders discard local marks.
  • If reconstruction is what narrows the semantic–scrambled gap, progressive post-training of the language head alone may not close remaining gaps on saturated parsing benches without stronger visual evidence.
  • Balancing MonkeyDoc-style supervision toward low-resource and historical scripts would be a direct test of whether the method generalizes beyond high-resource languages that dominate the current mix.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. MonkeyOCRv2 proposes a document-native visual encoder pretrained on MonkeyDoc v2 (113M images, 17 languages) with a joint objective of image-to-text generation and pixel-level reconstruction (Eqs. 1–11). The encoder is evaluated as a backbone substitution on five document analysis tasks (text recognition, formula recognition, text detection, tampering detection, overlapping text segmentation) and, frozen, as the vision tower of lightweight VLMs for document parsing and understanding. The paper reports consistent gains from encoder replacement (Tabs. 2–5, Fig. 4), open-source SOTA on MDPBench for a 0.7B frozen-encoder parser (+2.8 over 3B dots.mocr with a much smaller ViT; Tab. 6), and superior document-understanding scores versus CLIP/DINO/SAM/OpenVision under matched LLM, data, and training (Tab. 8). Reconstruction is further supported by scrambled-text and CHAOS-Bench analyses (Sec. 5.1–5.2).

Significance. If the transfer results hold under the stated controls, the work is a substantial contribution to document AI: it argues, with multi-task evidence, that character-level document pretraining can serve as a foundation rather than a domain adaptation of natural-image encoders. Strengths include a large multilingual corpus, a dual-objective recipe with explicit reconstruction ablations (MSE and structure-aware variants in Tab. 8; scrambled-text gap narrowing in Fig. 5), controlled frozen-encoder VLM comparisons on eight benchmarks (Tab. 8), and honest system-level caveats for OmniDocBench (Sec. 4.6, Tab. 7). The five-task backbone-swap protocol is particularly useful for the community. Code and data release is promised, which would further raise impact.

major comments (3)
  1. Sec. 4.6 and Tab. 6 present MonkeyOCRv2-Parsing as open-source SOTA on MDPBench (+2.8 over dots.mocr, ~11× smaller vision encoder). The same section correctly notes that OmniDocBench (Tab. 7) is system-level and that encoder attribution should use Tab. 8. That caveat is not applied to MDPBench: the parser couples a frozen encoder to autoregressive layout prediction, per-element re-crop recognition, and assembly, with no matched experiment that freezes a strong alternative encoder (e.g., OpenVision-B, RADIOv2.5-B, SigLIP 2) inside the identical parsing pipeline, data, and training recipe. Without that control, the headline +2.8 and size framing can credit pipeline design and training as much as document-native pretraining. Please either add the matched encoder swap in the parsing stack or reframe Tab. 6 as a system result and rest the encoder claim primarily on Tabs. 2–5 and Tab. 8.
  2. Sec. 3.1 (Expert Model Labeling; Data Filtering) relies on multi-expert OCR agreement and LLM layout/reading-order filters for large-scale supervision. The residual error structure of those automatic labels is not quantified against human gold or against the evaluation stacks (including MDPBench, which shares research-lineage with prior MonkeyOCR work). If label failure modes correlate with the pretraining objective or with same-lineage benchmarks, measured gains partly reflect label–model correlation. A short audit—e.g., human agreement rates on a stratified sample, or performance when pretraining only on fully public human-annotated subsets—would strengthen the claim that gains are representation quality rather than supervision artifacts.
  3. Tab. 8 is the cleanest encoder-level comparison, but input configurations differ substantially (App. D: CLIP 196 tokens vs SAM 4096 vs MonkeyOCRv2 ~1082). The paper states each encoder uses its native setting, which is reasonable, yet token budget and resolution are known confounders for document VQA. A sensitivity check that equalizes approximate visual-token count (or reports a fixed-token budget ablation for the top baselines) would make the 13.2-point gap over OpenVision-B more attributable to pretraining rather than resolution policy.
minor comments (6)
  1. Abstract and Fig. 2(a) lead with the MDPBench SOTA and 11× smaller encoder; after addressing the major comment on attribution, align abstract wording with the revised claim so abstract and Sec. 4.6 do not over-promise encoder-only causality.
  2. Eq. (11) sets λ=1.0 with α, β, T, τ fixed without tuning (Sec. 3.2). A brief sensitivity note (even one-dimensional in λ) would help readers assess robustness of the dual-objective balance.
  3. Table 1 lists MonkeyDoc v2 as 113M multi-type documents / 17 languages; App. A shows strong English/Chinese skew. Mentioning this imbalance earlier (not only in Limitations) would set expectations for low-resource scripts.
  4. Fig. 4 uses bar charts without numeric tables in the main text; adding exact F-measures in a small table or appendix would aid citation and reproducibility.
  5. Typo/consistency: abstract says “previous best 3B dots.mocr” while related work and Tab. 6 use “dots.mocr”; unify naming. Also “UniMERNet-T to outperform the 325M UniMERNet-B” is clear in Tab. 3 but the abstract’s “enabling the 110M” phrasing could state ExpRate/CDM explicitly.
  6. Sec. 5.1 correctly treats the semantic–scrambled gap as an operational proxy; consider moving one sentence of that caveat into the figure caption of Fig. 5 so casual readers do not over-interpret the gap as a pure hallucination metric.

Circularity Check

1 steps flagged · score 2.0 of 10

No derivation-by-construction circularity; mild same-lineage risk on MDPBench SOTA framing, while backbone swaps and matched VLM controls remain independent.

  1. self citation load bearing [Sec. 4.6 Document Parsing; Tab. 6 MDPBench; abstract SOTA claim]
    "Kept frozen and paired with a lightweight language model, it yields a 0.7B document parsing model that sets a new open-source state-of-the-art on MDPBench... surpassing the previous best 3B dots.mocr by 2.8% absolute with a vision encoder roughly 11× smaller. ... While MDPBench originates from the same research line as our prior benchmarks, the encoder, the LLM, and the data pipeline evaluated here are independent of its construction"

    The headline open-source SOTA and 11×-smaller-encoder framing rest on MDPBench, which the paper itself notes comes from the same research line. The paper does not freeze a strong alternative encoder inside the exact autoregressive-layout + re-crop parsing pipeline, so the load-bearing +2.8 claim partly leans on same-lineage benchmark construction rather than a fully independent external test. This is mild self-citation risk, not a definitional reduction of the pretraining objective.

full rationale

MonkeyOCRv2 is an empirical CV foundation-model paper, not a first-principles derivation. The dual objective L_pretrain = L_text + λ L_rec is a training recipe, not a claim that one quantity is mathematically forced by another. Gains are measured by encoder substitution into CRNN/PARSeq, UniMERNet-T, DBNet/PSENet/DPText-DETR, FFDN, Mask2Former/MOTS (Tabs. 2–5, Fig. 4) and by frozen-encoder VLMs under fixed LLM/data/optimization (Tab. 8). Reconstruction ablations (Fig. 5, Tab. 8 baseline vs MSE vs structure-aware, Tab. 9 CHAOS-Bench) compare trained variants rather than renaming a fit as a prediction. MDPBench shares authorship lineage with prior MonkeyOCR work, and Sec. 4.6/Tab. 7 explicitly warn that OmniDocBench is system-level, so the +2.8 open-source SOTA headline is not cleanly encoder-attributed without a matched alternative-encoder ablation inside the same parsing pipeline—this is attribution softness, not circular reduction of a claimed derivation. No self-definitional equations, fitted-input-as-prediction, uniqueness theorems, or ansatz-via-citation chains appear. Score 2 for mild self-citation load on the parsing headline only.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

This is an empirical systems paper. Load-bearing premises are standard deep-learning transfer assumptions plus specific data-labeling and loss-weight choices, not new physical entities. Free parameters are the dual-loss weights and training knobs fixed without extensive search. Axioms cover the claim that reconstruction preserves character-level evidence and that automatic multi-expert labels are adequate at 113M scale.

free parameters (4)
  • λ (reconstruction weight in L_pretrain)
    Balances text generation vs reconstruction; fixed at 1.0 without reported sweep beyond the presence/absence ablation.
  • α, β (structure-aware reconstruction weights)
    Fixed at 0.5 and 0.25 for edge/distance terms used in document-understanding variants; not systematically tuned.
  • T, τ (distance-to-edge iterations and edge temperature)
    Fixed at 16 and 0.08 for structure-aware loss; ad hoc numerical choices.
  • peak learning rate and batch size for pretraining
    1e-3 and global batch 256 on 64 A800s; standard but claim-dependent training hyperparameters.
assumptions (4)
  • domain assumption Pixel reconstruction (MSE and optional edge/distance matching) forces the encoder to retain character strokes and layout that pure text supervision discards.
    Core justification for the dual objective (Sec. 3.2 and Sec. 5.1); supported by ablations but not proven as the unique mechanism.
  • domain assumption Multi-expert agreement among OCR systems plus LLM layout/order filters yields sufficiently accurate labels for large-scale pretraining.
    Data Engine (Sec. 3.1) relies on this for real-document supervision quality.
  • domain assumption Frozen-encoder transfer under matched LLM, data, and optimization isolates visual representation quality.
    Used for document understanding (Tab. 8) and parsing claims; residual differences in native resolution/token counts remain (App. D).
  • standard math Standard transformer/ViT training dynamics and cross-entropy + MSE optimization are valid for learning transferable document features.
    Background ML practice assumed throughout pretraining and fine-tuning.
invented entities (2)
  • MonkeyDoc v2 corpus
    purpose: Provide 113M multilingual document images with dense text supervision for document-native pretraining.
    Constructed resource, not a physical postulate; independent value depends on public release and external reuse.
  • MonkeyOCRv2 dual-objective encoder family (S/B/AS)
    purpose: Serve as a reusable document vision backbone for recognition, detection, parsing, and understanding.
    New model family defined by architecture + pretraining recipe; evidence is internal benchmarks pending external reimplementation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MonkeyOCRv2: A Visual-Text Foundation Model for Document AI." pith.science (2026). https://pith.science/paper/NDLC6ZRV

@misc{pith2026260711562,
  author       = {Pith},
  title        = {Pith review of: MonkeyOCRv2: A Visual-Text Foundation Model for Document AI},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NDLC6ZRV}},
  note         = {Machine review of arXiv:2607.11562}
}
abstract

Mainstream visual encoders are pretrained on natural images and cannot be effectively applied to document images without document-oriented adaptation, as dense text and fine-grained character strokes demand character-level visual perception. We present MonkeyOCRv2, a visual-text pretrained model for document AI. First, we construct MonkeyDoc v2, to our knowledge the largest document-image pretraining corpus, comprising 113 million images spanning 17 languages. Second, we propose a pretraining strategy that jointly learns image-to-text generation and pixel-level document reconstruction: the former aligns visual representations with textual content, while the latter preserves character strokes and layout details. Extensive experiments are conducted on five representative document analysis tasks, including text recognition, formula recognition, text detection, document tampering detection, and overlapping text segmentation. Replacing the original encoders with MonkeyOCRv2 consistently improves performance across all five tasks. Finally, we validate its effectiveness as the vision encoder of multimodal large language models on the more challenging tasks of document parsing and document understanding. Kept frozen and paired with a lightweight language model, it yields a 0.7B document parsing model that sets a new open-source state-of-the-art on MDPBench, a recent benchmark spanning digital-born and photographed documents across 17 languages, surpassing the previous best 3B dots.mocr by 2.8% absolute with a vision encoder roughly 11$\times$ smaller. The frozen encoder also powers a document understanding model that outperforms counterparts built on CLIP, DINO, and SAM across eight benchmarks under identical training settings. These results suggest that document-oriented visual pretraining can serve as a foundation for document intelligence in its own right.

Figures

Figures reproduced from arXiv: 2607.11562 by the authors.

Figure 1
Figure 1. Overview of MonkeyOCRv2. Existing vision foundation models are primarily designed for natural images and emphasize object semantics, global alignment, semantic features, or region boundaries. MonkeyOCRv2 addresses the resulting representation mismatch by jointly learning text generation and pixel-level reconstruction, producing document-native visual representations that transfer across diverse document AI tasks. di… view at source ↗
Figure 2
Figure 2. Performance overview of MonkeyOCRv2. (a) Performance versus vision-encoder size on MDPBench [56], a challenging multilingual document parsing benchmark. MonkeyOCRv2 achieves 83.3%, outperforming the previous best open-source model, dots.mocr, with a vision encoder roughly 11× smaller. Bubble area indicates the total number of model parameters. (b) Absolute performance improvements across seven document analysis task… view at source ↗
Figure 3
Figure 3. MonkeyOCRv2 is pretrained on a large-scale corpus of multilingual, multi-type document [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: For text detection, MonkeyOCRv2 consistently delivers robust performance gains on [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Left: scrambled text recognition accuracy. Right: the accuracy gap between semantically [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Visualization comparisons with leading document parsing models on an Arabic document [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Visualization comparisons with leading document parsing models on a photographed [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: Visualization comparisons with popular vision foundation models on document understand [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Detailed data distribution of MonkeyDoc v2. [PITH_FULL_IMAGE:figures/full_fig_p031_9.png]
Figure 10
Figure 10. Figure 10: Visualization of Arabic, German, English, Spanish, French, and Hindi images in Monkey [PITH_FULL_IMAGE:figures/full_fig_p033_10.png]
Figure 11
Figure 11. Figure 11: Visualization of Indonesian, Italian, Japanese, Korean, Dutch, and Portuguese images in [PITH_FULL_IMAGE:figures/full_fig_p034_11.png]
Figure 12
Figure 12. Figure 12: Visualization of Russian, Thai, Vietnamese, Simplified Chinese, and Traditional Chinese [PITH_FULL_IMAGE:figures/full_fig_p035_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

137 extracted references · 20 linked inside Pith

  1. [1]

    Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan ...

  2. [2]

    Qwen2.5-vl technical report.arXiv preprint arXiv:2502.13923, 2025

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, 19 Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report...

  3. [3]

    BEiT: BERT pre-training of image transformers

    Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. BEiT: BERT pre-training of image transformers. InInternational Conference on Learning Representations, 2022

  4. [4]

    Scene text recognition with permuted autoregressive sequence models

    Darwin Bautista and Rowel Atienza. Scene text recognition with permuted autoregressive sequence models. InProceedings of the European Conference on Computer Vision, pages 178–196, 2022

  5. [5]

    Nougat: Neu- ral optical understanding for academic documents

    Lukas Blecher, Guillem Cucurull Preixens, Thomas Scialom, and Robert Stojnic. Nougat: Neu- ral optical understanding for academic documents. InInternational Conference on Learning Representations, 2024

  6. [6]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 9650–9660, 2021

  7. [7]

    Encoder-decoder with atrous separable convolution for semantic image segmentation

    Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European Conference on Computer Vision, pages 801–818, 2018

  8. [8]

    Enhancing tampered text detection through frequency feature fusion and decomposition

    Zhongxi Chen, Shen Chen, Taiping Yao, Ke Sun, Shouhong Ding, Xianming Lin, Liujuan Cao, and Rongrong Ji. Enhancing tampered text detection through frequency feature fusion and decomposition. InProceedings of the European Conference on Computer Vision, pages 200–217, 2024

Show all 137 references
  1. [9]

    Masked-attention mask transformer for universal image segmentation

    Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexander Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1290–1299, 2022

  2. [10]

    Per-pixel classification is not all you need for semantic segmentation.Advances in Neural Information Processing Systems, 34:17864–17875, 2021

    Bowen Cheng, Alex Schwing, and Alexander Kirillov. Per-pixel classification is not all you need for semantic segmentation.Advances in Neural Information Processing Systems, 34:17864–17875, 2021

  3. [11]

    M6doc: a large-scale multi-format, multi-type, multi-layout, multi-language, multi-annotation category dataset for modern document layout analysis

    Hiuyi Cheng, Peirong Zhang, Sihang Wu, Jiaxin Zhang, Qiyuan Zhu, Zecheng Xie, Jing Li, Kai Ding, and Lianwen Jin. M6doc: a large-scale multi-format, multi-type, multi-layout, multi-language, multi-annotation category dataset for modern document layout analysis. In Proceedings ...

  4. [12]

    Reproducible scaling laws for contrastive language-image learning

    Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev. Reproducible scaling laws for contrastive language-image learning. InProceedings of the IEEE/CVF Conference on Computer Vision ...

  5. [13]

    Total-text: A comprehensive dataset for scene text detection and recognition

    Chee Kheng Ch’ng and Chee Seng Chan. Total-text: A comprehensive dataset for scene text detection and recognition. InProceedings of the International Conference on Document Analysis and Recognition, pages 935–942, 2017

  6. [14]

    Icdar2019 robust reading challenge on arbitrary-shaped text-rrc-art

    Chee Kheng Chng, Yuliang Liu, Yipeng Sun, Chun Chet Ng, Canjie Luo, Zihan Ni, ChuanMing Fang, Shuaitao Zhang, Junyu Han, Errui Ding, Jingtuo Liu, Dimosthenis Karatzas, Chee Seng Chan, and Lianwen Jin. Icdar2019 robust reading challenge on arbitrary-shaped text-rrc-art. InProce...

  7. [15]

    Paddleocr-vl-1.5: Towards a multi-task 0.9b vlm for robust in-the-wild document parsing.arXiv preprint arXiv:2601.21957, 2026

    Cheng Cui, Ting Sun, Suyin Liang, Tingquan Gao, Zelun Zhang, Jiaxuan Liu, Xueqing Wang, Changda Zhou, Hongen Liu, Manhui Lin, Yue Zhang, Yubo Zhang, Yi Liu, Dianhai Yu, and Yanjun Ma. Paddleocr-vl-1.5: Towards a multi-task 0.9b vlm for robust in-the-wild document parsing.arXiv...

  8. [16]

    Boosting document parsing efficiency and performance with coarse-to-fine visual processing

    Cheng Cui, Ting Sun, Suyin Liang, Tingquan Gao, Zelun Zhang, Jiaxuan Liu, Xueqing Wang, Changda Zhou, Hongen Liu, Manhui Lin, Yue Zhang, Yubo Zhang, Jing Zhang, Jun Zhang, Xing Wei, Yi Liu, Dianhai Yu, and Yanjun Ma. Boosting document parsing efficiency and performance with co...

  9. [17]

    Vision grid transformer for document layout analysis

    Cheng Da, Chuwei Luo, Qi Zheng, and Cong Yao. Vision grid transformer for document layout analysis. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 19462–19472, 2023

  10. [18]

    Imagenet: A large- scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009

  11. [19]

    Decaf: A deep convolutional activation feature for generic visual recognition

    Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, and Trevor Darrell. Decaf: A deep convolutional activation feature for generic visual recognition. InProceedings of the International Conference on Machine Learning, pages 647–655, 2014

  12. [20]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at...

  13. [21]

    Out of length text recognition with sub-string matching

    Yongkun Du, Zhineng Chen, Caiyan Jia, Xieping Gao, and Yu-Gang Jiang. Out of length text recognition with sub-string matching. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 2798–2806, 2025

  14. [22]

    Context perception parallel decoder for scene text recognition.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    Yongkun Du, Zhineng Chen, Caiyan Jia, Xiaoting Yin, Chenxia Li, Yuning Du, and Yu-Gang Jiang. Context perception parallel decoder for scene text recognition.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

  15. [23]

    Instruction-guided scene text recognition.IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(4):2723–2738, 2025

    Yongkun Du, Zhineng Chen, Yuchen Su, Caiyan Jia, and Yu-Gang Jiang. Instruction-guided scene text recognition.IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(4):2723–2738, 2025

  16. [24]

    Svtrv2: Ctc beats encoder-decoder models in scene text recognition

    Yongkun Du, Zhineng Chen, Hongtao Xie, Caiyan Jia, and Yu-Gang Jiang. Svtrv2: Ctc beats encoder-decoder models in scene text recognition. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 20147–20156, 2025

  17. [25]

    Unirec-0.1 b: Unified text and formula recognition with 0.1 b parameters.arXiv preprint arXiv:2512.21095, 2025

    Yongkun Du, Zhineng Chen, Yazhen Xie, Weikang Bai, Hao Feng, Wei Shi, Yuchen Su, Can Huang, and Yu-Gang Jiang. Unirec-0.1 b: Unified text and formula recognition with 0.1 b parameters.arXiv preprint arXiv:2512.21095, 2025

  18. [26]

    Glm-ocr technical report.arXiv preprint arXiv:2603.10910, 2026

    Shuaiqi Duan, Yadong Xue, Weihan Wang, Zhe Su, Huan Liu, Sheng Yang, Guobing Gan, Guo Wang, Zihan Wang, Shengdong Yan, Dexin Jin, Yuxuan Zhang, Guohong Wen, Yanfeng Wang, Yutao Zhang, Xiaohan Zhang, Wenyi Hong, Yukuo Cen, Da Yin, Bin Chen, Wenmeng Yu, Xiaotao Gu, and Jie Tang....

  19. [27]

    Read like humans: Autonomous, bidirectional and iterative language modeling for scene text recognition

    Shancheng Fang, Hongtao Xie, Yuxin Wang, Zhendong Mao, and Yongdong Zhang. Read like humans: Autonomous, bidirectional and iterative language modeling for scene text recognition. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7098–7107, 2021

  20. [28]

    Mathwriting: A dataset for handwrit- ten mathematical expression recognition

    Philippe Gervais, Anastasiia Fadeeva, and Andrii Maksai. Mathwriting: A dataset for handwrit- ten mathematical expression recognition. InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V . 2, pages 5459–5469, 2025

  21. [29]

    White, Silvia C

    Ali Essam Ghareeb, Benjamin Chang, Ludovico Mitchener, Angela Yiu, Caralyn J Szostkiewicz, Dmytro Shved, Gavin J Gyimesi, Jon M Laurent, Samantha M Wright, Muhammed T Razzak, Andrew D. White, Silvia C. Finnemann, Michaela M. Hinks, and Samuel G. Rodriques. A multi-agent system...

  22. [30]

    Rich feature hierarchies for accurate object detection and semantic segmentation

    Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 580–587, 2014

  23. [31]

    Juraj Gottweis, Wei-Hung Weng, Alexander Daryin, Tao Tu, Petar Sirkovic, Artiom Myaskovsky, Grzegorz Glowaty, Felix Weissenberger, Alessio Orlandi, Dan Popovici, Anil Palepu, Keran Rong, Ryutaro Tanno, Khaled Saab, Fan Zhang, Jacob Blum, Andrew Carroll, Kavita Kulkarni, Nenad ...

  24. [32]

    Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks

    Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. InProceedings of the 23rd international conference on Machine learning, pages 369–376, 2006

  25. [33]

    Speech recognition with deep recurrent neural networks

    Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton. Speech recognition with deep recurrent neural networks. In2013 IEEE international conference on acoustics, speech and signal processing, pages 6645–6649. Ieee, 2013

  26. [34]

    Unimernet: A universal network for real-world mathematical expression recognition

    Zhuangcheng Gu, Guang Liang, Bin Wang, Zhiyuan Zhao, Qintong Zhang, Weijia Li, Chao Xu, Bo Zhang, Botian Shi, Jiang Wu, Wentao Zhang, and Conghui He. Unimernet: A universal network for real-world mathematical expression recognition. InProceedings of the IEEE/CVF Conference on ...

  27. [35]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

  28. [36]

    Icpr2018 contest on robust reading for multi-type web images

    Mengchao He, Yuliang Liu, Zhibo Yang, Sheng Zhang, Canjie Luo, Feiyu Gao, Qi Zheng, Yongpan Wang, Xin Zhang, and Lianwen Jin. Icpr2018 contest on robust reading for multi-type web images. InProceedings of the International Conference on Pattern Recognition, pages 7–12, 2018

  29. [37]

    Radiov2.5: Improved baselines for agglomerative vision founda- tion models

    Greg Heinrich, Mike Ranzinger, Hongxu Yin, Yao Lu, Jan Kautz, Andrew Tao, Bryan Catan- zaro, and Pavlo Molchanov. Radiov2.5: Improved baselines for agglomerative vision founda- tion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pa...

  30. [38]

    mplug-docowl 1.5: Unified structure learning for ocr-free document understanding

    Anwen Hu, Haiyang Xu, Jiabo Ye, Ming Yan, Liang Zhang, Bo Zhang, Ji Zhang, Qin Jin, Fei Huang, and Jingren Zhou. mplug-docowl 1.5: Unified structure learning for ocr-free document understanding. InFindings of the Association for Computational Linguistics: EMNLP 2024, pages 309...

  31. [39]

    Layoutlmv3: Pre-training for document ai with unified text and image masking

    Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, and Furu Wei. Layoutlmv3: Pre-training for document ai with unified text and image masking. InProceedings of the 30th ACM International Conference on Multimedia, pages 4083–4091, 2022

  32. [40]

    Revisiting scene text recognition: A data perspective

    Qing Jiang, Jiapeng Wang, Dezhi Peng, Chongyu Liu, and Lianwen Jin. Revisiting scene text recognition: A data perspective. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 20543–20554, 2023

  33. [41]

    Icdar 2015 competition on robust reading

    Dimosthenis Karatzas, Lluis Gomez-Bigorda, Anguelos Nicolaou, Suman Ghosh, Andrew Bagdanov, Masakazu Iwamura, Jiri Matas, Lukas Neumann, Vijay Ramaseshan Chandrasekhar, Shijian Lu, Faisal Shafait, Seiichi Uchida, and Ernest Valveny. Icdar 2015 competition on robust reading. In...

  34. [42]

    Icdar 2013 robust reading competition

    Dimosthenis Karatzas, Faisal Shafait, Seiichi Uchida, Masakazu Iwamura, Lluis Gomez i Bigorda, Sergi Robles Mestre, Joan Mas, David Fernandez Mota, Jon Almazan Almazan, and Lluis Pere De Las Heras. Icdar 2013 robust reading competition. InProceedings of the International Confe...

  35. [43]

    Ocr-free document understanding transformer

    Geewook Kim, Teakgyu Hong, Moonbin Yim, JeongYeon Nam, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, and Seunghyun Park. Ocr-free document understanding transformer. InProceedings of the European Conference on Computer Vision, pages 498–517, 2022

  36. [44]

    Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything. InProceedings of the IEEE/CVF International Conference on Computer Vision, ...

  37. [45]

    Open images v5 text annotation and yet another mask text spotter

    Ilya Krylov, Sergei Nosov, and Vladislav Sovrasov. Open images v5 text annotation and yet another mask text spotter. InAsian Conference on Machine Learning, pages 379–389, 2021

  38. [46]

    Cat-net: Compression artifact tracing network for detection and localization of image splicing

    Myung-Joon Kwon, In-Jae Yu, Seung-Hun Nam, and Heung-Kyu Lee. Cat-net: Compression artifact tracing network for detection and localization of image splicing. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 375–384, 2021

  39. [47]

    Towards better structured and less noisy web data: Oscar with register annotations

    Veronika Laippala, Anna Salmela, Samuel Rönnqvist, Alham Fikri Aji, Li-Hsin Chang, Asma Dhifallah, Larissa Goulart, Henna Kortelainen, Marc Pàmies, Deise Prina Dutra, Valtteri Skantsi, Lintang Sutawika, and Sampo Pyysalo. Towards better structured and less noisy web data: Osca...

  40. [48]

    Pix2struct: Screenshot parsing as pretraining for visual language understanding

    Kenton Lee, Mandar Joshi, Iulia Raluca Turc, Hexiang Hu, Fangyu Liu, Julian Martin Eisen- schlos, Urvashi Khandelwal, Peter Shaw, Ming-Wei Chang, and Kristina Toutanova. Pix2struct: Screenshot parsing as pretraining for visual language understanding. InProceedings of the Inter...

  41. [49]

    Building a test collection for complex document information processing

    David Lewis, Gady Agam, Shlomo Argamon, Ophir Frieder, David Grossman, and Jefferson Heard. Building a test collection for complex document information processing. InProceedings of the 29th annual international ACM SIGIR conference on Research and development in information re...

  42. [50]

    Hunyuanocr-1.5: Making lightweight ocr vlms faster and better.arXiv preprint arXiv:2607.04884, 2026

    Gengluo Li, Xingyu Wan, Shangpin Peng, Weinong Wang, Hao Feng, Yongkun Du, Binghong Wu, Zheng Ruan, Zhiqiong Lu, Liang Wu, Pengyuan Lyu, Huawen Shen, Zibin Lin, Shijing Hu, Jieneng Yang, Hongbing Wen, Guanghua Yu, Hong Liu, Bochao Wang, Can Ma, Han Hu, Chengquan Zhang, and Yu ...

  43. [51]

    Dit: Self-supervised pre-training for document image transformer

    Junlong Li, Yiheng Xu, Tengchao Lv, Lei Cui, Cha Zhang, and Furu Wei. Dit: Self-supervised pre-training for document image transformer. InProceedings of the 30th ACM International Conference on Multimedia, pages 3530–3539, 2022

  44. [52]

    Trocr: Transformer-based optical character recognition with pre- trained models

    Minghao Li, Tengchao Lv, Jingye Chen, Lei Cui, Yijuan Lu, Dinei Florencio, Cha Zhang, Zhoujun Li, and Furu Wei. Trocr: Transformer-based optical character recognition with pre- trained models. InProceedings of the AAAI conference on artificial intelligence, volume 37, pages 13...

  45. [53]

    Openvision: A fully-open, cost- effective family of advanced vision encoders for multimodal learning

    Xianhang Li, Yanqing Liu, Haoqin Tu, and Cihang Xie. Openvision: A fully-open, cost- effective family of advanced vision encoders for multimodal learning. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 3977–3987, 2025

  46. [54]

    Exploring plain vision transformer backbones for object detection

    Yanghao Li, Hanzi Mao, Ross Girshick, and Kaiming He. Exploring plain vision transformer backbones for object detection. InProceedings of the European Conference on Computer Vision, pages 280–296, 2022

  47. [55]

    dots.ocr: Multilingual document layout parsing in a single vision-language model.arXiv preprint arXiv:2512.02498, 2025

    Yumeng Li, Guang Yang, Hao Liu, Bowen Wang, and Colin Zhang. dots.ocr: Multilingual document layout parsing in a single vision-language model.arXiv preprint arXiv:2512.02498, 2025

  48. [56]

    Mdpbench: A benchmark for multilingual document parsing in real-world scenarios.arXiv preprint arXiv:2603.28130, 2026

    Zhang Li, Zhibo Lin, Qiang Liu, Ziyang Zhang, Shuo Zhang, Zidun Guo, Jiajun Song, Jiarui Zhang, Xiang Bai, and Yuliang Liu. Mdpbench: A benchmark for multilingual document parsing in real-world scenarios.arXiv preprint arXiv:2603.28130, 2026

  49. [57]

    Monkeyocr: Document parsing with a structure-recognition-relation triplet paradigm.arXiv preprint arXiv:2506.05218, 2025

    Zhang Li, Yuliang Liu, Qiang Liu, Zhiyin Ma, Ziyang Zhang, Shuo Zhang, Biao Yang, Zidun Guo, Jiarui Zhang, Xinyu Wang, and Xiang Bai. Monkeyocr: Document parsing with a structure-recognition-relation triplet paradigm.arXiv preprint arXiv:2506.05218, 2025

  50. [58]

    Monkey: Image resolution and text label are important things for large multi-modal models

    Zhang Li, Biao Yang, Qiang Liu, Zhiyin Ma, Shuo Zhang, Jingxu Yang, Yabo Sun, Yuliang Liu, and Xiang Bai. Monkey: Image resolution and text label are important things for large multi-modal models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognit...

  51. [59]

    Real-time scene text detection with differentiable binarization

    Minghui Liao, Zhaoyi Wan, Cong Yao, Kai Chen, and Xiang Bai. Real-time scene text detection with differentiable binarization. InProceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 11474–11481, 2020

  52. [60]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306, 2024

  53. [61]

    Xiaohong Liu, Yaojie Liu, Jun Chen, and Xiaoming Liu. Pscc-net: Progressive spatio-channel correlation network for image manipulation detection and localization.IEEE Transactions on Circuits and Systems for Video Technology, 32(11):7505–7517, 2022

  54. [62]

    Multi-scenario overlapping text segmen- tation with depth awareness

    Yang Liu, Xudong Xie, Yuliang Liu, and Xiang Bai. Multi-scenario overlapping text segmen- tation with depth awareness. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 17454–17463, 2025

  55. [63]

    Openvision 2: A family of generative pretrained visual encoders for multimodal learning

    Yanqing Liu, Xianhang Li, Letian Zhang, Zirui Wang, Zeyu Zheng, Yuyin Zhou, and Cihang Xie. Openvision 2: A family of generative pretrained visual encoders for multimodal learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 39164...

  56. [64]

    Multilingual denoising pre-training for neural machine translation.Transactions of the Association for Computational Linguistics, 8:726–742, 2020

    Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. Multilingual denoising pre-training for neural machine translation.Transactions of the Association for Computational Linguistics, 8:726–742, 2020

  57. [65]

    Curved scene text detection via transverse and longitudinal sequence connection.Pattern Recognition, 90:337–345, 2019

    Yuliang Liu, Lianwen Jin, Shuaitao Zhang, Canjie Luo, and Sheng Zhang. Curved scene text detection via transverse and longitudinal sequence connection.Pattern Recognition, 90:337–345, 2019

  58. [66]

    Ocrbench: on the hidden mystery of ocr in large multimodal models.Science China Information Sciences, 67(12):220102, 2024

    Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, Xu-Cheng Yin, Cheng-Lin Liu, Lianwen Jin, and Xiang Bai. Ocrbench: on the hidden mystery of ocr in large multimodal models.Science China Information Sciences, 67(12):220102, 2024

  59. [67]

    Textmonkey: An ocr-free large multimodal model for understanding document.IEEE Transac- tions on Pattern Analysis and Machine Intelligence, 48(5):6008–6019, 2026

    Yuliang Liu, Biao Yang, Qiang Liu, Zhang Li, Zhiyin Ma, Shuo Zhang, and Xiang Bai. Textmonkey: An ocr-free large multimodal model for understanding document.IEEE Transac- tions on Pattern Analysis and Machine Intelligence, 48(5):6008–6019, 2026

  60. [68]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 10012–10022, 2021

  61. [69]

    A convnet for the 2020s

    Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11976–11986, 2022

  62. [70]

    Towards end-to-end unified scene text detection and layout analysis

    Shangbang Long, Siyang Qin, Dmitry Panteleev, Alessandro Bissacco, Yasuhisa Fujii, and Michalis Raptis. Towards end-to-end unified scene text detection and layout analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022

  63. [71]

    Toward real text manipulation detection: New dataset and new solution.Pattern Recognition, 157:110828, 2025

    Dongliang Luo, Yuliang Liu, Rui Yang, Xianjin Liu, Jishen Zeng, Yu Zhou, and Xiang Bai. Toward real text manipulation detection: New dataset and new solution.Pattern Recognition, 157:110828, 2025

  64. [72]

    Chartqa: A benchmark for question answering about charts with visual and logical reasoning

    Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: A benchmark for question answering about charts with visual and logical reasoning. InFindings of the Association for Computational Linguistics: ACL 2022, pages 2263–2279, 2022

  65. [73]

    Infographicvqa

    Minesh Mathew, Viraj Bagal, Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny, and CV Jawa- har. Infographicvqa. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1697–1706, 2022

  66. [74]

    Docvqa: A dataset for vqa on document images

    Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. Docvqa: A dataset for vqa on document images. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2200–2209, 2021

  67. [75]

    Scene text recognition using higher order language priors

    Anand Mishra, Karteek Alahari, and CV Jawahar. Scene text recognition using higher order language priors. InBMVC-British Machine Vision Conference, 2012

  68. [76]

    Mineru2.5: A decoupled vision-language model for efficient high-resolution document parsing

    Junbo Niu, Zheng Liu, Zhuangcheng Gu, Bin Wang, Linke Ouyang, Zhiyuan Zhao, Tao Chu, Tianyao He, Fan Wu, Qintong Zhang, Zhenjiang Jin, Guang Liang, Rui Zhang, Wen- zheng Zhang, Yuan Qu, Zhifei Ren, Yuefeng Sun, Zirui Tang, Boyu Niu, Yuanhong Zheng, Dongsheng Ma, Ziyang Miao, H...

  69. [77]

    Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mahmoud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabb...

  70. [78]

    Omnidocbench: Benchmarking diverse pdf document parsing with comprehensive annotations

    Linke Ouyang, Yuan Qu, Hongbin Zhou, Jiawei Zhu, Rui Zhang, Qunshu Lin, Bin Wang, Zhiyuan Zhao, Man Jiang, Xiaomeng Zhao, Jin Shi, Fan Wu, Pei Chu, Minghao Liu, Zhenxiang Li, Chao Xu, Bo Zhang, Botian Shi, Zhongying Tu, and Conghui He. Omnidocbench: Benchmarking diverse pdf do...

  71. [79]

    Compositional semantic parsing on semi-structured tables

    Panupong Pasupat and Percy Liang. Compositional semantic parsing on semi-structured tables. InProceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Paper...

  72. [80]

    Doclaynet: A large human-annotated dataset for document-layout segmentation

    Birgit Pfitzmann, Christoph Auer, Michele Dolfi, Ahmed S Nassar, and Peter Staar. Doclaynet: A large human-annotated dataset for document-layout segmentation. InProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 3743–3751, 2022

  73. [81]

    Recogniz- ing text with perspective distortion in natural scenes

    Trung Quy Phan, Palaiahnakote Shivakumara, Shangxuan Tian, and Chew Lim Tan. Recogniz- ing text with perspective distortion in natural scenes. InProceedings of the IEEE International Conference on Computer Vision, pages 569–576, 2013

  74. [82]

    olmocr: Unlocking trillions of tokens in pdfs with vision language models.arXiv preprint arXiv:2502.18443, 2025

    Jake Poznanski, Aman Rangapur, Jon Borchardt, Jason Dunkelberger, Regan Huff, Daniel Lin, Christopher Wilhelm, Kyle Lo, and Luca Soldaini. olmocr: Unlocking trillions of tokens in pdfs with vision language models.arXiv preprint arXiv:2502.18443, 2025

  75. [83]

    olmocr 2: Unit test rewards for document ocr

    Jake Poznanski, Luca Soldaini, and Kyle Lo. olmocr 2: Unit test rewards for document ocr. arXiv preprint arXiv:2510.19817, 2025

  76. [84]

    Towards robust tampered text detection in document image: New dataset and new solution

    Chenfan Qu, Chongyu Liu, Yuliang Liu, Xinhong Chen, Dezhi Peng, Fengjun Guo, and Lianwen Jin. Towards robust tampered text detection in document image: New dataset and new solution. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5937...

  77. [85]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceedings of th...

  78. [86]

    Am-radio: Agglomerative vision foundation model reduce all domains into one

    Mike Ranzinger, Greg Heinrich, Jan Kautz, and Pavlo Molchanov. Am-radio: Agglomerative vision foundation model reduce all domains into one. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12490–12500, June 2024

  79. [87]

    Sam 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao-Yuan Wu, Ross Girshick, Piotr Dollar, and Christoph Feichtenh...

  80. [88]

    A robust arbitrary text detection system for natural scene images.Expert Systems with Applications, 41(18):8027–8048, 2014

    Anhar Risnumawan, Palaiahankote Shivakumara, Chee Seng Chan, and Chew Lim Tan. A robust arbitrary text detection system for natural scene images.Expert Systems with Applications, 41(18):8027–8048, 2014

  81. [89]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. InInternational Conference on Medical Image Computing and Computer-Assisted Intervention, pages 234–241, 2015

  82. [90]

    Wikimatrix: Mining 135m parallel sentences in 1620 language pairs from wikipedia

    Holger Schwenk, Vishrav Chaudhary, Shuo Sun, Hongyu Gong, and Francisco Guzmán. Wikimatrix: Mining 135m parallel sentences in 1620 language pairs from wikipedia. InPro- ceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Ma...

  83. [91]

    Baoguang Shi, Xiang Bai, and Cong Yao. An end-to-end trainable neural network for image- based sequence recognition and its application to scene text recognition.IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(11):2298–2304, 2017

  84. [92]

    Oriane Siméoni, Huy V . V o, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Michaël Ramamonjisoa, Francisco Massa, Daniel Haziza, Luca Wehrstedt, Jianyuan Wang, Timothée Darcet, Théo Moutakanni, Leonel Sentana, ...

  85. [93]

    Textocr: Towards large-scale end-to-end reasoning for arbitrary-shaped scene text

    Amanpreet Singh, Guan Pang, Mandy Toh, Jing Huang, Wojciech Galuba, and Tal Hassner. Textocr: Towards large-scale end-to-end reasoning for arbitrary-shaped scene text. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8802–8812, 2021

  86. [94]

    Kleister: key information extraction datasets involving long documents with complex layouts

    Tomasz Stanisławek, Filip Grali´nski, Anna Wróblewska, Dawid Lipi´nski, Agnieszka Kaliska, Paulina Rosalska, Bartosz Topolski, and Przemysław Biecek. Kleister: key information extraction datasets involving long documents with complex layouts. InProceedings of the International...

  87. [95]

    Icdar 2019 competition on large-scale street view text with partial labeling-rrc-lsvt

    Yipeng Sun, Zihan Ni, Chee-Kheng Chng, Yuliang Liu, Canjie Luo, Chun Chet Ng, Junyu Han, Errui Ding, Jingtuo Liu, Dimosthenis Karatzas, Chee Seng Chan, and Lianwen Jin. Icdar 2019 competition on large-scale street view text with partial labeling-rrc-lsvt. InProceedings of the ...

  88. [96]

    Deepform: Understand structured documents at scale.Weights & Biases report, 4, 2020

    Stacey Svetlichnaya. Deepform: Understand structured documents at scale.Weights & Biases report, 4, 2020

  89. [97]

    Hunyuanocr technical report.arXiv preprint arXiv:2511.19575, 2025

    Hunyuan Vision Team, Pengyuan Lyu, Xingyu Wan, Gengluo Li, Shangpin Peng, Weinong Wang, Liang Wu, Huawen Shen, Yu Zhou, Canhui Tang, Qi Yang, Qiming Peng, Bin Luo, Hower Yang, Xinsong Zhang, Jinnian Zhang, Houwen Peng, Hongming Yang, Senhao Xie, Longsha Zhou, Ge Pei, Binghong ...

  90. [98]

    Kimi k2.5: Visual agentic intelligence.arXiv preprint arXiv:2602.02276, 2026

    Kimi Team, Tongtong Bai, Yifan Bai, Yiping Bao, SH Cai, Yuan Cao, Y Charles, HS Che, Cheng Chen, Guanduo Chen, et al. Kimi k2.5: Visual agentic intelligence.arXiv preprint arXiv:2602.02276, 2026

  91. [99]

    Kwai keye-vl technical report.arXiv preprint arXiv:2507.01949, 2025

    Kwai Keye Team, Biao Yang, Bin Wen, Changyi Liu, Chenglong Chu, Chengru Song, Chongling Rao, Chuan Yi, Da Li, Dunju Zang, Fan Yang, Guorui Zhou, Hao Peng, Hao- jie Ding, Jiaming Huang, Jiangxia Cao, Jiankang Chen, Jingyun Hua, Jin Ouyang, Kaibing Chen, Kaiyu Jiang, Kaiyu Tang,...

  92. [100]

    Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features.arXiv preprint arXiv:2502.14786, 2025

    Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, Olivier Hénaff, Jeremiah Harmsen, Andreas Steiner, and Xiaohua Zhai. Siglip 2: Multilingual vision-language enc...

  93. [101]

    Coco-text: Dataset and benchmark for text detection and recognition in natural images.arXiv preprint arXiv:1601.07140, 2016

    Andreas Veit, Tomas Matera, Lukas Neumann, Jiri Matas, and Serge Belongie. Coco-text: Dataset and benchmark for text detection and recognition in natural images.arXiv preprint arXiv:1601.07140, 2016

  94. [102]

    Mineru2.5-pro: Pushing the limits of data-centric document parsing at scale.arXiv preprint arXiv:2604.04771, 2026

    Bin Wang, Tianyao He, Linke Ouyang, Fan Wu, Zhiyuan Zhao, Tao Chu, Yuan Qu, Zhenjiang Jin, Weijun Zeng, Ziyang Miao, Bangrui Xu, Junbo Niu, Mengzhang Cai, Jiantao Qiu, Qintong Zhang, Dongsheng Ma, Yuefeng Sun, Hejun Dong, Wenzheng Zhang, Jutao Xiao, Jiayong Shi, Pengyu Liao, X...

  95. [103]

    Image over text: Transforming formula recognition evaluation with character detection matching

    Bin Wang, Fan Wu, Linke Ouyang, Zhuangcheng Gu, Rui Zhang, Renqiu Xia, Botian Shi, Bo Zhang, and Conghui He. Image over text: Transforming formula recognition evaluation with character detection matching. InProceedings of the Computer Vision and Pattern Recognition Conference,...

  96. [104]

    End-to-end scene text recognition

    Kai Wang, Boris Babenko, and Serge Belongie. End-to-end scene text recognition. In Proceedings of the IEEE International Conference on Computer Vision, pages 1457–1464, 2011

  97. [105]

    Qwen2-vl: Enhancing vision- language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision- language model’s p...

  98. [106]

    Internvl3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency.arXiv preprint arXiv:2508.18265, 2025

    Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, Zhaokai Wang, Zhe Chen, Hongjie Zhang, Ganlin Yang, Haomin Wang, Qi Wei, Jinhui Yin, Wenhao Li, Erfei Cui, Guanzhou Chen, Zichen Ding, Changyao Tian, Z...

  99. [107]

    Internimage: Exploring large-scale vision foundation models with deformable convolutions

    Wenhai Wang, Jifeng Dai, Zhe Chen, Zhenhang Huang, Zhiqi Li, Xizhou Zhu, Xiaowei Hu, Tong Lu, Lewei Lu, Hongsheng Li, Xiaogang Wang, and Yu Qiao. Internimage: Exploring large-scale vision foundation models with deformable convolutions. InProceedings of the IEEE/CVF Conference ...

  100. [108]

    Shape robust text detection with progressive scale expansion network

    Wenhai Wang, Enze Xie, Xiang Li, Wenbo Hou, Tong Lu, Gang Yu, and Shuai Shao. Shape robust text detection with progressive scale expansion network. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9336–9345, 2019

  101. [109]

    From two to one: A new scene text recognizer with visual language modeling network

    Yuxin Wang, Hongtao Xie, Shancheng Fang, Jing Wang, Shenggao Zhu, and Yongdong Zhang. From two to one: A new scene text recognizer with visual language modeling network. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14194– 14203, 2021

  102. [110]

    Vary: Scaling up the vision vocabulary for large vision-language model

    Haoran Wei, Lingyu Kong, Jinyue Chen, Liang Zhao, Zheng Ge, Jinrong Yang, Jianjian Sun, Chunrui Han, and Xiangyu Zhang. Vary: Scaling up the vision vocabulary for large vision-language model. InProceedings of the European Conference on Computer Vision, pages 408–424, 2024

  103. [111]

    General ocr theory: Towards ocr-2.0 via a unified end-to-end model.arXiv preprint arXiv:2409.01704, 2024

    Haoran Wei, Chenglong Liu, Jinyue Chen, Jia Wang, Lingyu Kong, Yanming Xu, Zheng Ge, Liang Zhao, Jianjian Sun, Yuang Peng, Chunrui Han, and Xiangyu Zhang. General ocr theory: Towards ocr-2.0 via a unified end-to-end model.arXiv preprint arXiv:2409.01704, 2024

  104. [112]

    Deepseek-ocr: Contexts optical compression.arXiv preprint arXiv:2510.18234, 2025

    Haoran Wei, Yaofeng Sun, and Yukun Li. Deepseek-ocr: Contexts optical compression.arXiv preprint arXiv:2510.18234, 2025

  105. [113]

    Deepseek-ocr 2: Visual causal flow.arXiv preprint arXiv:2601.20552, 2026

    Haoran Wei, Yaofeng Sun, and Yukun Li. Deepseek-ocr 2: Visual causal flow.arXiv preprint arXiv:2601.20552, 2026

  106. [114]

    Convnext v2: Co-designing and scaling convnets with masked autoencoders

    Sanghyun Woo, Shoubhik Debnath, Ronghang Hu, Xinlei Chen, Zhuang Liu, In So Kweon, and Saining Xie. Convnext v2: Co-designing and scaling convnets with masked autoencoders. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16133–16142, 2023

  107. [115]

    Dstext v2: A comprehensive video text spotting dataset for dense and small text.Pattern Recognition, 149:110177, 2024

    Weijia Wu, Yiming Zhang, Yefei He, Luoming Zhang, Zhenyu Lou, Hong Zhou, and Xiang Bai. Dstext v2: A comprehensive video text spotting dataset for dense and small text.Pattern Recognition, 149:110177, 2024

  108. [116]

    Docgenome: An open large-scale scientific document benchmark for training and testing multi-modal large language models.arXiv preprint arXiv:2406.11633, 2024

    Renqiu Xia, Song Mao, Xiangchao Yan, Hongbin Zhou, Bo Zhang, Haoyang Peng, Jiahao Pi, Daocheng Fu, Wenjie Wu, Hancheng Ye, Shiyang Feng, Bin Wang, Chao Xu, Conghui He, Pinlong Cai, Min Dou, Botian Shi, Sheng Zhou, Yongwei Wang, Bin Wang, Junchi Yan, Fei Wu, and Yu Qiao. Docgen...

  109. [117]

    Unified perceptual parsing for scene understanding

    Tete Xiao, Yingcheng Liu, Bolei Zhou, Yuning Jiang, and Jian Sun. Unified perceptual parsing for scene understanding. InProceedings of the European Conference on Computer Vision, pages 418–434, 2018

  110. [118]

    Segformer: Simple and efficient design for semantic segmentation with transformers.Advances in Neural Information Processing Systems, 34:12077–12090, 2021

    Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M Alvarez, and Ping Luo. Segformer: Simple and efficient design for semantic segmentation with transformers.Advances in Neural Information Processing Systems, 34:12077–12090, 2021

  111. [119]

    Was: dataset and methods for artistic text segmentation

    Xudong Xie, Yuzhe Li, Yang Liu, Zhifei Zhang, Zhaowen Wang, Wei Xiong, and Xiang Bai. Was: dataset and methods for artistic text segmentation. InProceedings of the European Conference on Computer Vision, pages 237–254, 2024

  112. [120]

    Rethinking text segmentation: A novel dataset and a text-specific refinement approach

    Xingqian Xu, Zhifei Zhang, Zhaowen Wang, Brian Price, Zhonghao Wang, and Humphrey Shi. Rethinking text segmentation: A novel dataset and a text-specific refinement approach. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12045–12055, 2021

  113. [121]

    Language matters: A weakly supervised vision-language pre-training approach for scene text detection and spotting

    Chuhui Xue, Wenqing Zhang, Yu Hao, Shijian Lu, Philip HS Torr, and Song Bai. Language matters: A weakly supervised vision-language pre-training approach for scene text detection and spotting. InProceedings of the European Conference on Computer Vision, pages 284–302, 2022

  114. [122]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jia...

  115. [123]

    A large-scale dataset for end-to-end table recognition in the wild.Scientific Data, 10(1):110, 2023

    Fan Yang, Lei Hu, Xinwu Liu, Shuangping Huang, and Zhenghui Gu. A large-scale dataset for end-to-end table recognition in the wild.Scientific Data, 10(1):110, 2023

  116. [124]

    Reading and writing: Discriminative and generative modeling for self-supervised text recognition

    Mingkun Yang, Minghui Liao, Pu Lu, Jing Wang, Shenggao Zhu, Hualin Luo, Qi Tian, and Xiang Bai. Reading and writing: Discriminative and generative modeling for self-supervised text recognition. InProceedings of the 30th ACM International Conference on Multimedia, pages 4214–4223, 2022

  117. [125]

    Dptext-detr: Towards better scene text detection with dynamic points in transformer

    Maoyuan Ye, Jing Zhang, Shanshan Zhao, Juhua Liu, Bo Du, and Dacheng Tao. Dptext-detr: Towards better scene text detection with dynamic points in transformer. InProceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 3241–3249, 2023

  118. [126]

    Deepsolo: Let transformer decoder with explicit points solo for text spotting

    Maoyuan Ye, Jing Zhang, Shanshan Zhao, Juhua Liu, Tongliang Liu, Bo Du, and Dacheng Tao. Deepsolo: Let transformer decoder with explicit points solo for text spotting. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19348–19357, 2023

  119. [127]

    Benchmarking chinese text recognition: Datasets, baselines, and an empirical study.arXiv preprint arXiv:2112.15093, 2021

    Haiyang Yu, Jingye Chen, Bin Li, Jianqi Ma, Mengnan Guan, Xixi Xu, Xiaocong Wang, Shaobo Qu, and Xiangyang Xue. Benchmarking chinese text recognition: Datasets, baselines, and an empirical study.arXiv preprint arXiv:2112.15093, 2021

  120. [128]

    Eaformer: scene text segmentation with edge-aware transformers

    Haiyang Yu, Teng Fu, Bin Li, and Xiangyang Xue. Eaformer: scene text segmentation with edge-aware transformers. InProceedings of the European Conference on Computer Vision, pages 410–427, 2024

  121. [129]

    Icdar 2023 competition on structured text extraction from visually-rich document images

    Wenwen Yu, Chengquan Zhang, Haoyu Cao, Wei Hua, Bohan Li, Huang Chen, Mingyu Liu, Mingrui Chen, Jianfeng Kuang, Mengjun Cheng, Yuning Du, Shikun Feng, Xiaoguang Hu, Pengyuan Lyu, Kun Yao, Yuechen Yu, Yuliang Liu, Wanxiang Che, Errui Ding, Cheng-Lin Liu, Jiebo Luo, Shuicheng Ya...

  122. [130]

    Object-contextual representations for semantic segmentation

    Yuhui Yuan, Xilin Chen, and Jingdong Wang. Object-contextual representations for semantic segmentation. InProceedings of the European Conference on Computer Vision, pages 173–190, 2020

  123. [131]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 11975–11986, 2023

  124. [132]

    Vitaev2: Vision transformer advanced by exploring inductive bias for image recognition and beyond.International Journal of Computer Vision, 131(5):1141–1162, 2023

    Qiming Zhang, Yufei Xu, Jing Zhang, and Dacheng Tao. Vitaev2: Vision transformer advanced by exploring inductive bias for image recognition and beyond.International Journal of Computer Vision, 131(5):1141–1162, 2023

  125. [133]

    Icdar 2019 robust reading challenge on reading chinese text on signboard

    Rui Zhang, Yongsheng Zhou, Qianyi Jiang, Qi Song, Nan Li, Kai Zhou, Lei Wang, Dong Wang, Minghui Liao, Mingkun Yang, et al. Icdar 2019 robust reading challenge on reading chinese text on signboard. InProceedings of the International Conference on Document Analysis and Recognit...

  126. [134]

    Exploring the capabilities of large multimodal models on dense text

    Shuo Zhang, Biao Yang, Zhang Li, Zhiyin Ma, Yuliang Liu, and Xiang Bai. Exploring the capabilities of large multimodal models on dense text. InProceedings of the International Conference on Document Analysis and Recognition, pages 281–298, 2024

  127. [135]

    Llavar: Enhanced visual instruction tuning for text-rich image understanding.arXiv preprint arXiv:2306.17107, 2024

    Yanzhe Zhang, Ruiyi Zhang, Jiuxiang Gu, Yufan Zhou, Nedim Lipka, Diyi Yang, and Tong Sun. Llavar: Enhanced visual instruction tuning for text-rich image understanding.arXiv preprint arXiv:2306.17107, 2024

  128. [136]

    Paddleocr-vl-1.6: Expanding the frontier of document parsing with under- optimized region refinement and progressive post-training.arXiv preprint arXiv:2606.03264, 2026

    Zelun Zhang, Hongen Liu, Suyin Liang, Yubo Zhang, Yiqing Xiang, Jiaxuan Liu, Ting Sun, Manhui Lin, Yue Zhang, Changda Zhou, Tingquan Gao, Cheng Cui, Yi Liu, Dianhai Yu, and Yanjun Ma. Paddleocr-vl-1.6: Expanding the frontier of document parsing with under- optimized region ref...

  129. [137]

    Multimodal ocr: Parse anything from documents.arXiv preprint arXiv:2603.13032, 2026

    Handong Zheng, Yumeng Li, Kaile Zhang, Liang Xin, Guangwei Zhao, Hao Liu, Jiayu Chen, Jie Lou, Qi Fu, Rui Yang, Shuo Jiang, Weijian Luo, Weijie Su, Weijun Zhang, Xingyu Zhu, Yabin Li, Yiwei Ma, Yu Chen, Yuqiu Ji, Zhaohui Yu, Guang Yang, Colin Zhang, Lei Zhang, Yuliang Liu, and...

Pith tools

Reviewed July 14, 2026 · model on record in the stance chip above.