REVIEW 10 cited by
TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models
read the original abstract
Text recognition is a long-standing research problem for document digitalization. Existing approaches are usually built based on CNN for image understanding and RNN for char-level text generation. In addition, another language model is usually needed to improve the overall accuracy as a post-processing step. In this paper, we propose an end-to-end text recognition approach with pre-trained image Transformer and text Transformer models, namely TrOCR, which leverages the Transformer architecture for both image understanding and wordpiece-level text generation. The TrOCR model is simple but effective, and can be pre-trained with large-scale synthetic data and fine-tuned with human-labeled datasets. Experiments show that the TrOCR model outperforms the current state-of-the-art models on the printed, handwritten and scene text recognition tasks. The TrOCR models and code are publicly available at \url{https://aka.ms/trocr}.
Forward citations
Cited by 10 Pith papers
-
Koshur Pixel: a large-scale synthetic ocr dataset for kashmiri
Koshur Pixel is the first large-scale synthetic OCR dataset for Kashmiri with 613,078 image-text pairs generated via SynthOCR-Gen from the KS-PRET-5M corpus across multiple fonts and granularities with 25+ augmentations.
-
The Character Error Vector: Decomposable errors for page-level OCR evaluation
The Character Error Vector is a decomposable bag-of-characters evaluator for page-level OCR that remains defined under parsing errors and bridges parsing metrics with local CER.
-
CodeOCR: On the Effectiveness of Vision Language Models in Code Understanding
Multimodal LLMs process code as images to achieve up to 8x token compression, with visual cues like syntax highlighting aiding tasks and clone detection remaining resilient or even improving under compression.
-
RaV-IDP: A Reconstruction-as-Validation Framework for Faithful Intelligent Document Processing
RaV-IDP adds a reconstruction step after extraction and scores how faithfully the reconstruction matches the original document region to provide label-free validation for intelligent document processing.
-
Nougat: Neural Optical Understanding for Academic Documents
Nougat applies a visual transformer to convert academic PDFs into markup language while accurately handling mathematical content on a new scientific document dataset.
-
PaLM-E: An Embodied Multimodal Language Model
PaLM-E is a single 562B-parameter multimodal model that performs embodied reasoning tasks like robotic manipulation planning and visual question answering by interleaving vision, state, and text inputs with positive t...
-
From Handwriting to Structured Data: Benchmarking AI Digitisation of Handwritten Forms
Frontier multimodal LLMs achieve ~85% accuracy and ~90% weighted F1 on digitizing complex handwritten medical forms, with Gemini 3.1 strongest overall and prompt optimization lifting macro metrics over 60%.
-
Towards Selection of Large Multimodal Models as Engines for Burned-in Protected Health Information Detection in Medical Images
Empirical benchmark of GPT-4o, Gemini 2.5 Flash, and Qwen 2.5 7B finds superior OCR performance over EasyOCR but inconsistent gains in overall PHI detection accuracy, with strongest improvements on complex imprint patterns.
-
Cleansing Jewel: A Neural Spelling Correction Model Built On Google OCR-ed Tibetan Manuscripts
A Transformer augmented with a confidence score mechanism outperforms LSTM and GRU baselines on correcting OCR errors in paired Tibetan manuscript data.
-
Comparative Analysis of Liquid Neural Networks and LSTM for Sequential Pattern Recognition: Robustness, Efficiency, and Clinical Utility
Benchmarking study reports that Closed-form Continuous-time Liquid Neural Networks outperform LSTMs in parameter efficiency and robustness to temporal dropout on neuromorphic, drawing, handwriting, and sepsis predicti...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.