REVIEW 7 cited by
mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understanding
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Multimodel Large Language Models(MLLMs) have achieved promising OCR-free Document Understanding performance by increasing the supported resolution of document images. However, this comes at the cost of generating thousands of visual tokens for a single document image, leading to excessive GPU memory and slower inference times, particularly in multi-page document comprehension. In this work, to address these challenges, we propose a High-resolution DocCompressor module to compress each high-resolution document image into 324 tokens, guided by low-resolution global visual features. With this compression module, to strengthen multi-page document comprehension ability and balance both token efficiency and question-answering performance, we develop the DocOwl2 under a three-stage training framework: Single-image Pretraining, Multi-image Continue-pretraining, and Multi-task Finetuning. DocOwl2 sets a new state-of-the-art across multi-page document understanding benchmarks and reduces first token latency by more than 50%, demonstrating advanced capabilities in multi-page questioning answering, explanation with evidence pages, and cross-page structure understanding. Additionally, compared to single-image MLLMs trained on similar data, our DocOwl2 achieves comparable single-page understanding performance with less than 20% of the visual tokens. Our codes, models, and data are publicly available at https://github.com/X-PLUG/mPLUG-DocOwl/tree/main/DocOwl2.
Forward citations
Cited by 7 Pith papers
-
BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perception
BVS combines early-stop attention rollout priors with a scale-aware non-stationary kernel and GP-UCB to locate tiny objects in UHR images more accurately and with fewer MLLM queries than prior visual-search methods.
-
Improving MLLM's Document Image Machine Translation via Synchronously Self-reviewing Its OCR Proficiency
A fine-tuning paradigm that prompts MLLMs to self-generate OCR text before translating document images improves DIMT quality and reduces catastrophic forgetting of OCR.
-
Docopilot: Improving Multimodal Models for Document-Level Understanding
A new academic-paper dataset and a retrieval-free fine-tuned InternVL2 model improve multi-page document QA accuracy and latency on several benchmarks.
-
Spatially Grounded Explanations in Vision Language Models for Document Visual Question Answering
EaGERS uses a vision language model's own explanation to mask a document image to relevant grid regions, then re-asks the question, improving DocVQA accuracy while grounding answers spatially.
-
ExpliCIT-QA: Explainable Code-Based Image Table Question Answering
ExpliCIT-QA turns table-image question answering into a traceable sequence of table extraction, natural-language reasoning, code generation, execution, and explanation.
-
DRISHTIKON: Visual Grounding at Multiple Granularities in Documents
A combined OCR, LLM, and fuzzy-matching pipeline locates answer spans in document images at block, line, word, and point granularity, with line-level grounding F1 of 69.10 on a new 70-document benchmark.
-
Survey of Specialized Large Language Model
A survey of 24 specialized LLMs (2022-2025) claims a shift from domain fine-tuning to native architectures, but the synthesis is undermined by citation errors and selection bias.
Discussion (0). Sign in to comment.