Pith. sign in

REVIEW 10 cited by

mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understanding

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2409.03420 v2 pith:3NYXXJTO submitted 2024-09-05 cs.CV

mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understanding

Anwen Hu , Haiyang Xu , Liang Zhang , Jiabo Ye , Ming Yan , Ji Zhang , Qin Jin , Fei Huang
show 1 more author
Jingren Zhou
This is my paper
classification cs.CV
keywords documentmulti-pageunderstandingdocowl2high-resolutionperformancetokensvisual
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Multimodel Large Language Models(MLLMs) have achieved promising OCR-free Document Understanding performance by increasing the supported resolution of document images. However, this comes at the cost of generating thousands of visual tokens for a single document image, leading to excessive GPU memory and slower inference times, particularly in multi-page document comprehension. In this work, to address these challenges, we propose a High-resolution DocCompressor module to compress each high-resolution document image into 324 tokens, guided by low-resolution global visual features. With this compression module, to strengthen multi-page document comprehension ability and balance both token efficiency and question-answering performance, we develop the DocOwl2 under a three-stage training framework: Single-image Pretraining, Multi-image Continue-pretraining, and Multi-task Finetuning. DocOwl2 sets a new state-of-the-art across multi-page document understanding benchmarks and reduces first token latency by more than 50%, demonstrating advanced capabilities in multi-page questioning answering, explanation with evidence pages, and cross-page structure understanding. Additionally, compared to single-image MLLMs trained on similar data, our DocOwl2 achieves comparable single-page understanding performance with less than 20% of the visual tokens. Our codes, models, and data are publicly available at https://github.com/X-PLUG/mPLUG-DocOwl/tree/main/DocOwl2.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning

    cs.CV 2024-12 accept novelty 7.0

    OCRBench v2 is a new benchmark with four times more tasks than prior versions that reveals most large multimodal models score below 50 out of 100 on visual text tasks and share five specific weaknesses.

  2. VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents

    cs.IR 2024-10 conditional novelty 7.0

    VisRAG achieves 20-40% better end-to-end performance than text-based RAG by directly embedding and retrieving document images with VLMs.

  3. BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perception

    cs.CV 2026-07 conditional novelty 6.0

    BVS combines early-stop attention rollout priors with a scale-aware non-stationary kernel and GP-UCB to locate tiny objects in UHR images more accurately and with fewer MLLM queries than prior visual-search methods.

  4. Towards Real-World Document Parsing via Realistic Scene Synthesis and Document-Aware Training

    cs.CV 2026-03 unverdicted novelty 6.0

    A realistic scene synthesis strategy and document-aware training recipe enable a 1B-parameter MLLM to achieve superior accuracy and robustness in end-to-end parsing of real-world captured documents.

  5. MoDora: Tree-Based Semi-Structured Document Analysis System

    cs.IR 2026-02 unverdicted novelty 6.0

    MoDora introduces local-alignment aggregation, a Component-Correlation Tree, and question-type-aware retrieval to improve accuracy on semi-structured document QA by 5.97-61.07% over baselines.

  6. RPC-Bench: A Fine-grained Benchmark for Research Paper Comprehension

    cs.CL 2026-01 conditional novelty 6.0

    RPC-Bench supplies 15K verified QA pairs and a research-flow taxonomy that shows top foundation models still achieve only 68.2 percent correctness-completeness on academic paper comprehension.

  7. Unveil: Unified Visual-Textual Integration and Distillation for Multi-modal Document Retrieval

    cs.CL 2026-05 unverdicted novelty 4.0

    Unveil proposes a visual-textual embedding model for multi-modal documents that is distilled into an efficient visual-only retriever.

  8. MinerU: An Open-Source Solution for Precise Document Content Extraction

    cs.CV 2024-09 conditional novelty 4.0

    MinerU delivers an open-source pipeline for high-precision document content extraction by integrating specialized models with tuned preprocessing and postprocessing rules.

  9. A Survey on MLLM-based Visually Rich Document Understanding: Methods, Challenges, and Emerging Trends

    cs.CV 2025-07 unverdicted novelty 3.0

    A survey of MLLM-based Visually Rich Document Understanding covering feature integration techniques, training paradigms, challenges like data scarcity, and emerging trends such as RAG and agentic frameworks.

  10. Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extraction

    cs.MM 2024-10 unverdicted novelty 3.0

    Survey proposing a taxonomy for document parsing into pipeline-based systems and VLM-driven unified models, reviewing components, metrics, benchmarks, and challenges.