Pith. sign in

REVIEW 5 cited by

READoc: A Unified Benchmark for Realistic Document Structured Extraction

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2409.05137 v3 pith:TR6HWDPS submitted 2024-09-08 cs.CL cs.CV

READoc: A Unified Benchmark for Realistic Document Structured Extraction

Zichao Li , Aizier Abulaiti , Yaojie Lu , Xuanang Chen , Jia Zheng , Hongyu Lin , Xianpei Han , Le Sun This is my paper
classification cs.CL cs.CV
keywords evaluationreadocunifiedbenchmarkrealisticstructureddocumentdocuments
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
read the original abstract

Document Structured Extraction (DSE) aims to extract structured content from raw documents. Despite the emergence of numerous DSE systems, their unified evaluation remains inadequate, significantly hindering the field's advancement. This problem is largely attributed to existing benchmark paradigms, which exhibit fragmented and localized characteristics. To address these limitations and offer a thorough evaluation of DSE systems, we introduce a novel benchmark named READoc, which defines DSE as a realistic task of converting unstructured PDFs into semantically rich Markdown. The READoc dataset is derived from 3,576 diverse and real-world documents from arXiv, GitHub, and Zenodo. In addition, we develop a DSE Evaluation S$^3$uite comprising Standardization, Segmentation and Scoring modules, to conduct a unified evaluation of state-of-the-art DSE approaches. By evaluating a range of pipeline tools, expert visual models, and general VLMs, we identify the gap between current work and the unified, realistic DSE objective for the first time. We aspire that READoc will catalyze future research in DSE, fostering more comprehensive and practical solutions.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Do VLMs Read or Rewrite? On Transcription Faithfulness in Vision-Language Models

    cs.AI 2026-05 conditional novelty 7.0

    General-purpose VLMs systematically rewrite perturbed words back to the original — up to 4.5 WER points on English — with rewriting tied to representation similarity and word length.

  2. From PDF to RAG-Ready: Evaluating Document Conversion Frameworks for Domain-Specific Question Answering

    cs.IR 2026-03 unverdicted novelty 7.0

    Docling with hierarchical splitting reaches 94.1% RAG accuracy on domain documents, beating naive PDF loading but trailing manual Markdown curation at 97.1%.

  3. XtraGPT: Context-Aware and Controllable Academic Paper Revision via Human-AI Collaboration

    cs.CL 2025-05 conditional novelty 6.0

    XtraGPT is a suite of 1.5B-14B parameter open-source LLMs fine-tuned on 140,000 revision pairs from 7,000 top-tier papers to support controllable, context-aware academic paper editing.

  4. From PDF to RAG-Ready: Evaluating Document Conversion Frameworks for Domain-Specific Question Answering

    cs.IR 2026-03 conditional novelty 5.0

    Docling with hierarchical splitting and image descriptions reaches 94.1% RAG QA accuracy on Portuguese admin PDFs, beating manual Markdown and other open-source converters.

  5. Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extraction

    cs.MM 2024-10 unverdicted novelty 3.0

    Survey proposing a taxonomy for document parsing into pipeline-based systems and VLM-driven unified models, reviewing components, metrics, benchmarks, and challenges.