Pith. sign in

REVIEW 3 major objections 96 references

Scene-aware synthesis of document–schema–annotation triples, without hand-crafted templates, trains compact multimodal models to extract key fields more accurately from real visual documents.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 16:00 UTC pith:3RFPCY7M

load-bearing objection Solid data-centric KIE paper: template-free multi-agent synthesis plus error-driven hard cases lifts small Qwen3-VL models on UniKIE, with real scaling and field-error evidence, but same-family generator coupling and single-benchmark scope keep the transfer claim provisional. the 3 major comments →

arxiv 2607.04636 v1 pith:3RFPCY7M submitted 2026-07-06 cs.CV

Enhancing Large Multimodal Models in Key Information Extraction via Scene-Aware Document Synthesis

Zhipeng Xu , Zulong Chen , Qing Liu , Junhao Ji , Jinxin Hu , Yipeng Yu , Jianqiang Wan , Jun Tang
show 1 more author
Zhao Li
This is my paper
classification cs.CV
keywords key information extractiondocument synthesislarge multimodal modelson-device inferencescene-aware generationerror-driven trainingvisual document understandingschema-aware extraction
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Key information extraction turns invoices, receipts, forms, and contracts into structured data, but strong results usually need expensive server-side multimodal models, while small models that can run on-device lack enough supervised examples covering real layouts and schemas. The paper presents SAYRE: from a few real exemplars of a document category it perceives content patterns and layout conventions, then generates new document images paired with extraction schemas and field annotations—no manual template design. It also rewrites real model and production failures into many hard examples that keep the same difficult page structures. Fine-tuning Qwen3-VL 2B and 4B models on this data raises field-level F1 on both fixed-schema and open-schema settings, makes the 4B model the strongest among tested on-device systems, and continues to help as more synthetic data is added, especially for smaller models and open-category extraction. Error analysis attributes the gains to better schema-aware reading of dense tables, business identifiers, and contract clauses. A reader who needs private, low-latency extraction cares because better training data can narrow the capability gap without scaling model size or hand-labeling enterprise documents.

Core claim

Scene-aware document synthesis that produces document–schema–annotation triples from a few category exemplars, plus error-driven expansion of real failure cases while preserving their structures, supplies scalable supervision that consistently improves compact large multimodal models on constrained- and open-category key information extraction, yields the strongest overall on-device results on UniKIE, and reduces field-level errors by strengthening schema-aware extraction over dense tables, identifiers, and contract clauses.

What carries the argument

SAYRE: a multi-agent, template-free synthesis loop that perceives content and layout from few exemplars, samples a topic and persona, generates structured annotations and a schema, renders HTML into a document image (with optional schema adaptation), and an error-driven branch that turns real failure pages into rewritten hard examples with aligned labels.

Load-bearing premise

Documents made from a few exemplars and rewritten failure cases, using the same family of large models that help build them, are assumed to match the layout and schema diversity of real evaluation documents closely enough that measured gains will hold in true deployment.

What would settle it

Train the same backbone on equal volume of non-scene-aware or purely random synthetic data, or evaluate on a held-out real document corpus never seen by the synthesis models or failure logs; if open-category UniKIE F1 no longer rises by a similar margin, the central claim that scene-aware and error-driven synthesis is what transfers is falsified.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Compact on-device multimodal models can approach much larger server models on practical KIE when given this form of synthetic supervision.
  • Open-category extraction keeps improving as more synthesized data is added, so data scale remains a lever after architecture is fixed.
  • Field-level mistakes on line items, business IDs, and contract clauses fall when models train on structure-preserving hard examples.
  • Category coverage can be expanded without hand-crafted templates, lowering the cost of multi-domain enterprise KIE.
  • Smaller backbones gain disproportionately, improving the case for local deployment under cost and latency constraints.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same exemplar-plus-failure loop could supply curriculum data for other layout-heavy tasks such as table understanding or multi-page form filling.
  • Production error logs could become a continuous source of hard training examples without new human template design.
  • If dense-table and clause gains dominate, similar synthesis may help any multimodal extractor that currently fails on repeated rows or long contractual text.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 0 minor

Summary. The paper proposes SAYRE, a template-free, scene-aware multi-agent framework that synthesizes document–schema–annotation triples for Key Information Extraction (KIE). From a few category exemplars it perceives content and layout, samples topics/personas, generates structured labels and HTML-rendered documents (Eqs. 1–6), and further expands real failure cases by HTML templating, text rewrite, and label update (Eqs. 7–11), with optional visual degradation. Fine-tuning Qwen3-VL-2B/4B on ~1M such samples yields consistent gains on UniKIE under constrained- and open-category settings (Table 1: SAYRE-4B avg F1 72.92 vs foundation 64.25; SAYRE-2B 70.47 vs 60.34), the best overall among listed on-device LMMs, with upward scaling as synthetic volume grows (Fig. 1) and field-error reductions concentrated on line-items, IDs/amounts, and contract clauses (Fig. 2, −17.8% total).

Significance. If the gains transfer beyond the Qwen-centric synthesis stack, the work is a useful data-centric contribution for practical multimodal KIE: it targets the real bottleneck of scarce schema-aligned supervision for compact on-device LMMs, avoids hand-crafted templates, and shows both scaling and error-type evidence rather than only aggregate F1. The combination of exemplar-guided generation with error-driven hard-example expansion is a clear, deployable recipe. Strengths include multi-scale backbones, constrained vs open evaluation, data-volume curves, and field-level error breakdown. The main significance risk is that reported improvements may partly reflect generator–evaluator family coupling rather than fully transferable scene-aware synthesis; resolving that would make the result more decisive for the community.

major comments (3)
  1. Implementation Details and §3.1–3.2: perception (Qwen-VL-Max), content/HTML generation (Qwen3-Max), and error templates (Qwen3-VL-Plus) all come from the same family as the fine-tuned backbones and as strong on-server baselines in Table 1. Without a cross-generator ablation (e.g., non-Qwen synthesizer, or training on SAYRE data while evaluating a non-Qwen on-device model) or a real-document-only control, it is hard to rule out that gains partly reflect distributional alignment with Qwen-friendly layouts/phrasing rather than transferable scene-aware supervision. This is load-bearing for the central claim of practical, generalizable KIE improvement.
  2. §4–5 / Table 1: UniKIE is cited as Ji et al. (2026) with overlapping authorship. The paper should state clearly whether any UniKIE categories, exemplars, or schemas overlap the enterprise exemplars or failure corpus used for synthesis, and whether any evaluation documents (or near-duplicates) entered the synthetic pipeline. A short contamination/overlap audit is needed to support the open-category generalization narrative in §5.1 and Fig. 1(b).
  3. §3.1–3.2 and §5: there is no ablation separating general exemplar-guided synthesis from error-driven generation, nor a comparison against simpler non-scene-aware synthesis (template fill, content replacement, or random HTML). Table 1 and Fig. 1 therefore cannot attribute gains specifically to “scene-aware” perception of content/layout patterns versus volume of synthetic KIE triples or hard-example mining alone. At least one such control is needed for the method claim.

Circularity Check

0 steps flagged

No derivation circularity; empirical training gains on a co-authored but external benchmark are not forced by construction.

full rationale

SAYRE is an empirical data-synthesis + fine-tuning paper. The generation process (Eqs. 1–11) produces document–schema–annotation triples from exemplars and rewritten failure cases; those triples are then used to train Qwen3-VL backbones whose field-level F1 is measured on UniKIE. Nothing in the reported numbers is definitionally equal to a fitted input or to a self-cited uniqueness claim. The sole mild self-reference is the evaluation benchmark UniKIE (Ji et al., 2026), which shares authors with the present work; that is ordinary self-citation of a test set, not a load-bearing premise that forces the claimed improvements. Generation and evaluation both use Qwen-family models, but that is a potential distribution-shift concern, not a circular reduction of the form “prediction = input by construction.” No fitted parameter is renamed a prediction, no ansatz is smuggled via self-citation, and no uniqueness theorem is invoked. Score 1 reflects only the minor co-authorship of the benchmark; the central claim remains independently measured.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 2 invented entities

The central claim is empirical systems work: synthetic supervision improves compact LMMs on KIE. It rests on standard ML training assumptions, the fidelity of multi-agent document generation, exact-match field F1 after normalization, and several hand-chosen training/generation knobs. No new physical entities are postulated; the invented pieces are methodological constructs (the multi-agent pipeline and error-driven rewrite loop).

free parameters (4)
  • synthetic_data_volume
    About 1M synthesized instances are generated and used for 15K-step fine-tuning; scaling curves treat data quantity as a controllable lever that drives reported gains.
  • learning_rate_and_schedule
    LR 5e-6 with cosine schedule, 15K steps, global batch 64, max sequence 8192, max image resolution 1,605,632 — hand-chosen training hyperparameters that affect final F1.
  • n_exemplars_per_category
    General generation conditions on a few exemplars n; the paper does not fix or ablate n as a reported sensitivity, yet the claim depends on small-n scene capture working.
  • persona_sample_size
    1M elite personas from Persona Hub are sampled to diversify content; this is a large discrete design choice, not derived.
axioms (4)
  • domain assumption Exact-match field-level F1 after normalization is an adequate primary metric for practical KIE quality.
    Evaluation Metrics section follows Yang et al. (2025) / Hwang et al. (2019); soft matches and partial credit are not primary.
  • domain assumption VLM/LLM agents can accurately perceive category content patterns and layout conventions from few exemplars and render faithful HTML documents.
    Core of §3.1 equations (2)–(5); without this, synthetic triples are not valid supervision.
  • ad hoc to paper Rewriting failure-case text while preserving HTML structure yields hard examples that remain distributionally useful after de-identification.
    §3.2 error-driven pipeline (7)–(11); multi-model voting for unmapped fields is an additional paper-specific fix.
  • domain assumption Full-parameter fine-tuning of Qwen3-VL-2B/4B on synthetic data does not erase general capabilities in a way that invalidates UniKIE gains as KIE improvement.
    Implicit in experimental methodology; only KIE F1 is reported.
invented entities (2)
  • SAYRE multi-agent scene-aware synthesis framework no independent evidence
    purpose: Generate document–schema–annotation triples from exemplars without hand-crafted templates.
    Methodological system composed of perception, content, layout, document, and schema agents; not an external physical entity, but the paper’s central construct.
  • Error-driven generation loop over real failure cases no independent evidence
    purpose: Expand scarce hard layouts into many de-identified hard training samples.
    Defined operationally in §3.2; usefulness is evidenced only by the paper’s own ablations/overall gains, not by an external independent test of the loop alone.

pith-pipeline@v1.1.0-grok45 · 18668 in / 3622 out tokens · 31104 ms · 2026-07-11T16:00:59.184529+00:00 · methodology

0 comments
read the original abstract

Key Information Extraction (KIE) converts visually rich documents into structured data, but practical deployment remains challenging: strong performance often relies on costly on-server Large Multimodal Models (LMMs), while compact locally deployable models lack sufficient KIE supervision. We present SAYRE, a scene-aware document synthesis framework for generating scalable KIE training data without hand-crafted template design. Given a few exemplar documents, SAYRE captures category-specific content patterns and layout conventions to synthesize document-schema-annotation triples. It further introduces error-driven generation, which expands real-world failure cases into hard training examples while preserving their structural patterns. Experiments on constrained- and open-category KIE show that SAYRE consistently improves Qwen3-VL backbones and achieves the strongest overall performance among on-device LMMs. Data scaling experiments show an overall upward trend as more synthesized data is introduced, especially for smaller models and open-category extraction. Error analysis further shows that synthesized training reduces field-level errors by improving schema-aware extraction over dense tables, business identifiers, and contract clauses. These results establish scene-aware synthesis as an effective data-centric approach for improving practical multimodal KIE.

Figures

Figures reproduced from arXiv: 2607.04636 by Jianqiang Wan, Jinxin Hu, Junhao Ji, Jun Tang, Qing Liu, Yipeng Yu, Zhao Li, Zhipeng Xu, Zulong Chen.

Figure 2
Figure 2. Figure 2: Field-Level Error Analysis of Qwen3-VL-4B [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

96 extracted references · 27 linked inside Pith

  1. [1]

    arXiv preprint arXiv:2602.07038 , year=

    UNIKIE-BENCH: Benchmarking Large Multimodal Models for Key Information Extraction in Visual Documents , author=. arXiv preprint arXiv:2602.07038 , year=

    Pith/arXiv arXiv

  2. [2]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages=

    Cleaner pretraining corpus curation with neural web scraping , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages=

  3. [3]

    Proceedings of the 14th ACM international conference on Information and knowledge management , pages=

    ViPER: augmenting automatic information extraction with visual perceptions , author=. Proceedings of the 14th ACM international conference on Information and knowledge management , pages=

  4. [4]

    ACM Computing Surveys , volume=

    Deep learning based key information extraction from business documents: Systematic literature review , author=. ACM Computing Surveys , volume=. 2025 , publisher=

    2025

  5. [5]

    Artificial Intelligence Review , volume=

    A survey of recent approaches to form understanding in scanned documents , author=. Artificial Intelligence Review , volume=. 2024 , publisher=

    2024

  6. [6]

    arXiv preprint arXiv:2103.14470 , year=

    Spatial dual-modality graph reasoning for key information extraction , author=. arXiv preprint arXiv:2103.14470 , year=

    Pith/arXiv arXiv

  7. [7]

    Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , pages=

    Chargrid: Towards understanding 2d documents , author=. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , pages=

    2018

  8. [8]

    2019 International Conference on Document Analysis and Recognition , pages=

    ICDAR2019 Competition on Scanned Receipt OCR and Information Extraction , author=. 2019 International Conference on Document Analysis and Recognition , pages=. 2019 , organization=

    2019

  9. [9]

    arXiv preprint arXiv:2501.02235 , year=

    Survey on question answering over visually rich documents: Methods, challenges, and trends , author=. arXiv preprint arXiv:2501.02235 , year=

    Pith/arXiv arXiv

  10. [10]

    Proceedings of the 30th ACM international conference on multimedia , pages=

    Layoutlmv3: Pre-training for document ai with unified text and image masking , author=. Proceedings of the 30th ACM international conference on multimedia , pages=

  11. [11]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Information extraction from visually rich documents using LLM-based organization of documents into independent textual segments , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  12. [12]

    Artificial Intelligence Review , year=

    Deep learning based visually rich document content understanding: A survey , author=. Artificial Intelligence Review , year=

  13. [13]

    2020 25th International conference on pattern recognition (ICPR) , pages=

    Pick: processing key information extraction from documents using improved graph learning-convolutional networks , author=. 2020 25th International conference on pattern recognition (ICPR) , pages=. 2021 , organization=

    2020

  14. [14]

    arXiv preprint arXiv:2602.13235 , year=

    Lang2Act: Fine-Grained Visual Reasoning through Self-Emergent Linguistic Toolchains , author=. arXiv preprint arXiv:2602.13235 , year=

    Pith/arXiv arXiv

  15. [15]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    Vidorag: Visual document retrieval-augmented generation via dynamic iterative reasoning agents , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  16. [16]

    arXiv preprint arXiv:2503.13964 , year=

    Mdocagent: A multi-modal multi-agent framework for document understanding , author=. arXiv preprint arXiv:2503.13964 , year=

    Pith/arXiv arXiv

  17. [17]

    arXiv preprint arXiv:2505.22019 , year=

    Vrag-rl: Empower vision-perception-based rag for visually rich information understanding via iterative reasoning with reinforcement learning , author=. arXiv preprint arXiv:2505.22019 , year=

    Pith/arXiv arXiv

  18. [18]

    Proceedings of the 30th ACM international conference on multimedia , pages=

    Dit: Self-supervised pre-training for document image transformer , author=. Proceedings of the 30th ACM international conference on multimedia , pages=

  19. [19]

    Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages=

    Visually guided generative text-layout pre-training for document intelligence , author=. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages=

    2024

  20. [20]

    Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    MarkupLM: Pre-training of text and markup language for visually rich document understanding , author=. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  21. [21]

    5-VL Technical Report , author=

    Qwen2. 5-VL Technical Report , author=. arXiv preprint arXiv:2502.13923 , year=

    Pith/arXiv arXiv

  22. [22]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    A Bounding Box is Worth One Token-Interleaving Layout and Text in a Large Language Model for Document Understanding , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  23. [23]

    Proceedings of the 2024 conference on empirical methods in natural language processing , pages=

    Modeling layout reading order as ordering relations for visually-rich document understanding , author=. Proceedings of the 2024 conference on empirical methods in natural language processing , pages=

    2024

  24. [24]

    Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformer , booktitle =

    Rafa. Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformer , booktitle =. 2021 , publisher =

    2021

  25. [25]

    Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages=

    Vrdu: A benchmark for visually-rich document understanding , author=. Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages=

  26. [26]

    Advances in neural information processing systems , volume=

    Chain-of-thought prompting elicits reasoning in large language models , author=. Advances in neural information processing systems , volume=

  27. [27]

    IDPLeaderboard: A Unified Leaderboard for Intelligent Document Processing Tasks , author=

  28. [28]

    Science China Information Sciences , volume=

    Ocrbench: on the hidden mystery of ocr in large multimodal models , author=. Science China Information Sciences , volume=. 2024 , publisher=

    2024

  29. [29]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Instructdoc: A dataset for zero-shot generalization of visual document understanding with instructions , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  30. [30]

    Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    Lmdx: Language model-based document information extraction and localization , author=. Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    2024

  31. [31]

    What is the value of templates?

    “What is the value of templates?” Rethinking Document Information Extraction Datasets for LLMs , author=. Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=

    2024

  32. [32]

    Proceedings of the 31st International Conference on Computational Linguistics , pages=

    Relayout: Towards real-world document understanding via layout-enhanced pre-training , author=. Proceedings of the 31st International Conference on Computational Linguistics , pages=

  33. [33]

    arXiv preprint arXiv:2405.14295 , year=

    Focus anywhere for fine-grained multi-page document understanding , author=. arXiv preprint arXiv:2405.14295 , year=

    Pith/arXiv arXiv

  34. [34]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Dockylin: A large multimodal model for visual document understanding with efficient visual slimming , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  35. [35]

    proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Monkey: Image resolution and text label are important things for large multi-modal models , author=. proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  36. [36]

    arXiv preprint arXiv:2410.05970 , year=

    Wukong: A large multimodal model for efficient long pdf reading with end-to-end sparse sampling , author=. arXiv preprint arXiv:2410.05970 , year=

    Pith/arXiv arXiv

  37. [37]

    arXiv preprint arXiv:2410.21169 , year=

    Document parsing unveiled: Techniques, challenges, and prospects for structured information extraction , author=. arXiv preprint arXiv:2410.21169 , year=

    Pith/arXiv arXiv

  38. [38]

    ACM Transactions on Information Systems , volume=

    Large language models in document intelligence: A comprehensive survey, recent advances, challenges, and future trends , author=. ACM Transactions on Information Systems , volume=. 2025 , publisher=

    2025

  39. [39]

    Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=

    mplug-docowl 1.5: Unified structure learning for ocr-free document understanding , author=. Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=

    2024

  40. [40]

    Findings of the Association for Computational Linguistics: EMNLP 2023 , pages=

    Ureader: Universal ocr-free visually-situated language understanding with multimodal large language model , author=. Findings of the Association for Computational Linguistics: EMNLP 2023 , pages=

    2023

  41. [41]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , year=

    Textmonkey: An ocr-free large multimodal model for understanding document , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , year=

  42. [42]

    Neurocomputing , pages=

    See then tell: Enhancing key information extraction with vision grounding , author=. Neurocomputing , pages=. 2026 , publisher=

    2026

  43. [43]

    Findings of the Association for Computational Linguistics: EMNLP 2023 , pages=

    Genkie: Robust generative multimodal document key information extraction , author=. Findings of the Association for Computational Linguistics: EMNLP 2023 , pages=

    2023

  44. [44]

    Transactions of the Association for Computational Linguistics , volume=

    VILA: Improving structured content extraction from scientific PDFs using visual layout groups , author=. Transactions of the Association for Computational Linguistics , volume=. 2022 , publisher=

    2022

  45. [45]

    arXiv preprint arXiv:2603.02789 , year=

    OCR or Not? Rethinking Document Information Extraction in the MLLMs Era with Real-World Large-Scale Datasets , author=. arXiv preprint arXiv:2603.02789 , year=

    arXiv

  46. [46]

    arXiv preprint arXiv:2104.08836 , year=

    Layoutxlm: Multimodal pre-training for multilingual visually-rich document understanding , author=. arXiv preprint arXiv:2104.08836 , year=

    Pith/arXiv arXiv

  47. [47]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track) , pages=

    Revise: A framework for revising ocred text in practical information systems with data contamination strategy , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track) , pages=

  48. [48]

    Proceedings of the 31st International Conference on Computational Linguistics: Industry Track , pages=

    FS-DAG: Few shot domain adapting graph networks for visually rich document understanding , author=. Proceedings of the 31st International Conference on Computational Linguistics: Industry Track , pages=

  49. [49]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Selfdoc: Self-supervised document representation learning , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  50. [50]

    Findings of the Association for Computational Linguistics: EMNLP 2022 , pages=

    Ernie-layout: Layout knowledge enhanced pre-training for visually-rich document understanding , author=. Findings of the Association for Computational Linguistics: EMNLP 2022 , pages=

    2022

  51. [51]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Doctr: Document transformer for structured information extraction in documents , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  52. [52]

    Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Lilt: A simple yet effective language-independent layout transformer for structured document understanding , author=. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  53. [53]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Docllm: A layout-aware generative language model for multimodal document understanding , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  54. [54]

    Proceedings of the 29th ACM international conference on multimedia , pages=

    Structext: Structured text understanding with multi-modal transformers , author=. Proceedings of the 29th ACM international conference on multimedia , pages=

  55. [55]

    Proceedings of the IEEE/CVF international conference on computer vision , pages=

    Docformer: End-to-end transformer for document understanding , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=

  56. [56]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Bros: A pre-trained language model focusing on text and layout for better key information extraction from documents , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  57. [57]

    International Conference of the Cross-Language Evaluation Forum for European Languages , pages=

    Business document information extraction: Towards practical benchmarks , author=. International Conference of the Cross-Language Evaluation Forum for European Languages , pages=. 2022 , organization=

    2022

  58. [58]

    Proceedings of the 28th ACM International Conference on Multimedia , pages=

    TRIE: end-to-end text reading and information extraction for document understanding , author=. Proceedings of the 28th ACM International Conference on Multimedia , pages=

  59. [59]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    Reading order matters: information extraction from visually-rich documents by token path prediction , author=. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    2023

  60. [60]

    Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages=

    Cost-effective end-to-end information extraction for semi-structured document images , author=. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages=

    2021

  61. [61]

    2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR) , volume=

    Cloudscan-a configuration-free invoice analysis system using recurrent neural networks , author=. 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR) , volume=. 2017 , organization=

    2017

  62. [62]

    arXiv preprint arXiv:2111.08609 , year=

    Document ai: Benchmarks, models and applications , author=. arXiv preprint arXiv:2111.08609 , year=

    Pith/arXiv arXiv

  63. [63]

    Proceedings of the 2010 ACM SIGMOD International Conference on Management of data , pages=

    Enterprise information extraction: recent developments and open challenges , author=. Proceedings of the 2010 ACM SIGMOD International Conference on Management of data , pages=

    2010

  64. [64]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Cc-ocr: A comprehensive and challenging ocr benchmark for evaluating large multimodal models in literacy , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  65. [65]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Scaling text-rich image understanding via code-guided synthetic multimodal data generation , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  66. [66]

    arXiv preprint arXiv:2503.19786 , year =

    Gemma 3 Technical Report , author =. arXiv preprint arXiv:2503.19786 , year =

    Pith/arXiv arXiv

  67. [67]

    Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Industry Papers) , pages=

    Graph convolution for multimodal information extraction from visually rich documents , author=. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Industry Papers) , pages=

    2019

  68. [68]

    Knowledge and Information Systems , volume=

    Visual information extraction , author=. Knowledge and Information Systems , volume=. 2006 , publisher=

    2006

  69. [69]

    International summer school on information extraction , pages=

    Information extraction: Techniques and challenges , author=. International summer school on information extraction , pages=. 1997 , publisher=

    1997

  70. [70]

    arXiv preprint arXiv:2406.20094 , year=

    Scaling synthetic data creation with 1,000,000,000 personas , author=. arXiv preprint arXiv:2406.20094 , year=

    Pith/arXiv arXiv

  71. [71]

    arXiv preprint arXiv:2504.05299 , year=

    Smolvlm: Redefining small and efficient multimodal models , author=. arXiv preprint arXiv:2504.05299 , year=

    Pith/arXiv arXiv

  72. [72]

    arXiv preprint arXiv:2601.08584 , year=

    Ministral 3 , author=. arXiv preprint arXiv:2601.08584 , year=

    Pith/arXiv arXiv

  73. [73]

    arXiv preprint arXiv:2601.10611 , year=

    Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding , author=. arXiv preprint arXiv:2601.10611 , year=

    Pith/arXiv arXiv

  74. [74]

    5 technical report , author=

    Ovis2. 5 technical report , author=. arXiv preprint arXiv:2508.11737 , year=

    Pith/arXiv arXiv

  75. [75]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Improved baselines with visual instruction tuning , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  76. [76]

    arXiv preprint arXiv:2503.01743 , year=

    Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture-of-loras , author=. arXiv preprint arXiv:2503.01743 , year=

    Pith/arXiv arXiv

  77. [77]

    arXiv preprint arXiv:2509.18154 , year=

    Minicpm-v 4.5: Cooking efficient mllms via architecture, data, and training recipe , author=. arXiv preprint arXiv:2509.18154 , year=

    Pith/arXiv arXiv

  78. [78]

    arXiv preprint arXiv:2511.21631 , year=

    Qwen3-vl technical report , author=. arXiv preprint arXiv:2511.21631 , year=

    Pith/arXiv arXiv

  79. [79]

    5: Advancing open-source multimodal models in versatility, reasoning, and efficiency , author=

    Internvl3. 5: Advancing open-source multimodal models in versatility, reasoning, and efficiency , author=. arXiv preprint arXiv:2508.18265 , year=

    Pith/arXiv arXiv

  80. [80]

    Proceedings of the IEEE/CVF winter conference on applications of computer vision , pages=

    Docvqa: A dataset for vqa on document images , author=. Proceedings of the IEEE/CVF winter conference on applications of computer vision , pages=

Showing first 80 references.