Pith. sign in

REVIEW 3 major objections 6 minor 58 references

Information Extraction from Heterogeneous Documents without Ground Truth Labels using Synthetic Label Generation and Knowledge Distillation

T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A small model trained on synthetic labels matches a state-of-the-art commercial large multimodal model on expense-document extraction, with 85% lower cost and about 5x faster inference.

desk verdict A useful distillation pipeline whose at-par claim is only supported on noisy internal labels and contradicted by the exact-label benchmark. read the letter →

arxiv 2411.14957 v2 pith:HFPPP4MQ submitted 2024-11-22 cs.CL

Aniket Bhattacharyya , Anurag Tripathi This is my paper · ORCID
classification cs.CL
keywords visuallyrichdocumentunderstandingsyntheticlabelgenerationknowledgedistillationmultimodallargelanguagemodelsexpensereceiptextractiontask-awarepromptingnoisylabelsANLS
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that organizations can build accurate information-extraction systems for heterogeneous receipts and invoices even when no ground-truth labels exist. It introduces TAIL, a prompting protocol that gets a large commercial multimodal model (Claude 3 Sonnet) to produce synthetic labels in a required output format, and then fine-tunes a small open-source vision-language model, LLaVA-Net, on those labels using response-based knowledge distillation. On an internal expense dataset of a multinational company, the distilled model matches or slightly beats the teacher on merchant name, amount, and date extraction while costing about 85% less and running about five times faster. The paper also shows the student outperforms layout-aware baselines on heterogeneous documents, and demonstrates how the extracted fields can flag overpayment risk.

What carries the argument

The load-bearing mechanism is TAIL (Task Aware Instruction-based Labelling), a set of task-specific prompt templates that constrain the teacher's output to the exact format required for each field (e.g., 'only the value, no extra words'). These teacher outputs serve as soft labels for response-based knowledge distillation, in which a LLaVA model (CLIP vision encoder plus a Mistral-based language model) is fine-tuned with LoRA. The paper also uses the Laplacian variance of receipt images as a quality gate: the at-par claim holds when most images are sharp (variance ≥ 200), and the student's advantage over layout-aware models is attributed to its reasoning ability rather than template matching.

What would settle it

Take a random sample of, say, 500 internal receipts for which the employees typed merchant name, amount, and date; have human annotators transcribe the true values from the images; then compute ANLS for LLaVA-Net, Sonnet, and the noisy labels against these human-verified values. If LLaVA-Net's human-verified ANLS falls significantly below Sonnet's, the central at-par claim fails; if noisy labels score lower than both models, the paper's premise that noisy labels are a usable benchmark is also weakened.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that synthetic labels from a state-of-the-art LMM, generated without access to the teacher's weights or training data, can transfer enough task knowledge to a much smaller student model for it to reach parity with the teacher on in-domain receipt extraction. Specifically, LLaVA-Net achieves ANLS of 52% on merchant name, 70% on amount, and 83% on date against noisy employee-typed labels, versus Sonnet's 50%, 70%, and 76%, under the condition that most images have Laplacian variance at or above 200. The student also shows fewer refusals and fewer format violations than the teacher, and produces the desired output format exclusively.

Load-bearing premise

The at-par comparison on internal data relies on employee-typed noisy labels as the yardstick, even though the paper shows these labels often disagree with the receipt images; if those noisy labels are biased toward one model, the claimed parity could be an artifact.

Editorial extensions

If this is right

  • Expense teams can train custom extraction models for fields that employees do not type in, since TAIL supplies labels where no noisy labels exist.
  • Deploying the 7B-parameter student on a single GPU instance instead of calling a closed API reduces annual cost by roughly 85% and raises throughput about fivefold, making large-scale receipt auditing more feasible.
  • Layout-aware models such as LayoutLMv3 trained on the same synthetic labels lag the student by more than 10% ANLS on heterogeneous corpora, because they fail on rare receipt templates and rely on fragile OCR.
  • Because the student rarely refuses and rarely deviates from the requested format, downstream automation (policy checks, duplicate detection, overpayment flags) can trust its output structure.
  • An overpayment-detection case study shows that comparing model-extracted amounts to employee-typed amounts surfaces invoices where reimbursement likely exceeds the receipt, at about $2 per flagged document.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next step the paper leaves implicit is evaluating both teacher and student against human-verified ground truth on the internal corpus; without that, the at-par claim is only relative to noisy labels.
  • The same TAIL-and-distill recipe could transfer to other structured-document domains (contracts, medical records, bank forms) whenever a closed LMM is acceptable as a one-time labeler and a small internal model is wanted for production.
  • One could iterate the procedure: use LLaVA-Net itself to label fresh document streams and periodically distill back, which the paper does not explore.
  • A testable extension is to measure whether the student's higher ANLS on amount and date reflects genuinely better reading or simply overfitting to the noisy-label distribution.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes TAIL (Task Aware Instruction-based Labelling), which uses Claude 3 Sonnet to generate synthetic labels for visually rich documents without ground truth, and distills these labels into a 7B LLaVA-based student (LLaVA-Net) via response-based knowledge distillation without using the teacher's weights or training data. The authors evaluate LLaVA-Net on an internal ExpenseQA dataset (merchant name, amount, transaction date) against employee-typed noisy labels, and on the CORD benchmark with exact labels under M=1 and M=24 settings, reporting ANLS and tree-edit distance scores. They claim the student matches or exceeds Sonnet on ExpenseQA while being about 85% cheaper and roughly 5x faster, and they demonstrate an overpayment-detection use case.

Significance. If the central claims hold, the work would be practically valuable: it shows a feasible pipeline for training small open models on proprietary document collections with only API access to a teacher, without sharing sensitive data, and it quantifies cost and latency benefits. Strengths include the use of an external benchmark with exact labels (CORD), the explicit acknowledgment that employee-typed fields are noisy (Sec. 3, Fig. 1), the RQ4 analysis showing the student is not a literal replica of Sonnet (87/74/82% ANLS with Sonnet, zero refusals vs 3%), and the cost/speed comparison. However, the main 'at par or better' claim depends on a validation set whose ground truth is itself shown to be unreliable, and the exact-label evidence on CORD points in the opposite direction.

major comments (3)
  1. [Sec. 5.3.1, Table 3] The central claim that LLaVA-Net performs 'at par or better' than Sonnet on ExpenseQA is measured against employee-typed noisy labels, which the paper itself shows are frequently inconsistent with the receipt image (Sec. 3, Fig. 1, Tab. 2). Because TAIL labels are designed to match the image rather than the typed fields, ANLS against noisy labels does not measure true extraction accuracy; a high score can be achieved only when the noisy label happens to match the image. The reported differences (52% vs 50% merchant, 70% vs 70% amount, 83% vs 76% date) may reflect different disagreement patterns with noisy labels rather than better extraction. No human-verified ground truth is provided for ExpenseQA, so the abstract's at-par claim is not established.
  2. [Sec. 5.3.3, Table 5] The paper asserts that on images with Laplacian variance >= 200, LLaVA-Net's ANLS exceeds 90%, 'almost at par with Sonnet,' but Table 5 reports only LLaVA-Net and LayoutLMV3; Sonnet's scores on the same quality-stratified subsets are never given. On the only benchmark with exact labels (CORD), Table 3 shows LLaVA-Net at 84% vs Sonnet's 94% (M=1) and 69% vs 93% (M=24), far from at par. The abstract's claim that the external dataset demonstrates conditions of at-par performance is therefore unsupported. A quality-stratified exact-label comparison with Sonnet would be needed to substantiate the stated condition.
  3. [Tables 3-5] All results are reported as point estimates without confidence intervals, significance tests, or multiple seeds. Given the close margins on ExpenseQA (e.g., 52% vs 50% for merchant name) and the small validation set (n=1000), the differences may be within noise. The paper should report variance estimates or at least state the number of runs; this is especially important because the headline 'at par or better' claim depends on these comparisons.
minor comments (6)
  1. [Sec. 3] The notation M=1 for ExpenseQA is confusing, since three attributes (merchant name, amount, date) are extracted; please clarify whether M denotes the number of attributes per query or the number of output fields.
  2. [Table 3] The dash for LayoutLMV3 on transaction date is explained in the text, but a footnote in the table would improve readability and prevent readers from interpreting it as an omission.
  3. [Throughout] There are several naming inconsistencies: 'LLaV A-Net', 'LlaV A-Net', 'LLaVA-Net', and 'LLaV A' are used interchangeably; please unify the notation.
  4. [Introduction and Conclusion] Minor typos: 'V AT' should be 'VAT' in the introduction, and 'multimodel student model' in the conclusion should be 'multimodal student model'.
  5. [Sec. 5.3.5] The overpayment detection result (89 invoices, about $2 per document) is reported without context on the total number of invoices assessed or the distribution of discrepancies; adding this context would strengthen the use-case claim.
  6. [Sec. 4.1] The exact internal prompts are withheld for confidentiality, which is understandable, but the paper should state more explicitly which parts of the pipeline are reproducible by others (e.g., CORD prompts are in the supplementary material, while ExpenseQA prompts are not).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: evaluation targets are independent of the teacher, RQ4 checks replica behavior, and the main at-par gap is a missing Sonnet stratified control rather than a circular reduction.

full rationale

Most of the derivation chain is self-contained and not circular. TAIL labels are generated by Claude 3 Sonnet from images; the student LLaVA-Net is trained on those labels and then evaluated against (i) employee-typed noisy labels on ExpenseQA and (ii) crowdsourced exact labels on CORD. Neither evaluation target is the teacher's output, so the at-par claim is not a self-comparison. The paper explicitly tests the replica concern in RQ4 by reporting ANLS/TED between student and teacher (87/74/82 and 86/87) and showing zero refusals vs 3%, so the student is not forced to equal Sonnet. The CORD exact-label benchmark provides independent grounding, even though it shows the student below Sonnet (84% vs 94% M=1; 69% vs 93% M=24), which is an honest negative result. The only load-bearing gap is in RQ3 (Sec 5.3.3): the assertion that at Laplacian variance >=200 LLaVA-Net is 'almost at par with Sonnet' is not backed by Sonnet scores on the same stratified subsets (Table 5 lists only LLaVA-Net and LayoutLMV3), so the at-par-on-good-images condition is under-supported; but this is a missing control, not a circular reduction. The sentence 'TAIL labels for date were designed to match noisy labels instead of the date formats in invoices' could be read as a value-level alignment, but in context it refers to output format matching the noisy-label format (yyyymmdd) rather than invoice-native formats, and no equation or fitted parameter makes the date evaluation equal to the training target. No self-citation chain or imported uniqueness theorem is present.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim relies on four domain assumptions: teacher label quality, noisy labels as evaluation ground truth, Laplacian variance as image quality proxy, and the effectiveness of response-based distillation for reasoning transfer. Two hand-chosen design parameters, the TAIL prompt templates and the output formats, plus the post hoc image-quality threshold, are free parameters in the method. No invented entities are introduced.

free parameters (3)
  • Laplacian variance threshold for image quality = 200
    Chosen post hoc in Sec. 5.3.3 to define the condition under which LLaVA-Net approaches Sonnet performance; the threshold itself is not derived or pre-registered.
  • TAIL prompt templates = Hand-designed role, example, missing-example, and rules prompts (Table 1)
    Prompt format was iterated by hand to enforce output format; no systematic prompt search is reported, and internal prompts are withheld for confidentiality.
  • Internal output formats = Merchant: few words; amount: numeric without currency sign; date: yyyymmdd
    Formats were chosen to match noisy employee labels rather than invoice text, which affects the date task in particular and is the stated reason for excluding LayoutLMV3 on date.
assumptions (4)
  • domain assumption Claude 3 Sonnet's generated labels are accurate enough to serve as training targets
    The entire TAIL method assumes the closed-source LMM provides high-quality annotations; the paper reports 94% ANLS against CORD labels under M=1 but does not verify on internal documents.
  • domain assumption Employee-entered noisy labels are a usable evaluation ground truth for internal ExpenseQA
    Sec. 3 states noisy labels are 'the closest fields available for ground truth'; the central at-par claim is measured against these imperfect labels.
  • domain assumption Laplacian variance is a valid proxy for image quality relevant to LMM extraction performance
    Introduced in Sec. 5.3.3 without validation against other image quality metrics or human ratings of blur.
  • domain assumption Response-based knowledge distillation transfers reasoning and rare-format handling from teacher to student
    The paper assumes fine-tuning on teacher outputs alone, without teacher weights or training data, yields the student's reasoning ability; this is the premise of Sec. 4.2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Information Extraction from Heterogeneous Documents without Ground Truth Labels using Synthetic Label Generation and Knowledge Distillation." pith.science (2026). https://pith.science/paper/HFPPP4MQ

@misc{pith2026241114957,
  author       = {Pith},
  title        = {Pith review of: Information Extraction from Heterogeneous Documents without Ground Truth Labels using Synthetic Label Generation and Knowledge Distillation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HFPPP4MQ}},
  note         = {Machine review of arXiv:2411.14957}
}
read the original abstract

Invoices and receipts submitted by employees are visually rich documents (VRDs) with textual, visual and layout information. To protect against the risk of fraud and abuse, it is crucial for organizations to efficiently extract desired information from submitted receipts. This helps in the assessment of key factors such as appropriateness of the expense claim, adherence to spending and transaction policies, the validity of the receipt, as well as downstream anomaly detection at various levels. These documents are heterogeneous, with multiple formats and languages, uploaded with different image qualities, and often do not contain ground truth labels for the efficient training of models. In this paper we propose Task Aware Instruction-based Labelling (TAIL), a method for synthetic label generation in VRD corpuses without labels, and fine-tune a multimodal Visually Rich Document Understanding Model (VRDU) on TAIL labels using response-based knowledge distillation without using the teacher model's weights or training dataset to conditionally generate annotations in the appropriate format. Using a benchmark external dataset where ground truth labels are available, we demonstrate conditions under which our approach performs at par with Claude 3 Sonnet through empirical studies. We then show that the resulting model performs at par or better on the internal expense documents of a large multinational organization than state-of-the-art LMM (large multimodal model) Claude 3 Sonnet while being 85% less costly and ~5X faster, and outperforms layout-aware baselines by more than 10% in Average Normalized Levenshtein Similarity (ANLS) scores due to its ability to reason and extract information from rare formats. Finally, we illustrate the usage of our approach in overpayment prevention.

Figures

Figures reproduced from arXiv: 2411.14957 by the authors.

Figure 1
Figure 1. Two receipts where the manually typed merchant name [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. LLaVA-Net correctly extracts amount from an invoice [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Receipt where employee typed in 600 euros as the [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 49 canonical work pages

  1. [1]

    Form2Seq : A framework for higher-order form structure extraction

    Milan Aggarwal, Hiresh Gupta, Mausoom Sarkar, and Balaji Krishnamurthy. Form2Seq : A framework for higher-order form structure extraction. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Pro- cessing (EMNLP), pages 3830–3840, Online, Nov. 2020. As- sociation for Co...

    work page 2020

  2. [2]

    Data protection, 2024

    Amazon. Data protection, 2024. 4

    work page 2024

  3. [3]

    The claude 3 model family: Opus, sonnet, haiku,

    Anthropic. The claude 3 model family: Opus, sonnet, haiku,

  4. [4]

    Introducing the next generation of claude, 2024

    Anthropic. Introducing the next generation of claude, 2024. 2, 3

    work page 2024

  5. [5]

    Prompt engineering, 2024

    Anthropic. Prompt engineering, 2024. 4

    work page 2024

  6. [6]

    Manmatha

    Srikar Appalaraju, Bhavan Jasani, Bhargava Urala Kota, Yusheng Xie, and R. Manmatha. Docformer: End-to-end transformer for document understanding, 2021. 3

    work page 2021

  7. [7]

    Wukong-reader: Multi-modal pre-training for fine-grained visual document understanding, 2022

    Haoli Bai, Zhiguang Liu, Xiaojun Meng, Wentao Li, Shuang Liu, Nian Xie, Rongfu Zheng, Liangwei Wang, Lu Hou, Jiansheng Wei, Xin Jiang, and Qun Liu. Wukong-reader: Multi-modal pre-training for fine-grained visual document understanding, 2022. 3

    work page 2022

  8. [8]

    Unilmv2: Pseudo-masked lan- guage models for unified language model pre-training, 2020

    Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Xiaodong Liu, Yu Wang, Songhao Piao, Jianfeng Gao, Ming Zhou, and Hsiao-Wuen Hon. Unilmv2: Pseudo-masked lan- guage models for unified language model pre-training, 2020. 3

    work page 2020

Show all 58 references
  1. [9]

    Ali Furkan Biten, Ruben Tito, Andres Mafla, Lluis Gomez, Marc ¸al Rusi˜nol, Ernest Valveny, C. V . Jawahar, and Dimos- thenis Karatzas. Scene text visual question answering, 2019. 4, 6

    2019

  2. [10]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakan- tan, Pranav Shyam, Girish Sastry, Amanda Askell, Sand- hini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler...

    2020

  3. [11]

    Gonzalez, Ion Stoica, and Eric P

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhang- hao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yong- hao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023. 5

    2023

  4. [12]

    Laura Chiticariu, Yunyao Li, and Frederick R. Reiss. Rule- based information extraction is dead! long live rule-based information extraction systems! In David Yarowsky, Tim- othy Baldwin, Anna Korhonen, Karen Livescu, and Steven Bethard, editors, Proceedings of the 2013 Confer...

    2013

  5. [13]

    Binarized neural networks: Training deep neural networks with weights and activations constrained to +1 or -1, 2016

    Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. Binarized neural networks: Training deep neural networks with weights and activations constrained to +1 or -1, 2016. 3

    2016

  6. [14]

    Denk and Christian Reisswig

    Timo I. Denk and Christian Reisswig. Bertgrid: Contextual- ized embedding for 2d document representation and under- standing, 2019. 2, 3

    2019

  7. [15]

    Bert: Pre-training of deep bidirectional trans- formers for language understanding, 2019

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional trans- formers for language understanding, 2019. 3

    2019

  8. [16]

    Docparser: End-to-end ocr-free information extraction from visually rich documents, 2023

    Mohamed Dhouib, Ghassen Bettaieb, and Aymen Shabou. Docparser: End-to-end ocr-free information extraction from visually rich documents, 2023. 3

    2023

  9. [17]

    Haralick, and I.T

    Jaekyu Ha, R.M. Haralick, and I.T. Phillips. Recursive x- y cut using bounding boxes of connected components. In Proceedings of 3rd International Conference on Document Analysis and Recognition , volume 2, pages 952–955 vol.2,

  10. [18]

    A table detection method for pdf documents based on convo- lutional neural networks

    Leipeng Hao, Liangcai Gao, Xiaohan Yi, and Zhi Tang. A table detection method for pdf documents based on convo- lutional neural networks. In 2016 12th IAPR Workshop on Document Analysis Systems (DAS), pages 287–292, 2016. 2, 3

    2016

  11. [19]

    Ocr with tesseract, amazon textract, and google document ai: a benchmarking experiment

    Thomas Hegghammer. Ocr with tesseract, amazon textract, and google document ai: a benchmarking experiment. Jour- nal of Computational Social Science, 5, 05 2022. 2

    2022

  12. [20]

    Distilling the knowledge in a neural network, 2015

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network, 2015. 3

    2015

  13. [21]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021. 5

    2021

  14. [22]

    Cre- ating something from nothing: Unsupervised knowledge dis- tillation for cross-modal hashing, 2020

    Hengtong Hu, Lingxi Xie, Richang Hong, and Qi Tian. Cre- ating something from nothing: Unsupervised knowledge dis- tillation for cross-modal hashing, 2020. 3

    2020

  15. [23]

    Layoutlmv3: Pre-training for document ai with unified text and image masking, 2022

    Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, and Furu Wei. Layoutlmv3: Pre-training for document ai with unified text and image masking, 2022. 2, 3, 6

    2022

  16. [24]

    Spatial dependency parsing for semi-structured document information extraction, 2021

    Wonseok Hwang, Jinyeong Yim, Seunghyun Park, Sohee Yang, and Minjoon Seo. Spatial dependency parsing for semi-structured document information extraction, 2021. 3

    2021

  17. [25]

    Funsd: A dataset for form understanding in noisy scanned documents, 2019

    Guillaume Jaume, Hazim Kemal Ekenel, and Jean-Philippe Thiran. Funsd: A dataset for form understanding in noisy scanned documents, 2019. 3

    2019

  18. [26]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lam- ple, Lucile Saulnier, L ´elio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Tho...

    2023

  19. [27]

    Chargrid: Towards understanding 2d documents, 2018

    Anoop Raveendra Katti, Christian Reisswig, Cordula Guder, Sebastian Brarda, Steffen Bickel, Johannes H ¨ohne, and Jean Baptiste Faddoul. Chargrid: Towards understanding 2d documents, 2018. 2, 3

    2018

  20. [28]

    A diagram is worth a dozen images, 2016

    Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. A diagram is worth a dozen images, 2016. 4

    2016

  21. [29]

    Formnetv2: Multimodal graph contrastive learning for form document information extraction, 2023

    Chen-Yu Lee, Chun-Liang Li, Hao Zhang, Timothy Dozat, Vincent Perot, Guolong Su, Xiang Zhang, Kihyuk Sohn, Nikolai Glushnev, Renshen Wang, Joshua Ainslie, Shang- bang Long, Siyang Qin, Yasuhisa Fujii, Nan Hua, and Tomas Pfister. Formnetv2: Multimodal graph contrastive learning...

    2023

  22. [30]

    Lewis, G

    D. Lewis, G. Agam, S. Argamon, O. Frieder, D. Grossman, and J. Heard. Building a test collection for complex doc- ument information processing. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’06,...

    2006

  23. [31]

    Improved baselines with visual instruction tuning, 2023

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning, 2023. 2, 3, 5

    2023

  24. [32]

    Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang

    Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts, 2023. 4

    2023

  25. [33]

    Roberta: A robustly optimized bert pretraining approach, 2019

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettle- moyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach, 2019. 3

    2019

  26. [34]

    Repre- sentation learning for information extraction from form-like documents

    Bodhisattwa Prasad Majumder, Navneet Potti, Sandeep Tata, James Bradley Wendt, Qi Zhao, and Marc Najork. Repre- sentation learning for information extraction from form-like documents. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors, Proceedings of th...

    2020

  27. [35]

    Marinai, M

    S. Marinai, M. Gori, and G. Soda. Artificial neural networks for document analysis and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(1):23–35,

  28. [36]

    Chartqa: A benchmark for question an- swering about charts with visual and logical reasoning, 2022

    Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: A benchmark for question an- swering about charts with visual and logical reasoning, 2022. 4

    2022

  29. [37]

    Minesh Mathew, Dimosthenis Karatzas, and C. V . Jawahar. Docvqa: A dataset for vqa on document images, 2021. 3, 4

    2021

  30. [38]

    Open language learning for information extraction

    Mausam, Michael Schmitz, Stephen Soderland, Robert Bart, and Oren Etzioni. Open language learning for information extraction. In Jun’ichi Tsujii, James Henderson, and Mar- ius Pas ¸ca, editors,Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Pr...

    2012

  31. [39]

    doctr: Document text recognition

    Mindee. doctr: Document text recognition. https:// github.com/mindee/doctr, 2021. 6

    2021

  32. [40]

    O’Gorman

    L. O’Gorman. The document spectrum for page layout anal- ysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(11):1162–1173, 1993. 3

    1993

  33. [41]

    Gpt-4 is openai’s most advanced system, 2023

    OpenAI. Gpt-4 is openai’s most advanced system, 2023. 2, 3

    2023

  34. [42]

    Revisiting the tree edit distance and its backtracing: A tutorial, 2022

    Benjamin Paaßen. Revisiting the tree edit distance and its backtracing: A tutorial, 2022. 6

    2022

  35. [43]

    Cloud- scan - a configuration-free invoice analysis system using re- current neural networks, 2017

    Rasmus Berg Palm, Ole Winther, and Florian Laws. Cloud- scan - a configuration-free invoice analysis system using re- current neural networks, 2017. 2, 3

    2017

  36. [44]

    Cord: A con- solidated receipt dataset for post-ocr parsing

    Seunghyun Park, Seung Shin, Bado Lee, Junyeop Lee, Jae- heung Surh, Minjoon Seo, and Hwalsuk Lee. Cord: A con- solidated receipt dataset for post-ocr parsing. 2019. 2, 4, 5

    2019

  37. [45]

    Ernie-layout: Layout knowledge en- hanced pre-training for visually-rich document understand- ing, 2022

    Qiming Peng, Yinxu Pan, Wenjin Wang, Bin Luo, Zhenyu Zhang, Zhengjie Huang, Teng Hu, Weichong Yin, Yongfeng Chen, Yin Zhang, Shikun Feng, Yu Sun, Hao Tian, Hua Wu, and Haifeng Wang. Ernie-layout: Layout knowledge en- hanced pre-training for visually-rich document understand- i...

    2022

  38. [46]

    Lmdx: Language model-based document information ex- traction and localization, 2024

    Vincent Perot, Kai Kang, Florian Luisier, Guolong Su, Xiaoyu Sun, Ramya Sree Boppana, Zilong Wang, Zifeng Wang, Jiaqi Mu, Hao Zhang, Chen-Yu Lee, and Nan Hua. Lmdx: Language model-based document information ex- traction and localization, 2024. 4, 7

    2024

  39. [47]

    Learning transferable visual models from natural language supervision, 2021

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021. 5

    2021

  40. [48]

    Xnor-net: Imagenet classification using bi- nary convolutional neural networks, 2016

    Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, and Ali Farhadi. Xnor-net: Imagenet classification using bi- nary convolutional neural networks, 2016. 3

    2016

  41. [49]

    Simon, J.-C

    A. Simon, J.-C. Pret, and A.P. Johnson. A fast algorithm for bottom-up document layout analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence , 19(3):273–277,

  42. [50]

    Llama 2: Open foundation and fine- tuned chat models, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almahairi, Yasmine Babaei, Nikolay Bash- lykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fer- nandes, Jere...

    2023

  43. [51]

    Layout and task aware instruction prompt for zero-shot document image question answering, 2023

    Wenjin Wang, Yunhao Li, Yixin Ou, and Yin Zhang. Layout and task aware instruction prompt for zero-shot document image question answering, 2023. 4

    2023

  44. [52]

    Cogvlm: Visual expert for pretrained language models, 2024

    Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, Jiazheng Xu, Bin Xu, Juanzi Li, Yuxiao Dong, Ming Ding, and Jie Tang. Cogvlm: Visual expert for pretrained language models, 2024. 2, 3

    2024

  45. [53]

    Cnnpack: packing convolutional neural networks in the frequency domain

    Yunhe Wang, Chang Xu, Shan You, Dacheng Tao, and Chao Xu. Cnnpack: packing convolutional neural networks in the frequency domain. In Proceedings of the 30th Inter- national Conference on Neural Information Processing Sys- tems, NIPS’16, page 253–261, Red Hook, NY , USA, 2016. ...

    2016

  46. [54]

    Queryform: A simple zero-shot form entity query framework, 2023

    Zifeng Wang, Zizhao Zhang, Jacob Devlin, Chen-Yu Lee, Guolong Su, Hao Zhang, Jennifer Dy, Vincent Perot, and Tomas Pfister. Queryform: A simple zero-shot form entity query framework, 2023. 2, 3

    2023

  47. [55]

    Chain-of-thought prompting elicits reasoning in large language models, 2023

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models, 2023. 5

    2023

  48. [56]

    Layoutlm: Pre-training of text and layout for document image understanding

    Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, and Ming Zhou. Layoutlm: Pre-training of text and layout for document image understanding. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’20. ACM, Aug. 2020. 2, 3

    2020

  49. [57]

    Layoutlmv2: Multi-modal pre-training for visually-rich document under- standing, 2022

    Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, and Lidong Zhou. Layoutlmv2: Multi-modal pre-training for visually-rich document under- standing, 2022. 3

    2022

  50. [58]

    Data-free knowledge amalgamation via group-stack dual-gan, 2020

    Jingwen Ye, Yixin Ji, Xinchao Wang, Xin Gao, and Mingli Song. Data-free knowledge amalgamation via group-stack dual-gan, 2020. 3

    2020

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.