Pith. sign in

REVIEW 3 major objections 6 minor 60 references

DREAM: Document Reconstruction via End-to-end Autoregressive Model

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a single end-to-end autoregressive model can turn a document image into one token sequence carrying each element's category, bounding box, and transcription, and reports top scores on a new DocRec1K benchmark.

desk verdict Interesting architecture and credible subtask results, but the DSM metric in Eq. (4) is not a valid edit distance, so the headline reconstruction scores are uninterpretable. read the letter →

arxiv 2507.05805 v1 pith:WVA3E2F7 submitted 2025-07-08 cs.CV

Xin Li , Mingming Gong , Yunfei Wu , Jianxin Dai , Antai Guo , Xinghua Jiang , Haoyu Cao , Yinsong Liu
show 2 more authors
Deqiang Jiang Xing Sun
This is my paper
classification cs.CV
keywords documentreconstructionend-to-endautoregressivemodellayoutanalysistextrecognitiontablestructureformulareadingorderdetectionparalleldecoding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DREAM reframes document reconstruction as one sequence-to-sequence problem: a page image goes in, and a single token stream comes out in which each element carries its type, its bounding-box coordinates, and its content (paragraph text lines with coordinates, tables as HTML with cell boxes, formulas in LaTeX). The authors argue this end-to-end design avoids the error propagation that hurts multi-stage pipelines and preserves layout information that earlier end-to-end generators drop. To measure the task they introduce a Document Similarity Metric and the DocRec1K dataset, and report that DREAM scores 91.4 DSM versus 85.2 for a retrained Nougat base and 65.3 for a PaddleOCR pipeline. They also report competitive results on layout analysis, text recognition, table structure recognition, formula recognition, and reading order detection.

What carries the argument

The load-bearing design is the element-parallel decoding scheme: N element queries, matched to ground-truth elements by DETR-style bipartite matching, each generate a complete element record (category, discretized coordinates, then text tokens) in parallel rather than as one long serial markup string; a confidence threshold filters out negative queries. This shortens the effective autoregressive sequence and, the authors argue, prevents repetitive degeneration while cutting inference time. The other central object is the reconstruction format itself, the per-element tuple with <Sep> separators, which lets one sequence carry both physical layout and logical content.

What would settle it

Recompute DSM using a standard edit-distance recurrence where insertion and deletion transitions add a fixed cost without adding the substitution cost $\text{Cost}(i,j)$; if Table 1's rankings or margins change materially, the claimed superiority is an artifact of Eq. (4) rather than of reconstruction quality.

Watch

Extended reading notes

Core claim

The central claim is that document reconstruction should be defined as transforming an image into a sequence Y={y_k} where each element y_k=(c,b,t,<Sep>) includes the element category, a four-coordinate bounding box, and a transcription in a format suited to the element type. DREAM implements this with an image encoder, a feature aggregator that lets N learnable element queries cross-attend to image features, and a parallel decoder that autoregressively generates the category and coordinates for all retained elements and then each element's text. The model is trained with a document reconstruction loss combining element discrimination via bipartite matching, transcription cross-entropy, and a sequence-level cosine loss intended to enforce reading order. The paper introduces DSM to score reconstruction quality and DocRec1K, a 1,000-page benchmark derived from DocLayNet, and claims DREAM outperforms the compared multi-stage and end-to-end baselines on that benchmark while remaining competitive across five document-understanding subtasks.

Load-bearing premise

The load-bearing premise is that DSM, the paper's new similarity metric, correctly measures reconstruction quality; all headline comparisons use its numbers, so if the metric is miscalibrated the reported margins are not meaningful.

Editorial extensions

If this is right

  • A single DREAM-style model can replace a pipeline of separate detection, OCR, table, and formula models, removing stage-to-stage error propagation.
  • Document images become structured inputs for large language models: the output sequence contains both where each element is and what it says, so downstream systems get layout-aware content without extra parsing.
  • The new DSM and DocRec1K give the field a common yardstick for reconstruction quality, making future end-to-end models directly comparable.
  • Parallel decoding makes end-to-end reconstruction practical at 3.2 seconds per image with 270M parameters, versus 14.1 seconds for the adapted Nougat base.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An editor's check: Eq. (4) adds the element substitution cost to every dynamic-programming transition, including pure insertion and deletion moves; a standard edit-distance recurrence would not do that, so the DSM numbers in Table 1, and the claimed 6.2-point margin over Nougat*, should be recomputed with a corrected recurrence before being taken at face value.
  • Because DREAM is trained on LaTeX-rendered pages, its strong results likely transfer well to clean digital documents but less to photographed or handwritten pages; the paper itself notes difficulty with natural scenes, so a testable extension is to evaluate on a mixed-domain reconstruction benchmark.
  • The element-query design suggests each output element is grounded in a specific image region; a natural next probe is whether the predicted bounding boxes are accurate enough for layout-aware editing, retrieval, or question answering, not just for reconstruction scoring.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes DREAM, an end-to-end autoregressive model for document reconstruction that outputs, for each document element, a category, a bounding box, and a transcribed content string in a single sequence. The authors introduce a task definition, a new evaluation metric called Document Similarity Metric (DSM), and a 1,000-page benchmark DocRec1K built from DocLayNet. DREAM uses a Swin image encoder, a DETR-style feature aggregator with element queries, and a parallel decoder that generates the elements' physical and logical tokens in parallel. The paper reports a DSM of 91.4 on DocRec1K, outperforming a retrained Nougat-base variant (85.2) and a PaddleOCR pipeline (65.3), and also evaluates on document layout analysis, text recognition, table structure recognition, formula recognition, and reading order detection, showing competitive or slightly lower scores than specialized baselines.

Significance. If the headline result were sound, the paper would be a useful step toward a unified document reconstruction framework: it combines physical layout and logical transcription in one autoregressive sequence, proposes a metric that attempts to measure both, and demonstrates a parallel decoding scheme that reduces inference time. The ablation study is informative, and the authors are candid about limitations such as poor performance on natural scenes. However, the central evaluation rests on a newly introduced metric that is not a valid edit-distance dynamic program, and the benchmark is a new dataset with possible overlap with the training distribution. These issues make the reported state-of-the-art claim unverifiable and potentially non-meaningful as stated. The subtask evaluations on established public benchmarks provide partial independent evidence that the architecture is competitive, but they do not rescue the central document-reconstruction claim.

major comments (3)
  1. [3.2, Eqs. (4)-(5); Table 1] The DSM recurrence is not a valid edit-distance dynamic program. In standard sequence alignment, moving from D(i-1,j) or D(i,j-1) should incur an insertion/deletion cost without charging the substitution cost Cost(i,j). Eq. (4) adds Cost(i,j) to every transition, so pure insertions and deletions are penalized as if they were mismatched elements. Moreover, no base cases are specified, and the normalization in Eq. (5) cannot guarantee a value in [0,1] because the accumulated cost is bounded by the path length in the grid, which can exceed max(K, eK). Since the headline results in Table 1, including the 6.2-point margin over Nougat_base*, are computed with this recurrence, the central SOTA claim is uninterpretable until the metric is corrected or the raw scores are independently recomputed. I request corrected base cases and DP, or release of predicted outputs so reviewers can recompute the scores.
  2. [5.1 and Appendix B] DocRec1K is derived from DocLayNet, while DREAM is trained on approximately 10 million image/ground-truth pairs generated from arXiv LaTeX sources. DocLayNet is known to include pages of scientific articles, including arXiv-derived content, so there is a real risk of train/evaluation overlap. The paper does not report any deduplication between DocRec1K and the training set, nor does it provide a breakdown of DocRec1K by source. This is load-bearing because DocRec1K is the only direct benchmark for the document-reconstruction claim. I ask for an overlap analysis, a disjoint evaluation set, or per-source results to establish that the comparison is not inflated by training-set leakage.
  3. [5.4, Table 1] The end-to-end baselines Pix2Struct_base* and Nougat_base* are retrained by the authors on the same synthetic pipeline and evaluated only through in-house code, with no released checkpoints, predicted outputs, or uncertainty estimates. Since both the benchmark and the metric are introduced in this paper and all scores are point estimates on 1,000 images, the reported margin could be affected by scoring details, post-processing, or small-sample variance. I request release of the baseline predictions and the evaluation code, and the reporting of confidence intervals or significance tests, before the claim of 'unparalleled performance' can be accepted.
minor comments (6)
  1. [3.2, Eq. (2)] Cost_tran divides by Maxlen(t_i, t_j); if both strings are empty, the denominator is zero. Please specify a convention for empty transcriptions.
  2. [4.4 and Figure 3] The notation F_v is used both for the patch embeddings in Sec. 4.2 and for the duplicated visual tokens in Sec. 4.4; using a different symbol, such as F_v_dup, would avoid confusion.
  3. [4.4] The negative-query filtering threshold of 80% is stated without justification or ablation; a sensitivity analysis would clarify its effect on the DSM and NED scores.
  4. [5.1] The sentence 'DREAM is trained on ... approximately 10 million pairs' is immediately repeated with slightly different wording; please remove the duplication.
  5. [5.3] The input resolution, maximum sequence length, and number of queries are fixed, but the paper does not report how these choices interact with the maximum number of elements or the longest transcription; a brief discussion would be helpful.
  6. [Appendix E] The statement that DREAM 'struggles with natural scene' is vague; giving a quantitative example would make the stated limitation more actionable.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DREAM's derivations and comparisons do not reduce to their inputs by construction; metric concerns are validity issues, not circularity.

full rationale

The paper's claimed derivation chain—defining the document reconstruction task, the DSM metric, the DocRec1K benchmark, and then reporting DREAM's performance—does not contain a step in which an output is equivalent to an input by construction. The DSM is a new edit-distance-style metric over an output sequence; it is not fitted from DREAM's outputs and is applied identically to all compared models. Baselines (PaddleOCR, Pix2Struct, Nougat) were retrained on the same 10M-image synthetic pipeline and scored with the same published protocol, so the comparison is not a fitted parameter renamed as a prediction. External subtask benchmarks (PubLayNet, DocLayNet, FUNSD, SROIE, CORD, SciTSR, TableBank, IM2LATEX-100K, ReadingBank) provide independent grounding for the model's component capabilities. Self-citations such as DocReL [31] and NCGM [34] appear only as comparison baselines and are not load-bearing for the central claim. The reader-identified defect in the DSM recurrence (Eq. 4) is a metric-validity concern about what the numbers mean; it is not circularity, because the metric is not defined in terms of the model's training objective or outputs. Accordingly, no circular step can be quoted and exhibited, and the appropriate finding is no significant circularity.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central contribution rests on unverified data-generation assumptions (LaTeX-to-PDF rendering with automatic annotation tools) and on a new evaluation metric (DSM) that is not validated against human judgments or standard metrics. The headline SOTA number depends on the DSM recurrence, which appears mathematically incorrect as written.

free parameters (7)
  • Loss weights lambda1, lambda2, lambda3 = 1, 1, 1
    Chosen empirically in Sec. 5.3; all set to 1 with no sensitivity analysis.
  • Number of element queries N = 200
    Set in Sec. 4.3 and 5.3; determines maximum parallel elements and affects computation; no ablation on N.
  • Maximum sequence length L = 1536
    Set in Sec. 5.3; caps text generation per element.
  • Negative query filter threshold = 0.8 (80% confidence)
    Sec. 4.4: elements with confidence below 80% are removed; chosen ad hoc.
  • xy-cut tolerance range = not specified
    Used in DocRec1K annotation ordering (Appendix B); manual tolerance to group y-coordinates; value not reported.
  • Input image size = 1024x1024
    Sec. 5.3; a standard choice, but affects coordinate discretization.
  • Coordinate discretization granularity = not specified
    Coordinates are emitted as tokens (Sec. 4.4); the quantization scheme and bins are not described, which affects DSM and mAP.
assumptions (5)
  • domain assumption LaTeXML and PDFMiner, with SLANet for table cells, produce correct physical and logical annotations from PDFs
    Invoked in Sec. 5.1 to build 10M training pairs; errors propagate to model and eval.
  • domain assumption Fuzzy string matching and IoU alignment correctly associate text with layout elements
    Sec. 5.1; matching heuristics could misassign content.
  • ad hoc to paper DSM recurrence (Eq. 4) is a valid edit-distance style measure
    Eq. 4 adds Cost(i,j) to all transitions, so it does not match standard alignment recurrence; central metric is unproven.
  • domain assumption DocRec1K annotations are correct after human intervention
    Appendix B; pseudo-labels from PaddleOCR are corrected by human validation, but the process is not quantified.
  • standard math Bipartite matching / Hungarian algorithm can be reused for sequence element matching
    Sec. 4.5; DETR-style matching is assumed valid for matching predicted sequences to ground-truth elements.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DREAM: Document Reconstruction via End-to-end Autoregressive Model." pith.science (2026). https://pith.science/paper/WVA3E2F7

@misc{pith2026250705805,
  author       = {Pith},
  title        = {Pith review of: DREAM: Document Reconstruction via End-to-end Autoregressive Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WVA3E2F7}},
  note         = {Machine review of arXiv:2507.05805}
}
read the original abstract

Document reconstruction constitutes a significant facet of document analysis and recognition, a field that has been progressively accruing interest within the scholarly community. A multitude of these researchers employ an array of document understanding models to generate predictions on distinct subtasks, subsequently integrating their results into a holistic document reconstruction format via heuristic principles. Nevertheless, these multi-stage methodologies are hindered by the phenomenon of error propagation, resulting in suboptimal performance. Furthermore, contemporary studies utilize generative models to extract the logical sequence of plain text, tables and mathematical expressions in an end-to-end process. However, this approach is deficient in preserving the information related to element layouts, which are vital for document reconstruction. To surmount these aforementioned limitations, we in this paper present an innovative autoregressive model specifically designed for document reconstruction, referred to as Document Reconstruction via End-to-end Autoregressive Model (DREAM). DREAM transmutes the text image into a sequence of document reconstruction in a comprehensive, end-to-end process, encapsulating a broader spectrum of document element information. In addition, we establish a standardized definition of the document reconstruction task, and introduce a novel Document Similarity Metric (DSM) and DocRec1K dataset for assessing the performance of the task. Empirical results substantiate that our methodology attains unparalleled performance in the realm of document reconstruction. Furthermore, the results on a variety of subtasks, encompassing document layout analysis, text recognition, table structure recognition, formula recognition and reading order detection, indicate that our model is competitive and compatible with various tasks.

Figures

Figures reproduced from arXiv: 2507.05805 by the authors.

Figure 1
Figure 1. Illustration of motivation of the proposed DREAM. (a) [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The task definition of document reconstruction. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The architecture of our proposed DREAM, in which the gray color of boxes denotes the negative queries to be filtered while the [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Visualization results of our proposed DREAM. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Visualizations between DREAM∗ w/o PD and DREAM. and the inference time on the DocRec1K dataset. in Tab. 8. It is important to highlight that we have expanded the orig￾inal Nougat [2] model, which predicts markup texts, to the Nougatbase ∗version. This adaptation was ma…
Figure 6
Figure 6. Figure 6: More visualization results of DREAM. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

60 extracted references · 45 canonical work pages

  1. [1]

    Combining visual and textual features for semantic segmentation of historical newspapers

    Rapha ¨el Barman, Maud Ehrmann, Simon Clematide, Sofia Ares Oliveira, and Fr´ed´eric Kaplan. Combining visual and textual features for semantic segmentation of historical newspapers. Journal of Data Mining & Digital Humanities, (HistoInformatics), 2021. 10

    work page 2021

  2. [2]

    Nougat: Neural optical understanding for academic documents

    Lukas Blecher, Guillem Cucurull, Thomas Scialom, and Robert Stojnic. Nougat: Neural optical understanding for academic documents. arXiv preprint arXiv:2308.13418 ,

    arXiv

  3. [3]

    End-to- end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to- end object detection with transformers. In European confer- ence on computer vision, pages 213–229. Springer, 2020. 5, 6, 10

    work page 2020

  4. [4]

    Geometric layout analysis techniques for document image understanding: a review

    Roldano Cattoni, Tarcisio Coianiz, Stefano Messelodi, and Carla Maria Modena. Geometric layout analysis techniques for document image understanding: a review. ITC-irst Tech- nical Report, 9703(09), 1998. 10

    work page 1998

  5. [5]

    M6doc: A large-scale multi-format, multi-type, multi- layout, multi-language, multi-annotation category dataset for modern document layout analysis

    Hiuyi Cheng, Peirong Zhang, Sihang Wu, Jiaxin Zhang, Qiyuan Zhu, Zecheng Xie, Jing Li, Kai Ding, and Lianwen Jin. M6doc: A large-scale multi-format, multi-type, multi- layout, multi-language, multi-annotation category dataset for modern document layout analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15...

    work page 2023

  6. [6]

    Complicated table structure recognition

    Zewen Chi, Heyan Huang, Heng-Da Xu, Houjin Yu, Wanx- uan Yin, and Xian-Ling Mao. Complicated table structure recognition. arXiv preprint arXiv:1908.04729, 2019. 2, 7, 10, 11, 12

    arXiv 1908

  7. [7]

    Vision grid transformer for document layout analysis

    Cheng Da, Chuwei Luo, Qi Zheng, and Cong Yao. Vision grid transformer for document layout analysis. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 19462–19472, 2023. 7, 8

    work page 2023

  8. [8]

    Image-to-markup generation with coarse-to- fine attention

    Yuntian Deng, Anssi Kanervisto, Jeffrey Ling, and Alexan- der M Rush. Image-to-markup generation with coarse-to- fine attention. In International Conference on Machine Learning, pages 980–989, 2017. 2, 7, 8, 10, 11, 12

    work page 2017

Show all 60 references
  1. [9]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. 10

    2018 arXiv

  2. [10]

    Pp-ocr: A practical ultra lightweight ocr system

    Yuning Du, Chenxia Li, Ruoyu Guo, Xiaoting Yin, Weiwei Liu, Jun Zhou, Yifan Bai, Zilin Yu, Yehua Yang, Qingqing Dang, et al. Pp-ocr: A practical ultra lightweight ocr system. arXiv preprint arXiv:2009.09941, 2020. 7, 8, 11

    2009 arXiv

  3. [11]

    Unidoc: A uni- versal large multimodal model for simultaneous text detec- tion, recognition, spotting and understanding

    Hao Feng, Zijian Wang, Jingqun Tang, Jinghui Lu, Wen- gang Zhou, Houqiang Li, and Can Huang. Unidoc: A uni- versal large multimodal model for simultaneous text detec- tion, recognition, spotting and understanding. arXiv preprint arXiv:2308.11592, 2023. 2

    2023 arXiv

  4. [12]

    Icdar 2013 table competition

    Max G ¨obel, Tamir Hassan, Ermelinda Oro, and Giorgio Orsi. Icdar 2013 table competition. In 2013 12th International Conference on Document Analysis and Recognition , pages 1449–1453. IEEE, 2013. 2, 7, 10, 11, 12

    2013

  5. [13]

    Re- cursive xy cut using bounding boxes of connected compo- nents

    Jaekyu Ha, Robert M Haralick, and Ihsin T Phillips. Re- cursive xy cut using bounding boxes of connected compo- nents. In Proceedings of 3rd International Conference on Document Analysis and Recognition, pages 952–955. IEEE,

  6. [14]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Gir- shick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017. 10

    2017

  7. [15]

    Swintextspotter: Scene text spotting via bet- ter synergy between text detection and text recognition

    Mingxin Huang, Yuliang Liu, Zhenghao Peng, Chongyu Liu, Dahua Lin, Shenggao Zhu, Nicholas Yuan, Kai Ding, and Lianwen Jin. Swintextspotter: Scene text spotting via bet- ter synergy between text detection and text recognition. In proceedings of the IEEE/CVF conference on comput...

    2022

  8. [16]

    Layoutlmv3: Pre-training for document ai with unified text and image masking

    Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, and Furu Wei. Layoutlmv3: Pre-training for document ai with unified text and image masking. In Proceedings of the 30th ACM International Conference on Multimedia, pages 4083–4091,

  9. [17]

    Improving table structure recognition with visual-alignment sequential coordinate modeling

    Yongshuai Huang, Ning Lu, Dapeng Chen, Yibo Li, Zecheng Xie, Shenggao Zhu, Liangcai Gao, and Wei Peng. Improving table structure recognition with visual-alignment sequential coordinate modeling. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recogni...

    2023

  10. [18]

    Icdar2019 compe- tition on scanned receipt ocr and information extraction

    Zheng Huang, Kai Chen, Jianhua He, Xiang Bai, Dimosthe- nis Karatzas, Shijian Lu, and CV Jawahar. Icdar2019 compe- tition on scanned receipt ocr and information extraction. In 2019 International Conference on Document Analysis and Recognition (ICDAR), pages 1516–1520. IEEE, 20...

    2019

  11. [19]

    Document transformation system from pa- pers to xml data based on pivot xml document method

    Yasuto Ishitani. Document transformation system from pa- pers to xml data based on pivot xml document method. In Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings., pages 250–255. IEEE,

    2003

  12. [20]

    Data clustering: a review

    Anil K Jain, M Narasimha Murty, and Patrick J Flynn. Data clustering: a review. ACM computing surveys (CSUR) , 31 (3):264–323, 1999. 10

    1999

  13. [21]

    Funsd: A dataset for form understanding in noisy 14 scanned documents

    Guillaume Jaume, Hazim Kemal Ekenel, and Jean-Philippe Thiran. Funsd: A dataset for form understanding in noisy 14 scanned documents. In 2019 International Conference on Document Analysis and Recognition Workshops (ICDARW), pages 1–6. IEEE, 2019. 2, 7, 10, 11, 12

    2019

  14. [22]

    ultralytics/yolov5: v6

    Glenn Jocher, Alex Stoken, Ayush Chaurasia, Jirka Borovec, Yonghye Kwon, Kalen Michael, Liu Changyu, Jiacong Fang, Piotr Skalski, Adam Hogan, et al. ultralytics/yolov5: v6. 0- yolov5n’nano’models, roboflow integration, tensorflow ex- port, opencv dnn support. Zenodo, 2021. 10

    2021

  15. [23]

    Ocr-free document understanding transformer

    Geewook Kim, Teakgyu Hong, Moonbin Yim, JeongYeon Nam, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sang- doo Yun, Dongyoon Han, and Seunghyun Park. Ocr-free document understanding transformer. In European Confer- ence on Computer Vision , pages 498–517. Springer, 2022. 2

    2022

  16. [24]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 ,

    arXiv

  17. [25]

    Pix2struct: Screenshot parsing as pretraining for visual lan- guage understanding

    Kenton Lee, Mandar Joshi, Iulia Raluca Turc, Hexiang Hu, Fangyu Liu, Julian Martin Eisenschlos, Urvashi Khandel- wal, Peter Shaw, Ming-Wei Chang, and Kristina Toutanova. Pix2struct: Screenshot parsing as pretraining for visual lan- guage understanding. In International Confere...

    2023

  18. [26]

    Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension

    Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvinine- jad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461, 2019. 7

    1910 arXiv

  19. [27]

    When counting meets hmer: counting-aware network for handwritten math- ematical expression recognition

    Bohan Li, Ye Yuan, Dingkang Liang, Xiao Liu, Zhilong Ji, Jinfeng Bai, Wenyu Liu, and Xiang Bai. When counting meets hmer: counting-aware network for handwritten math- ematical expression recognition. In European Conference on Computer Vision, pages 197–214. Springer, 2022. 11

    2022

  20. [28]

    Pp- structurev2: A stronger document analysis system

    Chenxia Li, Ruoyu Guo, Jun Zhou, Mengtao An, Yuning Du, Lingfeng Zhu, Yi Liu, Xiaoguang Hu, and Dianhai Yu. Pp- structurev2: A stronger document analysis system. arXiv preprint arXiv:2210.05391, 2022. 7, 11

    2022 arXiv

  21. [29]

    Tablebank: Table benchmark for image-based table detection and recognition

    Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou, and Zhoujun Li. Tablebank: Table benchmark for image-based table detection and recognition. In Proceedings of the Twelfth Language Resources and Evaluation Confer- ence, pages 1918–1925, 2020. 2, 7, 10, 11, 12

    1918

  22. [30]

    Docbank: A bench- mark dataset for document layout analysis

    Minghao Li, Yiheng Xu, Lei Cui, Shaohan Huang, Furu Wei, Zhoujun Li, and Ming Zhou. Docbank: A bench- mark dataset for document layout analysis. arXiv preprint arXiv:2006.01038, 2020. 2, 7, 10, 11

    2006 arXiv

  23. [31]

    Relational represen- tation learning in visually-rich documents

    Xin Li, Yan Zheng, Yiqing Hu, Haoyu Cao, Yunfei Wu, De- qiang Jiang, Yinsong Liu, and Bo Ren. Relational represen- tation learning in visually-rich documents. In Proceedings of the 30th ACM International Conference on Multimedia , pages 4614–4624, 2022. 8

    2022

  24. [32]

    Tsrformer: Table structure recognition with transformers

    Weihong Lin, Zheng Sun, Chixiang Ma, Mingze Li, Jiawei Wang, Lei Sun, and Qiang Huo. Tsrformer: Table structure recognition with transformers. In Proceedings of the 30th ACM International Conference on Multimedia, pages 6473– 6482, 2022. 8

    2022

  25. [33]

    Show, read and reason: Table struc- ture recognition with flexible context aggregator

    Hao Liu, Xin Li, Bing Liu, Deqiang Jiang, Yinsong Liu, Bo Ren, and Rongrong Ji. Show, read and reason: Table struc- ture recognition with flexible context aggregator. InProceed- ings of the 29th ACM International Conference on Multime- dia, pages 1084–1092, 2021. 2, 8, 10

    2021

  26. [34]

    Neural collaborative graph machines for table structure recognition

    Hao Liu, Xin Li, Bing Liu, Deqiang Jiang, Yinsong Liu, and Bo Ren. Neural collaborative graph machines for table structure recognition. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 4533–4542, 2022. 2, 7, 8, 10

    2022

  27. [35]

    Roberta: A robustly optimized bert pretraining approach

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettle- moyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692,

    1907 arXiv

  28. [36]

    Abcnet: Real-time scene text spot- ting with adaptive bezier-curve network

    Yuliang Liu, Hao Chen, Chunhua Shen, Tong He, Lianwen Jin, and Liangwei Wang. Abcnet: Real-time scene text spot- ting with adaptive bezier-curve network. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9809–9818, 2020. 2, 10

    2020

  29. [37]

    Spts v2: single-point scene text spotting

    Yuliang Liu, Jiaxin Zhang, Dezhi Peng, Mingxin Huang, Xinyu Wang, Jingqun Tang, Can Huang, Dahua Lin, Chun- hua Shen, Xiang Bai, et al. Spts v2: single-point scene text spotting. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 2023. 2

    2023

  30. [38]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021. 5, 7

    2021

  31. [39]

    Kosmos-2.5: A multimodal literate model

    Tengchao Lv, Yupan Huang, Jingye Chen, Lei Cui, Shum- ing Ma, Yaoyao Chang, Shaohan Huang, Wenhui Wang, Li Dong, Weiyao Luo, et al. Kosmos-2.5: A multimodal literate model. arXiv preprint arXiv:2309.11419, 2023. 2, 3, 4, 6, 7, 8, 12

    2023 arXiv

  32. [40]

    Mask textspotter: An end-to-end trainable neural network for spotting text with arbitrary shapes

    Pengyuan Lyu, Minghui Liao, Cong Yao, Wenhao Wu, and Xiang Bai. Mask textspotter: An end-to-end trainable neural network for spotting text with arbitrary shapes. In Proceed- ings of the European conference on computer vision (ECCV), pages 67–83, 2018. 2, 10

    2018

  33. [41]

    Optimized xy-cut for determining a page read- ing order

    J-L Meunier. Optimized xy-cut for determining a page read- ing order. In Eighth International Conference on Docu- ment Analysis and Recognition (ICDAR’05), pages 347–351. IEEE, 2005. 10, 11

    2005

  34. [42]

    Icfhr 2014 competition on recognition of on-line handwritten mathematical expressions (crohme 2014)

    Harold Mouchere, Christian Viard-Gaudin, Richard Zanibbi, and Utpal Garain. Icfhr 2014 competition on recognition of on-line handwritten mathematical expressions (crohme 2014). In 2014 14th International Conference on Frontiers in Handwriting Recognition, pages 791–796. IEEE, ...

    2014

  35. [43]

    Tableformer: Table structure understanding with transformers

    Ahmed Nassar, Nikolaos Livathinos, Maksym Lysak, and Peter Staar. Tableformer: Table structure understanding with transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4614– 4623, 2022. 2, 10

    2022

  36. [44]

    Cord: a con- solidated receipt dataset for post-ocr parsing

    Seunghyun Park, Seung Shin, Bado Lee, Junyeop Lee, Jae- heung Surh, Minjoon Seo, and Hwalsuk Lee. Cord: a con- solidated receipt dataset for post-ocr parsing. In Workshop on Document Intelligence at NeurIPS 2019 , 2019. 2, 7, 11, 12 15

    2019

  37. [45]

    Doclaynet: A large human- annotated dataset for document-layout segmentation

    Birgit Pfitzmann, Christoph Auer, Michele Dolfi, Ahmed S Nassar, and Peter Staar. Doclaynet: A large human- annotated dataset for document-layout segmentation. InPro- ceedings of the 28th ACM SIGKDD Conference on Knowl- edge Discovery and Data Mining , pages 3743–3751, 2022. 2...

    2022

  38. [46]

    Faster r-cnn: Towards real-time object detection with region proposal networks

    Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information process- ing systems, 28, 2015. 10

    2015

  39. [47]

    Teaching machines to code: neural markup generation with visual attention

    Sumeet S Singh. Teaching machines to code: neural markup generation with visual attention. arXiv preprint arXiv:1802.05415, 2018. 8

    2018 arXiv

  40. [48]

    Optimal boxes: boosting end-to- end scene text recognition by adjusting annotated bounding boxes via reinforcement learning

    Jingqun Tang, Wenming Qian, Luchuan Song, Xiena Dong, Lan Li, and Xiang Bai. Optimal boxes: boosting end-to- end scene text recognition by adjusting annotated bounding boxes via reinforcement learning. In European Conference on Computer Vision, pages 233–248. Springer, 2022. 2

    2022

  41. [49]

    You can even annotate text with voice: Transcription-only-supervised text spotting

    Jingqun Tang, Su Qiao, Benlei Cui, Yuhang Ma, Sheng Zhang, and Dimitrios Kanoulas. You can even annotate text with voice: Transcription-only-supervised text spotting. In Proceedings of the 30th ACM International Conference on Multimedia, page 4154–4163, New York, NY , USA, 202...

    2022

  42. [50]

    Few could be better than all: Feature sampling and grouping for scene text detection

    Jingqun Tang, Wenqing Zhang, Hongye Liu, MingKun Yang, Bo Jiang, Guanglong Hu, and Xiang Bai. Few could be better than all: Feature sampling and grouping for scene text detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4563...

    2022

  43. [51]

    Layoutreader: Pre-training of text and layout for read- ing order detection

    Zilong Wang, Yiheng Xu, Lei Cui, Jingbo Shang, and Furu Wei. Layoutreader: Pre-training of text and layout for read- ing order detection. arXiv preprint arXiv:2108.11591, 2021. 2, 7, 8, 10, 11, 12

    2021 arXiv

  44. [52]

    Learning to extract semantic struc- ture from documents using multimodal fully convolutional neural networks

    Xiao Yang, Ersin Yumer, Paul Asente, Mike Kraley, Daniel Kifer, and C Lee Giles. Learning to extract semantic struc- ture from documents using multimodal fully convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pag...

    2017

  45. [53]

    Syntax-aware network for handwritten mathematical expression recognition

    Ye Yuan, Xiao Liu, Wondimu Dikubab, Hui Liu, Zhilong Ji, Zhongqin Wu, and Xiang Bai. Syntax-aware network for handwritten mathematical expression recognition. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4553–4562, 2022. 2, 10, 11

    2022

  46. [54]

    Reading order matters: Information extraction from visually-rich documents by to- ken path prediction

    Chong Zhang, Ya Guo, Yi Tu, Huan Chen, Jinyang Tang, Huijia Zhu, Qi Zhang, and Tao Gui. Reading order matters: Information extraction from visually-rich documents by to- ken path prediction. arXiv preprint arXiv:2310.11016, 2023. 8

    arXiv

  47. [55]

    Vsr: a unified framework for document layout analysis combining vision, semantics and relations

    Peng Zhang, Can Li, Liang Qiao, Zhanzhan Cheng, Shil- iang Pu, Yi Niu, and Fei Wu. Vsr: a unified framework for document layout analysis combining vision, semantics and relations. In Document Analysis and Recognition–ICDAR 2021: 16th International Conference, Lausanne, Switzer...

    2021

  48. [56]

    Tabpedia: Towards comprehensive visual ta- ble understanding with concept synergy.Advances in Neural Information Processing Systems, 37:7185–7212, 2025

    Weichao Zhao, Hao Feng, Qi Liu, Jingqun Tang, Binghong Wu, Lei Liao, Shu Wei, Yongjie Ye, Hao Liu, Wengang Zhou, et al. Tabpedia: Towards comprehensive visual ta- ble understanding with concept synergy.Advances in Neural Information Processing Systems, 37:7185–7212, 2025. 2

    2025

  49. [57]

    Multi-modal in-context learning makes an ego-evolving scene text recognizer

    Zhen Zhao, Jingqun Tang, Chunhui Lin, Binghong Wu, Can Huang, Hao Liu, Xin Tan, Zhizhong Zhang, and Yuan Xie. Multi-modal in-context learning makes an ego-evolving scene text recognizer. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pa...

    2024

  50. [58]

    Pub- laynet: largest dataset ever for document layout analysis

    Xu Zhong, Jianbin Tang, and Antonio Jimeno Yepes. Pub- laynet: largest dataset ever for document layout analysis. In 2019 International Conference on Document Analysis and Recognition (ICDAR), pages 1015–1022. IEEE, 2019. 2, 7, 10, 11

    2019

  51. [59]

    Image-based table recognition: data, model, and evaluation

    Xu Zhong, Elaheh ShafieiBavani, and Antonio Ji- meno Yepes. Image-based table recognition: data, model, and evaluation. In European conference on computer vision, pages 564–580. Springer, 2020. 2, 4, 10

    2020

  52. [60]

    Deformable detr: Deformable trans- formers for end-to-end object detection

    Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable trans- formers for end-to-end object detection. arXiv preprint arXiv:2010.04159, 2020. 10 16

    2010 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.