Pith. sign in

REVIEW 4 major objections 6 minor 48 references

LDP: Generalizing to Multilingual Visual Information Extraction by Language Decoupled Pretraining

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Pre-training on language-scrambled English document images lets a vision-layout model generalize to non-English form understanding, beating multilingual pre-trained models on XFUND and staying competitive on English benchmarks.

desk verdict Fresh idea, plausible direction, but the cross-lingual gain isn't cleanly isolated from the multilingual text encoder. read the letter →

arxiv 2412.14596 v1 pith:MISURGFL submitted 2024-12-19 cs.CV cs.CLcs.LG

Huawen Shen , Gengluo Li , Jinwen Zhong , Yu Zhou This is my paper · ORCID
classification cs.CVcs.CLcs.LG
keywords visualinformationextractionmultilingualtransferlanguagedecouplingdocumentpre-trainingvision-layoutmodelsformunderstandingdiffusiontexteditingXFUND
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the visual appearance and spatial layout of a document are language-invariant, so a model does not need readable text in every language to learn how to extract information from forms. It proposes Language Decoupled Pre-training (LDP): scramble the text in English document images with a text-editing diffusion model, pre-train a vision-layout model on those language-independent images, then fine-tune on downstream tasks with the original images and injected text embeddings. If the claim is right, multilingual visual information extraction no longer depends on large, balanced multi-language corpora. The reported numbers support the claim: fine-tuned on English FUNSD, the proposed LDM reaches 62.90 average F1 on seven non-English XFUND languages, compared with 57.24 for the best prior layout-only baseline, and reaches 88.21 average F1 when fine-tuned per language.

What carries the argument

The load-bearing mechanism is the language-decoupling operation: a text-editing diffusion model is asked to edit a 1x1 pixel corner with an empty prompt, so its learned editing behavior scrambles the small, dense text across the whole document image while leaving fonts, colors, backgrounds, ruling, and layout geometry intact. A 'decouple resolution' hyperparameter controls how much detail survives. The model that consumes these images, LDM, is built on a pre-trained image-segmentation backbone: an image encoder plus prompt encoder fed with bounding boxes, with the segmentation head replaced by an MLP classification head. Its Multi-Token Information Merging (MTIM) module exchanges features among the bounding boxes of one image so neighboring boxes can influence each other's labels, and in fine-tuning the Language Knowledge Inserting (LKI) module adds a frozen multilingual sentence embedding of each OCR text to the final features. Pre-training on the decoupled images, with pseudo-labels generated from the same English corpora, is what teaches the cross-lingual invariance.

What would settle it

Measure the OCR word accuracy of the decoupled images at the chosen resolution: if readable words remain at levels far above the reported ~3.5%, language bias is not removed and the cross-lingual result could come from residual English text. The sharper test is to pre-train the same model on the original, non-decoupled English images and zero-shot evaluate on XFUND; if that version matches or exceeds the 62.90 average F1, the decoupling operation is not what produces the generalization.

Watch

Extended reading notes

Core claim

The central discovery is that vision and layout alone—bold fonts, gray background blocks, relative positions, ruling lines—carry enough structure to identify semantic roles such as question, answer, and header across languages, while the identity of the glyphs themselves is a bias that hurts transfer. When the text is scrambled so thoroughly that OCR reads almost none of it (text recognition ratio near 3.5%) and a classifier cannot tell which language an image came from (21.67% vs 16.66% random), a vision-layout model becomes better at zero-shot cross-lingual extraction, not worse. Pre-trained on English-only corpora and fine-tuned on English FUNSD, the model outperforms multilingual models on non-English XFUND; fine-tuned per language or on all languages jointly, it also sets the best multilingual numbers while remaining competitive on English FUNSD and CORD.

Load-bearing premise

The load-bearing premise is that the diffusion text-edit operation destroys readable text in dense document images while preserving exactly the visual and layout cues a VIE model needs; the paper's own evidence is surrogate OCR and language-classification metrics plus one-epoch ablations, not a mechanism that guarantees preservation.

Editorial extensions

If this is right

  • Multilingual VIE can be built from English-only pre-training data, removing the need for large balanced corpora in every target language.
  • Scrambling text at a controlled resolution is a tunable data augmentation: cross-lingual F1 rises as more language bias is removed, until the resolution gets low enough that useful visual detail is destroyed too.
  • The model retains strong English performance (88.23 F1 on FUNSD, 95.80 on CORD), so the decoupling does not sacrifice the language it was pre-trained on.
  • In per-language and multitask settings the model also beats prior multilingual pre-trained models, with the largest margins on non-English languages.
  • Injecting language knowledge at fine-tuning (LKI) is complementary to the language-independent pre-training: adding frozen multilingual text embeddings improves downstream accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test this, but a natural extension is to apply the same decoupling to other visually rich document tasks such as table structure recognition or document question answering; the invariance claim would predict similar cross-lingual gains there.
  • The paper's surrogate metrics suggest a testable design rule: pick the decouple resolution by watching when a language classifier approaches chance while a perceptual similarity metric between original and decoupled images stays high, rather than fixing 1024 across corpora.
  • Because fine-tuning still needs OCR text through the frozen sentence encoder, the approach inherits OCR errors in low-resource languages; an OCR-free variant that conditions on visual text features only would be a stronger test of the invariance claim.
  • If the invariance generalizes beyond these seven languages, the method could be applied to under-resourced languages with no labeled forms at all, using English pre-training plus zero-shot transfer.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes LDP (Language Decoupled Pre-training), a paradigm that pre-trains a vision-layout document model on English-only images whose readable text has been scrambled by the AnyText diffusion editor, with the goal of removing language bias while retaining layout and visual appearance. The resulting model, LDM, is built on SAM with a Multi-Token Information Merging (MTIM) module and, at fine-tuning time, a Language Knowledge Inserting (LKI) module that fuses frozen multilingual Sentence-BERT embeddings. The authors report state-of-the-art cross-lingual results on XFUND under zero-shot, per-language, and multitask fine-tuning, plus strong results on SIBR, FUNSD, and CORD. The central claim is that decoupling language from document images enables monolingual pre-training to transfer to unseen languages.

Significance. If the central claim holds, the paper offers a practical and conceptually attractive way to build multilingual VIE models without multilingual pre-training corpora: a monolingual English corpus is sufficient once language bias is removed from images. The paper deserves credit for formulating a clear hypothesis about visual/layout invariance, for proposing concrete surrogate metrics (OCR recognition ratio and language classification accuracy) to measure decoupling, and for grounding the approach in extensive experiments across multiple benchmarks. However, the evidence as presented does not cleanly isolate the effect of language decoupling from other ingredients of the final model, especially the frozen multilingual text encoder inserted at fine-tuning and the architectural differences with the closest baseline. The paper's significance therefore rests on a plausible but not yet established attribution.

major comments (4)
  1. [Language Knowledge Inserting (LKI), Eq. (2), and Table 8] The final LDM fuses frozen multilingual Sentence-BERT embeddings into the classifier through LKI, so cross-lingual transfer can enter through the text branch even if the decoupled images contribute nothing. Table 8 shows that LKI itself improves XFUND from 61.36 to 62.90 (+1.54) in the zero-shot setting, which is a substantial fraction of the reported margins over LiLT and LayoutXLM in Table 1. To support the paper's central claim that language decoupling is what enables cross-lingual generalization, the authors should provide a 2x2 ablation crossing decoupled vs. original pre-training with LKI on vs. off, ideally with multiple random seeds. The current ablations do not separate these factors.
  2. [Ablation Study, Table 7] Table 7 is the only direct comparison of pre-training on decoupled vs. original images, and it is reported for a single pre-training epoch without error bars or significance tests. The claimed benefit at decouple_resolution=1024 is 60.68 vs. 57.65 XFUND F1, a 3-point gap that could be within run-to-run noise at one epoch. Moreover, the hyperparameter decouple_resolution is selected based on Table 7's XFUND results, meaning the test benchmark is used for model selection; this should be stated explicitly. The authors should report multi-seed mean and variance, and ideally verify the trend with a longer pre-training budget or on a validation split.
  3. [Quantitative Evaluation, Figure 2] The surrogate metrics support that AnyText removes readable text, but they do not directly demonstrate that layout and visual features are preserved at the chosen decouple_resolution=1024. The text reports language classification accuracy at 768 (21.67%) and 512 (20.33%) but does not report the value at 1024, which is the setting used in all main experiments. If language bias remains substantial at 1024, the mechanism underlying the claimed improvement is unclear. The authors should report the language classification accuracy at 1024 and, ideally, add a layout-preservation metric (e.g., layout detection consistency before/after decoupling) to support the claim that the decoupled images retain the cues on which VIE relies.
  4. [Comparison with ESP, Tables 1 and 2] ESP is the most direct control because it uses the same pre-training corpora (DocBank and RVL-CDIP), the same pseudo-label generation, and no language decoupling. Yet ESP is absent from the zero-shot Table 1, where its inclusion is necessary to interpret LDM's cross-lingual gains. In the per-language fine-tuning setting (Table 2), LDM beats ESP on the non-English average by only 1.45 points (88.21 vs. 86.76), and the two models differ in architecture, so this margin cannot be attributed to decoupling alone. The paper should include ESP in Table 1 and, ideally, a decoupled variant of ESP or an LDM variant pre-trained on original images without LKI to isolate the effect.
minor comments (6)
  1. [Implementation Details] The text states that 'all other parameters are randomly initialized' but also that SAM's pre-trained parameters are inherited. Please clarify exactly which modules inherit SAM weights (image encoder, prompt encoder, decoder) and which modules are randomly initialized (MTIM, prediction head).
  2. [Figure 2] The figure is referenced but not fully described in the text; please ensure axis labels, legends, and the values at decouple_resolution=1024 are visible and stated, since the main experiments use 1024.
  3. [Notation, Eq. (1)] In Eq. (1), the index k in F^SAM_nk is not precisely defined beyond 'k ∈ [0, K]'; please clarify whether K is the number of prompt tokens from the SAM decoder and whether the concatenation order of k matters.
  4. [Tables 2 and 3 captions] The captions use 'F1 accuracy'; F1 is not an accuracy measure, so 'F1 score' or 'F1' would be more precise.
  5. [Introduction] The claim of being 'the first to systematically study visual invariance in multilingual VIE' should be qualified by a discussion of prior vision-based multilingual work, or supported by a focused literature comparison, to avoid overclaiming.
  6. [Global] The term 'decouple resolution' is used interchangeably with 'decouple_resolution'; please unify the terminology in the text and figures.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the decoupling is produced by an external model, verified by independent surrogates, and cross-lingual gains are measured on held-out languages.

full rationale

The paper's central claim is that decoupling language bias from document images improves cross-lingual VIE. The decoupling is performed by an external, independently trained diffusion model (AnyText), and its effect is verified by two surrogate metrics (OCR edit distance and language classification accuracy) that are not part of the downstream VIE training objective. The cross-lingual transfer is evaluated on XFUND languages never seen during pre-training or fine-tuning, so the reported gains are genuine held-out measurements. The key ablation (Table 7) compares pre-training on original versus decoupled images while holding the rest of the pipeline fixed, showing a monotonic improvement in XFUND as decoupling increases up to a point, which directly supports the causal claim. The LKI module does inject multilingual text at fine-tuning, but Table 8 shows strong cross-lingual performance even without LKI (XFUND 61.36 versus 57.24 for LiLT), so the vision-layout contribution does not reduce to the text encoder. The choice of decouple_resolution=1024 from Table 7 is a hyperparameter selection on the evaluation set, a methodological weakness but not a circular reduction of the prediction to its inputs. Self-citations in related work are not load-bearing; each cited claim is either supported by the paper's own experiments or purely contextual. No equation or construction equates a predicted quantity with a fitted input, and no load-bearing argument relies on a self-citation chain. Thus the derivation is self-contained for the purposes of circularity analysis.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

No new physical or conceptual entities are introduced; the free parameters are the explicit decoupling resolution and the standard trained modules. The main burden is the domain assumption that diffusion-based text distortion preserves the cues needed for entity extraction.

free parameters (1)
  • decouple_resolution = 1024
    Controls how strongly AnyText scrambles text; chosen as the best trade-off in Table 7 using FUNSD and XFUND, so it is tuned on the same benchmarks used for the main claims.
assumptions (4)
  • domain assumption Vision and layout cues in document images are language-invariant.
    The paper's core hypothesis, argued via surrogate OCR and language-classification metrics and Figure 3; it is not proven and is the basis for treating language-decoupled images as equivalent to originals.
  • ad hoc to paper Prompting AnyText to edit a 1x1 blank region removes readable text from dense document images while preserving layout and visual appearance.
    This is the central data-generation trick; the paper relies on the model's distortion behavior rather than on a mechanism guaranteed by AnyText's design.
  • domain assumption ESP's pseudo-label generation on DocBank and RVL-CDIP provides sufficiently correct entity labels for pre-training.
    No validation of pseudo-label quality is reported; the method follows ESP, but errors in pseudo-labels could limit the decoupled pre-training signal.
  • domain assumption Frozen multilingual Sentence-BERT embeddings are useful for entity classification in downstream languages.
    LKI injects these embeddings in Equation (2); their quality determines part of cross-lingual transfer, so the model is not purely vision and layout at test time.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LDP: Generalizing to Multilingual Visual Information Extraction by Language Decoupled Pretraining." pith.science (2026). https://pith.science/paper/MISURGFL

@misc{pith2026241214596,
  author       = {Pith},
  title        = {Pith review of: LDP: Generalizing to Multilingual Visual Information Extraction by Language Decoupled Pretraining},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MISURGFL}},
  note         = {Machine review of arXiv:2412.14596}
}
read the original abstract

Visual Information Extraction (VIE) plays a crucial role in the comprehension of semi-structured documents, and several pre-trained models have been developed to enhance performance. However, most of these works are monolingual (usually English). Due to the extremely unbalanced quantity and quality of pre-training corpora between English and other languages, few works can extend to non-English scenarios. In this paper, we conduct systematic experiments to show that vision and layout modality hold invariance among images with different languages. If decoupling language bias from document images, a vision-layout-based model can achieve impressive cross-lingual generalization. Accordingly, we present a simple but effective multilingual training paradigm LDP (Language Decoupled Pre-training) for better utilization of monolingual pre-training data. Our proposed model LDM (Language Decoupled Model) is first pre-trained on the language-independent data, where the language knowledge is decoupled by a diffusion model, and then the LDM is fine-tuned on the downstream languages. Extensive experiments show that the LDM outperformed all SOTA multilingual pre-trained models, and also maintains competitiveness on downstream monolingual/English benchmarks.

Figures

Figures reproduced from arXiv: 2412.14596 by the authors.

Figure 1
Figure 1. (a) Original image. (b) The previous method, LiLT, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The text recognition ratio and language classifica [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. VIE performance on XFUND when applying language-decoupled images. Quantitative Evaluation AnyText model can decouple language bias from docu￾ment images. To measure the remaining language bias in decoupled images, we conduct experiments on XFUND and use two surrogate metrics: (i) text recognition ratio, and (ii) language classification accuracy. For the text recognition ra￾tio, we crop the image according to the bou… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: The overall illustration of LDM. LDM takes the image and bounding boxes as input, which exactly follows SAM’s [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 39 canonical work pages

  1. [1]

    Appalaraju, S.; Tang, P.; Dong, Q.; Sankaran, N.; Zhou, Y.; and Manmatha, R. 2024. DocFormerv2: Local Features for Document Understanding. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI)

    work page 2024

  2. [2]

    Chen, H.; Xu, Z.; Gu, Z.; Lan, J.; Zheng, X.; Li, Y.; Meng, C.; Zhu, H.; and Wang, W. 2023. DiffUTE: Universal Text Editing Diffusion Model. In Advances in Neural Information Processing Systems (NeurIPS)

    work page 2023

  3. [3]

    Chen, Y.; Wang, W.; Zhou, Y.; Yang, F.; Yang, D.; and Wang, W. 2020. Self-Training for Domain Adaptive Scene Text Detection. In International Conference on Pattern Recognition, (ICPR)

    work page 2020

  4. [4]

    Cheng, Z.; Zhang, P.; Li, C.; Liang, Q.; Xu, Y.; Li, P.; Pu, S.; Niu, Y.; and Wu, F. 2022. TRIE++: Towards End-to-End Information Extraction from Visually Rich Documents. arXiv:2207.06744

    work page Pith review arXiv 2022

  5. [5]

    Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; and Fei-Fei, L. 2009. ImageNet: A Large-Scale Hierarchical Image Database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    work page 2009

  6. [6]

    Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2019. BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)

    work page 2019

  7. [7]

    Fujitake, M. 2024. LayoutLLM: Large Language Model Instruction Tuning for Visually Rich Document Understanding. In Proceedings of the Joint International Conference on Computational Linguistics and Language Resources and Evaluation (LREC/COLING)

    work page 2024

  8. [8]

    Gu, Z.; Meng, C.; Wang, K.; Lan, J.; Wang, W.; Gu, M.; and Zhang, L. 2022. XYLayoutLM: Towards Layout-Aware Multimodal Networks for Visually-Rich Document Understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    work page 2022

Show all 48 references
  1. [9]

    W.; Ufkes, A.; and Derpanis, K

    Harley, A. W.; Ufkes, A.; and Derpanis, K. G. 2015. Evaluation of Deep Convolutional Nets for Document Image Classification and Retrieval. In Proceedings of the International Conference on Document Analysis and Recognition (ICDAR)

    2015

  2. [10]

    Hu, A.; Xu, H.; Ye, J.; Yan, M.; Zhang, L.; Zhang, B.; Li, C.; Zhang, J.; Jin, Q.; Huang, F.; and Zhou, J. 2024. mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understanding. arXiv:2403.12895

    2024 arXiv

  3. [11]

    Huang, Y.; Lv, T.; Cui, L.; Lu, Y.; and Wei, F. 2022. LayoutLMv3: Pre-Training for Document AI with Unified Text and Image Masking. In Proceedings of the ACM International Conference on Multimedia (MM)

    2022

  4. [12]

    K.; and Thiran, J.-P

    Jaume, G.; Ekenel, H. K.; and Thiran, J.-P. 2019. FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents. In Proceedings of the International Conference on Document Analysis and Recognition Workshops (ICDARW)

    2019

  5. [13]

    Kim, G.; Hong, T.; Yim, M.; Nam, J.; Park, J.; Yim, J.; Hwang, W.; Yun, S.; Han, D.; and Park, S. 2022. OCR-Free Document Understanding Transformer. In Proceedings of the European Conference on Computer Vision (ECCV)

    2022

  6. [14]

    P.; and Welling, M

    Kingma, D. P.; and Welling, M. 2014. Auto-Encoding Variational Bayes. In Proceedings of the International Conference on Learning Representations (ICLR)

    2014

  7. [15]

    C.; Lo, W.-Y.; et al

    Kirillov, A.; Mintun, E.; Ravi, N.; Mao, H.; Rolland, C.; Gustafson, L.; Xiao, T.; Whitehead, S.; Berg, A. C.; Lo, W.-Y.; et al. 2023. Segment Anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    2023

  8. [16]

    D.; Agam, G.; Argamon, S.; Frieder, O.; Grossman, D

    Lewis, D. D.; Agam, G.; Argamon, S.; Frieder, O.; Grossman, D. A.; and Heard, J. 2006. Building a Test Collection for Complex Document Information Processing. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR)

    2006

  9. [17]

    Li, M.; Xu, Y.; Cui, L.; Huang, S.; Wei, F.; Li, Z.; and Zhou, M. 2020. DocBank: A Benchmark Dataset for Document Layout Analysis. In Proceedings of the International Conference on Computational Linguistics (COLING)

    2020

  10. [18]

    Li, Y.; Qian, Y.; Yu, Y.; Qin, X.; Zhang, C.; Liu, Y.; Yao, K.; Han, J.; Liu, J.; and Ding, E. 2021. StrucTexT: Structured Text Understanding with Multi-Modal Transformers. In Proceedings of the ACM Multimedia Conference (MM)

    2021

  11. [19]

    Li, Z.; Shu, Y.; Zeng, W.; Yang, D.; and Zhou, Y. 2024. First Creating Backgrounds Then Rendering Texts: A New Paradigm for Visual Text Blending. In European Conference on Artificial Intelligence (ECAI)

    2024

  12. [20]

    Liu, H.; Li, C.; Wu, Q.; and Lee, Y. J. 2023. Visual Instruction Tuning. In Advances in Neural Information Processing Systems (NeurIPS)

    2023

  13. [21]

    Liu, Y.; Yang, B.; Liu, Q.; Li, Z.; Ma, Z.; Zhang, S.; and Bai, X. 2024. TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document. arXiv:2403.04473

    2024 arXiv

  14. [22]

    Park, S.; Shin, S.; Lee, B.; Lee, J.; Surh, J.; Seo, M.; and Lee, H. 2019. CORD: A Consolidated Receipt Dataset for Post-OCR Parsing. In Proceedings of the Workshop on Document Intelligence at NeurIPS

    2019

  15. [23]

    Qiao, Z.; Zhou, Y.; Wei, J.; Wang, W.; Zhang, Y.; Jiang, N.; Wang, H.; and Wang, W. 2021. PIMNet: A Parallel, Iterative and Mimicking Network for Scene Text Recognition. In Proceedings of the ACM International Conference on Multimedia (MM)

    2021

  16. [24]

    Qiao, Z.; Zhou, Y.; Yang, D.; Zhou, Y.; and Wang, W. 2020. SEED: Semantics Enhanced Encoder-Decoder Framework for Scene Text Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2020

  17. [25]

    Raffel, C.; Shazeer, N.; Roberts, A.; Lee, K.; Narang, S.; Matena, M.; Zhou, Y.; Li, W.; and Liu, P. J. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. Journal of Machine Learning Research (JMLR)

    2020

  18. [26]

    Reimers, N.; and Gurevych, I. 2019. Sentence-BERT: Sentence Embeddings Using Siamese BERT-Networks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)

    2019

  19. [27]

    Reimers, N.; and Gurevych, I. 2020. Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)

    2020

  20. [28]

    Shen, H.; Gao, X.; Wei, J.; Qiao, L.; Zhou, Y.; Li, Q.; and Cheng, Z. 2023. Divide Rows and Conquer Cells: Towards Structure Recognition for Large Tables. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, (IJCAI)

    2023

  21. [29]

    Shu, Y.; Zeng, W.; Li, Z.; Zhao, F.; and Zhou, Y. 2024. Visual Text Meets Low-level Vision: A Comprehensive Survey on Visual Text Processing. arXiv:2402.03082

    2024 arXiv

  22. [30]

    Tang, Z.; Yang, Z.; Wang, G.; Fang, Y.; Liu, Y.; Zhu, C.; Zeng, M.; Zhang, C.; and Bansal, M. 2023. Unifying Vision, Text, and Layout for Universal Document Processing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2023

  23. [31]

    Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.-A.; Lacroix, T.; Rozière, B.; Goyal, N.; Hambro, E.; Azhar, F.; Rodriguez, A.; Joulin, A.; Grave, E.; and Lample, G. 2023. LLaMA: Open and Efficient Foundation Language Models. arXiv:2302.13971

    2023 arXiv

  24. [32]

    Tuo, Y.; Xiang, W.; He, J.-Y.; Geng, Y.; and Xie, X. 2024. AnyText: Multilingual Visual Text Generation and Editing. In Proceedings of the International Conference on Learning Representations (ICLR)

    2024

  25. [33]

    Wang, J.; Jin, L.; and Ding, K. 2022. LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understanding. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)

    2022

  26. [34]

    Wei, H.; Kong, L.; Chen, J.; Zhao, L.; Ge, Z.; Yang, J.; Sun, J.; Han, C.; and Zhang, X. 2025. Vary: Scaling up the vision vocabulary for large vision-language model. In European Conference on Computer Vision (ECCV)

    2025

  27. [35]

    Xu, Y.; Li, M.; Cui, L.; Huang, S.; Wei, F.; and Zhou, M. 2020. LayoutLM: Pre-Training of Text and Layout for Document Image Understanding. In Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD)

    2020

  28. [36]

    Xu, Y.; Lv, T.; Cui, L.; Wang, G.; Lu, Y.; Florencio, D.; Zhang, C.; and Wei, F. 2021 a . LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understanding. arXiv:2104.08836

    2021 arXiv

  29. [37]

    Xu, Y.; Lv, T.; Cui, L.; Wang, G.; Lu, Y.; Florêncio, D. A. F.; Zhang, C.; and Wei, F. 2022. XFUND: A Benchmark Dataset for Multilingual Visually Rich Form Understanding. In Findings of the Association for Computational Linguistics (ACL)

    2022

  30. [38]

    Xu, Y.; Xu, Y.; Lv, T.; Cui, L.; Wei, F.; Wang, G.; Lu, Y.; Florêncio, D. A. F.; Zhang, C.; Che, W.; Zhang, M.; and Zhou, L. 2021 b . LayoutLMv2: Multi-Modal Pre-Training for Visually-Rich Document Understanding. In Proceedings of the Annual Meeting of the Association for Comp...

    2021

  31. [39]

    Yang, Z.; Long, R.; Wang, P.; Song, S.; Zhong, H.; Cheng, W.; Bai, X.; and Yao, C. 2023. Modeling Entities as Semantic Points for Visual Information Extraction in the Wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2023

  32. [40]

    Ye, J.; Hu, A.; Xu, H.; Ye, Q.; Yan, M.; Dan, Y.; Zhao, C.; Xu, G.; Li, C.; Tian, J.; Qi, Q.; Zhang, J.; and Huang, F. 2023 a . mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding. arXiv:2307.02499

    2023 arXiv

  33. [41]

    Ye, Q.; Xu, H.; Xu, G.; Ye, J.; Yan, M.; Zhou, Y.; Wang, J.; Hu, A.; Shi, P.; Shi, Y.; Li, C.; Xu, Y.; Chen, H.; Tian, J.; Qi, Q.; Zhang, J.; and Huang, F. 2023 b . mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality. arXiv:2304.14178

    2023 arXiv

  34. [42]

    Yu, Y.; Li, Y.; Zhang, C.; Zhang, X.; Guo, Z.; Qin, X.; Yao, K.; Han, J.; Ding, E.; and Wang, J. 2023. StrucTexTv2: Masked Visual-Textual Prediction for Document Image Pre-Training. In Proceedings of the International Conference on Learning Representations (ICLR)

    2023

  35. [43]

    Zeng, G.; Zhang, Y.; Wei, J.; Yang, D.; Zhang, P.; Gao, Y.; Qin, X.; and Zhou, Y. 2024 a . Focus, Distinguish, and Prompt: Unleashing CLIP for Efficient and Flexible Scene Text Retrieval. In Proceedings of the ACM International Conference on Multimedia (MM)

    2024

  36. [44]

    Zeng, G.; Zhang, Y.; Zhou, Y.; Yang, X.; Jiang, N.; Zhao, G.; Wang, W.; and Yin, X. 2023. Beyond OCR + VQA: Towards end-to-end reading and reasoning for robust and accurate textvqa. Pattern Recognit

    2023

  37. [45]

    Zeng, W.; Shu, Y.; Li, Z.; Yang, D.; and Zhou, Y. 2024 b . TextCtrl: Diffusion-based Scene Text Editing with Prior Guidance Control. In Advances in Neural Information Processing Systems (NeurIPS)

    2024

  38. [46]

    Zhang, P.; Xu, Y.; Cheng, Z.; Pu, S.; Lu, J.; Qiao, L.; Niu, Y.; and Wu, F. 2020. TRIE: End-to-End Text Reading and Information Extraction for Document Understanding. In Proceedings of the ACM International Conference on Multimedia (MM)

    2020

  39. [47]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...

  40. [48]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.