Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Class-Agnostic Region-of-Interest Matching in Document Images

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read RoI-Matching defines a new task: with only a reference image, a binary mask as the visual prompt, and a target document image, a model outputs the corresponding region in the target, with no class labels, and a proposed baseline reaches…

desk verdict A useful document-specific benchmark and a strong visual baseline, but the open-set claim is narrower than the benchmark's construction makes it look. read the letter →

arxiv 2506.21055 v1 pith:SCMAZZ2L submitted 2025-06-26 cs.CV

Demin Zhang , Jiahao Lyu , Zhijie Shen , Yu Zhou This is my paper · ORCID
classification cs.CV
keywords class-agnosticregion-of-interestmatchingdocumentimageanalysisvisualpromptcross-domainbenchmarksiamesenetworkcross-attention
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Existing document understanding tasks such as layout analysis and key information extraction assume fixed categories, so a user cannot ask for an arbitrary region that has no predefined name. This paper defines class-agnostic region-of-interest matching (RoI-Matching): given a reference document, a binary mask highlighting the desired region, and a target document, the model must output the corresponding mask in the target, using only the visual appearance of the prompt. To make the task testable, the authors build RoI-Matching-Bench with three difficulty levels (similar layout, shifted position, and different appearance) and report macro and micro metrics. They also propose RoI-Matcher, a siamese network with cross-attention and a pixel-aggregation-style head, which reaches 87.3% mIoU and 83.9% F-measure overall while running at 0.056 seconds per image. If this holds, document workflows can be customized by pointing at an example rather than retraining for each new region type.

What carries the argument

The load-bearing mechanism is the two-stage cross-granularity attention combined with a pixel-aggregation-style head (predicting region, kernel, and similarity vectors). A siamese visual encoder produces multi-level features for reference and target; the reference mask is applied early, before feature pyramid fusion, to preserve source semantics. Two multi-head cross-attention operations first align the masked reference prompt with reference features and then project it onto target features. The head predicts a six-channel mask, and the kernel-plus-vector post-processing separates regions that overlap in dense documents. Efficiency comes from multi-level grid sampling before attention and a lightweight feature pyramid variant.

What would settle it

Evaluate the trained RoI-Matcher on a held-out set of user-defined regions that are semantically specific but visually plain, such as the 'total amount' field on receipts that differ in font, size, and position; if mIoU drops well below the reported 87.3% overall, the visual-prompt-only assumption would be falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that RoI-Matching is a distinct, useful task and that a simple baseline can solve it well. RoI-Matcher encodes reference and target images with a shared visual encoder, multiplies the reference features by the user mask at each level, aligns the masked prompt with reference and target features through two cross-attention layers, and predicts a six-channel mask (region, kernel, and four similarity-vector channels) so that overlapping text regions can be separated during post-processing. On RoI-Matching-Bench, the method achieves 92.1% mIoU at Level I, 88.5% at Level II, and 85.0% at Level III, totaling 87.3% mIoU and 83.9% F-measure, whereas the strongest few-shot segmentation baseline reaches at most 59.5% mIoU and the best multimodal language model reaches 55.3% F-measure. The paper interprets this as evidence that a purely visual prompt carries enough information to transfer a customized region across documents when the region has distinctive visual features.

Load-bearing premise

The load-bearing premise is that the real regions users care about are visually distinctive enough to be defined by a mask alone, because the benchmark excludes plain text content like article paragraphs and gives the model no text or category information.

Editorial extensions

If this is right

  • A user can define a region of interest by example rather than by category, so no retraining is needed when the region type changes.
  • The three-level benchmark gives a standardized way to compare future RoI-Matching methods under real-world layout variability.
  • At 0.056 seconds per image, RoI-Matcher is fast enough for interactive document workflows, while multimodal language models are roughly 56 times slower on the same inputs.
  • Because the model is class-agnostic and mask-based, the same checkpoint handles word-level, line-level, and paragraph-level regions within one forward pass.
  • The label-generation and loss design (kernel shrink plus aggregation and dispersion terms) is what lets the baseline separate overlapping regions, a common case in dense documents.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the visual-prompt premise is accepted, the method is likely to struggle on RoIs defined by text semantics whose appearance varies, such as the 'total amount' field on receipts, because the benchmark explicitly excludes visually plain content; a text-aware extension would be needed for those cases.
  • The same cross-attention machinery could in principle transfer to other image domains, such as maps or medical scans, where a user points to a structure in one image and wants its counterpart in another.
  • The evaluation could be sharpened by testing on a held-out set of plain paragraphs, which would probe how much of the reported accuracy depends on the benchmark's selection of visually distinctive categories.
  • A multi-prompt variant that lets the user give several reference masks for the same region could give the model a self-correction mechanism, addressing the paper's stated one-shot limitation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a new task, class-agnostic Region-of-Interest Matching (RoI-Matching), in which a user provides a reference document image and a binary mask indicating a region of interest, and the model must locate the corresponding region in a target document image. To support this task, the authors construct RoI-Matching-Bench from six existing document datasets, organized into three difficulty levels, and introduce mIoU and F-measure as evaluation metrics. They also present RoI-Matcher, a Siamese-network baseline with multi-level feature extraction, cross-granularity attention, and a PAN-style segmentation head. Experiments report that RoI-Matcher achieves 87.3% mIoU and 83.9% F-measure on the benchmark, outperforming CD-FSS and MLLM baselines, with ablations on backbones, training strategies, losses, and efficiency. The paper releases code and explicitly lists limitations, including the model being one-shot, uni-modal, and not yet supporting repeated multi-RoI inference.

Significance. If the proposed benchmark and baseline are accepted, the paper would provide a useful task formulation for flexible document image understanding, with a reproducible baseline and a public benchmark built from existing datasets. The explicit difficulty levels, the macro/micro metrics, and the efficiency analysis (0.056 s per image) are practical contributions, and the code release is a strength. However, the central claims of flexibility, open-set generalization, and coverage of ``real-world conditions'' are currently supported only for a visually distinctive sub-population of document regions, because the benchmark construction deliberately excludes text-semantics-defined regions and the model receives no text or category information. In addition, a citation error in the main comparison table and an underspecified baseline training protocol weaken confidence in the empirical comparison. These issues are fixable but need to be addressed before the claims can be fully trusted.

major comments (3)
  1. [Section 3.1 and Section 4.1] The benchmark construction in Section 3.1 explicitly selects ``categories with distinctive features'' and excludes ``content lacking significant visual features, such as article paragraphs.'' Combined with Section 4.1, where ``there is no category information during the whole pipeline,'' and Section 6, where the model is admitted to be uni-modal and to ignore linguistic information, the evaluation covers only RoIs that are visually distinctive and visually similar across documents. The task-level claims of flexibility, open-set matching, and evaluation ``under real-world conditions'' are therefore only supported for a visually distinctive sub-population. Since many user-customized document RoIs are defined by text semantics (for example, ``total amount'' on a receipt or a specific paragraph in a paper), the benchmark should either include a diagnostic split of text-defined regions and report performance on it, or the claims should be explicitly scoped to visually defined RoIs. This is a load-bearing issue for the validity of the benchmark as a general evaluation of RoI-Matching.
  2. [Table 5.1 and References [66]] Reference [66] is cited in the main comparison table and in Section 5.1 as a CD-FSS baseline (DMTNet), but the reference list entry for [66] is ``DMT-Net: Deep Multiple Networks for Low-Light Image Enhancement Based on Retinex Model'' (IEEE Access, 2023), which is an image-enhancement paper rather than a cross-domain few-shot segmentation method. The reported DMTNet row in Table 5.1 (59.5% mIoU, 34.5% F-measure) therefore cannot be verified from the cited source. The authors must correct the citation or replace the baseline with the intended CD-FSS method, and re-verify the corresponding numbers; a wrong baseline in the central comparison table undermines confidence in the empirical claims.
  3. [Section 5.1] The experimental protocol for the CD-FSS and MLLM baselines is underspecified. The text states that the cross-entropy loss is replaced with Dice loss for CD-FSS methods, but it does not state whether these baselines are trained on the RoI-Matching-Bench training splits, fine-tuned from their original checkpoints, or evaluated without any training on the benchmark. Similarly, for the MLLM baselines, it is not stated whether the reported numbers come from in-context prompting alone, optional fine-tuning, or a specific decoding protocol. Because the main empirical claim is that RoI-Matcher outperforms these baselines, the authors must report the exact training and evaluation protocol for every baseline, including the data split used, number of training steps, and any prompt details, so that the comparison is reproducible and fair.
minor comments (5)
  1. [Section 3.2] The metric name ``Mean Interaction-of-Union'' should be ``Mean Intersection-over-Union''; the abbreviation mIoU is correct, but the spelled-out name is wrong.
  2. [Section 5.2] The sentence ``the CD-FSS method performs well in simple scenes, with ResNet achieving 73.2% mIoU and 63.6% F-Measure'' should refer to a specific method such as RestNet rather than ``ResNet,'' which is the backbone rather than the method.
  3. [Section 5] In the introductory sentence ``we compare our RoI-Matcher with other CD-DSS methods,'' the acronym ``CD-DSS'' should be ``CD-FSS'' to match the terminology used elsewhere.
  4. [Table 5.5 and Section 5.3] There is an inconsistency in the parameter count: the text says the first row reports ``4.5M parameters,'' but Table 5.5 lists 34.5M. Also, the table column labeled ``Throughput (GFlops)'' is mislabeled; GFlops is a measure of computational cost, not throughput. The text should say ``computational cost'' or ``FLOPs'' and should be interpreted accordingly.
  5. [Section 4.4] Equation (9) would be clearer if the channel dimensions of the four similarity-vector channels in M_v were stated explicitly, since the notation [M_r, M_k, M_v] with M_v as a multi-channel tensor may be ambiguous on first reading.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; the main results are held-out empirical comparisons against external baselines, with only benchmark-coverage limitations.

full rationale

The paper's central claim is empirical: Table 5.1 reports RoI-Matcher at 87.3% mIoU / 83.9% F-Measure versus CD-FSS and MLLM baselines. These numbers come from held-out splits of RoI-Matching-Bench, which is assembled from six external datasets (SVRD, POIE, SROIE, D4LA, M6Doc), and the comparison methods are external (PATNet, RestNet, DMTNet, ABCD-FSS, Qwen2-VL, InternVL). No equation defines the benchmark result in terms of the method's own fitted parameters; hyperparameters such as r=0.4, alpha=0.5, beta=0.25 are standard constants, not fitted to reproduce the headline performance. The closest issue to circularity is benchmark coverage: Section 3.1 selects 'categories with distinctive features' and excludes 'content lacking significant visual features, such as article paragraphs,' while Section 4.1 removes category and text information, so the evaluation under-represents text-semantics-defined RoIs. This is a genuine limitation for the 'flexible/open-set' generalization claim and is partly conceded in Section 6, but it is a sampling-coverage concern rather than a derivation that reduces to its inputs. Self-citations in the related work are not load-bearing for the central result. No exhibitable circular step is present.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No new physical or conceptual entities are postulated beyond the task definition itself. The central claims rest on standard training assumptions, the accuracy of derived and manual annotations, and faithful baseline comparisons.

free parameters (4)
  • shrink ratio r = 0.4
    Used in Eq. (11) to generate the kernel mask; set empirically.
  • loss balance factors alpha, beta = alpha=0.5, beta=0.25
    Set in Eq. (12) to weight kernel, aggregation, and discrimination losses.
  • F-measure IoU threshold = 0.5
    Standard instance-matching threshold in scene text detection, used to count a prediction as correct.
  • input resolution = 640x640
    All training and test images resized to 640x640 (Sec. 5.1); affects the feature grid sampling ratio.
assumptions (4)
  • domain assumption A binary mask over the reference image is a sufficient visual specification of the target RoI
    Task definition in Sec. 4.1 states no category information is used; Section 3.1 restricts the benchmark to visually distinctive categories, so this assumption is only tested under that restriction.
  • domain assumption Ground-truth masks from existing box annotations and manual annotation are accurate
    Section 3.1 describes generation and strict visual verification but reports no inter-annotator agreement.
  • domain assumption Compared baselines are implemented faithfully
    Section 5.1 discloses modifications (Dice loss for CD-FSS, in-context learning for MLLMs) but no hyperparameter tuning or seed averaging is reported; the DMTNet [66] citation mismatch raises uncertainty.
  • standard math Deep learning training generalizes from the training split to the test split of the benchmark
    Adopted implicitly in all experiments in Section 5.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Class-Agnostic Region-of-Interest Matching in Document Images." pith.science (2026). https://pith.science/paper/SCMAZZ2L

@misc{pith2026250621055,
  author       = {Pith},
  title        = {Pith review of: Class-Agnostic Region-of-Interest Matching in Document Images},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SCMAZZ2L}},
  note         = {Machine review of arXiv:2506.21055}
}
read the original abstract

Document understanding and analysis have received a lot of attention due to their widespread application. However, existing document analysis solutions, such as document layout analysis and key information extraction, are only suitable for fixed category definitions and granularities, and cannot achieve flexible applications customized by users. Therefore, this paper defines a new task named ``Class-Agnostic Region-of-Interest Matching'' (``RoI-Matching'' for short), which aims to match the customized regions in a flexible, efficient, multi-granularity, and open-set manner. The visual prompt of the reference document and target document images are fed into our model, while the output is the corresponding bounding boxes in the target document images. To meet the above requirements, we construct a benchmark RoI-Matching-Bench, which sets three levels of difficulties following real-world conditions, and propose the macro and micro metrics to evaluate. Furthermore, we also propose a new framework RoI-Matcher, which employs a siamese network to extract multi-level features both in the reference and target domains, and cross-attention layers to integrate and align similar semantics in different domains. Experiments show that our method with a simple procedure is effective on RoI-Matching-Bench, and serves as the baseline for further research. The code is available at https://github.com/pd162/RoI-Matching.

Figures

Figures reproduced from arXiv: 2506.21055 by the authors.

Figure 1.1
Figure 1.1. The task definition of RoI-Matching. (a) shows the pipeline of RoI-Matching. Given some document images by the user, RoI-Matcher responds to the correspond￾ing mask, even if the user only provides visual prompt without class labels (“name”, “number of calories”, and “figure” from left to right). (b) demonstrates the difference of RoI-Matching between other Tasks. like processing [1, 2], detection [3–5], recognition … view at source ↗
Figure 3.1
Figure 3.1. The visualizations of RoI-Matching-bench. The reference image (left) and the target image (right) are image pairs. Hint: these visual prompts indicate semantic labels, which are “Tax Ratio”, “Items in Receipt”, “Equation” [PITH_FULL_IMAGE:figures/full_fig_p006_3_1.png] view at source ↗
Figure 3.2
Figure 3.2. The position distribution of RoI-Matching benchmark. There are two images in each group, the left is the visualization of the reference images, and the right is the target images. 3.2 Evaluation Protocols We report two metrics: one from the pixel-wise protocol for common semantic segmentation and another from the instance-wise protocol for scene text detec￾tion. Mean Interaction-of-Union (mIoU). Inspired by [58], we… view at source ↗
Figures from the paper (2 more)
Figure 4.1
Figure 4.1. Figure 4.1: The architecture of RoI-Matcher. The background colors represent three stages of our model, blue for Visual Perception, green for Cross-granularity Atten￾tion, and yellow for Segmentation Head, respectively. The colors of arrows indicate flows of different inputs, bl…
Figure 5.1
Figure 5.1. Figure 5.1: The visualization of RoI-Matcher. The red bounding boxes of each image are ground truths and the green ones are predictions. The difficulty levels I - III are shown from top to bottom in this figure [PITH_FULL_IMAGE:figures/full_fig_p013_5_1.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Uni-DocDiff: A Unified Document Restoration Model Based on Diffusion

    cs.CV 2025-08 conditional novelty 6.0 of 10

    A dual-stream diffusion model with a handcrafted prior pool and a prior fusion module unifies six document restoration tasks and matches task-specific specialists.

Reference graph

Works this paper leans on

69 extracted references · 63 canonical work pages · cited by 1 Pith paper

  1. [66]

    Dmt-net: Deep multiple networks for low-light image enhancement based on retinex model.IEEE Access, 11:132147–132161, 2023

    Minh-Thien Duong, Seongsoo Lee, and Min-Cheol Hong. Dmt-net: Deep multiple networks for low-light image enhancement based on retinex model.IEEE Access, 11:132147–132161, 2023

    work page 2023

  2. [1]

    Visual text processing: A comprehensive review and unified evaluation, 2025

    Yan Shu, Weichao Zeng, Fangmin Zhao, Zeyu Chen, Zhenhang Li, Xiaomeng Yang, Yu Zhou, Paolo Rota, Xiang Bai, Lianwen Jin, Xu-Cheng Yin, and Nicu Sebe. Visual text processing: A comprehensive review and unified evaluation, 2025. 16 D. Zhang et al

    work page 2025

  3. [2]

    TextCtrl: Diffusion-based scene text editing with prior guidance control

    Weichao Zeng, Yan Shu, Zhenhang Li, Dongbao Yang, and Yu Zhou. TextCtrl: Diffusion-based scene text editing with prior guidance control. InNeurIPS, pages 138569–138594, 2024

    work page 2024

  4. [3]

    The devil is in fine-tuning and long-tailed problems: A new benchmark for scene text detection

    Tianjiao Cao, Jiahao Lyu, Weichao Zeng, Weimin Mu, and Yu Zhou. The devil is in fine-tuning and long-tailed problems: A new benchmark for scene text detection. InIJCAI, 2025

    work page 2025

  5. [4]

    Perceiving ambiguity and semantics without recognition: An efficientandeffectiveambiguousscenetextdetector

    Yan Shu, Wei Wang, Yu Zhou, Shaohui Liu, Aoting Zhang, Dongbao Yang, and Weipinng Wang. Perceiving ambiguity and semantics without recognition: An efficientandeffectiveambiguousscenetextdetector. InACM MM,page1851–1862, 2023

    work page 2023

  6. [5]

    Self-training for domain adaptive scene text detection

    Yudi Chen, Wei Wang, Yu Zhou, Fei Yang, Dongbao Yang, and Weiping Wang. Self-training for domain adaptive scene text detection. InICPR, pages 850–857. IEEE, 2021

    work page 2021

  7. [6]

    SEED: Semantics enhanced encoder-decoder framework for scene text recognition

    Zhi Qiao, Yu Zhou, Dongbao Yang, Yucan Zhou, and Weiping Wang. SEED: Semantics enhanced encoder-decoder framework for scene text recognition. In CVPR, pages 13528–13537, 2020

    work page 2020

  8. [7]

    PIMNet: A parallel, iterative and mimicking network for scene text recognition

    Zhi Qiao, Yu Zhou, Jin Wei, Wei Wang, Yuan Zhang, Ning Jiang, Hongbin Wang, and Weiping Wang. PIMNet: A parallel, iterative and mimicking network for scene text recognition. InACM MM, pages 2046–2055, 2021

    work page 2021

Show all 69 references
  1. [8]

    IPAD: Iterative, parallel, and diffusion- based network for scene text recognition.IJCV, 2025

    Xiaomeng Yang, Zhi Qiao, and Yu Zhou. IPAD: Iterative, parallel, and diffusion- based network for scene text recognition.IJCV, 2025

    2025

  2. [9]

    Linguistics-aware masked image modeling for self-supervised scene text recognition

    Yifei Zhang, Chang Liu, Jin Wei, Xiaomeng Yang, Yu Zhou, Can Ma, and Xi- angyang Ji. Linguistics-aware masked image modeling for self-supervised scene text recognition. InCVPR, pages 9318–9328, 2025

    2025

  3. [10]

    Divide rows and conquer cells: Towards structure recognition for large tables

    Huawen Shen, Xiang Gao, Jin Wei, Liang Qiao, Yu Zhou, Qiang Li, and Zhanzhan Cheng. Divide rows and conquer cells: Towards structure recognition for large tables. InIJCAI, pages 1369–1377, 2023

    2023

  4. [11]

    Arbitrary reading order scene text spotter with local semantics guidance

    Jiahao Lyu, Wei Wang, Dongbao Yang, Jinwen Zhong, and Yu Zhou. Arbitrary reading order scene text spotter with local semantics guidance. InAAAI, vol- ume 39, pages 5919–5927, 2025

    2025

  5. [12]

    TPSNet: Reverse thinking of thin plate splines for arbitrary shape scene text representation

    Wei Wang, Yu Zhou, Jiahao Lv, Dayan Wu, Guoqing Zhao, Ning Jiang, and Weip- inng Wang. TPSNet: Reverse thinking of thin plate splines for arbitrary shape scene text representation. InACM MM, pages 5014–5025, 2022

    2022

  6. [13]

    TextBlockV2: Towards precise-detection-free scene text spotting with pre-trained language model.TOMM, 2025

    Jiahao Lyu, Jin Wei, Gangyan Zeng, Zeng Li, Enze Xie, Wei Wang, Can Ma, and Yu Zhou. TextBlockV2: Towards precise-detection-free scene text spotting with pre-trained language model.TOMM, 2025

    2025

  7. [14]

    Beyond cropped regions: New benchmark and corresponding baseline for chinese scene text retrieval in diverse layouts

    Gengluo Li, Huawen Shen, and Yu Zhou. Beyond cropped regions: New benchmark and corresponding baseline for chinese scene text retrieval in diverse layouts. In ICML, 2025

    2025

  8. [15]

    Focus, distinguish, and prompt: Unleashing clip for efficient and flexible scene text retrieval

    Gangyan Zeng, Yuan Zhang, Jin Wei, Dongbao Yang, Peng Zhang, Yiwen Gao, Xugong Qin, and Yu Zhou. Focus, distinguish, and prompt: Unleashing clip for efficient and flexible scene text retrieval. InACM MM, pages 2525–2534, 2024

    2024

  9. [16]

    LDP: Generalizing to multilingual visual information extraction by language decoupled pretraining

    Huawen Shen, Gengluo Li, Jinwen Zhong, and Yu Zhou. LDP: Generalizing to multilingual visual information extraction by language decoupled pretraining. In AAAI, volume 39, pages 6805–6813, 2025

    2025

  10. [17]

    Beyond OCR+ VQA: Involving ocr into the flow for robust and accurate textvqa

    Gangyan Zeng, Yuan Zhang, Yu Zhou, and Xiaomeng Yang. Beyond OCR+ VQA: Involving ocr into the flow for robust and accurate textvqa. InACM MM, pages 376–385, 2021

    2021

  11. [18]

    Publaynet: largest dataset ever for document layout analysis

    Xu Zhong, Jianbin Tang, and Antonio Jimeno Yepes. Publaynet: largest dataset ever for document layout analysis. InICDAR, pages 1015–1022. IEEE, 2019. Class-Agnostic Region-of-Interest Matching in Document Images 17

    2019

  12. [19]

    Learning to extract semantic structure from documents using multimodal fully convolutional neural networks

    Xiao Yang, Ersin Yumer, Paul Asente, Mike Kraley, Daniel Kifer, and C Lee Giles. Learning to extract semantic structure from documents using multimodal fully convolutional neural networks. InCVPR, pages 5315–5324, 2017

    2017

  13. [20]

    VSR: a unified framework for document layout analysis combining vision, semantics and relations

    Peng Zhang, Can Li, Liang Qiao, Zhanzhan Cheng, Shiliang Pu, Yi Niu, and Fei Wu. VSR: a unified framework for document layout analysis combining vision, semantics and relations. InICDAR, pages 115–130. Springer, 2021

    2021

  14. [21]

    Attention where it matters: Rethinking visualdocumentunderstandingwithselectiveregionconcentration

    Haoyu Cao, Changcun Bao, Chaohu Liu, Huang Chen, Kun Yin, Hao Liu, Yin- song Liu, Deqiang Jiang, and Xing Sun. Attention where it matters: Rethinking visualdocumentunderstandingwithselectiveregionconcentration. InICCV,pages 19517–19527, 2023

    2023

  15. [22]

    Bros: A pre-trained language model focusing on text and layout for better key information extraction from documents

    Teakgyu Hong, Donghyun Kim, Mingi Ji, Wonseok Hwang, Daehyun Nam, and Sungrae Park. Bros: A pre-trained language model focusing on text and layout for better key information extraction from documents. InAAAI, volume 36, pages 10767–10775, 2022

    2022

  16. [23]

    StrucText: Structured text under- standing with multi-modal transformers

    Yulin Li, Yuxi Qian, Yuechen Yu, Xiameng Qin, Chengquan Zhang, Yan Liu, Kun Yao, Junyu Han, Jingtuo Liu, and Errui Ding. StrucText: Structured text under- standing with multi-modal transformers. InACM MM, pages 1912–1920, 2021

    1912

  17. [24]

    Cross-domain few-shot semantic segmentation

    Shuo Lei, Xuchao Zhang, Jianfeng He, Fanglan Chen, Bowen Du, and Chang- Tien Lu. Cross-domain few-shot semantic segmentation. InECCV, pages 73–90. Springer, 2022

    2022

  18. [25]

    Pixel-by- pixel cross-domain alignment for few-shot semantic segmentation

    Antonio Tavera, Fabio Cermelli, Carlo Masone, and Barbara Caputo. Pixel-by- pixel cross-domain alignment for few-shot semantic segmentation. InW ACV, pages 1626–1635, 2022

    2022

  19. [26]

    Restnet: Boosting cross-domain few-shot segmentation with residual transformation network

    Xinyang Huang, Chuang Zhu, and Wenkai Chen. Restnet: Boosting cross-domain few-shot segmentation with residual transformation network. InBMVC, 2023

    2023

  20. [27]

    Adapt before comparison: A new perspective on cross-domain few- shot segmentation

    Jonas Herzog. Adapt before comparison: A new perspective on cross-domain few- shot segmentation. InCVPR, pages 23605–23615, 2024

    2024

  21. [28]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv, 2024

    PengWang,ShuaiBai,SinanTan,ShijieWang,ZhihaoFan,JinzeBai,KeqinChen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv, 2024

    2024

  22. [29]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. InCVPR, pages 24185–24198, 2024

    2024

  23. [30]

    Faster R-CNN: To- wards real-time object detection with region proposal networks.IEEE TPAMI, 39(6):1137–1149, 2016

    Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster R-CNN: To- wards real-time object detection with region proposal networks.IEEE TPAMI, 39(6):1137–1149, 2016

    2016

  24. [31]

    Mask R-CNN

    Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask R-CNN. In ICCV, pages 2961–2969, 2017

    2017

  25. [32]

    M6doc: A large-scale multi-format, multi- type, multi-layout, multi-language, multi-annotation category dataset for modern document layout analysis

    Hiuyi Cheng, Peirong Zhang, Sihang Wu, Jiaxin Zhang, Qiyuan Zhu, Zecheng Xie, Jing Li, Kai Ding, and Lianwen Jin. M6doc: A large-scale multi-format, multi- type, multi-layout, multi-language, multi-annotation category dataset for modern document layout analysis. InCVPR, pages ...

    2023

  26. [33]

    Swindocseg- menter: An end-to-end unified domain adaptive transformer for document instance segmentation

    Ayan Banerjee, Sanket Biswas, Josep Lladós, and Umapada Pal. Swindocseg- menter: An end-to-end unified domain adaptive transformer for document instance segmentation. InICDAR, pages 307–325. Springer, 2023

    2023

  27. [34]

    Dino: Detr with improved denoising anchor boxes for end-to- end object detection

    Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel Ni, and Heung-Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to- end object detection. InICLR, 2023. 18 D. Zhang et al

    2023

  28. [35]

    Selfdocseg: A self-supervised vision- based approach towards document segmentation

    Subhajit Maity, Sanket Biswas, Siladittya Manna, Ayan Banerjee, Josep Lladós, Saumik Bhattacharya, and Umapada Pal. Selfdocseg: A self-supervised vision- based approach towards document segmentation. InICDAR, pages 342–360. Springer, 2023

    2023

  29. [36]

    Bootstrap your own latent-a new approach to self-supervised learning.NeurIPS, 33:21271–21284, 2020

    Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond,ElenaBuchatskaya,CarlDoersch,BernardoAvilaPires,ZhaohanGuo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning.NeurIPS, 33:21271–21284, 2020

    2020

  30. [37]

    LayoutLM: Pre-training of text and layout for document image understanding

    Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, and Ming Zhou. LayoutLM: Pre-training of text and layout for document image understanding. In ACM SIGKDD, pages 1192–1200, 2020

    2020

  31. [38]

    LayoutLMv2: Multi-modal pre-training for visually-rich document understanding

    Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, et al. LayoutLMv2: Multi-modal pre-training for visually-rich document understanding. InACL, pages 2579–2591, 2021

    2021

  32. [39]

    LayoutLMv3: Pre-training for document ai with unified text and image masking

    Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, and Furu Wei. LayoutLMv3: Pre-training for document ai with unified text and image masking. InACM MM, pages 4083–4091, 2022

    2022

  33. [40]

    BEiT: Bert pre-training of image transformers

    Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. BEiT: Bert pre-training of image transformers. InICLR, 2022

    2022

  34. [41]

    Dit: Self-supervised pre-training for document image transformer

    Junlong Li, Yiheng Xu, Tengchao Lv, Lei Cui, Cha Zhang, and Furu Wei. Dit: Self-supervised pre-training for document image transformer. InACM MM, pages 3530–3539, 2022

    2022

  35. [42]

    Unidoc: Unified pretraining framework for document understanding.NeurIPS, 34:39–50, 2021

    Jiuxiang Gu, Jason Kuen, Vlad I Morariu, Handong Zhao, Rajiv Jain, Nikolaos Barmpalios, Ani Nenkova, and Tong Sun. Unidoc: Unified pretraining framework for document understanding.NeurIPS, 34:39–50, 2021

    2021

  36. [43]

    StrucTexTv2: Masked visual-textual prediction for document image pre-training

    Yuechen Yu, Yulin Li, Chengquan Zhang, Xiaoqiang Zhang, Zengyuan Guo, Xia- meng Qin, Kun Yao, Junyu Han, Errui Ding, and Jingdong Wang. StrucTexTv2: Masked visual-textual prediction for document image pre-training. InICLR, 2023

    2023

  37. [44]

    Apseg: auto-prompt network for cross-domain few-shot semantic seg- mentation

    Weizhao He, Yang Zhang, Wei Zhuo, Linlin Shen, Jiaqi Yang, Songhe Deng, and Liang Sun. Apseg: auto-prompt network for cross-domain few-shot semantic seg- mentation. InCVPR, pages 23762–23772, 2024

    2024

  38. [45]

    Layoutreader: Pre-training of text and layout for reading order detection.arXiv preprint arXiv:2108.11591, 2021

    Zilong Wang, Yiheng Xu, Lei Cui, Jingbo Shang, and Furu Wei. Layoutreader: Pre-training of text and layout for reading order detection.arXiv preprint arXiv:2108.11591, 2021

    2021 arXiv

  39. [46]

    Reading order matters: Information extraction from visually-rich documents by token path prediction

    Chong Zhang, Ya Guo, Yi Tu, Huan Chen, Jinyang Tang, Huijia Zhu, Qi Zhang, and Tao Gui. Reading order matters: Information extraction from visually-rich documents by token path prediction. InEMNLP, 2023

    2023

  40. [47]

    Layoutxlm: Multimodal pre-training for multilingual visually-rich document understanding.arXiv preprint arXiv:2104.08836, 2021

    Yiheng Xu, Tengchao Lv, Lei Cui, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, and Furu Wei. Layoutxlm: Multimodal pre-training for multilingual visually-rich document understanding.arXiv preprint arXiv:2104.08836, 2021

    2021 arXiv

  41. [48]

    Query-driven generative network for document infor- mation extraction in the wild

    Haoyu Cao, Xin Li, Jiefeng Ma, Deqiang Jiang, Antai Guo, Yiqing Hu, Hao Liu, Yinsong Liu, and Bo Ren. Query-driven generative network for document infor- mation extraction in the wild. InACM MM, pages 4261–4271, 2022

    2022

  42. [49]

    Unifying vision, text, and layout for universal document processing

    Zineng Tang, Ziyi Yang, Guoxin Wang, Yuwei Fang, Yang Liu, Chenguang Zhu, Michael Zeng, Cha Zhang, and Mohit Bansal. Unifying vision, text, and layout for universal document processing. InCVPR, pages 19254–19264, 2023

    2023

  43. [50]

    Pix2struct: Screenshot parsing as pretraining for visual lan- guage understanding

    Kenton Lee, Mandar Joshi, Iulia Raluca Turc, Hexiang Hu, Fangyu Liu, Ju- lian Martin Eisenschlos, Urvashi Khandelwal, Peter Shaw, Ming-Wei Chang, and Kristina Toutanova. Pix2struct: Screenshot parsing as pretraining for visual lan- guage understanding. InICML, pages 18893–1891...

    2023

  44. [51]

    Prestu: Pre-training for scene-text understanding

    Jihyung Kil, Soravit Changpinyo, Xi Chen, Hexiang Hu, Sebastian Goodman, Wei- Lun Chao, and Radu Soricut. Prestu: Pre-training for scene-text understanding. InICCV, pages 15270–15280, 2023

    2023

  45. [52]

    Ocr-free document understanding transformer

    Geewook Kim, Teakgyu Hong, Moonbin Yim, JeongYeon Nam, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, and Seunghyun Park. Ocr-free document understanding transformer. InECCV, pages 498–517. Springer, 2022

    2022

  46. [53]

    Omniparser: A unified framework for text spotting key information extraction and table recognition

    Jianqiang Wan, Sibo Song, Wenwen Yu, Yuliang Liu, Wenqing Cheng, Fei Huang, Xiang Bai, Cong Yao, and Zhibo Yang. Omniparser: A unified framework for text spotting key information extraction and table recognition. InCVPR, pages 15641–15653, 2024

    2024

  47. [54]

    Icdar 2023 competition on structured text extraction from visually-rich document images

    Wenwen Yu, Chengquan Zhang, Haoyu Cao, Wei Hua, Bohan Li, Huang Chen, Mingyu Liu, Mingrui Chen, Jianfeng Kuang, Mengjun Cheng, et al. Icdar 2023 competition on structured text extraction from visually-rich document images. In ICDAR, pages 536–552. Springer, 2023

    2023

  48. [55]

    Visual information extraction in the wild: practical dataset and end-to-end solution

    JianfengKuang,WeiHua,DingkangLiang,MingkunYang,DeqiangJiang,BoRen, and Xiang Bai. Visual information extraction in the wild: practical dataset and end-to-end solution. InICDAR, pages 36–53. Springer, 2023

    2023

  49. [56]

    Towards robust visual information extraction in real world: new dataset and novel solution

    Jiapeng Wang, Chongyu Liu, Lianwen Jin, Guozhi Tang, Jiaxin Zhang, Shuaitao Zhang, Qianying Wang, Yaqiang Wu, and Mingxiang Cai. Towards robust visual information extraction in real world: new dataset and novel solution. InAAAI, volume 35, pages 2738–2745, 2021

    2021

  50. [57]

    Vision grid transformer for document layout analysis

    Cheng Da, Chuwei Luo, Qi Zheng, and Cong Yao. Vision grid transformer for document layout analysis. InICCV, pages 19462–19472, 2023

    2023

  51. [58]

    Fully convolutional networks for semantic segmentation

    Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. InCVPR, pages 3431–3440, 2015

    2015

  52. [59]

    Real-time scene text detection with differentiable binarization

    Minghui Liao, Zhaoyi Wan, Cong Yao, Kai Chen, and Xiang Bai. Real-time scene text detection with differentiable binarization. InAAAI, volume 34, pages 11474– 11481, 2020

    2020

  53. [60]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. NeurIPS, 30, 2017

    2017

  54. [61]

    Efficientandaccuratearbitrary-shapedtextdetectionwith pixel aggregation network

    Wenhai Wang, Enze Xie, Xiaoge Song, Yuhang Zang, WenjiaWang, Tong Lu, Gang Yu,andChunhuaShen. Efficientandaccuratearbitrary-shapedtextdetectionwith pixel aggregation network. InICCV, pages 8440–8449, 2019

    2019

  55. [62]

    Shape robust text detection with progressive scale expansion network

    Wenhai Wang, Enze Xie, Xiang Li, Wenbo Hou, Tong Lu, Gang Yu, and Shuai Shao. Shape robust text detection with progressive scale expansion network. In CVPR, pages 9336–9345, 2019

    2019

  56. [63]

    A generic solution to polygon clipping.Communications of the ACM, 35(7):56–63, 1992

    Bala R Vatti. A generic solution to polygon clipping.Communications of the ACM, 35(7):56–63, 1992

    1992

  57. [64]

    V-net: Fully convolu- tional neural networks for volumetric medical image segmentation

    Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. V-net: Fully convolu- tional neural networks for volumetric medical image segmentation. In3DV, pages 565–571. Ieee, 2016

    2016

  58. [65]

    Training region-based object detectors with online hard example mining

    Abhinav Shrivastava, Abhinav Gupta, and Ross Girshick. Training region-based object detectors with online hard example mining. InCVPR, pages 761–769, 2016

    2016

  59. [67]

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gre- gory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 20 D. Zhang et al. Pytorch: An imperative style, high-performance deep learning library.NeurIPS, 32, 2019

    2019

  60. [68]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InCVPR, pages 770–778, 2016

    2016

  61. [69]

    A survey on curriculum learning.IEEE TPAMI, 44(9):4555–4576, 2021

    Xin Wang, Yudong Chen, and Wenwu Zhu. A survey on curriculum learning.IEEE TPAMI, 44(9):4555–4576, 2021

    2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.