Efficient Medical VIE via Reinforcement Learning

ChengLin Zhu; Chong Li; Jian Xie; Jiehan Cheng; Lijun Liu; Qiang Ju; Ruiyang Li; Zhaocheng Liu

REVIEW 3 major objections 5 minor 61 references

Efficient Medical VIE via Reinforcement Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fine-tuning a 7B vision-language model with verifiable-reward reinforcement learning on just 100 annotated medical report images reaches 77.81 field-level F1, the best among compared systems on the medical VIE test set.

desk verdict A plausible RLVR-for-medical-VIE result with real gains, but the undisclosed reward weight and private test set make the SOTA claim hard to verify. read the letter →

arxiv 2506.13363 v1 pith:C43SIUQI submitted 2025-06-16 cs.CL

Lijun Liu , Ruiyang Li , Zhaocheng Liu , Chenglin Zhu , Chong Li , Jiehan Cheng , Qiang Ju , Jian Xie This is my paper · ORCID

classification cs.CL

keywords visualinformationextractionmedicaldocumentunderstandingreinforcementlearningwithverifiablerewardsGRPOmultimodallargelanguagemodelsJSONprecision-recallrewardfew-shot

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that medical visual information extraction can be made data-efficient: 100 carefully chosen annotated report images, used with outcome-verifiable reinforcement learning, are enough to fine-tune a general 7B vision-language model into a top performer on medical JSON extraction. It argues that the key is a reward that weighs precision and recall separately, so the model is punished both for hallucinating fields and for missing real fields, together with query sampling that exposes the model to varied subsets of the schema. On a 203-image medical test set the resulting model reaches 77.81 F1, 79.85 precision, and 75.88 recall, ahead of pipeline OCR systems, larger multimodal models, and a supervised fine-tuned version. The same model also improves on receipt-like and form-like general benchmarks, but loses to the base model on dissimilar ticket and receipt tasks, which the paper reads as evidence that VIE rewards are strongly domain-specific.

What carries the argument

The mechanism is a rule-based reward from Eq. (5): $r_{\mathrm{matching}} = \alpha\, n_{\mathrm{matched}}/|S_p| + (1-\alpha)\, n_{\mathrm{matched}}/|S_g|$, where $S_p$ and $S_g$ are the flattened leaf key-value sets of predicted and ground-truth JSON, and $n_{\mathrm{matched}}$ counts exactly matched pairs. The weight $\alpha$ trades hallucination suppression against field coverage, and the paper combines this with a group-relative advantage estimator and the Clip-Higher and token-level policy-gradient loss modifications. Query sampling, which asks the model to extract random subsets of the schema keys rather than always the full schema, is the second load-bearing component because it shortens responses and speeds reward growth.

What would settle it

Train RLVR on the same 100 images with $\alpha$ set to 0.2, 0.5, and 0.8 while holding everything else fixed, and evaluate each model on the 203-image medical test set; if the F1 spread across $\alpha$ exceeds about two points, the headline 77.81 F1 is not robust to the undisclosed hyperparameter, and if the spread is negligible, the method's data-efficiency claim stands independently of $\alpha$.

Watch

Extended reading notes

Core claim

The central claim is that RLVR with a rule-based matching reward, applied to only 100 diverse medical report images, outperforms supervised fine-tuning and much larger or commercial baselines on medical VIE. The reward flattens predicted and ground-truth JSON into leaf key-value pairs and scores them with a weighted combination of precision and recall; when the weight is set to emphasize both, the model is discouraged from emitting invented fields while still being pushed to cover the schema. The paper reports 77.81 F1, 79.85 precision, and 75.88 recall on its 203-image private medical test set, and shows the thinking-augmented model beats its no-thinking counterpart by 1.91 F1 points. On general benchmarks, the gains appear on CORD and FUNSD, which resemble the medical schema, but not on SROIE and Ticket.

Load-bearing premise

The reward-balancing weight $\alpha$ is never reported for the final models, and the paper only illustrates the $\alpha=0$ and $\alpha=1$ extremes; if $\alpha$ was chosen by looking at test-set performance, the reported gains over baselines could be partly an artifact of that selection.

Editorial extensions

If this is right

Domain-specific VIE may no longer require thousands of labeled documents: 100 diverse images with verifiable JSON targets can produce a specialist extractor, provided the reward penalizes both missing and invented fields.
The precision-recall reward gives practitioners a direct dial: raising $\alpha$ pushes toward hallucination-averse output, lowering it pushes toward full field coverage, and the paper shows the two extremes behave very differently.
Reasoning-chain supervision is not needed; the model learns to think during training because RLVR rewards the final JSON, and the think version outperforms the no-think version on F1.
Transfer of medical VIE models to other document types is limited: improvements are large on structurally similar benchmarks like CORD and FUNSD, and negative on dissimilar ones like SROIE and Ticket, implying VIE remains domain-bound.
RLVR handles unordered JSON targets better than SFT, because the reward is invariant to key ordering while cross-entropy on fixed JSON order can introduce ambiguity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper does not report the $\alpha$ value used for the final models; readers cannot tell from the text whether 77.81 F1 is robust to $\alpha$ or partly a product of tuning on the private test set.
A direct extension would apply the same 100-image RLVR recipe to other verticals such as legal, insurance, or scientific PDFs, to see whether the gain comes from RLVR generally or from the specific medical schema and reward design.
Because the reward matches flattened leaf pairs, it ignores nested structure; a prediction that places correct leaf values under the wrong parent keys could score well on F1 while failing TED accuracy, so production use should probably keep both metrics.
The 203-image private test set is small; reporting per-report-type F1 or confidence intervals would show whether the headline gain is spread across CT, ultrasound, lab, and pathology reports or driven by a few types.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The useful core is applying a standard RLVR recipe to medical VIE with only 100 annotated images. That is a new application, not a new method, but it is a genuinely useful data point: the balanced precision/recall reward is a sensible way to handle the missing-field and hallucination trade-off, and the field-sampling strategy is a reasonable way to create query diversity. I also credit the authors for reporting that performance drops on dissimilar tasks, an honest negative result that many papers omit. The case studies are illustrative, not rigorous, but they do support the claim that the think step helps. The soft spots are in the evidence. The stress-test note is on target: alpha in Eq. (5) is the one knob that directly trades off the exact metrics reported in Table 1, and the paper never says what value was used for RL-100 or the other RL models. Figure 2 only shows alpha = 0 and alpha = 1, the degenerate extremes. If alpha was chosen by looking at the private 203-image test set, the reported F1, precision, and recall are partly selected statistics, and the comparison to baselines that did not get that tuning is not apples-to-apples. The absence of error bars or multiple seeds makes this worse: one favorable run cannot be distinguished from the expected best of several. The private test set and the undocumented TextIn baseline only add to the verification gap. These are not fatal flaws in the method; they are missing information that a thoughtful referee would demand. Overall, the central idea is sound and the direction is consistent with the broader RLVR literature, but the quantitative claims are not yet established. The paper deserves a serious referee, not a desk reject. The authors should be asked to disclose alpha, run multiple seeds with variance, evaluate on at least one public VIE benchmark or release the test set, and document all baselines. If they can do that, this becomes a useful contribution for anyone working on low-resource document extraction. I would not cite it in its current form, but I would put it on the reading list for the group working on multimodal RL.

Referee Report

3 major / 5 minor

Summary. The paper proposes an RLVR-based method for medical Visual Information Extraction (VIE) using only 100 annotated images. The method fine-tunes Qwen2.5-VL-7B with GRPO and a rule-based reward that combines a format score and a weighted precision-recall matching score, along with a key-sampling strategy that varies queries during training. On a private medical VIE test set of 203 images, the RL-100 model is reported to achieve 77.81 F1, 79.85 precision, and 75.88 recall, exceeding pipeline, expert, commercial, and general MLLM baselines. The paper also evaluates the trained models on four general VIE benchmarks (CORD, FUNSD, SROIE, Ticket) and reports an ablation on the effect of the model's thinking process.

Significance. If the results hold, the paper's contribution is practical and relevant: a purely outcome-driven RL approach that improves domain-specific VIE with very few annotations, avoiding RLHF, and with a reward that directly controls hallucination versus coverage. The authors provide transparent reasoning for the reward design, compare RLVR with SFT, and show generalization behavior on dissimilar tasks. However, the central SOTA claim rests on a private test set with no error bars, an undisclosed value of the critical reward weight alpha, and an incomplete baseline description. The paper does not release code, data, or model weights, which limits reproducibility. These issues are fixable and do not invalidate the method's promise, but they currently prevent the reader from verifying that the reported gains are not artifacts of test-set hyperparameter selection.

major comments (3)

[§3.3 and §4.4, Eq. (5)] The value of α in Eq. (5) is never disclosed for RL-100, RL-100(w/o sample), or OCR-SFT-17K-RL-100. Because Eq. (5) is literally a weighted combination of the field-level precision and recall that Table 1 reports as the headline metrics, choosing α by inspecting the private test set would be equivalent to selecting the evaluation statistic. Figure 2 only shows α=0 and α=1 extremes and does not indicate the value used in the final models. Please report the exact α used for each model, the protocol for selecting it (e.g., a held-out validation split), and ideally a sensitivity analysis of F1/precision/recall across a range of α values.
[§4.2 and Table 1] Table 1 lists a commercial baseline 'TextIn' with an OCR output, but §4.2, which introduces all baselines (MinerU, Marker, GOT-OCR, GPT4o, Qwen2.5-VL, InternVL), does not describe TextIn or its configuration. Moreover, the entire medical VIE evaluation is on a private 203-image test set with no public medical VIE benchmark; the 'state-of-the-art' claim is therefore only relative to this private test set and the authors' own selection of fields and ground truth. Please add a proper description of TextIn and its settings, and, if no public medical VIE benchmark is used, state this limitation explicitly and justify why the private set is sufficient for the SOTA claim.
[§4.4, Tables 1 and 2] All results in Tables 1 and 2 are single runs with no error bars, confidence intervals, or multiple seeds. Because RL training and rollout sampling are stochastic, the reader cannot distinguish a robust improvement from a single favorable run, especially given the undisclosed α and the private test set. Please report results over at least a few seeds (or provide the seed policy and the number of runs), and, if multiple seeds are too expensive, state this cost explicitly and report the variance over the evaluation set in some other way (e.g., bootstrap confidence intervals on the 203 test images).

minor comments (5)

[§4.3] In the description of JSON-SFT-100, 'model leans to extract' should be 'model learns to extract', and 'the learning rate starts from 1e-6 and decays to 0 following the liner schedule' should be 'linear schedule'.
[§4.6] There are typos in the paragraph introducing general VIE benchmarks: 'widerly used' should be 'widely used', and 'comparision' should be 'comparison'.
[§4.6 and Table 3] The evaluation protocol for CORD, FUNSD, SROIE, and Ticket is not described. Please specify how the field-level F1 and TED accuracy are computed for these public benchmarks, in particular how the ground-truth key-value trees are derived from the native annotations of each dataset, so that the results are reproducible.
[§2.1 and References] The citation '(Wang et al.; Li et al., 2024)' in the first paragraph of §2.1 lacks a year for the Wang et al. work; the corresponding reference list entry also appears incomplete. Please correct the citation.
[Appendix C] The two case-study tables present model outputs entirely in Chinese without English translations. If the journal expects English, please provide translations or at least an English summary of what each case demonstrates, since the tables are intended to support the claim that thinking improves extraction.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the RLVR reward is metric-aligned by design, but evaluation is on a held-out private test set, and no load-bearing self-citation or definitional reduction is present.

full rationale

The paper's derivation chain is empirical rather than definitional. The reward in Eq. (5) is a weighted combination of field-level precision and recall computed against ground truth on 100 training samples, while the reported F1, precision, and recall in Table 1 are measured on a separately described private 203-image test set (Section 4.4). Training a model with a reward that belongs to the same metric family as the evaluation metric is standard reinforcement learning, not circular reasoning, because the test-set results reflect generalization to unseen images and are not forced by construction from the training reward. The undisclosed balancing weight alpha is a legitimate reproducibility and hyperparameter-selection concern, but the paper never claims that alpha is predicted or derived from the test result, and there is no quoted reduction showing that the reported test metrics are equivalent to the reward by definition. The only self-citation, Liu et al. (2025b), is used to motivate the medical application context of online consultations and is not load-bearing for the RLVR method or the SOTA claim. No uniqueness theorem, ansatz smuggled via citation, or renamed known result is invoked. Accordingly, the paper is self-contained against external benchmarks for its central empirical claim, and no significant circularity is present.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claims rest on an unreported reward-weighting parameter, the adequacy of the flattened precision/recall metric, and two private datasets that are not released. No brand-new physical or mathematical entities are introduced.

free parameters (1)

alpha = Not reported
Eq. (5) defines the matching score as alpha times precision plus (1-alpha) times recall; the paper never states the value used for RL-100 or other final models. Figure 2 shows only alpha=0 and alpha=1, and no sensitivity analysis is provided.

assumptions (4)

standard math GRPO and DAPO update rules (Eqs. 1-4) are correctly implemented.
The paper adopts these formulas from DeepSeek-R1 and DAPO and does not re-derive them; correctness depends on the cited implementations.
domain assumption Flattening JSON to leaf key-value pairs and measuring precision/recall is an adequate proxy for VIE quality.
Section 3.3 flattens the nested JSON into a dictionary; this discards structural nesting and treats each leaf independently, which may ignore semantic dependencies between fields.
domain assumption The 100 selected training images and the GPT-4o-assisted ground truth are representative and accurate for the medical VIE task.
Section 3.2 describes manual selection and GPT-4o annotation; the test set in Section 4.4 also relies on GPT-4o with manual correction, so shared annotation biases could inflate results.
domain assumption The private 203-image test set is disjoint from the 100 training images.
The paper does not explicitly state that the test images are disjoint from the training pool; if they overlap, gains could reflect memorization.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Efficient Medical VIE via Reinforcement Learning." pith.science (2026). https://pith.science/paper/C43SIUQI

@misc{pith2026250613363,
  author       = {Pith},
  title        = {Pith review of: Efficient Medical VIE via Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/C43SIUQI}},
  note         = {Machine review of arXiv:2506.13363}
}

read the original abstract

Visual Information Extraction (VIE) converts unstructured document images into structured formats like JSON, critical for medical applications such as report analysis and online consultations. Traditional methods rely on OCR and language models, while end-to-end multimodal models offer direct JSON generation. However, domain-specific schemas and high annotation costs limit their effectiveness in medical VIE. We base our approach on the Reinforcement Learning with Verifiable Rewards (RLVR) framework to address these challenges using only 100 annotated samples. Our approach ensures dataset diversity, a balanced precision-recall reward mechanism to reduce hallucinations and improve field coverage, and innovative sampling strategies to enhance reasoning capabilities. Fine-tuning Qwen2.5-VL-7B with our RLVR method, we achieve state-of-the-art performance on medical VIE tasks, significantly improving F1, precision, and recall. While our models excel on tasks similar to medical datasets, performance drops on dissimilar tasks, highlighting the need for domain-specific optimization. Case studies further demonstrate the value of reasoning during training and inference for VIE.

Figures

Figures reproduced from arXiv: 2506.13363 by the authors.

**Figure 1.** Overview of our proposed method. A sampling strategy is employed to randomly select key-value pairs from the annotated JSON data associated with each image, serving as the ground truth. The descriptions of these keys are incorporated into a predefined prompt to generate queries. These queries, along with the corresponding images, are input into the model to produce responses. The JSON predictions are then extracted … view at source ↗

**Figure 2.** Impact of the hyperparameter α on response length when the Sampling Strategy is enabled. The semi-transparent and the solid lines indicate raw samples and the smoothed trend. 3.4 Sampling Strategy To assess the impact of query diversity on experimental outcomes, we employ two data construction strategies. The first strategy involves random sampling of keys corresponding to the JSON data of an image, thereby generati… view at source ↗

**Figure 3.** Comparison of Reward and Response Length Trends Between Sampling and Non-Sampling Strategies During Training. The semi-transparent and the solid lines indicate raw samples and the smoothed trend. 4.2 VIE Baselines To compare our results with existing works, we introduce models with various types and different outputs. Pipeline models. The pipeline models for OCR task are always composed of layout recognizer and OCR … view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 24 canonical work pages

[1]

Wiam Adnan, Joel Tang, Yassine Bel Khayat Zouggari, Seif Edinne Laatiri, Laurent Lam, and Fabien Caspani. 2024. A layoutlmv3-based model for enhanced relation extraction in visually-rich documents. In International Conference on Document Analysis and Recognition, pages 160--174. Springer

work page 2024
[2]

Srikar Appalaraju, Bhavan Jasani, Bhargava Urala Kota, Yusheng Xie, and R Manmatha. 2021. Docformer: End-to-end transformer for document understanding. In Proceedings of the IEEE/CVF international conference on computer vision, pages 993--1003

work page 2021
[3]

PaddlePaddle Authors. 2020. Paddleocr, awesome multilingual ocr toolkits based on paddlepaddle. https://github.com/PaddlePaddle/PaddleOCR

2020
[4]

Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923

arXiv 2025
[5]

Haoyu Cao, Xin Li, Jiefeng Ma, Deqiang Jiang, Antai Guo, Yiqing Hu, Hao Liu, Yinsong Liu, and Bo Ren. 2022. Query-driven generative network for document information extraction in the wild. In Proceedings of the 30th ACM International Conference on Multimedia, pages 4261--4271

work page 2022
[6]

Jingye Chen, Tengchao Lv, Lei Cui, Cha Zhang, and Furu Wei. 2022. Xdoc: Unified pre-training for cross-format document understanding. arXiv preprint arXiv:2210.02849

arXiv 2022
[7]

Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, Lixin Gu, Xuehui Wang, Qingyun Li, Yimin Ren, Zixuan Chen, Jiapeng Luo, Jiahao Wang, Tan Jiang, Bo Wang, and 23 others. 2025. https://arxiv.org/abs/2412.05271 Expanding performance boundaries of open-source multimodal models with model,...

arXiv 2025
[8]

Chengqi Duan, Rongyao Fang, Yuqing Wang, Kun Wang, Linjiang Huang, Xingyu Zeng, Hongsheng Li, and Xihui Liu. 2025. Got-r1: Unleashing reasoning capability of mllm for visual generation with reinforcement learning. arXiv preprint arXiv:2505.17022

arXiv 2025

Show all 61 references

[9]

Hao Feng, Shu Wei, Xiang Fei, Wei Shi, Yingdong Han, Lei Liao, Jinghui Lu, Binghong Wu, Qi Liu, Chunhui Lin, and 1 others. 2025. Dolphin: Document image parsing via heterogeneous anchor prompting. arXiv preprint arXiv:2505.14059

2025 arXiv
[10]

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

2025 arXiv
[11]

He Guo, Xiameng Qin, Jiaming Liu, Junyu Han, Jingtuo Liu, and Errui Ding. 2019. Eaten: Entity-aware attention for single shot visual text extraction. In 2019 International Conference on Document Analysis and Recognition (ICDAR), pages 254--259. IEEE

2019
[12]

Teakgyu Hong, Donghyun Kim, Mingi Ji, Wonseok Hwang, Daehyun Nam, and Sungrae Park. 2022. Bros: A pre-trained language model focusing on text and layout for better key information extraction from documents. In Proceedings of the AAAI Conference on Artificial Intelligence, volu...

2022
[13]

Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. 2025. Vision-r1: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749

2025 arXiv
[14]

Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, and Furu Wei. 2022. Layoutlmv3: Pre-training for document ai with unified text and image masking. In Proceedings of the 30th ACM international conference on multimedia, pages 4083--4091

2022
[15]

Zheng Huang, Kai Chen, Jianhua He, Xiang Bai, Dimosthenis Karatzas, Shijian Lu, and C.V. Jawahar. 2019 a . https://doi.org/10.1109/ICDAR.2019.00244 ICDAR 2019 competition on scanned receipt OCR and information extraction ( SROIE ) . In 2019 International Conference on Document...

2019
[16]

Zheng Huang, Kai Chen, Jianhua He, Xiang Bai, Dimosthenis Karatzas, Shijian Lu, and CV Jawahar. 2019 b . Icdar2019 competition on scanned receipt ocr and information extraction. In 2019 International Conference on Document Analysis and Recognition (ICDAR), pages 1516--1520. IEEE

2019
[17]

Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, and 1 others. 2024. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186

2024 arXiv
[18]

Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others. 2024. Openai o1 system card. arXiv preprint arXiv:2412.16720

2024 arXiv
[19]

Guillaume Jaume, Haz m Kemal Ekenel, and Jean - Philippe Thiran. 2019. https://doi.org/10.48550/ARXIV.1905.13538 FUNSD : A dataset for form understanding in noisy scanned documents . arXiv preprint arXiv:1905.13538

2019 doi arXiv
[20]

Geewook Kim, Teakgyu Hong, Moonbin Yim, JeongYeon Nam, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, and Seunghyun Park. 2022. Ocr-free document understanding transformer. In European Conference on Computer Vision, pages 498--517. Springer

2022
[21]

Geewook Kim, Teakgyu Hong, Moonbin Yim, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, and Seunghyun Park. 2021. Donut: Document understanding transformer without ocr. arXiv preprint arXiv:2111.15664, 7(15):2

2021 arXiv
[22]

Jianfeng Kuang, Wei Hua, Dingkang Liang, Mingkun Yang, Deqiang Jiang, Bo Ren, and Xiang Bai. 2023. Visual information extraction in the wild: practical dataset and end-to-end solution. In International Conference on Document Analysis and Recognition, pages 36--53. Springer

2023
[23]

Yiming Li, Qiang Wei, Xinghan Chen, Jianfu Li, Cui Tao, and Hua Xu. 2024. Improving tabular data extraction in scanned laboratory reports using deep learning models. Journal of Biomedical Informatics, 159:104735

2024
[24]

Yuqi Liu, Bohao Peng, Zhisheng Zhong, Zihao Yue, Fanbin Lu, Bei Yu, and Jiaya Jia. 2025 a . Seg-zero: Reasoning-chain guided segmentation via cognitive reinforcement. arXiv preprint arXiv:2503.06520

2025 arXiv
[25]

Zhaocheng Liu, Quan Tu, Wen Ye, Yu Xiao, Zhishou Zhang, Hengfu Cui, Yalun Zhu, Qiang Ju, Shizheng Li, and Jian Xie. 2025 b . Exploring the inquiry-diagnosis relationship with advanced patient simulators. arXiv preprint arXiv:2501.09484

2025 arXiv
[26]

Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. 2025 c . Visual-rft: Visual reinforcement fine-tuning. arXiv preprint arXiv:2503.01785

2025 arXiv
[27]

Chuwei Luo, Changxu Cheng, Qi Zheng, and Cong Yao. 2023. Geolayoutlm: Geometric pre-training for visual information extraction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7092--7101

2023
[28]

Ming-Wei Ma, Xian-Shu Gao, Ze-Yu Zhang, Shi-Yu Shang, Ling Jin, Pei-Lin Liu, Feng Lv, Wei Ni, Yu-Chen Han, and Hui Zong. 2023. Extracting laboratory test information from paper-based reports. BMC Medical Informatics and Decision Making, 23(1):251

2023
[29]

OpenAI. 2024 a . GPT-4o system card. https://openai.com/index/gpt-4o-system-card/

2024
[30]

OpenAI. 2024 b . Openai o1 system card. https://openai.com/index/openai-o1-system-card/

2024
[31]

Linke Ouyang, Yuan Qu, Hongbin Zhou, Jiawei Zhu, Rui Zhang, Qunshu Lin, Bin Wang, Zhiyuan Zhao, Man Jiang, Xiaomeng Zhao, and 1 others. 2025. Omnidocbench: Benchmarking diverse pdf document parsing with comprehensive annotations. In Proceedings of the Computer Vision and Patte...

2025
[32]

Hantian Pang. 2024. https://github.com/ppaanngggg/layoutreader Faster LayoutReader based on LayoutLMv3

2024
[33]

Seunghyun Park, Seung Shin, Bado Lee, Junyeop Lee, Jaeheung Surh, Minjoon Seo, and Hwalsuk Lee. 2019. Cord: A consolidated receipt dataset for post-ocr parsing. In Document Intelligence Workshop at NeurIPS 2019

2019
[34]

Rafa Powalski, ukasz Borchmann, Dawid Jurkiewicz, Tomasz Dwojak, Micha Pietruszka, and Gabriela Pa ka. 2021. Going full-tilt boogie on document understanding with text-image-layout transformer. In Document Analysis and Recognition--ICDAR 2021: 16th International Conference, La...

2021
[35]

Jake Poznanski, Jon Borchardt, Jason Dunkelberger, Regan Huff, Daniel Lin, Aman Rangapur, Christopher Wilhelm, Kyle Lo, and Luca Soldaini. 2025. olmocr: Unlocking trillions of tokens in pdfs with vision language models. arXiv preprint arXiv:2502.18443

2025
[36]

Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300

2024 arXiv
[37]

Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, and 1 others. 2025. Vlm-r1: A stable and generalizable r1-style large vision-language model. arXiv preprint arXiv:2504.07615

2025 arXiv
[38]

Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, and 1 others. 2025. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599

2025 arXiv
[39]

Jianqiang Wan, Sibo Song, Wenwen Yu, Yuliang Liu, Wenqing Cheng, Fei Huang, Xiang Bai, Cong Yao, and Zhibo Yang. 2024. Omniparser: A unified framework for text spotting key information extraction and table recognition. In Proceedings of the IEEE/CVF Conference on Computer Visi...

2024
[40]

Bin Wang, Zhuangcheng Gu, Guang Liang, Chao Xu, Bo Zhang, Botian Shi, and Conghui He. 2024 a . https://arxiv.org/abs/2404.15254 Unimernet: A universal network for real-world mathematical expression recognition . Preprint, arXiv:2404.15254

2024 arXiv
[41]

Bin Wang, Chao Xu, Xiaomeng Zhao, Linke Ouyang, Fan Wu, Zhiyuan Zhao, Rui Xu, Kaiwen Liu, Yuan Qu, Fukai Shang, and 1 others. 2024 b . Mineru: An open-source solution for precise document content extraction. arXiv preprint arXiv:2409.18839

2024 arXiv
[42]

Chatschema: development and validation of a pipeline for extracting structured nursing information with large multimodal models

Fei Wang, Yuewen Zheng, Jingyi Wu, Qing Li, Pengfei Li, and Luxia Zhang. Chatschema: development and validation of a pipeline for extracting structured nursing information with large multimodal models. Interdisciplinary Nursing Research, pages 10--1097
[43]

Jiapeng Wang, Chongyu Liu, Lianwen Jin, Guozhi Tang, Jiaxin Zhang, Shuaitao Zhang, Qianying Wang, Yaqiang Wu, and Mingxiang Cai. 2021 a . Towards robust visual information extraction in real world: new dataset and novel solution. In Proceedings of the AAAI Conference on Artifi...

2021
[44]

Pengfei Wang, Chengquan Zhang, Fei Qi, Shanshan Liu, Xiaoqiang Zhang, Pengyuan Lyu, Junyu Han, Jingtuo Liu, Errui Ding, and Guangming Shi. 2021 b . Pgnet: Real-time arbitrarily-shaped text spotting with point gathering network. In Proceedings of the AAAI Conference on Artifici...

2021
[45]

Haoran Wei, Chenglong Liu, Jinyue Chen, Jia Wang, Lingyu Kong, Yanming Xu, Zheng Ge, Liang Zhao, Jianjian Sun, Yuang Peng, and 1 others. 2024. General ocr theory: Towards ocr-2.0 via a unified end-to-end model

2024
[46]

Renqiu Xia, Song Mao, Xiangchao Yan, Hongbin Zhou, Bo Zhang, Haoyang Peng, Jiahao Pi, Daocheng Fu, Wenjie Wu, Hancheng Ye, Shiyang Feng, Bin Wang, Chao Xu, Conghui He, Pinlong Cai, Min Dou, Botian Shi, Sheng Zhou, Yongwei Wang, and 4 others. 2024. https://arxiv.org/abs/2406.11...

2024
[47]

Guowei Xu, Peng Jin, Li Hao, Yibing Song, Lichao Sun, and Li Yuan. 2024. Llava-o1: Let vision language models reason step-by-step. arXiv preprint arXiv:2411.10440

2024 arXiv
[48]

Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, and 1 others. 2020 a . Layoutlmv2: Multi-modal pre-training for visually-rich document understanding. arXiv preprint arXiv:2012.14740

2020 arXiv
[49]

Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, and Ming Zhou. 2020 b . Layoutlm: Pre-training of text and layout for document image understanding. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1192--1200

2020
[50]

Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, and 1 others. 2025. R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization. arXiv preprint arXiv:2503.10615

2025 arXiv
[51]

Minghong Yao, Liansheng Zhuang, Houqiang Li, and Jiuchang Wei. 2024. Learning label dependencies for visual information extraction. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, pages 6615--6623

2024
[52]

Huaiyuan Ying, Shuo Zhang, Linyang Li, Zhejian Zhou, Yunfan Shao, Zhaoye Fei, Yichuan Ma, Jiawei Hong, Kuikun Liu, Ziyi Wang, and 1 others. 2024. Internlm-math: Open math large language models toward verifiable reasoning. arXiv preprint arXiv:2402.06332

2024 arXiv
[53]

En Yu, Kangheng Lin, Liang Zhao, Jisheng Yin, Yana Wei, Yuang Peng, Haoran Wei, Jianjian Sun, Chunrui Han, Zheng Ge, and 1 others. 2025 a . Perception-r1: Pioneering perception policy with reinforcement learning. arXiv preprint arXiv:2504.07954

2025 arXiv
[54]

Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, and 1 others. 2025 b . Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476

2025 arXiv
[55]

Wenwen Yu, Ning Lu, Xianbiao Qi, Ping Gong, and Rong Xiao. 2021. Pick: processing key information extraction from documents using improved graph learning-convolutional networks. In 2020 25th International conference on pattern recognition (ICPR), pages 4363--4370. IEEE

2021
[56]

Peng Zhang, Yunlu Xu, Zhanzhan Cheng, Shiliang Pu, Jing Lu, Liang Qiao, Yi Niu, and Fei Wu. 2020. Trie: end-to-end text reading and information extraction for document understanding. In Proceedings of the 28th ACM International Conference on Multimedia, pages 1413--1422

2020
[57]

Zhiyuan Zhao, Hengrui Kang, Bin Wang, and Conghui He. 2024. https://arxiv.org/abs/2410.12628 Doclayout-yolo: Enhancing document layout analysis through diverse synthetic data and global-to-local adaptive perception . Preprint, arXiv:2410.12628

2024 arXiv
[58]

Lianchi Zheng, Xiaoming Liu, Zhihui Sun, and Yuxiang He. 2022. Improving medical ocr information extraction with integrated bert and layoutxlm models. In China Health Information Processing Conference, pages 164--173. Springer

2022
[59]

aha moment

Hengguang Zhou, Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsieh. 2025. R1-zero’s" aha moment" in visual reasoning on a 2b non-sft model, 2025. URL https://arxiv. org/abs/2503.05132

2025 arXiv
[60]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...
[61]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Wiam Adnan, Joel Tang, Yassine Bel Khayat Zouggari, Seif Edinne Laatiri, Laurent Lam, and Fabien Caspani. 2024. A layoutlmv3-based model for enhanced relation extraction in visually-rich documents. In International Conference on Document Analysis and Recognition, pages 160--174. Springer

work page 2024

[2] [2]

Srikar Appalaraju, Bhavan Jasani, Bhargava Urala Kota, Yusheng Xie, and R Manmatha. 2021. Docformer: End-to-end transformer for document understanding. In Proceedings of the IEEE/CVF international conference on computer vision, pages 993--1003

work page 2021

[3] [3]

PaddlePaddle Authors. 2020. Paddleocr, awesome multilingual ocr toolkits based on paddlepaddle. https://github.com/PaddlePaddle/PaddleOCR

2020

[4] [4]

Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923

arXiv 2025

[5] [5]

Haoyu Cao, Xin Li, Jiefeng Ma, Deqiang Jiang, Antai Guo, Yiqing Hu, Hao Liu, Yinsong Liu, and Bo Ren. 2022. Query-driven generative network for document information extraction in the wild. In Proceedings of the 30th ACM International Conference on Multimedia, pages 4261--4271

work page 2022

[6] [6]

Jingye Chen, Tengchao Lv, Lei Cui, Cha Zhang, and Furu Wei. 2022. Xdoc: Unified pre-training for cross-format document understanding. arXiv preprint arXiv:2210.02849

arXiv 2022

[7] [7]

Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, Lixin Gu, Xuehui Wang, Qingyun Li, Yimin Ren, Zixuan Chen, Jiapeng Luo, Jiahao Wang, Tan Jiang, Bo Wang, and 23 others. 2025. https://arxiv.org/abs/2412.05271 Expanding performance boundaries of open-source multimodal models with model,...

arXiv 2025

[8] [8]

Chengqi Duan, Rongyao Fang, Yuqing Wang, Kun Wang, Linjiang Huang, Xingyu Zeng, Hongsheng Li, and Xihui Liu. 2025. Got-r1: Unleashing reasoning capability of mllm for visual generation with reinforcement learning. arXiv preprint arXiv:2505.17022

arXiv 2025