Pith. sign in

REVIEW 3 major objections 5 minor 59 references

A single, reinforcement-selected visual patch per text instance, refined against the full image, is the most accurate grounding interface for MLLM-based scene text spotting.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-04 03:15 UTC pith:N226Z26H

load-bearing objection Genuine single-patch grounding idea with a clean RL formulation, but the headline claims overreach and the Multi-Patch baseline is likely undercooked. the 3 major comments →

arxiv 2607.27902 v2 pith:N226Z26H submitted 2026-07-30 cs.CV

One Patch Is Enough: Reinforcement-Optimized Visual Token Grounding for MLLM-Based Scene Text Spotting

Rui Tang , Wentao Yang , Peirong Zhang , Yongxin Shi , Shun Zhang , Huiguo He , Lianwen Jin This is my paper
classification cs.CV
keywords Scene Text SpottingMultimodal Large Language ModelsVisual Token GroundingSingle-Patch SelectionReinforcement LearningVisual Patch DecodingDirectional Embedding Alignment
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper claims that the noise and ambiguity in MLLM-based scene text spotting come largely from grounding on multiple image patches at once, and that routing each text instance through a single anchor visual token—then recovering geometry by attending to the whole image—is both more accurate and more reliable. To find that anchor without extra labels, the authors train the model with reinforcement learning rewards that measure final spotting quality, and they stabilize the selected patch features by separating feature direction from magnitude. If correct, this means a lightweight 2B or 4B model can outperform far larger general-purpose MLLMs and specialized OCR systems on text spotting benchmarks, shifting the bottleneck from model scale to the grounding format.

Core claim

SPaTS is built on the claim that the correct granularity for visual grounding in scene text spotting is one anchor image patch per text instance, not a set of patches or a set of coordinates. The paper's central discovery is that this single-patch interface is learnable without oracle labels: SPaSO formulates patch selection as a discrete reinforcement-learning problem, rewarding both the final spotting outcome (a harmonic mean of F1 and average IoU) and the presence of a ground-truth-overlapping patch among the top-k predictions. Two architectural pieces make the single patch usable: DEA parameterizes each visual prototype as a normalized direction times a learned scale, so token selection

What carries the argument

The central machinery is the learned single-patch selector under the SPaSO objective: the model autoregressively emits an index into the image's patch-token vocabulary for each text instance, and the policy is optimized with two patch-level rewards — a generation reward that measures end-to-end spotting quality and a dominance reward that keeps ground-truth patches in the top-k. DEA ensures that the visual-token logits reflect directional alignment rather than raw magnitude, and PED turns the selected token into an object query that reads from the full image to decode boxes, Bézier curves, points, and scores. The single anchor is therefore the point where recognition and localization are cou

Load-bearing premise

The load-bearing premise is that overlap with the ground-truth polygon mask is a faithful proxy for how informative an image patch is for recognition and localization; if the reward reinforces patches that only skim the polygon but are visually uninformative, the selection policy can settle on suboptimal anchors.

What would settle it

Train the Multi-Patch formulation with the same SPaSO rewards and the same full-image decoder used in Single-Patch, then compare end-to-end F-measure on Total-Text, CTW1500, and ICDAR 2015; if the multi-patch policy matches or beats the single-patch policy, the central claim collapses.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Multi-patch grounding, a common approach in patch-based MLLM spotting, is actively harmful for scene text: a single routed patch yields higher precision, recall, and F-measure on the tested benchmarks.
  • Discrete patch selection can be trained end-to-end with task-level rewards, so no oracle best-patch annotation is needed; the model discovers which patch is informative from spotting quality alone.
  • Decoupling feature magnitude from direction in the unified head is a transferable mechanism: it can prevent high-norm patches from dominating any visual-vocabulary selection task, not just text spotting.
  • The reported results imply that a 2B or 4B MLLM with a well-designed grounding interface can beat much larger general MLLMs and specialized OCR models on end-to-end spotting, making the grounding format a primary design lever.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If single-anchor grounding is as general as the paper suggests, the same routing-plus-full-image-refinement recipe should transfer to other dense multimodal tasks, such as document key-information extraction or referring-expression grounding, where multi-patch aggregation is currently the default.
  • A label-free extension would be to learn the patch-selection reward from self-consistency instead of ground-truth polygon overlap: sample several candidate patches, decode geometry from each, and reward patches whose transcriptions and boxes agree; this would remove the need for a fixed overlap threshold.
  • A natural stress test is to evaluate SPaTS on images with severe occlusion or specular highlights, where the most informative patch may not overlap the ground-truth polygon; if the magnitude branch is frozen after warmup, feature norms under extreme illumination could become a failure mode that current benchmarks do not expose.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper proposes SPaTS, an MLLM-based scene text spotter that routes each text instance through a single anchor visual token and recovers geometry by full-image cross-attention refinement. To learn the anchor without oracle labels, the authors introduce SPaSO, a GRPO-based reinforcement-learning stage with two patch-level rewards (generation quality and top-k patch dominance), together with DEA, which decouples feature magnitude from direction, and PED, which fuses the routed patch with language hidden states. The method is evaluated on Total-Text, CTW1500, and ICDAR 2015 against closed-source MLLMs, open-source MLLMs, OCR-specialized MLLMs, and expert spotters, with ablations of the grounding formulation, reward components, and architectural modules. The claimed central finding is that a single patch, selected by RL and refined by full-image decoding, is a better grounding interface than coordinate text or multi-patch routing.

Significance. If the central comparison were fully supported, the single-anchor design would be a useful and parameter-efficient contribution to visual-token grounding for scene text spotting. The paper provides a concrete RL formulation for discrete visual-token selection, a reasonable architectural rationale for DEA and PED, extensive ablations, efficiency measurements, and a public code release. However, the advertised 'state-of-the-art' claim is contradicted by the paper's own tables, and the controlled evidence for 'one patch is enough' rests on a Multi-Patch baseline whose behavior is not yet explained. The conceptual idea is promising, but the evidence as presented is not yet at the level of the claims.

major comments (3)
  1. [Abstract; §4.2, Table 1; Appendix B.1, Table 10] The central claim that SPaTS 'consistently and significantly outperforms both frontier closed-source MLLMs and OCR MLLMs' is contradicted by the paper's own tables. In Table 1, on CTW1500 'None', HunyuanOCR reaches F=63.8, while SPaTS-2B scores 47.0 and SPaTS-4B 51.2; on 'Full', HunyuanOCR (77.7) beats SPaTS-2B (75.1). Table 10 shows OmniParser ahead of both SPaTS variants on all reported metrics (e.g., Total-Text None 84.0 vs. 64.9; ICDAR 2015 S 89.6 vs. 73.4). The abstract and §4.2 should be rewritten to state the actual comparison scope, and the acknowledgment in Appendix B.1 that SPaTS trails the strongest specialist systems should be reflected in the main text. This is not merely wording: the advertised result is not the result the tables establish.
  2. [Appendix A.5; Table 2; Table 11] The controlled comparison for the central 'one patch is enough' claim depends on the Multi-Patch baseline. After SFT, Multi-Patch gives F=21.6 on Total-Text (Table 2). Table 11 reports PaDT-3B and PaDT-7B at F=30.7 and 32.2 on the same benchmark, and Appendix A.5 states that Multi-Patch uses the same SPaTS architecture with five routed patches per instance following PaDT under the same SFT setting. A correct implementation should not fall roughly 9–11 F points below PaDT. Please provide the PaDT-equivalent configuration, or an explanation of why DEA/PED, designed for single-patch routing, cause such a large drop when applied to multi-patch routing. Without this, Table 2's conclusion that Single-Patch consistently outperforms Multi-Patch is not a controlled comparison.
  3. [§3.2, Eq. (9); Appendix A.6] The Patch Dominance Reward uses a candidate set G_i defined as every grid cell whose overlap with the ground-truth polygon mask exceeds a fixed threshold. The threshold value is never given. Since Figure 6 demonstrates that peripheral or mismatched patches can cause immediate grounding degradation, the reward may reinforce boundary-overlapping but visually uninformative patches whenever the overlap proxy is not faithful to patch informativeness. Please specify the threshold, report sensitivity to it, and ideally validate the overlap proxy against an alternative (e.g., crop-based recognition accuracy). This is load-bearing for the claim that SPaSO learns to select the most informative patch without oracle labels.
minor comments (5)
  1. [Table 4 vs. Table 1] Table 4 reports CTW1500 Patch F=81.2, whereas Table 1 reports SPaTS-4B CTW1500 Full F=79.6. Appendix A.7 says different matching protocols are used for format-specific ablations, but the main text does not say which stage or protocol Table 4 uses. Please clarify.
  2. [Table 5] The table mixes two factor groups (L2 Norm/Scale/DEA and h_t/p_att/PED) in a way that makes it unclear whether the lower-block rows also include DEA. Please separate the two ablations explicitly.
  3. [Tables 2 and 11] The Total-Text F=63.7 in Table 2 for Single-Patch is identical to the SPaTS-4B row in Table 11. Readers cannot tell which model variant Table 2 uses. Please state the model size and training stage in the caption.
  4. [Table 8] The edit-distance threshold in r_gen is reported as 0.2, but the normalization (character-level? word-level? normalized by length?) is not defined. Please specify the exact metric so the reward is reproducible.
  5. [Figure 6] The qualitative claim that patch selection drives grounding quality would be stronger with a quantitative plot (e.g., selection distance to GT center vs. final F/IoU) over the validation set.

Circularity Check

0 steps flagged

No significant circularity: rewards are optimization signals, not fitted predictions; PaDT is external, not a self-citation chain.

full rationale

SPaTS does not derive its central result from its own fitted terms. The SPaSO rewards (Eq. 8 and Eq. 9) are computed from ground-truth polygons, F1, and IoU; they are training objectives, not predictions claimed to be independent evidence. Eq. 13 is a standard cross-entropy expression for normalized visual logits, and it follows algebraically from the DEA parameterization rather than encoding the conclusion. The closest concern is Appendix A.6, where the correct-patch candidate set G_i is defined by overlap with ground-truth polygon masks. This means the phrase 'without oracle labels' should be read as 'without oracle patch labels'; it is still supervision derived from standard spotting annotations, not circularity, because the model must still learn to select and decode geometry. The large gap between the Multi-Patch baseline (Table 2, F=21.6) and PaDT (Table 11, F=30.7/32.2) is a possible fairness or implementation issue in the controlled comparison, but it does not make any equation or fitted parameter equivalent to its input; it is an experimental-validity concern, not a circular-derivation concern. No load-bearing self-citation chain is present: PaDT and ClawMachine are external prior works, no uniqueness theorem is imported from the authors' own prior papers, and no fitted value is renamed as a prediction. Therefore the claimed derivation chain is self-contained with respect to circularity.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

No new physical entities are postulated. The 'visual prototype' and 'anchor token' are internal architectural constructs of the model; their behavior is measured on standard benchmarks. The free parameters listed are hand-chosen training/reward constants that the central claims depend on.

free parameters (5)
  • overlap threshold for candidate patch set G_i
    Appendix A.6 defines G_i as grid cells whose overlap with the GT polygon exceeds a 'fixed threshold'; the value is never given, yet it fully determines which patches are rewarded as correct in Eq. 9.
  • edit-distance threshold in r_gen = 0.2
    Table 8: edit-distance threshold 0.2 controls the definition of a True Positive transcription in the generation reward; chosen by hand.
  • Top-K in r_dom = min(10, |G_i|)
    Table 8 and Appendix A.6: the dominance reward checks membership in the top-k predictions; k=10 cap is a hand-chosen schedule.
  • GRPO group size G = 4
    Table 8: the RL group size is 4; this affects advantage normalization and the variance of the policy-gradient estimate.
  • DEA scale warmup ratio = 0.05
    Table 8: the scale branch is warmed up for 0.05 of training then frozen; this is central to DEA's design but its value is chosen by experiment.
axioms (5)
  • domain assumption GT-polygon overlap is a faithful proxy for patch informativeness
    Appendix A.6 uses overlap with GT polygons to define the candidate set. If interior patches can be visually uninformative (as Fig. 6 suggests), the dominance reward mis-specifies the selection objective.
  • domain assumption Any polygon-interior patch is an acceptable anchor during SFT
    Appendix A.6: during SFT a random patch from G_i is sampled per step, assuming all such patches are equivalent. Fig. 6 contradicts this by showing peripheral patches degrade localization.
  • domain assumption The pre-trained Qwen3-VL vision-language space preserves enough spatial precision for fine text boundaries
    The vision encoder is frozen during RL (Table 8); if its patch features blur small text, the decoder cannot recover precise curves or boxes.
  • domain assumption Harmonic mean of F1 and IoU_avg is a sufficient task-level reward
    Eq. 8 assumes that F1 plus matched IoU captures all that matters for spotting; edit-distance tolerance and thresholding could hide systematic errors.
  • domain assumption GRPO with group size 4 gives a stable advantage estimate for discrete patch selection
    The RL stage relies on GRPO's normalization over a small group of 4 samples; the paper does not analyze variance or sensitivity to this choice.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of One Patch Is Enough: Reinforcement-Optimized Visual Token Grounding for MLLM-Based Scene Text Spotting." pith.science (2026). https://pith.science/paper/N226Z26H

@misc{pith2026260727902,
  author       = {Pith},
  title        = {Pith review of: One Patch Is Enough: Reinforcement-Optimized Visual Token Grounding for MLLM-Based Scene Text Spotting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N226Z26H}},
  note         = {Machine review of arXiv:2607.27902}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Scene text spotting requires high-precision alignment between textual recognition and spatial localization. While visual-token grounding has emerged as a promising formulation for Multimodal Large Language Models (MLLMs), the previous multi-patch paradigm often introduces redundant noise and localization ambiguity, particularly for dense or small text instances. To address this, we propose Single-Patch Text Spotting (SPaTS), a vision-centric framework that routes each text instance through a single anchor visual token and then recovers geometry via full-image refinement. To accurately identify this anchor without oracle labels, we introduce Single-Patch Selective Optimization (SPaSO), a reinforcement learning framework that optimizes discrete visual-token selection using patch-level rewards. To further improve representation robustness and localization precision, we introduce Directional Embedding Alignment (DEA) to suppress unstable norm bias by decoupling feature magnitude and direction, and Patch-Enhanced Decoding (PED) to fuse the routed anchor with language semantics and cross-attend over the full-image feature map for geometry-aware boundary regression beyond coordinate-space surrogates. Extensive experiments demonstrate that SPaTS consistently and significantly outperforms both frontier closed-source MLLMs and OCR MLLMs. Code is available at https://github.com/eeNickTang/SPaTS.

Figures

Figures reproduced from arXiv: 2607.27902 by Huiguo He, Lianwen Jin, Peirong Zhang, Rui Tang, Shun Zhang, Wentao Yang, Yongxin Shi.

Figure 1
Figure 1. Figure 1: Motivation for SPaTS. In current multi-patch [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of SPaTS. (a) Autoregressive spotting architecture. (b) Two-stage training with supervised fine-tuning and [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Illustration of SPaSO. (a) Selection Optimization & Reward Modeling: The model selects a visual patch from candidates [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative comparison of localization representa [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: Localization quality varies across different sampled [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Prompt template for unified text spotting and grounding. The template includes the SYSTEM prompt, USER query, [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Effect of normalization strategy on geometry con [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative comparisons on scene text spotting. We compare SPaTS with three representative baselines from different [PITH_FULL_IMAGE:figures/full_fig_p014_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: SPaSO reward and visual-metric convergence over [PITH_FULL_IMAGE:figures/full_fig_p014_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: More qualitative comparisons of different grounding representations. Columns show the input image together [PITH_FULL_IMAGE:figures/full_fig_p015_11.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

59 extracted references · 9 linked inside Pith

  1. [1]

    Anthropic. 2026. Introducing Claude Sonnet 4.6. https://www.anthropic.com/ news/claude-sonnet-4-6

    2026

  2. [2]

    Youngmin Baek, Seung Shin, Jeonghun Baek, Sungrae Park, Junyeop Lee, Dae- hyun Nam, and Hwalsuk Lee. 2020. Character Region Attention for Text Spotting. InComputer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXIX (Lecture Notes in Computer Science), Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-M...

    doi:10.1007/978-3-030-58526-6_30 2020

  3. [3]

    Bytedance. 2026. Seed 2.0 Official Launch. https://seed.bytedance.com/zh/blog/ seed-2-0-official-launch?view_from=content_recommend

    2026

  4. [4]

    Keqin Chen, Zhao Zhang, Weili Zeng, Richong Zhang, Feng Zhu, and Rui Zhao

  5. [5]

    Chee Kheng Chng and Chee Seng Chan. 2017. Total-Text: A Comprehensive Dataset for Scene Text Detection and Recognition. In14th IAPR International Conference on Document Analysis and Recognition, ICDAR 2017, Kyoto, Japan, November 9-15, 2017. IEEE, 935–942. doi:10.1109/ICDAR.2017.157

    doi:10.1109/icdar.2017.157 2017

  6. [6]

    Cheng Cui, Ting Sun, Suyin Liang, Tingquan Gao, Zelun Zhang, Jiaxuan Liu, Xueqing Wang, Changda Zhou, Hongen Liu, Manhui Lin, Yue Zhang, Yubo Zhang, Yi Liu, Dianhai Yu, and Yanjun Ma. 2026. PaddleOCR-VL-1.5: Towards a Multi- Task 0.9B VLM for Robust In-the-Wild Document Parsing.CoRRabs/2601.21957 (2026). arXiv:2601.21957 doi:10.48550/ARXIV.2601.21957

    Pith/arXiv arXiv doi:10.48550/arxiv.2601.21957 2026

  7. [7]

    Cheng Cui, Ting Sun, Suyin Liang, Tingquan Gao, Zelun Zhang, Jiaxuan Liu, Xueqing Wang, Changda Zhou, Hongen Liu, Manhui Lin, Yue Zhang, Yubo Zhang, Handong Zheng, Jing Zhang, Jun Zhang, Yi Liu, Dianhai Yu, and Yanjun Ma

  8. [8]

    Rongyao Fang, Chengqi Duan, Kun Wang, Hao Li, Hao Tian, Xingyu Zeng, Rui Zhao, Jifeng Dai, Hongsheng Li, and Xihui Liu. 2024. PUMA: Empowering Unified MLLM with Multi-granular Visual Generation.CoRRabs/2410.13861 (2024). arXiv:2410.13861 doi:10.48550/ARXIV.2410.13861

    Pith/arXiv arXiv doi:10.48550/arxiv.2410.13861 2024

  9. [9]

    Wei Feng, Wenhao He, Fei Yin, Xu-Yao Zhang, and Cheng-Lin Liu. 2019. TextDragon: An End-to-End Framework for Arbitrary Shaped Text Spotting. In2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019. IEEE, 9075–9084. doi:10.1109/ICCV. 2019.00917

    arXiv 2019

  10. [10]

    Gemini Team. 2025. Gemini 3 Flash: frontier intelligence built for speed. https: //blog.google/products/gemini/gemini-3-flash/

    2025

  11. [11]

    Daya Guo, Dejian Yang, Haowei Zhang, et al. 2025. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning.Nature645 (2025), 633–638. doi:10.1038/s41586-025-09422-z

    doi:10.1038/s41586-025-09422-z 2025

  12. [12]

    Jiaxing Huang, Jingyi Zhang, Kai Jiang, Han Qiu, Xiaoqin Zhang, Ling Shao, Shijian Lu, and Dacheng Tao. 2025. Visual Instruction Tuning towards General- Purpose Multimodal Large Language Model: A Survey.Int. J. Comput. Vis.133, 11 (2025), 8151–8189. doi:10.1007/S11263-025-02572-7

    doi:10.1007/s11263-025-02572-7 2025

  13. [13]

    Mingxin Huang, Yuliang Liu, Zhenghao Peng, Chongyu Liu, Dahua Lin, Shenggao Zhu, Nicholas Jing Yuan, Kai Ding, and Lianwen Jin. 2022. SwinTextSpotter: Scene Text Spotting via Better Synergy between Text Detection and Text Recognition. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022. IEEE,...

    arXiv 2022

  14. [14]

    Zilin Huang, Zihao Sheng, Yansong Qu, Junwei You, and Sikai Chen. 2025. VLM- RL: A Unified Vision Language Models and Reinforcement Learning Framework for Safe Autonomous Driving.Transportation Research Part C: Emerging Tech- nologies(2025). doi:10.1016/j.trc.2025.105321

    arXiv 2025

  15. [15]

    2026.TRL: Transformers Reinforcement Learning

    Hugging Face. 2026.TRL: Transformers Reinforcement Learning. Accessed: 2026- 04-09. https://huggingface.co/docs/trl

    2026

  16. [16]

    Ghosh, Andrew D

    Dimosthenis Karatzas, Lluis Gomez-Bigorda, Anguelos Nicolaou, Suman K. Ghosh, Andrew D. Bagdanov, Masakazu Iwamura, Jiri Matas, Lukás Neumann, Vijay Ramaseshan Chandrasekhar, Shijian Lu, Faisal Shafait, Seiichi Uchida, and Ernest Valveny. 2015. ICDAR 2015 competition on Robust Reading. In 13th International Conference on Document Analysis and Recognition,...

    arXiv 2015

  17. [17]

    Dimosthenis Karatzas, Faisal Shafait, Seiichi Uchida, Masakazu Iwamura, Lluis Gomez i Bigorda, Sergi Robles Mestre, Joan Mas, David Fernández Mota, Jon Almazán, and Lluís-Pere de las Heras. 2013. ICDAR 2013 Robust Reading Competition. In12th International Conference on Document Analysis and Recog- nition, ICDAR 2013, Washington, DC, USA, August 25-28, 201...

    doi:10.1109/icdar.2013.221 2013

  18. [18]

    Jaakkola

    Tao Lei, Regina Barzilay, and Tommi S. Jaakkola. 2016. Rationalizing Neural Predictions. InProceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, Jian Su, Xavier Carreras, and Kevin Duh (Eds.). The Association for Computational Linguistics, 107–117. doi:10.18653/V1/D16-1011

    doi:10.18653/v1/d16-1011 2016

  19. [20]

    Minghui Liao, Pengyuan Lyu, Minghang He, Cong Yao, Wenhao Wu, and Xiang Bai. 2021. Mask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapes.IEEE Trans. Pattern Anal. Mach. Intell.43, 2 (2021), 532–548. doi:10.1109/TPAMI.2019.2937086

    arXiv 2021

  20. [21]

    Minghui Liao, Zhisheng Zou, Zhaoyi Wan, Cong Yao, and Xiang Bai. 2023. Real- Time Scene Text Detection With Differentiable Binarization and Adaptive Scale Fusion.IEEE Trans. Pattern Anal. Mach. Intell.45, 1 (2023), 919–931. doi:10.1109/ TPAMI.2022.3155612

    arXiv 2023

  21. [22]

    Xuebo Liu, Ding Liang, Shi Yan, Dagui Chen, Yu Qiao, and Junjie Yan. 2018. FOTS: Fast Oriented Text Spotting With a Unified Network. In2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018. Computer Vision Foundation / IEEE Computer Society, 5676–5685. doi:10.1109/CVPR.2018.00595

    arXiv 2018

  22. [23]

    Yuliang Liu, Hao Chen, Chunhua Shen, Tong He, Lianwen Jin, and Liangwei Wang. 2020. ABCNet: Real-Time Scene Text Spotting With Adaptive Bezier- Curve Network. In2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, W A, USA, June 13-19, 2020. Computer Vision Foundation / IEEE, 9806–9815. doi:10.1109/CVPR42600.2020.00983

    arXiv 2020

  23. [24]

    Yuliang Liu, Lianwen Jin, Shuaitao Zhang, Canjie Luo, and Sheng Zhang. 2019. Curved scene text detection via transverse and longitudinal sequence connection. Pattern Recognit.90 (2019), 337–345. doi:10.1016/J.PATCOG.2019.02.002

    doi:10.1016/j.patcog.2019.02.002 2019

  24. [25]

    Yuliang Liu, Chunhua Shen, Lianwen Jin, Tong He, Peng Chen, Chongyu Liu, and Hao Chen. 2022. ABCNet v2: Adaptive Bezier-Curve Network for Real-Time End-to-End Text Spotting.IEEE Trans. Pattern Anal. Mach. Intell.44, 11 (2022), 8048–8064. doi:10.1109/TPAMI.2021.3107437

    arXiv 2022

  25. [26]

    Yuliang Liu, Jiaxin Zhang, Dezhi Peng, Mingxin Huang, Xinyu Wang, Jingqun Tang, Can Huang, Dahua Lin, Chunhua Shen, Xiang Bai, and Lianwen Jin. 2023. SPTS v2: Single-Point Scene Text Spotting.IEEE Trans. Pattern Anal. Mach. Intell. 45, 12 (2023), 15665–15679. doi:10.1109/TPAMI.2023.3312285

    arXiv 2023

  26. [27]

    Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. 2025. Visual-RFT: Visual Reinforcement Fine-Tuning. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 2034–2044. https://openaccess.thecvf.com/content/ICCV2025/html/Liu_Visual- RFT_Visual_Reinforcement_Fine-Tuning_ICCV_2025_paper.html

    2025

  27. [28]

    Shangbang Long, Xin He, and Cong Yao. 2021. Scene Text Detection and Recog- nition: The Deep Learning Era.Int. J. Comput. Vis.129, 1 (2021), 161–184. doi:10.1007/S11263-020-01369-0

    doi:10.1007/s11263-020-01369-0 2021

  28. [29]

    Chuofan Ma, Yi Jiang, Jiannan Wu, Zehuan Yuan, and Xiaojuan Qi. 2024. Groma: Localized Visual Tokenization for Grounding Multimodal Large Language Models. InComputer Vision - ECCV 2024 - 18th European Conference, Milan, Italy, September 29-October 4, 2024, Proceedings, Part VI (Lecture Notes in Computer Science), Ales Leonardis, Elisa Ricci, Stefan Roth, ...

    doi:10.1007/978-3-031-72658-3_24 2024

  29. [30]

    Tianren Ma, Lingxi Xie, Yunjie Tian, Boyu Yang, and Qixiang Ye. 2025. Claw- Machine: Learning to Fetch Visual Tokens for Referential Comprehension. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net. https://openreview.net/forum?id= TOtk9dTYGG

    2025

  30. [31]

    Zhibin Ma, Pengwen Dai, Wei Zhuo, and Xugong Qin. 2026. One2Seq: One- Token Wise Decoder for Efficient Scene Text Recognition.Proceedings of the AAAI Conference on Artificial Intelligence(2026). https://api.semanticscholar.org/ CorpusID:286656590

    2026

  31. [32]

    Nibal Nayef, Fei Yin, Imen Bizid, Hyunsoo Choi, Yuan Feng, Dimosthenis Karatzas, Zhenbo Luo, Umapada Pal, Christophe Rigaud, Joseph Chazalon, Wafa Khlif, Muhammad Muzzamil Luqman, Jean-Christophe Burie, Cheng-Lin Liu, and Jean- Marc Ogier. 2017. ICDAR2017 Robust Reading Challenge on Multi-Lingual Scene Text Detection and Script Identification - RRC-MLT. I...

    2017

  32. [33]

    OpenAI. 2026. Introducing GPT-5.4. https://openai.com/index/introducing-gpt- 5-4/ MM ’26, November 10–14, 2026, Rio de Janeiro, Brazil. Rui Tang et al

    2026

  33. [34]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022. Train- ing language models to follow instructions with h...

    2022

  34. [35]

    Dezhi Peng, Xinyu Wang, Yuliang Liu, Jiaxin Zhang, Mingxin Huang, Songxuan Lai, Jing Li, Shenggao Zhu, Dahua Lin, Chunhua Shen, Xiang Bai, and Lianwen Jin. 2022. SPTS: Single-Point Text Spotting. InMM ’22: The 30th ACM International Conference on Multimedia, Lisboa, Portugal, October 10 - 14, 2022, João Magalhães, Alberto Del Bimbo, Shin’ichi Satoh, Nicu ...

    doi:10.1145/3503161 2022

  35. [36]

    Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. 2024. Kosmos-2: Grounding Multimodal Large Language Models to the World. InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. https://openreview.net/forum?id= lLmqxkfSIw

    2024

  36. [37]

    Girshick, and Jian Sun

    Shaoqing Ren, Kaiming He, Ross B. Girshick, and Jian Sun. 2017. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.IEEE Trans. Pattern Anal. Mach. Intell.39, 6 (2017), 1137–1149. doi:10.1109/TPAMI.2016. 2577031

    doi:10.1109/tpami.2016 2017

  37. [38]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models.CoRRabs/2402.03300 (2024). arXiv:2402.03300 doi:10.48550/ARXIV.2402.03300

    Pith/arXiv arXiv doi:10.48550/arxiv.2402.03300 2024

  38. [39]

    Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, Ruochen Xu, and Tiancheng Zhao. 2025. VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model.CoRRabs/2504.07615 (2025). arXiv:2504.07615 doi:10.48550/ARXIV.2504. 07615

    Pith/arXiv arXiv doi:10.48550/arxiv.2504.07615 2025

  39. [40]

    Chen, Shuicheng Yan, Xulei Yang, and Xun Xu

    Yongyi Su, Haojie Zhang, Shijie Li, Nanqing Liu, Jingyi Liao, Junyi Pan, Yuan Liu, Xiaofen Xing, Chong Sun, Chen Li, Nancy F. Chen, Shuicheng Yan, Xulei Yang, and Xun Xu. 2026. Patch-as-Decodable-Token: Towards Unified Multi- Modal Vision Tasks in MLLMs. InThe International Conference on Learning Representations (ICLR). https://openreview.net/forum?id=xF0Dcmvsl0

    2026

  40. [41]

    Hunyuan Vision Team, Pengyuan Lyu, Xingyu Wan, Gengluo Li, Shangpin Peng, Weinong Wang, Liang Wu, Huawen Shen, Yu Zhou, Canhui Tang, Qi Yang, Qiming Peng, Bin Luo, Hower Yang, Xinsong Zhang, Jinnian Zhang, Houwen Peng, Hongming Yang, Senhao Xie, Longsha Zhou, Ge Pei, Binghong Wu, Rui Yan, Kan Wu, Jieneng Yang, Bochao Wang, Kai Liu, Jianchen Zhu, Jie Jiang...

    doi:10.48550/arxiv.2511.19575 2025

  41. [42]

    Kimi Team. 2026. Kimi K2.5: Visual Agentic Intelligence.CoRRabs/2602.02276 (2026). arXiv:2602.02276 doi:10.48550/ARXIV.2602.02276

    Pith/arXiv arXiv doi:10.48550/arxiv.2602.02276 2026

  42. [43]

    Qwen Team. 2025. Qwen3-VL Technical Report.CoRRabs/2511.21631 (2025). arXiv:2511.21631 doi:10.48550/ARXIV.2511.21631

    Pith/arXiv arXiv doi:10.48550/arxiv.2511.21631 2025

  43. [44]

    Qwen Team. 2026. Qwen3. 5: Towards native multimodal agents.URL: https://qwen. ai/blog(2026)

    2026

  44. [45]

    Haoqin Tu, Chenhang Cui, Zijun Wang, Yiyang Zhou, Bingchen Zhao, Junlin Han, Wangchunshu Zhou, Huaxiu Yao, and Cihang Xie. 2024. How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMs. InProceedings of the European Conference on Computer Vision (ECCV). https://eccv.ecva.net/ virtual/2024/poster/1855

    2024

  45. [47]

    Hao Wang, Pu Lu, Hui Zhang, Mingkun Yang, Xiang Bai, Yongchao Xu, Mengchao He, Yongpan Wang, and Wenyu Liu. 2020. All You Need Is Boundary: Toward Arbitrary-Shaped Text Spotting. InThe Thirty-Fourth AAAI Conference on Artifi- cial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth...

    2020

  46. [48]

    Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, Zhaokai Wang, Zhe Chen, Hongjie Zhang, Ganlin Yang, Haomin Wang, Qi Wei, Jinhui Yin, Wenhao Li, Erfei Cui, Guanzhou Chen, Zichen Ding, Changyao Tian, Zhenyu Wu, JingJing Xie, Zehao Li, Bowen Yang, Yuchen Duan, Xuehui Wang, Zhi Hou,...

    Pith/arXiv arXiv doi:10.48550/arxiv.2508.18265 2025

  47. [49]

    Xue Yang, Jirui Yang, Junchi Yan, Yue Zhang, Tengfei Zhang, Zhi Guo, Xian Sun, and Kun Fu. 2019. SCRDet: Towards More Robust Detection for Small, Cluttered and Rotated Objects. In2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019. IEEE, 8231–8240. doi:10.1109/ICCV.2019.00832

    arXiv 2019

  48. [50]

    Maoyuan Ye, Jing Zhang, Shanshan Zhao, Juhua Liu, Tongliang Liu, Bo Du, and Dacheng Tao. 2023. DeepSolo: Let Transformer Decoder with Explicit Points Solo for Text Spotting. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023. IEEE, 19348– 19357. doi:10.1109/CVPR52729.2023.01854

    arXiv 2023

  49. [51]

    Yuqian Yuan, Wentong Li, Jian Liu, Dongqi Tang, Xinjie Luo, Chi Qin, Lei Zhang, and Jianke Zhu. 2024. Osprey: Pixel Understanding with Visual Instruction Tuning. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, W A, USA, June 16-22, 2024. IEEE, 28202–28211. doi:10.1109/ CVPR52733.2024.02664

    arXiv 2024

  50. [52]

    Z.ai Team. 2025. GLM-4.6V: Open Source Multimodal Models with Native Tool Use. https://z.ai/blog/glm-4.6v

    2025

  51. [53]

    Peirong Zhang, Haowei Xu, Jiaxin Zhang, Xuhan Zheng, Guitao Xu, Yuyi Zhang, Junle Liu, Zhenhua Yang, Wei Zhou, and Lianwen Jin. 2025. OCRGenBench: A Comprehensive Benchmark for Evaluating OCR Generative Capabilities.arXiv preprint arXiv:2507.15085(2025)

    arXiv 2025

  52. [54]

    Peirong Zhang, Jiaxin Zhang, Jiahuan Cao, Hongliang Li, and Lianwen Jin. 2025. Smaller But Better: Unifying Layout Generation with Smaller Large Language Models.International Journal of Computer Vision (IJCV)133 (2025), 3891–3917

    2025

  53. [55]

    Shaolei Zhang, Qingkai Fang, Zhe Yang, and Yang Feng. 2025. LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Token. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net. https://openreview.net/forum?id= UQJ7CDW8nb

    2025

  54. [56]

    Xiang Zhang, Yongwen Su, Subarna Tripathi, and Zhuowen Tu. 2022. Text Spotting Transformers. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022. IEEE, 9509–9518. doi:10.1109/CVPR52688.2022.00930

    arXiv 2022

  55. [57]

    Handong Zheng, Yumeng Li, Kaile Zhang, Liang Xin, Guangwei Zhao, Hao Liu, Jiayu Chen, Jie Lou, Jiyu Qiu, Qi Fu, et al. 2026. Multimodal OCR: Parse Anything from Documents.arXiv preprint arXiv:2603.13032(2026)

    arXiv 2026

  56. [58]

    text"]. SYSTEM：You are an expert system specialized in scene text spotting, with performance exceeding that of human experts. ASSISTANT：There are N

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, Zhangwei Gao, Erfei Cui, Xuehui Wang, Yue Cao, Yangzhou Liu, Xingguang Wei, Hongjie Zhang, Haomin Wang, Weiye Xu, Hao Li, Jiahao Wang, Nianchen Deng, Songze Li, Yinan He, Tan Jiang, Jiapeng Luo, Yi Wang, Conghui He, Botian Shi, Xingcheng Zh...

    Pith/arXiv arXiv doi:10.48550/arxiv.2504.10479 2025

  57. [2020]

    doi:10.1609/AAAI.V34I07.6896

    AAAI Press, 12160–12167. doi:10.1609/AAAI.V34I07.6896

    doi:10.1609/aaai.v34i07.6896

  58. [2023]

    arXiv:2306.15195 doi:10.48550/ARXIV.2306.15195

    Shikra: Unleashing Multimodal LLM’s Referential Dialogue Magic.CoRR abs/2306.15195 (2023). arXiv:2306.15195 doi:10.48550/ARXIV.2306.15195

    Pith/arXiv arXiv doi:10.48550/arxiv.2306.15195 2023

  59. [2025]

    arXiv:2510.14528 doi:10.48550/ARXIV.2510.14528

    PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra- Compact Vision-Language Model.CoRRabs/2510.14528 (2025). arXiv:2510.14528 doi:10.48550/ARXIV.2510.14528

    doi:10.48550/arxiv.2510.14528 2025

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.