Pith. sign in

REVIEW 4 major objections 4 minor 72 references

ESTR-CoT: Towards Explainable and Accurate Event Stream based Scene Text Recognition with Chain-of-Thought Reasoning

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Chain-of-thought supervision makes event-camera text recognition both more accurate and more explainable.

desk verdict Useful dataset, untested claims: label-derived CoT means the paper's interpretability and accuracy gains are not established. read the letter →

arxiv 2507.02200 v1 pith:QZHKVBVQ submitted 2025-07-02 cs.CV cs.AIcs.CL

Xiao Wang , Jingtao Jiang , Qiang Chen , Lan Chen , Lin Zhu , Yaowei Wang , Yonghong Tian , Jin Tang This is my paper
classification cs.CVcs.AIcs.CL
keywords eventcamerascenetextrecognitionchain-of-thoughtreasoninglargelanguagemodelinterpretabilityvision-languageCoTdatasetlow-light
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that giving an event-stream scene text recognizer chain-of-thought supervision—reasoning text attached to each recognized word—improves both accuracy and interpretability under the low-light, fast-motion conditions where event cameras operate. The proposed ESTR-CoT framework connects an event image encoder to a large language model decoder and is trained end-to-end with two objectives: output the recognized text and output a step-by-step rationale. To get supervision, the authors build a 16,222-pair CoT dataset through generation, automatic evaluation and rewriting, and expert review. On the EventSTR, WordArt*, and IC15* benchmarks, the method reports consistent gains over the SimC-ESTR baseline while producing rationales that cite letter shapes and semantic context. A sympathetic reader would care because this is a concrete step toward OCR systems that say why they read what they read.

What carries the argument

The load-bearing mechanism is CoT supervision: each training pair joins an event-based text image, its ground-truth answer, and a rationale that explains the reading by comparing letter shapes and word meanings. The architecture that carries it is a prompt-based control design: a shared Q-former-aligned visual input with separate <answer> and <thinking> suffixes guides the pretrained Vicuna-7B decoder to emit either the final text or the reasoning chain, and the two outputs are trained jointly with two cross-entropy losses.

What would settle it

Take a fixed ground-truth text label and two different event images that share that label but contain different visually similar distractor letters; if the model's generated rationales name the same distractors in both cases rather than the ones actually present in each image, the reasoning is driven by the label and not by visual evidence.

Watch

Extended reading notes

Core claim

The central claim is that a chain-of-thought reasoning target, generated from ground-truth OCR labels and refined by automatic and human filtering, can be used to supervise an event-stream scene text recognizer so that one model simultaneously produces the recognized text and a transparent reasoning chain. ESTR-CoT encodes the event image with a CLIP-style vision encoder (EVA-CLIP ViT-G/14), aligns the visual tokens to the Vicuna-7B decoder through a Q-former, and steers generation with separate <answer> and <thinking> prompt suffixes. Trained with the unweighted sum of answer and reasoning cross-entropy losses, the model raises BLEU-1 on EventSTR from 0.638 to 0.648 and accuracy on WordArt* and IC15* from 65.1 and 56.8 to 65.6 and 57.1 relative to the SimC-ESTR baseline, while generating rationales that name visually similar distractors and semantic context.

Load-bearing premise

The load-bearing premise is that rationales generated from the ground-truth text label alone teach a model genuine visual-semantic reasoning that transfers to new event images, rather than merely restating the answer.

Editorial extensions

If this is right

  • On the EventSTR benchmark, ESTR-CoT improves BLEU-1 from 0.638 to 0.648 over the SimC-ESTR baseline and matches its BLEU-4 at 0.430, while also emitting a reasoning chain for each prediction.
  • The three-stage CoT filtering pipeline raises BLEU-1 from 0.632 (unfiltered) to 0.648 (filtered), indicating that the quality of the reasoning supervision directly affects recognition performance.
  • Using separate <answer> and <thinking> prompt suffixes outperforms separate projection layers (BLEU-1 0.648 vs 0.638), and directly summing the two losses is at least as good as any weighted combination tested.
  • If correct, ESTR-CoT is the first event-stream scene text recognizer that produces human-readable step-by-step rationales, making it applicable to autonomous driving and other settings where OCR decisions need to be auditable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural testable extension is to generate CoT rationales directly from the event image rather than from the ground-truth label alone; if that changes the rationales on visually altered inputs, it would confirm that the reasoning is visually grounded.
  • The reported accuracy gains (about +0.01 BLEU-1 and +0.5 to +0.3 accuracy points) are small, so the method's primary value may be interpretability rather than raw accuracy; future work could measure rationale faithfulness directly with human judges or attribution metrics.
  • The same label-to-rationale recipe could apply to other event-based perception tasks such as object detection or tracking, where a language model could turn labels into explanatory text for human review.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes ESTR-CoT, a chain-of-thought reasoning framework for event stream based scene text recognition (STR). The approach uses an EVA-CLIP vision encoder, a Q-Former, and the Vicuna-7B large language model to generate both the recognized text and a step-by-step reasoning chain. A three-stage pipeline (generation, automatic filtering, and expert review) produces a CoT dataset of 16,222 image-reasoning pairs, where the reasoning is initially generated by DeepSeek-V3 from ground-truth OCR labels. The model is trained end-to-end with a sum of answer and thinking cross-entropy losses. Experiments on EventSTR, WordArt*, and IC15* report small accuracy gains over the SimC-ESTR baseline, and qualitative examples of reasoning chains are shown.

Significance. If the central claims were substantiated, the paper would contribute a useful dataset and a plausible demonstration that chain-of-thought supervision can improve event-based STR. The idea of adding explicit reasoning supervision to an LLM-based STR pipeline is timely, and the constructed CoT dataset could be a resource for future work. However, the evidence presented does not establish the claims: the reported accuracy improvements are very small, the interpretability claim is not grounded because the CoT supervision is generated from labels alone, and the ablations do not isolate the effect of the reasoning task. The paper deserves credit for the dataset construction effort and the clear system description, but the validation is currently insufficient.

major comments (4)
  1. [III-B Stage 1] The CoT supervision is generated from the ground-truth label only, not from the event image: the text states, "Given a raw dataset of OCR labels or question-answer pairs ... we first utilize a powerful LLM ... to generate an initial CoT explanation." Consequently, the rationales are post-hoc text conditioned solely on the answer, and training the model to reproduce them cannot demonstrate visually grounded reasoning. The interpretability claim in Sections IV.E and V is therefore circular. Please provide evidence that the model's reasoning is actually influenced by visual content, for example by perturbing or masking input event images and showing that the generated rationales change accordingly, or by generating CoT from event images directly.
  2. [IV-C Tables I and II] The accuracy improvements over the SimC-ESTR baseline are very small: BLEU-1 rises by 0.010, WordArt* accuracy by 0.5 percentage points, and IC15* accuracy by 0.3 percentage points. No error bars, multiple seeds, or significance tests are reported, so these differences may be within run-to-run variance. Please report means and standard deviations over at least three runs and apply a paired significance test (e.g., bootstrap) for the main comparisons.
  3. [IV-D Table IV and III-D Eq. (15)] The ablations compare filtered versus unfiltered CoT data and different loss weighting schemes, but no experiment trains the model with answer-only supervision (that is, removing the L_thinking term or setting lambda=1). Without such a baseline, the central claim that CoT supervision improves recognition accuracy is not isolated from the effect of additional fine-tuning of the vision-language model on the same data. Please add an answer-only ablation trained on the same event images with the same architecture.
  4. [IV-D Table IV] The filtering effect on BLEU-1 is 0.016 (0.648 versus 0.632), and the paper describes this as "noticeable" and "significant". Given that no variance estimates or significance tests are provided, this conclusion is unsupported. Please report the statistical significance of the filtered-versus-unfiltered difference as well.
minor comments (4)
  1. [III-A] The word "interoperability" appears twice where "interpretability" is intended, which is confusing in a paper about explainable STR.
  2. [IV.C and V] The phrases "fully validated the effectiveness and interpretability" and "fully demonstrate" are stronger than the reported results warrant; please soften these to match the limited evidence.
  3. [IV.A and IV.D] The paper does not report dataset statistics for CoT_ESTR (e.g., number of training/validation/test samples, overlap with EventSTR/WordArt*/IC15*, and language distribution). Please clarify how the 16,222 pairs are split and how they relate to the evaluation datasets.
  4. [IV.A] BLEU is used as the primary metric for EventSTR, computed at character level for Chinese and word level for English. BLEU is a sequence-matching metric and may not reflect recognition accuracy; please also report word accuracy or edit distance on EventSTR, or justify why BLEU alone is appropriate.

Circularity Check

2 steps flagged · score 6.0 of 10

CoT rationales are generated from the ground-truth answer label alone, so the interpretability claim reduces to reproducing label-derived text rather than visual reasoning; the accuracy comparison itself is not definitionally circular.

  1. self definitional [Section III-B, Stage 1 (Chain-of-Thought Data Generation), Eq. (1) and Eq. (5)]
    "Given a raw dataset of OCR labels or question-answer pairs (e.g., the text string “LOVEL” predicted from an image), we first utilize a powerful LLM (e.g., Deepseek-V3) to generate an initial CoT explanation. ... D1 = {(ai, c(0)i)}Ni=1 ... Each final sample in D3 is formatted as: xi = <answer>ai</answer><thinking>c∗i</thinking>"

    The CoT target c_i is constructed from the answer label a_i alone (D1 = {(a_i, c_i^(0))}), with no event image used in generation, rewriting, or expert review. The model is then trained with L_thinking = CE(y^thinking_j, c_j) to output rationales that are by construction paraphrases of the label. Consequently, the claimed interpretability (Fig. 5: rationales 'excluding visually similar distractors' and 'matching word meaning with context') cannot demonstrate image-grounded reasoning: the supervision never conditions the rationale on the visual input. The reasoning output is a post-hoc text restatement of the answer, so the interpretability claim is defined into the training target rather than derived from the event image.

  2. renaming known result [Section I, Contribution 2]
    "We build a large-scale reasoning dataset for the event stream based scene text recognition. It contains 16,222 image-reasoning pairs, each consisting of an event-based scene text image, the corresponding recognized text (answer), and a detailed CoT rationale explaining how the model arrives at the final answer."

    The dataset is called 'image-reasoning pairs,' but the rationale in each pair is generated from the recognized text/answer, not from the event-based image (Section III-B Stage 1). Renaming label-conditioned LLM text as 'image-reasoning' presents post-hoc answer rationalization as if it were visual evidence-based reasoning, so the claimed explanatory content of the dataset is equivalent to its text-label input by construction.

full rationale

The central accuracy experiments are self-contained against external and prior-consistency benchmarks: ESTR-CoT is fine-tuned on standard splits and compared with BLIVA, SimC-ESTR, and other STR methods, so the reported BLEU/accuracy numbers are not definitionally forced by the CoT data. However, the paper's second central claim, interpretable visual-semantic reasoning, is circular in an important respect. The CoT supervision is produced by DeepSeek-V3 from OCR labels only (Section III-B Stage 1), and the dataset is later described as 'image-reasoning pairs' (Section I). The training loss L_thinking (Eq. 17) teaches the model to reproduce text that is a deterministic function of the answer label, not of the event image. Therefore the rationales shown in Fig. 5 and the claim that the model 'provides step-by-step reasoning to justify the prediction' do not provide independent evidence of visual grounding; they show imitation of label-derived rationalization. The paper's own Limitation Analysis (Section IV-F) does not disclose this label-only generation issue, instead citing inference speed and VQA pretraining as the main limitations. No self-citation chain or uniqueness theorem forces the architecture choice, so the self-citations in [18] are not themselves load-bearing. The small accuracy gains and absence of error bars are correctness concerns rather than circularity. The score of 6 reflects partial circularity: the accuracy comparison has independent content, but the interpretability claim reduces, by the paper's own data-generation equations, to reproducing answer-conditioned text.

Assumptions & free parameters 3 free parameters · 5 assumptions · 1 invented entities

The central empirical results rest on three domain assumptions that are not independently verified: simulated event datasets stand in for real event data, label-derived LLM rationales are treated as genuine visual reasoning supervision, and BLEU overlap is used as a proxy for reasoning quality. There are no fitted physical parameters; the hand-chosen CoT filtering thresholds and loss weights are the main free choices. The CoT dataset is claimed but not released, so it functions as an entity without independent evidence.

free parameters (3)
  • Lmax (CoT length cutoff) = e.g., 100 tokens
    Hand-chosen maximum token length in the automatic evaluation function Eval(c) (Eq. 4); it filters the CoT training data and influences the curated dataset, which is central to the claimed accuracy gains.
  • HasVisual / HasSemantic predicates = not specified
    Boolean criteria in Eval(c) that decide whether a generated CoT passes filtering; the exact definitions are not given, and the choice of these criteria directly determines which 16,222 samples form the training set.
  • Loss weighting lambda = 0.3, 0.5, 0.7 explored; unweighted sum adopted
    Loss weights for Lanswer and Lthinking were tuned by hand; the final choice of direct summation is justified only by comparable BLEU scores (Section IV-D).
assumptions (5)
  • domain assumption WordArt* and IC15*, converted from RGB datasets with the ESIM event simulator, are valid proxies for real event-camera text recognition.
    Used in Section IV-A as evaluation benchmarks; no real event-camera sequences for these two datasets, so conclusions about event-camera robustness rest on simulated event data.
  • domain assumption LLM-generated CoT explanations derived from ground-truth labels are a valid supervisory signal for visual reasoning.
    The data pipeline in Section III-B generates reasoning only from the answer string, not from the image; the assumption that this teaches visual reasoning is never tested directly.
  • domain assumption BLEU scores on EventSTR measure recognition quality and reasoning quality.
    Section IV-A uses BLEU-1 to BLEU-4 as the main metric on EventSTR; BLEU measures lexical overlap with a reference text, not the validity of the reasoning chain, and the reference is the single ground-truth label.
  • domain assumption Pretrained EVA-CLIP, Vicuna-7B, and BLIVA weights provide a sound starting point for fine-tuning.
    Used in Sections III-C and IV-B; the paper does not evaluate biases or failure modes of these base models in the event-text setting.
  • standard math Cross-entropy loss and BLEU are standard metrics whose mathematical properties are assumed.
    Used in Eqs. 15-17 and Section IV-A; no novel mathematics is introduced.
invented entities (1)
  • CoT_ESTR dataset (16,222 image-reasoning pairs)
    purpose: Training supervision for chain-of-thought scene text recognition; the paper's main data contribution.
    The dataset is described and illustrated but no download link or release is provided; it cannot be independently inspected or tested, so it currently provides no falsifiable handle outside the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ESTR-CoT: Towards Explainable and Accurate Event Stream based Scene Text Recognition with Chain-of-Thought Reasoning." pith.science (2026). https://pith.science/paper/QZHKVBVQ

@misc{pith2026250702200,
  author       = {Pith},
  title        = {Pith review of: ESTR-CoT: Towards Explainable and Accurate Event Stream based Scene Text Recognition with Chain-of-Thought Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QZHKVBVQ}},
  note         = {Machine review of arXiv:2507.02200}
}
read the original abstract

Event stream based scene text recognition is a newly arising research topic in recent years which performs better than the widely used RGB cameras in extremely challenging scenarios, especially the low illumination, fast motion. Existing works either adopt end-to-end encoder-decoder framework or large language models for enhanced recognition, however, they are still limited by the challenges of insufficient interpretability and weak contextual logical reasoning. In this work, we propose a novel chain-of-thought reasoning based event stream scene text recognition framework, termed ESTR-CoT. Specifically, we first adopt the vision encoder EVA-CLIP (ViT-G/14) to transform the input event stream into tokens and utilize a Llama tokenizer to encode the given generation prompt. A Q-former is used to align the vision token to the pre-trained large language model Vicuna-7B and output both the answer and chain-of-thought (CoT) reasoning process simultaneously. Our framework can be optimized using supervised fine-tuning in an end-to-end manner. In addition, we also propose a large-scale CoT dataset to train our framework via a three stage processing (i.e., generation, polish, and expert verification). This dataset provides a solid data foundation for the development of subsequent reasoning-based large models. Extensive experiments on three event stream STR benchmark datasets (i.e., EventSTR, WordArt*, IC15*) fully validated the effectiveness and interpretability of our proposed framework. The source code and pre-trained models will be released on https://github.com/Event-AHU/ESTR-CoT.

Figures

Figures reproduced from arXiv: 2507.02200 by the authors.

Figure 1
Figure 1. Comparison between the (a) traditional end-to-end encoder-decoder STR, (b) LLM-based STR, and (c) our newly proposed chain-of-thought reasoning based event stream STR. Traditional STR models rely on task-specific architectures and direct visual-text mappings. LLM-based approaches introduce language understanding but still lack explicit reasoning. Our framework incorporates chain-of-thought reasoning to enable interp… view at source ↗
Figure 2
Figure 2. Data Generation Pipeline of ESTR-CoT. The figure illustrates a three-stage pipeline for constructing high-quality CoT data in ESTR-CoT. Beginning with a raw dataset, a generator produces initial CoT responses (D1). An evaluator filters these outputs: high-quality samples proceed to D2, while suboptimal ones are revised by a rewriter based on evaluator feedback. All refined samples are added to D2. Finally, human exp… view at source ↗
Figure 3
Figure 3. Representative samples from our newly proposed CoT_ESTR Dataset. Each example consists of an event stream scene image, a recognition result <Answer>, and an accompanying reasoning chain <Thinking>. both generating the answer and the reasoning chain in the subsequent architecture. • Why Separate Outputs for answer and thinking? The decision to output both answer and thinking arises from the need to balance high recog… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: An overview of the proposed ESTR-CoT framework for the reasoning-based event stream scene text recognition. This figure illustrates the prompt-based control architecture for generating answers and reasoning chains from event-based scene text inputs. A Q-Former extracts…
Figure 5
Figure 5. Figure 5: Visualization of rationales. Our model can generate rationales that not only recognize the correct textual content from event-based images but also provide step-by-step reasoning to justify the prediction. These rationales typically include visual disambiguation cues (…
Figure 6
Figure 6. Figure 6: Comparison of text recognition results: Baseline, Our Proposed Approach, and Ground Truth (GT). that explains how it arrives at the final output. These exam￾ples demonstrate the effectiveness of our model in handling complex text recognition tasks, generating coherent …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

72 extracted references · 42 canonical work pages

  1. [1]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al. , “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774 , 2023

    arXiv 2023

  2. [2]

    Gpt-4o system card,

    A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radford et al., “Gpt-4o system card,” arXiv preprint arXiv:2410.21276 , 2024

    arXiv 2024

  3. [3]

    Deepseek llm: Scaling open-source language models with longtermism,

    X. Bi, D. Chen, G. Chen, S. Chen, D. Dai, C. Deng, H. Ding, K. Dong, Q. Du, Z. Fu et al. , “Deepseek llm: Scaling open-source language models with longtermism,” arXiv preprint arXiv:2401.02954 , 2024

    arXiv 2024

  4. [4]

    Qwen technical report,

    J. Bai, S. Bai, Y . Chu, Z. Cui, K. Dang, X. Deng, Y . Fan, W. Ge, Y . Han, F. Huang et al. , “Qwen technical report,” arXiv preprint arXiv:2309.16609, 2023

    arXiv 2023

  5. [5]

    Vqa: Visual question answering,

    S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. L. Zitnick, and D. Parikh, “Vqa: Visual question answering,” inProceedings of the IEEE international conference on computer vision , 2015, pp. 2425–2433

    2015

  6. [6]

    Flamingo: a visual language model for few-shot learning,

    J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y . Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds et al. , “Flamingo: a visual language model for few-shot learning,” Advances in neural information processing systems, vol. 35, pp. 23 716–23 736, 2022

    2022

  7. [7]

    Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models,

    J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models,” in International conference on machine learning . PMLR, 2023, pp. 19 730–19 742

    2023

  8. [8]

    Bliva: A simple multimodal llm for better handling of text-rich visual questions,

    W. Hu, Y . Xu, Y . Li, W. Li, Z. Chen, and Z. Tu, “Bliva: A simple multimodal llm for better handling of text-rich visual questions,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 3, 2024, pp. 2256–2264

    work page 2024

Show all 72 references
  1. [9]

    On the automatic generation of medical imaging reports,

    B. Jing, P. Xie, and E. Xing, “On the automatic generation of medical imaging reports,” arXiv preprint arXiv:1711.08195 , 2017

    2017 arXiv

  2. [10]

    Mmtn: multi- modal memory transformer network for image-report consistent medical report generation,

    Y . Cao, L. Cui, L. Zhang, F. Yu, Z. Li, and Y . Xu, “Mmtn: multi- modal memory transformer network for image-report consistent medical report generation,” in Proceedings of the AAAI conference on artificial intelligence, vol. 37, no. 1, 2023, pp. 277–285

    2023

  3. [11]

    Medblip: Bootstrapping language-image pre- training from 3d medical images and texts,

    Q. Chen and Y . Hong, “Medblip: Bootstrapping language-image pre- training from 3d medical images and texts,” in Proceedings of the Asian Conference on Computer Vision , 2024, pp. 2404–2420

    2024

  4. [12]

    Cxpmrg-bench: Pre-training and benchmarking for x-ray medical report generation on chexpert plus dataset,

    X. Wang, F. Wang, Y . Li, Q. Ma, S. Wang, B. Jiang, and J. Tang, “Cxpmrg-bench: Pre-training and benchmarking for x-ray medical report generation on chexpert plus dataset,” in Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR) , June 2025, pp. 5123–5133

    2025

  5. [13]

    Textmonkey: An ocr-free large multimodal model for understanding document,

    Y . Liu, B. Yang, Q. Liu, Z. Li, Z. Ma, S. Zhang, and X. Bai, “Textmonkey: An ocr-free large multimodal model for understanding document,” arXiv preprint arXiv:2403.04473 , 2024

    2024 arXiv

  6. [14]

    Docpedia: Un- leashing the power of large multimodal model in the frequency domain for versatile document understanding,

    H. Feng, Q. Liu, H. Liu, W. Zhou, H. Li, and C. Huang, “Docpedia: Un- leashing the power of large multimodal model in the frequency domain for versatile document understanding,”arXiv preprint arXiv:2311.11810, 2023

    2023 arXiv

  7. [15]

    Vary: Scaling up the vision vocabulary for large vision-language model,

    H. Wei, L. Kong, J. Chen, L. Zhao, Z. Ge, J. Yang, J. Sun, C. Han, and X. Zhang, “Vary: Scaling up the vision vocabulary for large vision-language model,” in European Conference on Computer Vision . Springer, 2025, pp. 408–424

    2025

  8. [16]

    mplug-docowl 1.5: Unified structure learning for ocr-free document understanding,

    A. Hu, H. Xu, J. Ye, M. Yan, L. Zhang, B. Zhang, C. Li, J. Zhang, Q. Jin, F. Huang et al. , “mplug-docowl 1.5: Unified structure learning for ocr-free document understanding,” arXiv preprint arXiv:2403.12895, 2024

    2024 arXiv

  9. [17]

    General ocr theory: Towards ocr-2.0 via a unified end-to-end model,

    H. Wei, C. Liu, J. Chen, J. Wang, L. Kong, Y . Xu, Z. Ge, L. Zhao, J. Sun, Y . Penget al., “General ocr theory: Towards ocr-2.0 via a unified end-to-end model,” arXiv preprint arXiv:2409.01704 , 2024

    2024 arXiv

  10. [18]

    Eventstr: A benchmark dataset and baselines for event stream based scene text recognition,

    X. Wang, J. Jiang, D. Li, F. Wang, L. Zhu, Y . Wang, Y . Tian, and J. Tang, “Eventstr: A benchmark dataset and baselines for event stream based scene text recognition,” arXiv preprint arXiv:2502.09020 , 2025

    2025 arXiv

  11. [19]

    Recurrent vision transformers for object detection with event cameras,

    M. Gehrig and D. Scaramuzza, “Recurrent vision transformers for object detection with event cameras,” in Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition , 2023, pp. 13 884– 13 893

    2023

  12. [20]

    Spatiotemporal aggregation trans- former for object detection with neuromorphic vision sensors,

    Z. Guo, J. Gao, G. Ma, and J. Xu, “Spatiotemporal aggregation trans- former for object detection with neuromorphic vision sensors,” IEEE Sensors Journal, vol. 24, no. 12, pp. 19 397–19 406, 2024

    2024

  13. [21]

    Scene adaptive sparse transformer for event-based object detection,

    Y . Peng, H. Li, Y . Zhang, X. Sun, and F. Wu, “Scene adaptive sparse transformer for event-based object detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 16 794–16 804

    2024

  14. [22]

    Object detection using event camera: A moe heat conduction based detector and a new benchmark dataset,

    X. Wang, Y . Jin, W. Wu, W. Zhang, L. Zhu, B. Jiang, and Y . Tian, “Object detection using event camera: A moe heat conduction based detector and a new benchmark dataset,” in Proceedings of the Computer Vision and Pattern Recognition Conference , 2025, pp. 29 321–29 330

    2025

  15. [23]

    Dynamic graph induced contour-aware heat conduction network for event-based object detection,

    X. Wang, Y . Jin, L. Chen, B. Jiang, L. Zhu, Y . Tian, J. Tang, and B. Luo, “Dynamic graph induced contour-aware heat conduction network for event-based object detection,” arXiv preprint arXiv:2505.12908 , 2025

    2025 arXiv

  16. [24]

    Frame- event alignment and fusion network for high frame rate tracking,

    J. Zhang, Y . Wang, W. Liu, M. Li, J. Bai, B. Yin, and X. Yang, “Frame- event alignment and fusion network for high frame rate tracking,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 9781–9790

    2023

  17. [25]

    Asynchronous tracking-by-detection on adaptive time surfaces for event-based object tracking,

    H. Chen, Q. Wu, Y . Liang, X. Gao, and H. Wang, “Asynchronous tracking-by-detection on adaptive time surfaces for event-based object tracking,” in Proceedings of the 27th ACM International Conference on Multimedia, 2019, pp. 473–481

    2019

  18. [26]

    Object tracking by jointly exploiting frame and event domain,

    J. Zhang, X. Yang, Y . Fu, X. Wei, B. Yin, and B. Dong, “Object tracking by jointly exploiting frame and event domain,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2021, pp. 13 043–13 052. IEEE TRANSACTIONS ON ***, 2025 12

    2021

  19. [27]

    Event stream-based visual object tracking: A high-resolution bench- mark dataset and a novel baseline,

    X. Wang, S. Wang, C. Tang, L. Zhu, B. Jiang, Y . Tian, and J. Tang, “Event stream-based visual object tracking: A high-resolution bench- mark dataset and a novel baseline,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 19 248–19 257

    2024

  20. [28]

    Eva-clip: Improved training techniques for clip at scale,

    Q. Sun, Y . Fang, L. Wu, X. Wang, and Y . Cao, “Eva-clip: Improved training techniques for clip at scale,” arXiv preprint arXiv:2303.15389 , 2023

    2023 arXiv

  21. [29]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar et al. , “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  22. [30]

    Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality,

    W.-L. Chiang, Z. Li, Z. Lin, Y . Sheng, Z. Wu, H. Zhang, L. Zheng, S. Zhuang, Y . Zhuang, J. E. Gonzalez et al., “Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality,” See https://vicuna. lmsys. org (accessed 14 April 2023) , vol. 2, no. 3, p. 6, 2023

    2023

  23. [31]

    Toward understand- ing wordart: Corner-guided transformer for scene text recognition,

    X. Xie, L. Fu, Z. Zhang, Z. Wang, and X. Bai, “Toward understand- ing wordart: Corner-guided transformer for scene text recognition,” in European conference on computer vision. Springer, 2022, pp. 303–321

    2022

  24. [32]

    Icdar 2015 competition on robust reading,

    D. Karatzas, L. Gomez-Bigorda, A. Nicolaou, S. Ghosh, A. Bagdanov, M. Iwamura, J. Matas, L. Neumann, V . R. Chandrasekhar, S. Lu et al., “Icdar 2015 competition on robust reading,” in 2015 13th international conference on document analysis and recognition (ICDAR) . IEEE, 2015,...

    2015

  25. [33]

    Large-scale multi-modal pre-trained models: A comprehensive survey,

    X. Wang, G. Chen, G. Qian, P. Gao, X.-Y . Wei, Y . Wang, Y . Tian, and W. Gao, “Large-scale multi-modal pre-trained models: A comprehensive survey,” Machine Intelligence Research , vol. 20, no. 4, pp. 447–482, 2023

    2023

  26. [34]

    Towards reasoning in large language models: A survey,

    J. Huang and K. C.-C. Chang, “Towards reasoning in large language models: A survey,” in Findings of the Association for Computational Linguistics: ACL 2023 , 2023, pp. 1049–1065

    2023

  27. [35]

    Scene text detection and recognition: The deep learning era,

    S. Long, X. He, and C. Yao, “Scene text detection and recognition: The deep learning era,” International Journal of Computer Vision , vol. 129, no. 1, pp. 161–184, 2021

    2021

  28. [36]

    End-to-end scene text recog- nition,

    K. Wang, B. Babenko, and S. Belongie, “End-to-end scene text recog- nition,” in 2011 International conference on computer vision . IEEE, 2011, pp. 1457–1464

    2011

  29. [37]

    An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition,

    B. Shi, X. Bai, and C. Yao, “An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition,” IEEE transactions on pattern analysis and machine intelligence, vol. 39, no. 11, pp. 2298–2304, 2016

    2016

  30. [38]

    Scene text recognition from two-dimensional perspective,

    M. Liao, J. Zhang, Z. Wan, F. Xie, J. Liang, P. Lyu, C. Yao, and X. Bai, “Scene text recognition from two-dimensional perspective,” in Proceedings of the AAAI conference on artificial intelligence , vol. 33, no. 01, 2019, pp. 8714–8721

    2019

  31. [39]

    Spotlight text detector: Spotlight on candidate regions like a camera,

    X. Han, J. Gao, C. Yang, Y . Yuan, and Q. Wang, “Spotlight text detector: Spotlight on candidate regions like a camera,” IEEE Transactions on Multimedia, 2024

    2024

  32. [40]

    Multi-modal in-context learning makes an ego-evolving scene text recognizer,

    Z. Zhao, J. Tang, C. Lin, B. Wu, C. Huang, H. Liu, X. Tan, Z. Zhang, and Y . Xie, “Multi-modal in-context learning makes an ego-evolving scene text recognizer,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 15 567–15 576

    2024

  33. [41]

    Self- supervised character-to-character distillation for text recognition,

    T. Guan, W. Shen, X. Yang, Q. Feng, Z. Jiang, and X. Yang, “Self- supervised character-to-character distillation for text recognition,” in 2023 IEEE/CVF International Conference on Computer Vision (ICCV) , 2023, pp. 19 416–19 427

    2023

  34. [42]

    Self- supervised implicit glyph attention for text recognition,

    T. Guan, C. Gu, J. Tu, X. Yang, Q. Feng, Y . Zhao, and W. Shen, “Self- supervised implicit glyph attention for text recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, 2023, pp. 15 285–15 294

    2023

  35. [43]

    Cdistnet: Perceiving multi-domain character distance for robust text recognition,

    T. Zheng, Z. Chen, S. Fang, H. Xie, and Y .-G. Jiang, “Cdistnet: Perceiving multi-domain character distance for robust text recognition,” International Journal of Computer Vision , vol. 132, no. 2, pp. 300–318, 2024

    2024

  36. [44]

    V olter: Visual collaboration and dual-stream fusion for scene text recognition,

    J.-N. Li, X.-Q. Liu, X. Luo, and X.-S. Xu, “V olter: Visual collaboration and dual-stream fusion for scene text recognition,” IEEE Transactions on Multimedia, 2024

    2024

  37. [45]

    Image as a language: Revisiting scene text recognition via balanced, unified and synchronized vision-language reasoning network,

    J. Wei, H. Zhan, Y . Lu, X. Tu, B. Yin, C. Liu, and U. Pal, “Image as a language: Revisiting scene text recognition via balanced, unified and synchronized vision-language reasoning network,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 6, 202...

    2024

  38. [46]

    Multi-modal text recognition networks: Interactive enhancements between visual and semantic features,

    B. Na, Y . Kim, and S. Park, “Multi-modal text recognition networks: Interactive enhancements between visual and semantic features,” in European Conference on Computer Vision . Springer, 2022, pp. 446– 463

    2022

  39. [47]

    Levenshtein ocr,

    C. Da, P. Wang, and C. Yao, “Levenshtein ocr,” in European Conference on Computer Vision . Springer, 2022, pp. 322–338

    2022

  40. [48]

    Read like humans: Autonomous, bidirectional and iterative language modeling for scene text recognition,

    S. Fang, H. Xie, Y . Wang, Z. Mao, and Y . Zhang, “Read like humans: Autonomous, bidirectional and iterative language modeling for scene text recognition,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2021, pp. 7098–7107

    2021

  41. [49]

    Language models are unsupervised multitask learners,

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al., “Language models are unsupervised multitask learners,” OpenAI blog , vol. 1, no. 8, p. 9, 2019

    2019

  42. [50]

    Language mod- els are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., “Language mod- els are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020

    1901

  43. [51]

    Palm: Scal- ing language modeling with pathways,

    A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann et al., “Palm: Scal- ing language modeling with pathways,” Journal of Machine Learning Research, vol. 24, no. 240, pp. 1–113, 2023

    2023

  44. [52]

    Training language models to follow instructions with human feedback,

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al. , “Training language models to follow instructions with human feedback,” Advances in neural information processing systems , vol. 35, pp. 27 730–27 744, 2022

    2022

  45. [53]

    Linin: Logic integrated neural inference network for explanatory visual question answering,

    D. Xue, S. Qian, Q. Fang, and C. Xu, “Linin: Logic integrated neural inference network for explanatory visual question answering,” IEEE Transactions on Multimedia , vol. 27, pp. 16–27, 2025

    2025

  46. [54]

    Large lan- guage models are zero-shot reasoners,

    T. Kojima, S. S. Gu, M. Reid, Y . Matsuo, and Y . Iwasawa, “Large lan- guage models are zero-shot reasoners,” Advances in neural information processing systems, vol. 35, pp. 22 199–22 213, 2022

    2022

  47. [55]

    Tree of thoughts: Deliberate problem solving with large language models,

    S. Yao, D. Yu, J. Zhao, I. Shafran, T. Griffiths, Y . Cao, and K. Narasimhan, “Tree of thoughts: Deliberate problem solving with large language models,” Advances in neural information processing systems , vol. 36, pp. 11 809–11 822, 2023

    2023

  48. [56]

    Topologies of reasoning: Demystifying chains, trees, and graphs of thoughts,

    M. Besta, F. Memedi, Z. Zhang, R. Gerstenberger, N. Blach, P. Nyczyk, M. Copik, G. Kwasniewski, J. Müller, L. Gianinazzi et al., “Topologies of reasoning: Demystifying chains, trees, and graphs of thoughts,”CoRR, 2024

    2024

  49. [57]

    Self-consistency improves chain of thought reasoning in language models,

    X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, S. Narang, A. Chowdh- ery, and D. Zhou, “Self-consistency improves chain of thought reasoning in language models,” arXiv preprint arXiv:2203.11171 , 2022

    2022 arXiv

  50. [58]

    Improve vision language model chain-of-thought reasoning,

    R. Zhang, B. Zhang, Y . Li, H. Zhang, Z. Sun, Z. Gan, Y . Yang, R. Pang, and Y . Yang, “Improve vision language model chain-of-thought reasoning,” arXiv preprint arXiv:2410.16198 , 2024

    2024 arXiv

  51. [59]

    Minigpt-4: Enhancing vision-language understanding with advanced large language models,

    D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny, “Minigpt-4: Enhancing vision-language understanding with advanced large language models,” arXiv preprint arXiv:2304.10592 , 2023

    2023 arXiv

  52. [60]

    Event- based vision: A survey,

    G. Gallego, T. Delbrück, G. Orchard, C. Bartolozzi, B. Taba, A. Censi, S. Leutenegger, A. J. Davison, J. Conradt, K. Daniilidis et al., “Event- based vision: A survey,” IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 1, pp. 154–180, 2020

    2020

  53. [61]

    Evcslr: Event-guided continuous sign language recognition and benchmark,

    Y . Jiang, Y . Wang, S. Li, Y . Zhang, Q. Guo, Q. Chu, and Y . Gao, “Evcslr: Event-guided continuous sign language recognition and benchmark,” IEEE Transactions on Multimedia , 2024

    2024

  54. [62]

    Masked autoen- coders in 3d point cloud representation learning,

    J. Jiang, X. Lu, L. Zhao, R. Dazaley, and M. Wang, “Masked autoen- coders in 3d point cloud representation learning,” IEEE Transactions on Multimedia, 2023

    2023

  55. [63]

    Hardvs: Revisiting human activity recognition with dynamic vision sensors,

    X. Wang, Z. Wu, B. Jiang, Z. Bao, L. Zhu, G. Li, Y . Wang, and Y . Tian, “Hardvs: Revisiting human activity recognition with dynamic vision sensors,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 6, 2024, pp. 5615–5623

    2024

  56. [64]

    Semantic-aware frame-event fusion based pattern recognition via large vision–language models,

    D. Li, J. Jin, Y . Zhang, Y . Zhong, Y . Wu, L. Chen, X. Wang, and B. Luo, “Semantic-aware frame-event fusion based pattern recognition via large vision–language models,” Pattern Recognition , vol. 158, p. 111080, 2025

    2025

  57. [65]

    Scene text recognition with permuted autoregressive sequence models,

    D. Bautista and R. Atienza, “Scene text recognition with permuted autoregressive sequence models,” in European conference on computer vision. Springer, 2022, pp. 178–196

    2022

  58. [66]

    Multi-granularity prediction for scene text recognition,

    P. Wang, C. Da, and C. Yao, “Multi-granularity prediction for scene text recognition,” in European Conference on Computer Vision . Springer, 2022, pp. 339–355

    2022

  59. [67]

    Lister: Neighbor decoding for length-insensitive scene text recognition,

    C. Cheng, P. Wang, C. Da, Q. Zheng, and C. Yao, “Lister: Neighbor decoding for length-insensitive scene text recognition,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 19 541–19 551

    2023

  60. [68]

    Reading and writing: Discriminative and generative modeling for self-supervised text recognition,

    M. Yang, M. Liao, P. Lu, J. Wang, S. Zhu, H. Luo, Q. Tian, and X. Bai, “Reading and writing: Discriminative and generative modeling for self-supervised text recognition,” in Proceedings of the 30th ACM International Conference on Multimedia , 2022, pp. 4214–4223. IEEE TRANSACT...

    2022

  61. [69]

    Esim: an open event camera simulator,

    H. Rebecq, D. Gehrig, and D. Scaramuzza, “Esim: an open event camera simulator,” in Conference on robot learning . PMLR, 2018, pp. 969– 982

    2018

  62. [70]

    Decoupled weight decay regularization,

    I. Loshchilov, “Decoupled weight decay regularization,” arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  63. [71]

    Synthetic data and artificial neural networks for natural scene text recognition,

    M. Jaderberg, K. Simonyan, A. Vedaldi, and A. Zisserman, “Synthetic data and artificial neural networks for natural scene text recognition,” arXiv preprint arXiv:1406.2227 , 2014

    2014 arXiv

  64. [72]

    Synthetic data for text localisation in natural images,

    A. Gupta, A. Vedaldi, and A. Zisserman, “Synthetic data for text localisation in natural images,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 2315–2324

    2016

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.