Pith. sign in

REVIEW 3 major objections 6 minor 57 references

Vision-language models often rewrite imperfect text into more plausible words instead of transcribing what is actually written—a failure mode that clean-text OCR benchmarks cannot detect.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 12:40 UTC pith:6XVM2WQN

load-bearing objection Solid behavioral result, oversold mechanism—worth a serious referee. the 3 major comments →

arxiv 2607.21617 v1 pith:6XVM2WQN submitted 2026-05-29 cs.AI cs.CL

Do VLMs Read or Rewrite? On Transcription Faithfulness in Vision-Language Models

Gwang Gook Lee , Kenan Emir Ak , Jay Mohta , Yan Xu , Dimitrios Dimitriadis This is my paper
classification cs.AI cs.CL
keywords vision-language modelsOCR faithfulnesstranscriptiontext perturbationlanguage priorFaithC4representation similarityrewriting behavior
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that vision-language models (VLMs) treat imperfect text as a prediction problem rather than a transcription problem: when a word contains a typo, a scramble, or a visually ambiguous character, the model often outputs the word it expects to see instead of the word that is written. To demonstrate this, the authors build FaithC4, a multilingual benchmark of 1,455 rendered document pages in English, Chinese, and Korean, perturbing a small fraction of words and measuring word error rate degradation across 15 systems. The result is a consistent ordering: traditional OCR degrades by under 0.6 points, OCR-specialized VLMs by 0.2–2 points, and general-purpose VLMs by up to 4.5 points on English. Layer-by-layer probing of one model shows rewriting is gated by how much the perturbation shifts the word's internal representation, and that short words are rewritten far more often than long ones, with a sharp cutoff at 8 characters. The paper matters because clean-text OCR benchmarks cannot see this failure mode, and domains that require literal transcription—legal records, medical notes, historical manuscripts—would silently get 'corrected' output.

Core claim

The paper establishes that VLMs are not faithful transcribers of imperfect text. Under controlled perturbations of rendered documents, general-purpose VLMs degrade by up to 4.5 WER points on English, OCR-specialized VLMs by 0.2–2 points, and traditional OCR by under 0.6 points, with the same three-tier ordering in Chinese and Korean. Probing Qwen3-VL-4B layer by layer shows rewriting fires only when a perturbed word's final-layer feed-forward representation stays close to the original encoding: top-quartile similarity samples are rewritten at 4.66%, bottom-quartile samples never, with attention playing no decisive role. Word length is a step function: 4–6 character words are rewritten up to

What carries the argument

The central instruments are two. First, FaithC4—a benchmark of 1,455 single-page documents rendered from perturbed text (scramble, random substitution, visually similar substitution) in English, Chinese, and Korean, using the perturbed text as ground truth so that any output matching the original unperturbed word counts as a 'rewrite.' Second, a layer-wise probing method on Qwen3-VL-4B that records the cosine similarity between feed-forward network (FFN) representations of the original and perturbed inputs at the final token, together with attention to the highlighted word's image patches; the FFN similarity at the final layer is the quantity that gates whether rewriting occurs. The probe is

Load-bearing premise

The load-bearing premise for the mechanistic explanation is that the representation-similarity gating observed in one model (Qwen3-VL-4B) with a single-word probe is the actual cause of rewriting across architectures; the paper itself notes that internal dynamics may differ across models, so if that correlation does not hold in other VLMs, the explanation fails even if the behavioral degradation remains.

What would settle it

A direct test: run the same highlighted-word probe on a second general-purpose VLM family (e.g., a different architecture) and check whether rewritten cases are again confined to the top FFN-similarity quartile. If a model rewrites words whose final-layer representation has diverged far from the original, or fails to rewrite words whose representation stays close, the gating claim is falsified. A simpler behavioral check: find any general-purpose VLM whose WER degradation under scramble perturbation is below 0.6 points on English, matching traditional OCR, which would break the three-tier orde

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Clean-text OCR benchmarks systematically overstate VLM transcription faithfulness, because they contain no imperfect text for rewriting to act on.
  • General-purpose VLMs are a poor default for literal-transcription tasks such as legal, medical, or historical document processing; the paper concludes traditional OCR or carefully selected OCR-specialized VLMs remain the safest choice.
  • A small corruption rate has outsized cost: corrupting about 5% of words multiplies errors on unperturbed text by 5–10× in general VLMs, so degradation cannot be predicted from perturbation rate alone.
  • Explicit no-correction prompting roughly halves the degradation on tested models but does not eliminate it, and slightly raises baseline WER.
  • Rewriting requires the language prior to recognize a plausible target: random substitutions that share no lexical overlap are never rewritten, so the failure mode is specifically about recovering known words.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the representation-gating mechanism generalizes, then decoding-time interventions that push the final-layer representation away from the clean encoding—rather than prompt instructions—might be a more direct mitigation than the no-correction prompt tested here.
  • The 7-to-8-character cutoff suggests a testable prediction for other scripts and models: rewriting should vanish for words whose perturbation exceeds the model's typical 'neighborhood' size in representation space; measuring that neighborhood could predict rewrite rates without running probes.
  • The same mechanism that produces rewriting may explain context leakage (outputting a salient document word instead of the target), implying that both errors are one phenomenon—representation collapse toward the closest known token—rather than separate failure modes.
  • For downstream tasks like document QA, faithful transcription may not always be the goal; rewriting could sometimes improve answer quality, so deployments should decide explicitly whether literalness or plausibility is the objective.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper studies whether vision-language models (VLMs) faithfully transcribe imperfect text or silently rewrite it into more plausible forms. It introduces FaithC4, a multilingual benchmark of 1,455 synthetic single-page documents in English, Chinese, and Korean, with three perturbation families (scramble, random, visual) at 8% of eligible words. The authors evaluate 15 systems across three categories—general-purpose VLMs, OCR-specialized VLMs, and traditional OCR—reporting WER/CER degradation relative to clean baselines, an error-type taxonomy distinguishing rewrites from ordinary OCR errors, and evidence of non-local error propagation to unperturbed words. They also probe Qwen3-VL-4B layer-by-layer to argue that rewriting is gated by the cosine similarity of final-layer FFN representations between original and perturbed inputs, and that word length affects rewriting with a sharp cutoff at 8 characters. The central behavioral claim is that general-purpose VLMs are substantially less faithful transcribers than OCR-specialized models and traditional OCR when text is imperfect.

Significance. If the behavioral finding holds, the paper addresses an important and underexplored failure mode: clean-text OCR benchmarks overstate VLM transcription reliability for degraded or noisy real-world documents. The controlled perturbation benchmark, the 15-system comparison, and the error taxonomy are valuable contributions. The non-local error propagation result is practically important. The mechanistic probing, while suggestive, is preliminary and not yet at the level of evidence claimed in the abstract and conclusion. The paper also ships falsifiable empirical results: the three-tier ordering of degradation, the rewrite-rate differences, and the word-length cutoff can be tested by other groups. Overall this is a solid empirical contribution whose significance is currently tempered by overgeneralized interpretive claims.

major comments (3)
  1. [Abstract / §4.1, Table 2] The abstract states that OCR-specialized VLMs degrade by 0.2–2 points, but Table 2 reports olmOCR-2-7B at +2.32 pp under scramble, which is outside this range. Also, PaddleOCR-VL-1.5 (OCR-specialized) degrades more than Qwen3.5-4B (general-purpose) under random substitution (+1.45 vs +0.74), so the three-tier ordering is not exception-free even in English. Please either report per-model ranges with exceptions or soften the category-level claim to 'most OCR-specialized VLMs'.
  2. [§4.1 / Table 3 / Conclusion] The paper claims the three-tier ordering (traditional OCR < OCR-specialized VLM < general-purpose VLM) 'holds in all three languages.' Table 3 contradicts this for Chinese: docTR degrades by +9.31 pp under visual substitution, higher than most general VLMs (e.g., Gemma4-E2B +6.66, InternVL3.5 +7.72), and Tesseract under random degrades +4.63 pp, comparable to Qwen3.5-4B +4.82. Section 6 acknowledges traditional OCR has limited multilingual support, but the conclusion still asserts the cross-lingual ordering. The authors should restrict the claim to English or provide a per-script analysis that accounts for baseline failures/truncation (e.g., docTR's high failure rate on Chinese in Table A.3).
  3. [§4.2 / Tables 8–9 / §6] The mechanistic conclusion that rewriting is gated by layer-35 FFN representation similarity is based on a single model (Qwen3-VL-4B), a single-word highlighted probe, and very few rewritten samples: Table 9 reports 19 rewritten words in Q4, 1 in Q3, and 0 in Q1/Q2. No confidence intervals or significance tests are reported, and the quartile partition is defined on the same similarity variable whose association with rewriting is being tested. Moreover, the probe rewrite rate (4.9–6.0%, Table 8) is an order of magnitude below the full-document rewrite rates (29–65%, Table 5), so the mechanism may not transfer to the setting that motivates the paper. The paper itself concedes that 'the internal dynamics may differ across architectures' (§6). At minimum, the abstract and conclusion should rephrase this as a hypothesis specific to Qwen3-VL, or the authors should add a second model and a prop
minor comments (6)
  1. [Abstract] Missing space: 'We introduceFaithC4' should be 'We introduce FaithC4'.
  2. [Table 6] The column arrangement for Orig/Scramble/Random/Visual with paired Pert/Non-pert columns is very difficult to parse. Please reformat, e.g., with separate subheaders or a wide-to-long layout.
  3. [References / Table A.2] The PaddleOCR-VL-1.5 reference describes the model as 0.9B, while Table A.2 and Table A.3 list it as 1.5B. Please reconcile.
  4. [§3.3, Eq. (1)] Equation (1) defines WER with N as the reference token count, but for Chinese the metric is character-level. Please state explicitly that N is the reference character count in that case.
  5. [Figure 2] The caption mentions panels (a) and (b), but the panels are not labeled in the figure. Add panel labels.
  6. [Table 9] The 'Highlight Attn.' values (e.g., 0.00012) are not directly comparable to the attention ratios in Table 7. Consider reporting the same ratio format for consistency.

Circularity Check

0 steps flagged

No significant circularity: behavioral findings are externally anchored measurements; the mechanistic probe is correlational, not derivational.

full rationale

The paper's central claims are empirical and externally anchored. The benchmark starts from raw text, applies perturbations, renders images, and declares the perturbed text as ground truth: 'The perturbed text is the ground truth, which lets us measure whether models faithfully transcribe what they see or silently "rewrite" it.' The 'Rewritten' category is defined as a prediction matching the original unperturbed word, an independent reference constructed before any model is run. No parameter is fitted to a subset of the model outputs and then used to predict the same or a closely related quantity; the three-tier ordering (traditional OCR < OCR-specialized VLM < general-purpose VLM) is a direct measurement across 15 systems. The mechanistic probe is correlational: the paper computes FFN cosine similarity and attention, then observes co-occurrence with rewriting; it does not derive rewriting from the similarity by construction. The acknowledged limitation that probing covers only Qwen3-VL and that 'the internal dynamics may differ across architectures' affects generalization of the explanation, not circularity. No load-bearing self-citation or imported uniqueness theorem is invoked. Accordingly, no circular step is identified.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

No new physical or conceptual entities are introduced; the benchmark FaithC4 is a dataset, not an invented entity in the sense of a new particle or force. The free parameters listed are experimental design choices, not fitted parameters. The axioms are the key assumptions about representativeness and interpretability of neural activations.

free parameters (3)
  • perturbation_rate = 8% of eligible words
    Hand-chosen rate; affects the magnitude of degradation but not the qualitative three-tier ordering. The effective rate differs by language (4.7% EN, 7.9% ZH, 2.2% KO).
  • min_word_length = 4 characters
    Eligibility filter for perturbation; reduces perturbed-word counts in Korean (2.2%) versus English (4.7%) and affects the word-length analysis.
  • visual_confusion_map = Appendix Table A.1
    Manually curated character substitutions (e.g., m→rn, d→cl, 大→太); the specific confusions influence visual-perturbation results and are a design choice rather than fitted to data.
axioms (4)
  • domain assumption Rendering with 9pt serif font and 1.2 line-height produces document images representative of real single-page documents.
    The paper uses synthetically rendered PDFs; Limitations admit they may not capture the visual complexity of scanned documents or low-resolution capture.
  • domain assumption The three perturbation families (scramble, random, visual) represent the kinds of imperfect text that matter for OCR deployment.
    Limitations explicitly exclude natural degradations such as scanning artifacts, handwriting variation, and low-resolution capture, so this assumption is load-bearing for the applied conclusion.
  • domain assumption Cosine similarity between FFN outputs at the last input token measures the perturbation's effect on the model's contextual representation.
    The probe interprets layer-35 FFN similarity as 'how far the perturbation has moved its internal representation' (Section 4.2); this is an interpretive assumption about neural activations.
  • standard math WER and EDS are appropriate metrics for transcription faithfulness.
    Standard edit-distance metrics; no controversy, but they are an operational choice that affects all numeric results.

pith-pipeline@v1.3.0-alltime-deepseek · 19453 in / 11755 out tokens · 103660 ms · 2026-08-02T12:40:55.926335+00:00 · methodology

0 comments
read the original abstract

Vision Language Models (VLMs) are increasingly used in place of traditional OCR pipelines for document understanding. In this paper, we show they do not always act as faithful transcribers: when text is imperfect, they often tend to rewrite it into a more plausible form - a behavior that clean-text OCR benchmarks cannot detect. We introduce FaithC4, a multilingual perturbation benchmark of 1,455 single-page documents (English, Chinese, Korean) with three perturbation families: scramble, random substitution, and visually similar substitution. We use the benchmark to evaluate 15 systems spanning general-purpose VLMs, OCR-specialized VLMs, and traditional OCR pipelines. These three categories differ in WER degradation under perturbation: general-purpose VLMs degrade by up to 4.5 points, OCR-specialized VLMs by 0.2-2 points, and traditional OCR by less than 0.6 points on English. Probing Qwen3-VL-4B layer-by-layer, we identify a consistent pattern: rewriting fires only when a perturbed word's final layer FFN representation stays close to the original encoding; when the representation diverges sufficiently, the model transcribes faithfully. Word length affects rewriting rate: short words (4-6 characters) are rewritten up to 10% of the time, with a sharp cutoff at 8 characters above which rewriting drops to 0%.

Figures

Figures reproduced from arXiv: 2607.21617 by Dimitrios Dimitriadis, Gwang Gook Lee, Jay Mohta, Kenan Emir Ak, Yan Xu.

Figure 1
Figure 1. Figure 1: Dataset creation process. Source text from C4 is perturbed at the word level (scramble, visual, or random), [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Model failure counts by (a) language and (b) [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: WER degradation (∆pp) across models and perturbation types for English. its language prior. Close predictions fall within edit distance 2 of the perturbed text, the regime of single-character substitutions, deletions, or trans￾positions that any OCR system produces on noisy input. Moderate predictions share at least half their characters with the perturbed text (edit-distance ratio ≥ 0.5) but are not close… view at source ↗
Figure 4
Figure 4. Figure 4: Error amplification on non-perturbed words. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Probing setup. A single word is highlighted in [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Per-layer FFN activation similarity and high [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

57 extracted references · 14 linked inside Pith

  1. [1]

    FirstName LastName , title =

  2. [2]

    FirstName Alpher , title =

  3. [3]

    Journal of Foo , volume = 13, number = 1, pages =

    FirstName Alpher and FirstName Fotheringham-Smythe , title =. Journal of Foo , volume = 13, number = 1, pages =

  4. [4]

    Journal of Foo , volume = 14, number = 1, pages =

    FirstName Alpher and FirstName Fotheringham-Smythe and FirstName Gamow , title =. Journal of Foo , volume = 14, number = 1, pages =

  5. [5]

    Trends in Cognitive Sciences , volume =

    Does the human mind read words as a whole? , author =. Trends in Cognitive Sciences , volume =. 2004 , doi =

    2004

  6. [6]

    FirstName Alpher and FirstName Gamow , title =

  7. [7]

    arXiv preprint arXiv:2601.20552 , year=

    DeepSeek-OCR 2: Visual Causal Flow , author=. arXiv preprint arXiv:2601.20552 , year=

    arXiv

  8. [8]

    ZAI Organization , title =

  9. [9]

    5: A decoupled vision-language model for efficient high-resolution document parsing , author=

    Mineru2. 5: A decoupled vision-language model for efficient high-resolution document parsing , author=. The 64th Annual Meeting of the Association for Computational Linguistics--Industry Track , year=

  10. [10]

    arXiv preprint arXiv:2409.05137 , year=

    READoc: A Unified Benchmark for Realistic Document Structured Extraction , author=. arXiv preprint arXiv:2409.05137 , year=

    Pith/arXiv arXiv

  11. [11]

    arXiv preprint arXiv:2308.12966 , year=

    Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond , author=. arXiv preprint arXiv:2308.12966 , year=

    Pith/arXiv arXiv

  12. [12]

    arXiv preprint arXiv:2511.21631 , year=

    Qwen3-vl technical report , author=. arXiv preprint arXiv:2511.21631 , year=

    Pith/arXiv arXiv

  13. [13]

    Belval, Edouard and Delteil, Thomas and Schade, Martin and Radhakrishna, Srividhya , title =

  14. [14]

    2026 , eprint=

    PaddleOCR-VL-1.5: Towards a Multi-Task 0.9B VLM for Robust In-the-Wild Document Parsing , author=. 2026 , eprint=

    2026

  15. [15]

    Proceedings of the IEEE international conference on computer vision , pages=

    Vqa: Visual question answering , author=. Proceedings of the IEEE international conference on computer vision , pages=

  16. [16]

    2021 , eprint=

    Learning Transferable Visual Models From Natural Language Supervision , author=. 2021 , eprint=

    2021

  17. [17]

    2024 , eprint=

    InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks , author=. 2024 , eprint=

    2024

  18. [18]

    5: Advancing open-source multimodal models in versatility, reasoning, and efficiency , author=

    Internvl3. 5: Advancing open-source multimodal models in versatility, reasoning, and efficiency , author=. arXiv preprint arXiv:2508.18265 , year=

    Pith/arXiv arXiv

  19. [19]

    Visual Instruction Tuning , url =

    Liu, Haotian and Li, Chunyuan and Wu, Qingyang and Lee, Yong Jae , booktitle =. Visual Instruction Tuning , url =

  20. [20]

    arXiv preprint arXiv:2303.08774 , year=

    Gpt-4 technical report , author=. arXiv preprint arXiv:2303.08774 , year=

    Pith/arXiv arXiv

  21. [21]

    2023 , eprint=

    Nougat: Neural Optical Understanding for Academic Documents , author=. 2023 , eprint=

    2023

  22. [22]

    OCRBench: on the hidden mystery of OCR in large multimodal models , volume=

    Liu, Yuliang and Li, Zhang and Huang, Mingxin and Yang, Biao and Yu, Wenwen and Li, Chunyuan and Yin, Xu-Cheng and Liu, Cheng-Lin and Jin, Lianwen and Bai, Xiang , year=. OCRBench: on the hidden mystery of OCR in large multimodal models , volume=. Science China Information Sciences , publisher=. doi:10.1007/s11432-024-4235-6 , number=

    doi:10.1007/s11432-024-4235-6

  23. [23]

    2025 , eprint=

    OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning , author=. 2025 , eprint=

    2025

  24. [24]

    2021 , eprint=

    DocVQA: A Dataset for VQA on Document Images , author=. 2021 , eprint=

    2021

  25. [25]

    2024 , eprint=

    OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations , author=. 2024 , eprint=

    2024

  26. [26]

    2020 , eprint=

    On Faithfulness and Factuality in Abstractive Summarization , author=. 2020 , eprint=

    2020

  27. [27]

    Survey of Hallucination in Natural Language Generation , volume=

    Ji, Ziwei and Lee, Nayeon and Frieske, Rita and Yu, Tiezheng and Su, Dan and Xu, Yan and Ishii, Etsuko and Bang, Ye Jin and Madotto, Andrea and Fung, Pascale , year=. Survey of Hallucination in Natural Language Generation , volume=. ACM Computing Surveys , publisher=. doi:10.1145/3571730 , number=

    doi:10.1145/3571730

  28. [28]

    2023 , eprint=

    Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks , author=. 2023 , eprint=

    2023

  29. [29]

    Forty-second International Conference on Machine Learning , year=

    Textural or textual: How vision-language models read text in images , author=. Forty-second International Conference on Machine Learning , year=

  30. [30]

    Linux Journal , volume=

    Tesseract: an open-source optical character recognition engine , author=. Linux Journal , volume=. 2007 , publisher=

    2007

  31. [31]

    National Science Review , volume=

    A survey on multimodal large language models , author=. National Science Review , volume=. 2024 , publisher=

    2024

  32. [32]

    IEEE transactions on pattern analysis and machine intelligence , volume=

    An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition , author=. IEEE transactions on pattern analysis and machine intelligence , volume=. 2016 , publisher=

    2016

  33. [33]

    LayoutLM: Pre-training of Text and Layout for Document Image Understanding , url=

    Xu, Yiheng and Li, Minghao and Cui, Lei and Huang, Shaohan and Wei, Furu and Zhou, Ming , year=. LayoutLM: Pre-training of Text and Layout for Document Image Understanding , url=. doi:10.1145/3394486.3403172 , booktitle=

    arXiv

  34. [34]

    arXiv preprint arXiv:2501.15558 , year=

    Ocean-ocr: Towards general ocr application via a vision-language model , author=. arXiv preprint arXiv:2501.15558 , year=

    Pith/arXiv arXiv

  35. [35]

    arXiv preprint arXiv:2502.18443 , year=

    olmocr: Unlocking trillions of tokens in pdfs with vision language models , author=. arXiv preprint arXiv:2502.18443 , year=

    arXiv

  36. [36]

    arXiv preprint arXiv:2409.18839 , year=

    Mineru: An open-source solution for precise document content extraction , author=. arXiv preprint arXiv:2409.18839 , year=

    Pith/arXiv arXiv

  37. [37]

    arXiv preprint arXiv:2009.09941 , year=

    Pp-ocr: A practical ultra lightweight ocr system , author=. arXiv preprint arXiv:2009.09941 , year=

    Pith/arXiv arXiv 2009

  38. [38]

    Ninth international conference on document analysis and recognition (ICDAR 2007) , volume=

    An overview of the Tesseract OCR engine , author=. Ninth international conference on document analysis and recognition (ICDAR 2007) , volume=. 2007 , organization=

    2007

  39. [39]

    Proceedings of the 33rd ACM International Conference on Multimedia , pages=

    Deciphering Functions of Neurons in Vision-Language Models , author=. Proceedings of the 33rd ACM International Conference on Multimedia , pages=

  40. [40]

    Journal of machine learning research , volume=

    Exploring the limits of transfer learning with a unified text-to-text transformer , author=. Journal of machine learning research , volume=

  41. [41]

    2021 , publisher =

    docTR: Document Text Recognition , author=. 2021 , publisher =

    2021

  42. [42]

    2025 , howpublished =

    LightOnOCR-1B: End-to-End and Efficient Domain-Specific Vision-Language Models for OCR , author =. 2025 , howpublished =

    2025

  43. [43]

    arXiv preprint arXiv:2312.11805 , year=

    Gemini: a family of highly capable multimodal models , author=. arXiv preprint arXiv:2312.11805 , year=

    Pith/arXiv arXiv

  44. [44]

    ArXiv , year=

    Gemma 3 Technical Report , author=. ArXiv , year=

  45. [45]

    arXiv preprint arXiv:1711.02173 , year=

    Synthetic and natural noise both break neural machine translation , author=. arXiv preprint arXiv:1711.02173 , year=

    Pith/arXiv arXiv

  46. [46]

    Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages=

    Hotflip: White-box adversarial examples for text classification , author=. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages=

  47. [47]

    Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages=

    Combating adversarial misspellings with robust word recognition , author=. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages=

  48. [48]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    Unnatural error correction: Gpt-4 can almost perfectly handle unnatural scrambled text , author=. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    2023

  49. [49]

    arXiv preprint arXiv:2604.15804 , year=

    Qwen3.5-Omni Technical Report , author=. arXiv preprint arXiv:2604.15804 , year=

    Pith/arXiv arXiv

  50. [50]

    arXiv preprint arXiv:2510.17771 , year=

    Seeing but not believing: Probing the disconnect between visual attention and answer correctness in vlms , author=. arXiv preprint arXiv:2510.17771 , year=

    arXiv

  51. [51]

    arXiv preprint arXiv:2505.23941 , year=

    Vision language models are biased , author=. arXiv preprint arXiv:2505.23941 , year=

    Pith/arXiv arXiv

  52. [52]

    Findings of the Association for Computational Linguistics: NAACL 2025 , pages=

    Vlind-bench: Measuring language priors in large vision-language models , author=. Findings of the Association for Computational Linguistics: NAACL 2025 , pages=

    2025

  53. [53]

    Proceedings of the 2023 conference on empirical methods in natural language processing , pages=

    Evaluating object hallucination in large vision-language models , author=. Proceedings of the 2023 conference on empirical methods in natural language processing , pages=

    2023

  54. [54]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  55. [55]

    arXiv preprint arXiv:2311.07397 , year=

    Amber: An llm-free multi-dimensional benchmark for mllms hallucination evaluation , author=. arXiv preprint arXiv:2311.07397 , year=

    Pith/arXiv arXiv

  56. [56]

    2026 , howpublished=

    Gemma 4: Open Multimodal Models , author=. 2026 , howpublished=

    2026

  57. [57]

    arXiv preprint arXiv:2601.03267 , year=

    Openai gpt-5 system card , author=. arXiv preprint arXiv:2601.03267 , year=

    Pith/arXiv arXiv