Pith Number

pith:7YTNYMNU

pith:2023:7YTNYMNUELMI4SR5F3BAH7D5KB

not attested not anchored not stored refs resolved

OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models

Biao Yang, Cheng-Lin Liu, Chunyuan Li, Lianwen Jin, Mingxin Huang, Wenwen Yu, Xiang Bai, Xucheng Yin, Yuliang Liu, Zhang Li

OCRBench evaluates large multimodal models on 29 OCR datasets to expose their specific weaknesses in text recognition tasks.

arxiv:2305.07895 v7 · 2023-05-13 · cs.CV · cs.CL

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{7YTNYMNUELMI4SR5F3BAH7D5KB}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

To facilitate the assessment of Optical Character Recognition (OCR) capabilities in Large Multimodal Models, we propose OCRBench, a comprehensive evaluation benchmark. OCRBench contains 29 datasets, making it the most comprehensive OCR evaluation benchmark available.

C2weakest assumption

That the 29 chosen datasets together form a representative and non-redundant sample of all text-related visual challenges that large multimodal models will encounter in practice.

C3one line summary

OCRBench provides the largest evaluation suite yet for OCR capabilities in large multimodal models, revealing gaps in multilingual, handwritten, and mathematical text handling.

References

122 extracted · 122 resolved · 12 Pith anchors

[1] OpenAI. ChatGPT. https://openai.com/blog/chatgpt/, 2023 2023

[2] Gpt-4 technical report 2023

[3] LLaMA: Open and Efficient Foundation Language Models 2023 · arXiv:2302.13971

[4] Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-following llama model. https://github.co 2023

[5] Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality 2023

Formal links

2 machine-checked theorem links

Cited by

29 papers in Pith

Judge a Book by its Cover: Investigating Multi-Modal LLMs for Multi-Page Handwritten Document Transcription

AdaMMS: Model Merging for Heterogeneous Multimodal Large Language Models with Unsupervised Coefficient Optimization

DocAtlas: Multilingual Document Understanding Across 80+ Languages

Unlocking Dense Metric Depth Estimation in VLMs

Robust Checkpoint Selection for Multimodal LLMs via Agentic Evaluation and Stability-Aware Ranking

Receipt and verification

First computed	2026-05-17T23:38:14.445847Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

fe26dc31b422d88e4a3d2ec203fc7d5061286d2fad3a439a3f0fa3536dda8d6a

Aliases

arxiv: 2305.07895 · arxiv_version: 2305.07895v7 · doi: 10.48550/arxiv.2305.07895 · pith_short_12: 7YTNYMNUELMI · pith_short_16: 7YTNYMNUELMI4SR5 · pith_short_8: 7YTNYMNU

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/7YTNYMNUELMI4SR5F3BAH7D5KB \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: fe26dc31b422d88e4a3d2ec203fc7d5061286d2fad3a439a3f0fa3536dda8d6a

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "29d2b49f308511804352e18309295d4a3fa4576924d2b23a7b2a012b54df781f",
    "cross_cats_sorted": [
      "cs.CL"
    ],
    "license": "http://creativecommons.org/licenses/by-sa/4.0/",
    "primary_cat": "cs.CV",
    "submitted_at": "2023-05-13T11:28:37Z",
    "title_canon_sha256": "752f574bb06201a05cecfb174a1559187447237c80e2df7daf18a0cc7aad28f9"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2305.07895",
    "kind": "arxiv",
    "version": 7
  }
}