Lost in OCR Translation? Vision-Based Approaches to Robust Document Retrieval

Alexander Most; Ayan Biswas; Dan O'Malley; Joseph Winjum; Manish Bhattarai; Nishath Rajiv Ranasinghe; Shawn Jones

REVIEW 4 major objections 5 minor 23 references

Lost in OCR Translation? Vision-Based Approaches to Robust Document Retrieval

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Without task-specific fine-tuning, OCR-based retrieval-augmented generation outperforms vision-language retrieval on documents of varying quality.

desk verdict A useful new degraded-document benchmark and a credible retrieval comparison, but the paper's generation claim overreaches because the semantic evaluation is OCR-based for both pipelines. read the letter →

arxiv 2505.05666 v1 pith:SZDAG57X submitted 2025-05-08 cs.CV cs.AI

Alexander Most , Joseph Winjum , Ayan Biswas , Shawn Jones , Nishath Rajiv Ranasinghe , Dan O'Malley , Manish Bhattarai This is my paper · ORCID

classification cs.CVcs.AI

keywords retrieval-augmentedgenerationvision-languagemodelsOCRdocumentretrievaldegradationlateinteractionsemanticanswerevaluationDocDeg

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to settle which retrieval paradigm—direct visual embedding of document images or traditional OCR-then-embed—better serves retrieval-augmented generation on real scanned documents of mixed quality. It compares a ColPali-style vision pipeline (ColQwen2, 7B) with OCR pipelines that use Nougat or Llama 3.2 (90B) for text extraction followed by Qwen2 dense embeddings, on a new 4,196-document dataset with four hand-labeled degradation levels and a semantic question-answering benchmark. The central finding is that, without fine-tuning on the target corpus, OCR-based RAG achieves better retrieval and answer-generation scores at every degradation level, while vision-based retrieval only wins on the clean ViDoRe benchmark after being fine-tuned on it. The paper concludes that practitioners should prefer modern-OCR pipelines for heterogeneous scanned collections, accepting slower indexing and larger memory in exchange for accuracy and faster query-time latency.

What carries the argument

The load-bearing object is the paired pipeline comparison built around three components: the VLM pipeline embeds non-overlapping image patches with ColQwen2 and scores query–patch similarity through ColBERT-style late interaction; the OCR pipeline converts each page to text with Llama 3.2 or Nougat and scores pages by cosine similarity of Qwen2 text embeddings; and the evaluation stack measures retrieval with MRR, Recall@5, and NDCG@5, then measures downstream answer quality with Exact Match, BLEU, and ROUGE, then measures speed and memory. The DocDeg dataset (4,196 pages, four manually graded degradation levels, ten Llama-generated QA pairs per page) is the instrument that makes the comparison possible, and the ViDoRe/DocVQA subset is the clean-domain control that isolates fine-tuning effects.

What would settle it

Run a vision-language retriever at roughly 90B parameter scale (or fine-tune ColQwen2 on DocDeg) and evaluate it on the same degradation-level splits; if its MRR or ROUGE-L matches or beats the Llama-OCR pipeline without task-specific fine-tuning, the paper's central claim fails.

Watch

Extended reading notes

Core claim

The paper's core claim is that the robustness advantage commonly attributed to vision-language retrieval does not survive contact with unseen, degraded documents. On the DocDeg dataset, the OCR pipeline with Llama 3.2 OCR and Qwen2 embeddings outperformed the VLM pipeline (ColQwen2) on MRR, Recall@5, and NDCG@5 at all four degradation levels, and also produced higher Exact Match, BLEU, ROUGE-1, and ROUGE-L scores in end-to-end question answering. The VLM pipeline's only win came on the ViDoRe/DocVQA benchmark, where it had been fine-tuned on that benchmark's own query-page pairs—evidence that its strength is in-domain memorization rather than general visual robustness. The paper also reports that OCR-based retrieval is faster at query time despite slower indexing, while the VLM pipeline uses less memory.

Load-bearing premise

The conclusion that OCR-based RAG generalizes better rests on treating a 90B-parameter OCR model as the representative of OCR pipelines and a 7B vision model as the representative of VLM pipelines; if a comparably large vision encoder existed at acceptable cost, the ranking might reverse.

Editorial extensions

If this is right

In production RAG over scanned or heterogeneous document collections, a modern OCR pipeline should be the default choice when no fine-tuning budget exists for a vision retriever.
Vision-language retrieval, used out of the box, cannot be assumed to handle degraded documents better than OCR; its measured advantage is specific to the distribution it was fine-tuned on.
The OCR engine itself matters: Llama 3.2 OCR substantially beat Nougat OCR on the same retrieval tasks, so pipeline comparisons should report which OCR is used.
Query-time latency favors the OCR pipeline, while indexing time and memory favor the VLM pipeline; the choice is a resource trade-off, not a one-sided win.
Retrieval accuracy drops as degradation increases, but semantic answer quality does not decline monotonically, suggesting the generator compensates for some retrieval noise.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If a vision encoder at the same parameter scale as the 90B OCR model became practical, the generalization ranking could plausibly reverse; the paper's comparison bundles architecture choice with model size.
Because the VLM pipeline's answers were generated from OCR text of retrieved images, the 'end-to-end' comparison still contains an OCR step; a vision-language QA model used directly on retrieved images might narrow the semantic gap.
DocDeg could serve as a reusable robustness benchmark for future retrievers, especially for measuring zero-shot transfer across degradation levels and for testing whether fine-tuning closes the gap.
The near-identical Nougat and Llama scores on ViDoRe suggest that on clean documents the binding constraint is query–embedding semantic alignment rather than OCR fidelity, pointing future work toward better encoders rather than better OCR.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: the paper is worth reading for the DocDeg dataset and the retrieval comparison, but the central 'generation' claim is not backed by the experiment as run. The semantic evaluation in Table 3 uses OCR text for both pipelines, so it only tests which pages were retrieved, not whether a vision model can generate better answers from images directly.

What's genuinely new: DocDeg is a 4,196-page manually annotated degradation benchmark from real DOE documents, with four quality levels and 12 structural features. That is a useful resource for the document retrieval community. The paper is also the first to compare ColQwen2 against a modern Llama 3.2 90B OCR pipeline, rather than Tesseract. The retrieval protocol is careful: they report MRR, Recall@5, NDCG@5 across levels, and they include a slide-removal robustness check that changes the ranking on level-0 documents, which is honest. The retrieval result itself—Llama OCR pipeline beats ColQwen2 on degraded docs—is credible for the model sizes tested, and the ViDoRe control (ColQwen2 wins when fine-tuned) is a good sanity check.

The soft spots are real. First and most important: the abstract and conclusion say OCR-based approaches offer improved retrieval and generation performance. The retrieval half is supported; the generation half is not. Section 4.1 states that for semantic evaluation they fed the OCR text from the images retrieved by ColPali into the LLM. So Table 3 is a second retrieval measurement using a shared OCR+Qwen reader. A VLM-based QA model could plausibly do better, especially on degraded scans, and the paper's own future-work section admits this. The claim as written overreaches.

Second, the latency numbers are internally inconsistent: Table 4 gives VLM retrieval latency as 0.04252s, while Section 4.4 and the radar plot say 0.0010s. That is a factor of forty, and needs fixing before anyone trusts the efficiency trade-off.

Third, the model-size confound is acknowledged but not tested: 7B VLM vs 90B OCR. The practical argument that a 90B VLM is too slow may be right, but the conclusion is conditional on it. The paper should say this more loudly.

Minor: DocDeg is not actually released (no link or DOI); the QA prompts and annotation protocol should be included if the dataset is to be reusable.

Bottom line: the retrieval comparison and the dataset deserve referee time. The generation claim should either be removed or re-run with a vision-language QA model. With that fix, this would be a solid contribution to the document retrieval RAG literature.

Referee Report

4 major / 5 minor

Summary. The paper compares a vision-based RAG pipeline (ColPali/ColQwen2, 7B) against OCR-based pipelines (Nougat OCR and Llama 3.2 90B OCR followed by Qwen2 7B embeddings) on a new dataset, DocDeg, of 4,196 manually labeled documents at four degradation levels. It reports retrieval metrics (MRR, Recall@5, NDCG@5) and semantic answer metrics (EM, BLEU, ROUGE-1, ROUGE-L) under a no-fine-tuning protocol, plus a supplementary ViDoRe/DocVQA evaluation and computational efficiency measurements. The central claim is that OCR-based RAG generalizes better to unseen degraded documents than VLM-based retrieval, and that this advantage extends to generation quality.

Significance. If the central claim holds, this is a practically important result: it would tell RAG practitioners that OCR-dependent pipelines with strong OCR models are preferable to lightweight VLM-based retrieval for collections containing degraded scans, and it would temper claims made for ColPali-style direct visual embedding. The paper contributes a new public-domain-derived dataset with explicit degradation labels, a dual evaluation protocol, and a slide-removal robustness check. The retrieval comparison is, on its face, a reasonable out-of-the-box comparison of representative model sizes, and the supplementary ViDoRe experiment usefully shows that fine-tuning on the target benchmark changes the ranking. However, the paper's generation claim is not supported by the experiment as designed, because the 'VLM pipeline' answer generation is performed on OCR text rather than on images; and the computational efficiency analysis contains an internal contradiction that must be resolved before the trade-off conclusions can be trusted.

major comments (4)

[§4.1, §4.2, Table 3, §5] The claim that OCR-based RAG offers 'improved retrieval and generation performance in all evaluated settings' is not supported for the generation component. Section 4.1 states that for the semantic answer evaluation, 'we used the OCR text from the images retrieved by ColPali as context for the LLM, in order to enable a fair comparison across pipelines.' Thus Table 3 compares two OCR+LLM readers applied to different retrieved page sets; the VLM pipeline's generation is never tested. The observed BLEU/ROUGE/EM differences therefore reflect retrieval differences, not vision-based generation ability. The conclusion in Section 5 that OCR-based approaches offer improved 'generation performance' is an overstatement. The authors should either add a condition in which a vision-language QA model answers from retrieved images, or explicitly restrict the claim to retrieval performance and reframe Table 3 as a downstream-retrieval-utility measure.
[§4.3, Table 4, §4.4, Figure 4] The computational efficiency numbers are internally inconsistent. Table 4 reports VLM retrieval latency of 0.04252s/query versus OCR latency of 0.0311s/query, which means the OCR pipeline is faster at query time. However, Section 4.3 states that 'the VLM-based system achieved ... substantially reduced retrieval latency (0.0010s per query vs. 0.0311s per query),' and Section 4.4 repeats 'significantly faster retrieval latency (0.001s vs 0.031s).' The conclusion then says OCR pipelines are 'actually faster at query time compared to VLM-based pipelines like ColPali,' which agrees with Table 4 but contradicts the section 4.3/4.4 text. Since the latency comparison feeds directly into the radar plot and the practical guidance, these values must be reconciled and the correct measurement reported with a clear description of what is timed (end-to-end RAG query, retrieval-only, or embedding-only).
[§4.1, Table 2] The slide-removal robustness check in Table 2 contains implausible values that appear internally inconsistent with Table 1. For Level 3, VLM-Based Recall@5 is 0.3112 and MRR is 0.2098—identical to Table 1—but NDCG@5 changes from 0.2350 (Table 1) to 0.2997 (Table 2). For OCR-Based (Llama), Recall@5 is 0.5925 and MRR is 0.4520 (identical to Table 1), but NDCG@5 drops from 0.4872 to 0.2474. NDCG@5 is determined by the same ranking and relevance labels that produce MRR and Recall@5, so such large changes with unchanged MRR/Recall are not coherent. This suggests a copy or aggregation error. The slide-removal claim (that both ColQwen and Llama degrade on level-0 when slides are excluded) depends on this table and should be re-verified.
[§1, §5] The paper's central generalization claim is conditioned on a model-size disparity that is acknowledged but not analyzed. The vision pipeline uses ColQwen2 (7B) while the OCR pipeline uses Llama 3.2 (90B) for OCR plus Qwen2 7B for embedding; the conclusion that 'OCR-based approaches offer improved retrieval ... in all evaluated settings' should be stated only for the compared configurations. Because the authors argue that a 90B VLM is impractical (Section 1), the claim is a statement about deployable configurations, but it should be explicitly labeled as such in the abstract and conclusion. As written, a reader could infer that OCR-based retrieval is intrinsically better than vision-based retrieval, which is not established by this design.

minor comments (5)

[§4.1] The paragraph after the first table contains a broken reference: 'Nougat outperformed the VLM-based RAG pipeline on levels one through three and on the weighted average. Interestingly, Nougat performed worse on the highest quality documents. 1 shows retrieval results with slideshows removed.' The '1' should read 'Table 2' and the sentence structure should be repaired.
[Table 5] The ViDoRe results show exactly identical NDCG@5, MAP@5, MRR@5, and Recall@5 for OCR+Nougat and OCR+Llama (0.3373, 0.3147, 0.3164, 0.4058). The text attributes this to the OCR output not being the bottleneck, but identical four-decimal values across all four metrics merit a comment on whether the OCR outputs were actually identical or whether the same embedding was used by accident.
[§3.3.1] The NDCG@k formula renders with garbled symbols ('˝' instead of summation and fraction notation). Please replace with a standard typeset formula.
[§4.2] The manual review of approximately 40 question-answer pairs is described as qualitative verification, but no details are given on how the review was performed, inter-annotator agreement, or what was confirmed; a sentence on criteria would help.
[§4.4] The radar plot normalization is described as a 'faithful, monotonic transformation,' but since the normalization uses the mean and standard deviation across all runs, the 'relative strengths' visualization depends on the particular mix of pipelines; this should be noted in the caption or text to avoid overinterpreting absolute differences.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: semantic-evaluation caveat is a disclosed validity limitation, not a circular derivation.

full rationale

No significant circularity. The paper's empirical claims are supported by an independent, externally grounded evaluation protocol: DocDeg degradation levels are manual annotations; Q&A pairs are generated by Llama 3.3 (70B), which is separate from the Qwen2 answer generator and the retrieval models; retrieval metrics (MRR, Recall@k, NDCG@k) are standard; and the radar plot is an explicitly defined monotonic rescaling of the raw values in Tables 1 and 4. The only in-scope caveat is the disclosed semantic-evaluation design: Section 4.1 states that the VLM column's answers are produced by running OCR on ColPali-retrieved images and passing that text to an LLM, so Table 3 measures retrieval-driven answer quality rather than a vision-language generation capability. This is a validity limitation of the 'generation' generalization claim, not a circular derivation: the result is not defined in terms of itself, no parameter is fitted to the target, and no load-bearing self-citation is used. References [3]-[5] are background citations for RAG applications and do not support the central conclusion.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the quality of the human annotations, the LLM-generated QA pairs, and the representativeness of the OSTI source. No fitted parameters are used; the evaluation is a direct comparison of model outputs.

assumptions (4)

domain assumption The DocDeg degradation levels (0-3) assigned by two expert annotators are consistent and meaningful for retrieval difficulty.
All results are segmented by these labels; no inter-annotator agreement is reported.
domain assumption Llama 3.3 70B generated QA pairs that are factual and not answerable from other documents.
Manual review covered only about 40 of 4,196 documents.
domain assumption OSTI documents are representative of real-world degraded document collections.
Dataset is drawn from a single source, DOE OSTI.
standard math Lexical overlap metrics (BLEU, ROUGE) are adequate proxies for answer semantic quality.
Standard metrics used to measure semantic quality; they are accepted in the field but do not capture paraphrastic equivalence fully.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Lost in OCR Translation? Vision-Based Approaches to Robust Document Retrieval." pith.science (2026). https://pith.science/paper/SZDAG57X

@misc{pith2026250505666,
  author       = {Pith},
  title        = {Pith review of: Lost in OCR Translation? Vision-Based Approaches to Robust Document Retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SZDAG57X}},
  note         = {Machine review of arXiv:2505.05666}
}

read the original abstract

Retrieval-Augmented Generation (RAG) has become a popular technique for enhancing the reliability and utility of Large Language Models (LLMs) by grounding responses in external documents. Traditional RAG systems rely on Optical Character Recognition (OCR) to first process scanned documents into text. However, even state-of-the-art OCRs can introduce errors, especially in degraded or complex documents. Recent vision-language approaches, such as ColPali, propose direct visual embedding of documents, eliminating the need for OCR. This study presents a systematic comparison between a vision-based RAG system (ColPali) and more traditional OCR-based pipelines utilizing Llama 3.2 (90B) and Nougat OCR across varying document qualities. Beyond conventional retrieval accuracy metrics, we introduce a semantic answer evaluation benchmark to assess end-to-end question-answering performance. Our findings indicate that while vision-based RAG performs well on documents it has been fine-tuned on, OCR-based RAG is better able to generalize to unseen documents of varying quality. We highlight the key trade-offs between computational efficiency and semantic accuracy, offering practical guidance for RAG practitioners in selecting between OCR-dependent and vision-based document retrieval systems in production environments.

Figures

Figures reproduced from arXiv: 2505.05666 by the authors.

**Figure 2.** 3.2.2 Q&A pair generation. We constructed a rigorous evaluation benchmark by generating ten unique, nuanced question-answer pairs per document using Llama 3.3 (70B). We leveraged Sambanova Systems’ API calls to deploy the LLaMA mod- els for Q&A metadata synthesis and retrieval evaluation. These pairs were carefully designed to capture specific factual details from each document, explicitly avoiding direct verbatim … view at source ↗

**Figure 2.** Zoomed-in view of documents with different degradation levels [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Figures from the paper (2 more)

**Figure 3.** Figure 3: Retrieval performance across document quality levels [PITH_FULL_IMAGE:figures/full_fig_p007_3.png]

**Figure 4.** Figure 4: Comparative Analysis of RAG System Capabilities [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 9 canonical work pages

[1]

Manmatha

Srikar Appalaraju, Bhavan Jasani, Bhargava Urala Kota, Yusheng Xie, and R. Manmatha. 2021. DocFormer: End-to-End Transformer for Document Under- standing. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 973–983. arXiv:2106.11539 [cs.CV] DocEng ’25, August 2025, Location TBD Alexander Most, Joseph Winjum, Ayan Biswas, Shaw...

arXiv 2021
[2]

Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschan- nen, Emanuele Bugliarello, Thomas Unterthiner, Daniel Keysers, Skanda Kop- pula, Fangyu Liu, Adam Grycner, Alexey Gritsenko, Neil Houlsby, Manoj Kumar, Keran Rong, Julian Eisenschlos, Rishabh Kabra, Matthias...

arXiv 2024
[3]

Eren, Minh N

Manish Bhattarai, Ryan Barron, Maksim E. Eren, Minh N. Vu, Vesselin Grantcharov, Ismael Ismael, Valentin Stanev, Cynthia Matuszek, Vladimir I Valtchinov, Kim Rasmussen, and Boian S. Alexandrov. 2025. HEAL: Hierarchical Embedding Alignment Loss for Improved Retrieval and Representation Learn- ing. InProceedings of the 4th International Workshop on Knowledg...

work page 2025
[4]

Santos, Shawn Jones, Ayan Biswas, Boian Alexandrov, and Daniel O’Malley

Manish Bhattarai, Javier E. Santos, Shawn Jones, Ayan Biswas, Boian Alexandrov, and Daniel O’Malley. 2024. Enhancing Code Translation in Language Models with Few-Shot Learning via Retrieval-Augmented Generation. In2024 IEEE High Performance Extreme Computing Conference (HPEC). 1–8. doi:10.1109/HPEC62836. 2024.10938485

arXiv 2024
[5]

Vu, Javier E

Manish Bhattarai, Minh N. Vu, Javier E. Santos, Ismael Ismael, and Daniel O’Malley. 2025. Enhancing Cross-Language Code Translation via Task-Specific Embedding Alignment in Retrieval-Augmented Generation. InProceedings of the 4th International Workshop on Knowledge-Augmented Methods for Natural Language Processing. Association for Computational Linguistic...

work page 2025
[6]

Manuel Faysse, Hugues Sibille, Tony Wu, Bilel Omrani, Gautier Viaud, Céline Hudelot, and Pierre Colombo. 2024. ColPali: Efficient Document Retrieval with Vision Language Models. InThe Twelfth International Conference on Learning Representations (ICLR). arXiv:2402.01410 [cs.CV]

work page Pith review arXiv 2024
[7]

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, et al. 2024. The Llama 3 Herd of Models. arXiv preprint. arXiv:2407.21783 [cs.AI]

arXiv 2024
[8]

Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, and Furu Wei. 2022. Lay- outLMv3: Pre-training for Document AI with Unified Text and Image Masking. InProceedings of the 30th ACM International Conference on Multimedia (ACM MM). 4233—-4242. arXiv:2204.08387 [cs.CV]

arXiv 2022

Show all 23 references

[9]

Omar Khattab and Matei Zaharia. 2020. ColBERT: Efficient Passage Search via Contextualized Late Interaction over BERT. InProceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 39–

2020
[10]

Tony Lee, Haoqin Tu, Chi Heem Wong, Wenhao Zheng, Yiyang Zhou, Yifan Mai, Josselin Somerville Roberts, Michihiro Yasunaga, Huaxiu Yao, Cihang Xie, and Percy Liang. 2024. VHELM: A Holistic Evaluation of Vision Language Models. arXiv preprint. arXiv:2410.07112 [cs.CV]

2024 arXiv
[11]

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. InAdvances in N...

2020 arXiv
[12]

Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michi- hiro Yasunaga, Yian Zhang, Karan Santhanam, Jared Quincy Davis, Yura Alexandr Perov, Yacine Jernite, et al. 2022. Holistic Evaluation of Language Models. In Advances in Neural Information Processing...

2022
[13]

Bhawna Piryani, Jamshid Mozafari, Abdelrahman Abdallah, Antoine Doucet, and Adam Jatowt. 2025. MultiOCR-QA: Dataset for Evaluating Robustness of LLMs in Question Answering on Multilingual OCR Texts. arXiv preprint. arXiv:2502.16781 [cs.CL]

2025
[14]

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. InProceedings ...

2021 arXiv
[15]

Ray Smith. 2007. An Overview of the Tesseract OCR Engine. InICDAR ’07: Proceedings of the Ninth International Conference on Document Analysis and Recognition. IEEE Computer Society, Washington, DC, USA, 629–633. https: //storage.googleapis.com/pub-tools-public-publication-data...

2007
[16]

Zineng Tang, Ziyi Yang, Guoxin Wang, Yuwei Fang, Yang Liu, Chenguang Zhu, Michael Zeng, Cha Zhang, and Mohit Bansal. 2023. Unifying Vision, Text, and Layout for Universal Document Processing. InCVPR. 19254–19264

2023
[17]

Roger Waleffe, Wonmin Byeon, Duncan Riach, Brandon Norick, Vijay Korthikanti, Tri Dao, Albert Gu, Ali Hatamizadeh, Sudhakar Singh, Deepak Narayanan, Garvit Kulshreshtha, Vartika Singh, Jared Casper, Jan Kautz, Mohammad Shoeybi, and Bryan Catanzaro. 2024. An Empirical Study of ...

2024 arXiv
[18]

Peng Xu, Wenqi Shao, Kaipeng Zhang, Peng Gao, Shuo Liu, Meng Lei, Fanqing Meng, Siyuan Huang, Yu Qiao, and Ping Luo. 2023. LVLM-eHub: A Comprehen- sive Evaluation Benchmark for Large Vision-Language Models. arXiv preprint. arXiv:2306.09265 [cs.CV]

2023 arXiv
[19]

Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, and Ming Zhou. 2020. LayoutLM: Pre-training of Text and Layout for Document Image Understanding. InProceedings of ACM SIGKDD. 1192–1200

2020
[20]

Qian Yang, Weixiang Yan, and Aishwarya Agrawal. 2024. Decompose and Com- pare Consistency: Measuring VLMs’ Answer Reliability via Task-Decomposition Consistency Comparison. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP). Associ...

2024
[21]

Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. Sig- moid Loss for Language Image Pre-Training. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 11941–11952. doi:10.1109/ ICCV51070.2023.01100 arXiv:2303.15343 [cs.CV]

2023
[22]

Junyuan Zhang, Qintong Zhang, Bin Wang, Linke Ouyang, Zichen Wen, Ying Li, Ka-Ho Chow, Conghui He, and Wentao Zhang. 2024. OCR Hinders RAG: Evaluating the Cascading Impact of OCR on Retrieval-Augmented Generation. arXiv preprint. arXiv:2412.02592 [cs.CL]

2024 arXiv
[48]

arXiv:2004.12832 [cs.IR]

2004 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Manmatha

Srikar Appalaraju, Bhavan Jasani, Bhargava Urala Kota, Yusheng Xie, and R. Manmatha. 2021. DocFormer: End-to-End Transformer for Document Under- standing. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 973–983. arXiv:2106.11539 [cs.CV] DocEng ’25, August 2025, Location TBD Alexander Most, Joseph Winjum, Ayan Biswas, Shaw...

arXiv 2021

[2] [2]

Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschan- nen, Emanuele Bugliarello, Thomas Unterthiner, Daniel Keysers, Skanda Kop- pula, Fangyu Liu, Adam Grycner, Alexey Gritsenko, Neil Houlsby, Manoj Kumar, Keran Rong, Julian Eisenschlos, Rishabh Kabra, Matthias...

arXiv 2024

[3] [3]

Eren, Minh N

Manish Bhattarai, Ryan Barron, Maksim E. Eren, Minh N. Vu, Vesselin Grantcharov, Ismael Ismael, Valentin Stanev, Cynthia Matuszek, Vladimir I Valtchinov, Kim Rasmussen, and Boian S. Alexandrov. 2025. HEAL: Hierarchical Embedding Alignment Loss for Improved Retrieval and Representation Learn- ing. InProceedings of the 4th International Workshop on Knowledg...

work page 2025

[4] [4]

Santos, Shawn Jones, Ayan Biswas, Boian Alexandrov, and Daniel O’Malley

Manish Bhattarai, Javier E. Santos, Shawn Jones, Ayan Biswas, Boian Alexandrov, and Daniel O’Malley. 2024. Enhancing Code Translation in Language Models with Few-Shot Learning via Retrieval-Augmented Generation. In2024 IEEE High Performance Extreme Computing Conference (HPEC). 1–8. doi:10.1109/HPEC62836. 2024.10938485

arXiv 2024

[5] [5]

Vu, Javier E

Manish Bhattarai, Minh N. Vu, Javier E. Santos, Ismael Ismael, and Daniel O’Malley. 2025. Enhancing Cross-Language Code Translation via Task-Specific Embedding Alignment in Retrieval-Augmented Generation. InProceedings of the 4th International Workshop on Knowledge-Augmented Methods for Natural Language Processing. Association for Computational Linguistic...

work page 2025

[6] [6]

Manuel Faysse, Hugues Sibille, Tony Wu, Bilel Omrani, Gautier Viaud, Céline Hudelot, and Pierre Colombo. 2024. ColPali: Efficient Document Retrieval with Vision Language Models. InThe Twelfth International Conference on Learning Representations (ICLR). arXiv:2402.01410 [cs.CV]

work page Pith review arXiv 2024

[7] [7]

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, et al. 2024. The Llama 3 Herd of Models. arXiv preprint. arXiv:2407.21783 [cs.AI]

arXiv 2024

[8] [8]

Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, and Furu Wei. 2022. Lay- outLMv3: Pre-training for Document AI with Unified Text and Image Masking. InProceedings of the 30th ACM International Conference on Multimedia (ACM MM). 4233—-4242. arXiv:2204.08387 [cs.CV]

arXiv 2022