{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2024:RBKAZ7ADGTBMDY7K2SR4DB7UD5","short_pith_number":"pith:RBKAZ7AD","schema_version":"1.0","canonical_sha256":"88540cfc0334c2c1e3ead4a3c187f41f59f6489d63cba4485a298a737bd882eb","source":{"kind":"arxiv","id":"2407.01449","version":6},"attestation_state":"computed","paper":{"title":"ColPali: Efficient Document Retrieval with Vision Language Models","license":"http://creativecommons.org/publicdomain/zero/1.0/","headline":"Directly embedding images of document pages with a vision language model outperforms text extraction pipelines in retrieval tasks.","cross_cats":["cs.CL","cs.CV"],"primary_cat":"cs.IR","authors_text":"Bilel Omrani, C\\'eline Hudelot, Gautier Viaud, Hugues Sibille, Manuel Faysse, Pierre Colombo, Tony Wu","submitted_at":"2024-06-27T15:45:29Z","abstract_excerpt":"Documents are visually rich structures that convey information through text, but also figures, page layouts, tables, or even fonts. Since modern retrieval systems mainly rely on the textual information they extract from document pages to index documents -often through lengthy and brittle processes-, they struggle to exploit key visual cues efficiently. This limits their capabilities in many practical document retrieval applications such as Retrieval Augmented Generation (RAG). To benchmark current systems on visually rich document retrieval, we introduce the Visual Document Retrieval Benchmark"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":false,"formal_links_present":true},"canonical_record":{"source":{"id":"2407.01449","kind":"arxiv","version":6},"metadata":{"license":"http://creativecommons.org/publicdomain/zero/1.0/","primary_cat":"cs.IR","submitted_at":"2024-06-27T15:45:29Z","cross_cats_sorted":["cs.CL","cs.CV"],"title_canon_sha256":"fe7ac4341d12ea95464629e8bc1ed64f6ef6f6341694a7fe2fbe9923c7a430fd","abstract_canon_sha256":"d5f28713d6a2afcb63490758f7208bc4f97ffac8fcf8140a266c97ffe4f0ecda"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-05-17T23:38:53.808067Z","signature_b64":"mS8d8uD63bLEqjCFb7U6mdHnav5ianRSfKkq84+MKnU/9olpMJr2YEZp25BHFpKjDsKPtlpIWrXamyKWUgRNAQ==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"88540cfc0334c2c1e3ead4a3c187f41f59f6489d63cba4485a298a737bd882eb","last_reissued_at":"2026-05-17T23:38:53.807452Z","signature_status":"signed_v1","first_computed_at":"2026-05-17T23:38:53.807452Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"ColPali: Efficient Document Retrieval with Vision Language Models","license":"http://creativecommons.org/publicdomain/zero/1.0/","headline":"Directly embedding images of document pages with a vision language model outperforms text extraction pipelines in retrieval tasks.","cross_cats":["cs.CL","cs.CV"],"primary_cat":"cs.IR","authors_text":"Bilel Omrani, C\\'eline Hudelot, Gautier Viaud, Hugues Sibille, Manuel Faysse, Pierre Colombo, Tony Wu","submitted_at":"2024-06-27T15:45:29Z","abstract_excerpt":"Documents are visually rich structures that convey information through text, but also figures, page layouts, tables, or even fonts. Since modern retrieval systems mainly rely on the textual information they extract from document pages to index documents -often through lengthy and brittle processes-, they struggle to exploit key visual cues efficiently. This limits their capabilities in many practical document retrieval applications such as Retrieval Augmented Generation (RAG). To benchmark current systems on visually rich document retrieval, we introduce the Visual Document Retrieval Benchmark"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"ColPali largely outperforms modern document retrieval pipelines while being drastically simpler, faster and end-to-end trainable.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That direct image embeddings from a vision-language model capture all necessary semantic and layout information better than text extraction pipelines across the tested domains and languages.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"ColPali embeds document page images with a vision-language model and late interaction to outperform text-based retrieval pipelines on a new visual document benchmark while being simpler and faster.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Directly embedding images of document pages with a vision language model outperforms text extraction pipelines in retrieval tasks.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"c55c656f01ef3c2cc60171e80457a9f9bcb352ccce0771ed29df5d7e88d0abbc"},"source":{"id":"2407.01449","kind":"arxiv","version":6},"verdict":{"id":"da0a4371-b525-469e-ac83-2d944729d746","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-15T02:32:05.572657Z","strongest_claim":"ColPali largely outperforms modern document retrieval pipelines while being drastically simpler, faster and end-to-end trainable.","one_line_summary":"ColPali embeds document page images with a vision-language model and late interaction to outperform text-based retrieval pipelines on a new visual document benchmark while being simpler and faster.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That direct image embeddings from a vision-language model capture all necessary semantic and layout information better than text extraction pipelines across the tested domains and languages.","pith_extraction_headline":"Directly embedding images of document pages with a vision language model outperforms text extraction pipelines in retrieval tasks."},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":2,"snapshot_sha256":"be80503df53eb046728f77a4a7100b93f594ebf97972885e0e178eaeb500b404"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2407.01449","created_at":"2026-05-17T23:38:53.807543+00:00"},{"alias_kind":"arxiv_version","alias_value":"2407.01449v6","created_at":"2026-05-17T23:38:53.807543+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2407.01449","created_at":"2026-05-17T23:38:53.807543+00:00"},{"alias_kind":"pith_short_12","alias_value":"RBKAZ7ADGTBM","created_at":"2026-05-18T12:33:37.589309+00:00"},{"alias_kind":"pith_short_16","alias_value":"RBKAZ7ADGTBMDY7K","created_at":"2026-05-18T12:33:37.589309+00:00"},{"alias_kind":"pith_short_8","alias_value":"RBKAZ7AD","created_at":"2026-05-18T12:33:37.589309+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":33,"internal_anchor_count":33,"sample":[{"citing_arxiv_id":"2605.22829","citing_title":"LFRAG: Layout-oriented Fine-grained Retrieval-Augmented Generation on Multimodal Document Understanding","ref_index":7,"is_internal_anchor":true},{"citing_arxiv_id":"2605.18818","citing_title":"Operationalizing Document AI: A Microservice Architecture for OCR and LLM Pipelines in Production","ref_index":7,"is_internal_anchor":true},{"citing_arxiv_id":"2605.16638","citing_title":"TTE-Flash: Accelerating Reasoning-based Multimodal Representations via Think-Then-Embed Tokens","ref_index":14,"is_internal_anchor":true},{"citing_arxiv_id":"2605.14311","citing_title":"Beyond Binary: Reframing GUI Critique as Continuous Semantic Alignment","ref_index":79,"is_internal_anchor":true},{"citing_arxiv_id":"2507.04590","citing_title":"VLM2Vec-V2: Advancing Multimodal Embedding for Videos, Images, and Visual Documents","ref_index":5,"is_internal_anchor":true},{"citing_arxiv_id":"2511.13131","citing_title":"MM-Telco: Benchmarks and Multimodal Large Language Models for Telecom Applications","ref_index":10,"is_internal_anchor":true},{"citing_arxiv_id":"2512.13511","citing_title":"Adapting MLLMs for Nuanced Video Retrieval","ref_index":26,"is_internal_anchor":true},{"citing_arxiv_id":"2410.10594","citing_title":"VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents","ref_index":5,"is_internal_anchor":true},{"citing_arxiv_id":"2604.09552","citing_title":"MCERF: Advancing Multimodal LLM Evaluation of Engineering Documentation with Enhanced Retrieval","ref_index":4,"is_internal_anchor":true},{"citing_arxiv_id":"2602.23061","citing_title":"MoDora: Tree-Based Semi-Structured Document Analysis System","ref_index":15,"is_internal_anchor":true},{"citing_arxiv_id":"2605.14311","citing_title":"Beyond Binary: Reframing GUI Critique as Continuous Semantic Alignment","ref_index":79,"is_internal_anchor":true},{"citing_arxiv_id":"2605.14581","citing_title":"A Picture is Worth a Thousand Words? An Empirical Study of Aggregation Strategies for Visual Financial Document Retrieval","ref_index":5,"is_internal_anchor":true},{"citing_arxiv_id":"2605.12882","citing_title":"CiteVQA: Benchmarking Evidence Attribution for Trustworthy Document Intelligence","ref_index":7,"is_internal_anchor":true},{"citing_arxiv_id":"2604.02073","citing_title":"PLUME: Latent Reasoning Based Universal Multimodal Embedding","ref_index":6,"is_internal_anchor":true},{"citing_arxiv_id":"2504.05299","citing_title":"SmolVLM: Redefining small and efficient multimodal models","ref_index":12,"is_internal_anchor":true},{"citing_arxiv_id":"2604.27724","citing_title":"Iterative Multimodal Retrieval-Augmented Generation for Medical Question Answering","ref_index":4,"is_internal_anchor":true},{"citing_arxiv_id":"2604.12812","citing_title":"DocSeeker: Structured Visual Reasoning with Evidence Grounding for Long Document Understanding","ref_index":18,"is_internal_anchor":true},{"citing_arxiv_id":"2605.10120","citing_title":"MicroWorld: Empowering Multimodal Large Language Models to Bridge the Microscopic Domain Gap with Multimodal Attribute Graph","ref_index":25,"is_internal_anchor":true},{"citing_arxiv_id":"2604.13710","citing_title":"SLQ: Bridging Modalities via Shared Latent Queries for Retrieval with Frozen MLLMs","ref_index":9,"is_internal_anchor":true},{"citing_arxiv_id":"2604.22280","citing_title":"Beyond Chain-of-Thought: Rewrite as a Universal Interface for Generative Multimodal Embeddings","ref_index":9,"is_internal_anchor":true},{"citing_arxiv_id":"2605.05806","citing_title":"Retrieval from Within: An Intrinsic Capability of Attention-Based Models","ref_index":8,"is_internal_anchor":true},{"citing_arxiv_id":"2604.11095","citing_title":"Bottleneck Tokens for Unified Multimodal Retrieval","ref_index":3,"is_internal_anchor":true},{"citing_arxiv_id":"2604.10167","citing_title":"Visual Late Chunking: An Empirical Study of Contextual Chunking for Efficient Visual Document Retrieval","ref_index":5,"is_internal_anchor":true},{"citing_arxiv_id":"2604.12812","citing_title":"DocSeeker: Structured Visual Reasoning with Evidence Grounding for Long Document Understanding","ref_index":18,"is_internal_anchor":true},{"citing_arxiv_id":"2604.07220","citing_title":"HIVE: Query, Hypothesize, Verify An LLM Framework for Multimodal Reasoning-Intensive Retrieval","ref_index":10,"is_internal_anchor":true}]},"formal_canon":{"evidence_count":2,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/RBKAZ7ADGTBMDY7K2SR4DB7UD5","json":"https://pith.science/pith/RBKAZ7ADGTBMDY7K2SR4DB7UD5.json","graph_json":"https://pith.science/api/pith-number/RBKAZ7ADGTBMDY7K2SR4DB7UD5/graph.json","events_json":"https://pith.science/api/pith-number/RBKAZ7ADGTBMDY7K2SR4DB7UD5/events.json","paper":"https://pith.science/paper/RBKAZ7AD"},"agent_actions":{"view_html":"https://pith.science/pith/RBKAZ7ADGTBMDY7K2SR4DB7UD5","download_json":"https://pith.science/pith/RBKAZ7ADGTBMDY7K2SR4DB7UD5.json","view_paper":"https://pith.science/paper/RBKAZ7AD","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2407.01449&json=true","fetch_graph":"https://pith.science/api/pith-number/RBKAZ7ADGTBMDY7K2SR4DB7UD5/graph.json","fetch_events":"https://pith.science/api/pith-number/RBKAZ7ADGTBMDY7K2SR4DB7UD5/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/RBKAZ7ADGTBMDY7K2SR4DB7UD5/action/timestamp_anchor","attest_storage":"https://pith.science/pith/RBKAZ7ADGTBMDY7K2SR4DB7UD5/action/storage_attestation","attest_author":"https://pith.science/pith/RBKAZ7ADGTBMDY7K2SR4DB7UD5/action/author_attestation","sign_citation":"https://pith.science/pith/RBKAZ7ADGTBMDY7K2SR4DB7UD5/action/citation_signature","submit_replication":"https://pith.science/pith/RBKAZ7ADGTBMDY7K2SR4DB7UD5/action/replication_record"}},"created_at":"2026-05-17T23:38:53.807543+00:00","updated_at":"2026-05-17T23:38:53.807543+00:00"}