{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2022:ROYX62NCB72JQHDVNJK6MR3MCN","short_pith_number":"pith:ROYX62NC","schema_version":"1.0","canonical_sha256":"8bb17f69a20ff4981c756a55e6476c13441e2bbbd55d7f5c78db51a1e6549e0a","source":{"kind":"arxiv","id":"2209.06794","version":4},"attestation_state":"computed","paper":{"title":"PaLI: A Jointly-Scaled Multilingual Language-Image Model","license":"http://creativecommons.org/licenses/by/4.0/","headline":"PaLI jointly scales a 4-billion-parameter vision transformer with a language model on a 10B multilingual image-text set to reach state-of-the-art on captioning, VQA and scene-text tasks.","cross_cats":["cs.CL"],"primary_cat":"cs.CV","authors_text":"Adam Grycner, AJ Piergiovanni, Alexander Kolesnikov, Andreas Steiner, Anelia Angelova, Ashish Thapliyal, Basil Mustafa, Burcu Karagol Ayan, Carlos Riquelme, Chao Jia, Daniel Salz, Gaurav Mishra, Hassan Akbari, James Bradbury, Joan Puigcerver, Keran Rong, Linting Xue, Lucas Beyer, Mojtaba Seyedhosseini, Nan Ding, Neil Houlsby, Piotr Padlewski, Radu Soricut, Sebastian Goodman, Soravit Changpinyo, Weicheng Kuo, Xiaohua Zhai, Xiao Wang, Xi Chen","submitted_at":"2022-09-14T17:24:07Z","abstract_excerpt":"Effective scaling and a flexible task interface enable large language models to excel at many tasks. We present PaLI (Pathways Language and Image model), a model that extends this approach to the joint modeling of language and vision. PaLI generates text based on visual and textual inputs, and with this interface performs many vision, language, and multimodal tasks, in many languages. To train PaLI, we make use of large pre-trained encoder-decoder language models and Vision Transformers (ViTs). This allows us to capitalize on their existing capabilities and leverage the substantial cost of tra"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":true,"formal_links_present":true},"canonical_record":{"source":{"id":"2209.06794","kind":"arxiv","version":4},"metadata":{"license":"http://creativecommons.org/licenses/by/4.0/","primary_cat":"cs.CV","submitted_at":"2022-09-14T17:24:07Z","cross_cats_sorted":["cs.CL"],"title_canon_sha256":"08a218ce080d71719c57e13038c8be63fb06bfe224ac17e3c8cb281586c15081","abstract_canon_sha256":"2a50651767b6289fbf279711ac7379d502692af9f7b0932b728ccd5beb6987f9"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-05-17T23:38:48.354921Z","signature_b64":"SiN5QnjnQeaKy+JQRIEX1ENhuz1wjJx8o4p3V7y8KjQZRJik1g4i0WjQNapFUhdSxeEoHQfbUJtyCOdR36uvDw==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"8bb17f69a20ff4981c756a55e6476c13441e2bbbd55d7f5c78db51a1e6549e0a","last_reissued_at":"2026-05-17T23:38:48.354221Z","signature_status":"signed_v1","first_computed_at":"2026-05-17T23:38:48.354221Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"PaLI: A Jointly-Scaled Multilingual Language-Image Model","license":"http://creativecommons.org/licenses/by/4.0/","headline":"PaLI jointly scales a 4-billion-parameter vision transformer with a language model on a 10B multilingual image-text set to reach state-of-the-art on captioning, VQA and scene-text tasks.","cross_cats":["cs.CL"],"primary_cat":"cs.CV","authors_text":"Adam Grycner, AJ Piergiovanni, Alexander Kolesnikov, Andreas Steiner, Anelia Angelova, Ashish Thapliyal, Basil Mustafa, Burcu Karagol Ayan, Carlos Riquelme, Chao Jia, Daniel Salz, Gaurav Mishra, Hassan Akbari, James Bradbury, Joan Puigcerver, Keran Rong, Linting Xue, Lucas Beyer, Mojtaba Seyedhosseini, Nan Ding, Neil Houlsby, Piotr Padlewski, Radu Soricut, Sebastian Goodman, Soravit Changpinyo, Weicheng Kuo, Xiaohua Zhai, Xiao Wang, Xi Chen","submitted_at":"2022-09-14T17:24:07Z","abstract_excerpt":"Effective scaling and a flexible task interface enable large language models to excel at many tasks. We present PaLI (Pathways Language and Image model), a model that extends this approach to the joint modeling of language and vision. PaLI generates text based on visual and textual inputs, and with this interface performs many vision, language, and multimodal tasks, in many languages. To train PaLI, we make use of large pre-trained encoder-decoder language models and Vision Transformers (ViTs). This allows us to capitalize on their existing capabilities and leverage the substantial cost of tra"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"PaLI achieves state-of-the-art in multiple vision and language tasks (such as captioning, visual question-answering, scene-text understanding), while retaining a simple, modular, and scalable design.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That joint scaling of the vision and language components on the new 10B multilingual dataset will produce the claimed performance gains without major issues from data quality, language imbalance, or overfitting.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"PaLI jointly scales a 4B-parameter vision transformer with language models on a new 10B multilingual image-text dataset to reach state-of-the-art results on vision-language tasks while keeping a simple modular design.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"PaLI jointly scales a 4-billion-parameter vision transformer with a language model on a 10B multilingual image-text set to reach state-of-the-art on captioning, VQA and scene-text tasks.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"403798ecb94f7cc80e113e1a6d067177d84ed9dceb0ac53306ffc5d55b618eae"},"source":{"id":"2209.06794","kind":"arxiv","version":4},"verdict":{"id":"4e01e5e3-5592-48cb-ad4d-82aa25360fc7","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-16T09:25:21.335193Z","strongest_claim":"PaLI achieves state-of-the-art in multiple vision and language tasks (such as captioning, visual question-answering, scene-text understanding), while retaining a simple, modular, and scalable design.","one_line_summary":"PaLI jointly scales a 4B-parameter vision transformer with language models on a new 10B multilingual image-text dataset to reach state-of-the-art results on vision-language tasks while keeping a simple modular design.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That joint scaling of the vision and language components on the new 10B multilingual dataset will produce the claimed performance gains without major issues from data quality, language imbalance, or overfitting.","pith_extraction_headline":"PaLI jointly scales a 4-billion-parameter vision transformer with a language model on a 10B multilingual image-text set to reach state-of-the-art on captioning, VQA and scene-text tasks."},"references":{"count":185,"sample":[{"doi":"","year":2019,"title":"Tallyqa: Answering complex counting questions","work_id":"70f8998a-d9b0-40d5-b598-a9c726ba4c8e","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2019,"title":"nocaps : Novel object captioning at scale","work_id":"ec4e26ac-4d58-40fc-ac24-6dfaecc71822","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2021,"title":"Crossvqa: Scalably generating benchmarks for systematically testing vqa generalization","work_id":"11d18ed5-36cb-4baa-9f46-6fa140666ecd","ref_index":3,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2020,"title":"On the cross-lingual transferability of monolingual representations","work_id":"0b1b73b4-170f-4bcb-83fd-61f3cab7db23","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2019,"title":"ObjectNet : a large-scale bias-controlled dataset for pushing the limits of object recognition models","work_id":"a9a36037-ee68-4859-ba58-22f9e38dd293","ref_index":6,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":185,"snapshot_sha256":"2e7f278ed328831f0c03dfa4cc8f32ab6bcf58a31ec615c3c377584f629855eb","internal_anchors":12},"formal_canon":{"evidence_count":1,"snapshot_sha256":"5473604ba031f2fd17202c1d2e5c4f9638d8cfacf0abbb3df412dc5f84ef3b64"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2209.06794","created_at":"2026-05-17T23:38:48.354333+00:00"},{"alias_kind":"arxiv_version","alias_value":"2209.06794v4","created_at":"2026-05-17T23:38:48.354333+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2209.06794","created_at":"2026-05-17T23:38:48.354333+00:00"},{"alias_kind":"pith_short_12","alias_value":"ROYX62NCB72J","created_at":"2026-05-18T12:33:33.725879+00:00"},{"alias_kind":"pith_short_16","alias_value":"ROYX62NCB72JQHDV","created_at":"2026-05-18T12:33:33.725879+00:00"},{"alias_kind":"pith_short_8","alias_value":"ROYX62NC","created_at":"2026-05-18T12:33:33.725879+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":34,"internal_anchor_count":34,"sample":[{"citing_arxiv_id":"2312.11805","citing_title":"Gemini: A Family of Highly Capable Multimodal Models","ref_index":11,"is_internal_anchor":true},{"citing_arxiv_id":"2505.16819","citing_title":"Character-Centered Dialogue Generation from Scene-Level Prompts","ref_index":14,"is_internal_anchor":true},{"citing_arxiv_id":"2505.23678","citing_title":"Grounded Reinforcement Learning for Visual Reasoning","ref_index":7,"is_internal_anchor":true},{"citing_arxiv_id":"2605.16603","citing_title":"Controlla: Learning Controllability via Graph-Constrained Latent Geometry","ref_index":7,"is_internal_anchor":true},{"citing_arxiv_id":"2408.04840","citing_title":"mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Models","ref_index":32,"is_internal_anchor":true},{"citing_arxiv_id":"2605.16834","citing_title":"Learning Relative Representations for Fine-Grained Multimodal Alignment with Limited Data","ref_index":18,"is_internal_anchor":true},{"citing_arxiv_id":"2605.16972","citing_title":"WhiteTesseract: Reframing the Interpretation of Cultural Heritage through XR and Conversational AI","ref_index":22,"is_internal_anchor":true},{"citing_arxiv_id":"2509.10026","citing_title":"LaV-CoT: Language-Aware Visual CoT with Multi-Aspect Reward Optimization for Real-World Multilingual VQA","ref_index":10,"is_internal_anchor":true},{"citing_arxiv_id":"2509.19207","citing_title":"Long Story Short: Disentangling Compositionality and Long-Caption Understanding in Contrastive VLMs","ref_index":8,"is_internal_anchor":true},{"citing_arxiv_id":"2510.12710","citing_title":"Reflection-Based Task Adaptation for Self-Improving VLA","ref_index":28,"is_internal_anchor":true},{"citing_arxiv_id":"2311.04257","citing_title":"mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration","ref_index":11,"is_internal_anchor":true},{"citing_arxiv_id":"2302.11550","citing_title":"Scaling Robot Learning with Semantically Imagined Experience","ref_index":7,"is_internal_anchor":true},{"citing_arxiv_id":"2512.10821","citing_title":"Agile Deliberation: Concept Deliberation for Subjective Visual Classification","ref_index":3,"is_internal_anchor":true},{"citing_arxiv_id":"2408.16500","citing_title":"CogVLM2: Visual Language Models for Image and Video Understanding","ref_index":13,"is_internal_anchor":true},{"citing_arxiv_id":"2312.16886","citing_title":"MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devices","ref_index":19,"is_internal_anchor":true},{"citing_arxiv_id":"2303.15343","citing_title":"Sigmoid Loss for Language Image Pre-Training","ref_index":13,"is_internal_anchor":true},{"citing_arxiv_id":"2602.01738","citing_title":"Simplicity Prevails: The Emergence of Generalizable AIGI Detection in Visual Foundation Models","ref_index":7,"is_internal_anchor":true},{"citing_arxiv_id":"2310.06114","citing_title":"Learning Interactive Real-World Simulators","ref_index":112,"is_internal_anchor":true},{"citing_arxiv_id":"2603.00655","citing_title":"Mema: Memory-Augmented Adapter for Enhanced Vision-Language Understanding","ref_index":6,"is_internal_anchor":true},{"citing_arxiv_id":"2404.18416","citing_title":"Capabilities of Gemini Models in Medicine","ref_index":15,"is_internal_anchor":true},{"citing_arxiv_id":"2603.08942","citing_title":"BiCLIP: Domain Canonicalization via Structured Geometric Transformation","ref_index":3,"is_internal_anchor":true},{"citing_arxiv_id":"2603.09921","citing_title":"WikiCLIP: An Efficient Contrastive Baseline for Open-domain Visual Entity Recognition","ref_index":5,"is_internal_anchor":true},{"citing_arxiv_id":"2412.03555","citing_title":"PaliGemma 2: A Family of Versatile VLMs for Transfer","ref_index":14,"is_internal_anchor":true},{"citing_arxiv_id":"2308.01390","citing_title":"OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Models","ref_index":6,"is_internal_anchor":true},{"citing_arxiv_id":"2605.09948","citing_title":"LoopVLA: Learning Sufficiency in Recurrent Refinement for Vision-Language-Action Models","ref_index":6,"is_internal_anchor":true}]},"formal_canon":{"evidence_count":1,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/ROYX62NCB72JQHDVNJK6MR3MCN","json":"https://pith.science/pith/ROYX62NCB72JQHDVNJK6MR3MCN.json","graph_json":"https://pith.science/api/pith-number/ROYX62NCB72JQHDVNJK6MR3MCN/graph.json","events_json":"https://pith.science/api/pith-number/ROYX62NCB72JQHDVNJK6MR3MCN/events.json","paper":"https://pith.science/paper/ROYX62NC"},"agent_actions":{"view_html":"https://pith.science/pith/ROYX62NCB72JQHDVNJK6MR3MCN","download_json":"https://pith.science/pith/ROYX62NCB72JQHDVNJK6MR3MCN.json","view_paper":"https://pith.science/paper/ROYX62NC","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2209.06794&json=true","fetch_graph":"https://pith.science/api/pith-number/ROYX62NCB72JQHDVNJK6MR3MCN/graph.json","fetch_events":"https://pith.science/api/pith-number/ROYX62NCB72JQHDVNJK6MR3MCN/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/ROYX62NCB72JQHDVNJK6MR3MCN/action/timestamp_anchor","attest_storage":"https://pith.science/pith/ROYX62NCB72JQHDVNJK6MR3MCN/action/storage_attestation","attest_author":"https://pith.science/pith/ROYX62NCB72JQHDVNJK6MR3MCN/action/author_attestation","sign_citation":"https://pith.science/pith/ROYX62NCB72JQHDVNJK6MR3MCN/action/citation_signature","submit_replication":"https://pith.science/pith/ROYX62NCB72JQHDVNJK6MR3MCN/action/replication_record"}},"created_at":"2026-05-17T23:38:48.354333+00:00","updated_at":"2026-05-17T23:38:48.354333+00:00"}