{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2023:PREMJTZC4J7RNK4IHH6M6UBNEE","short_pith_number":"pith:PREMJTZC","schema_version":"1.0","canonical_sha256":"7c48c4cf22e27f16ab8839fccf502d21084bf52b5499072da4555157a99911e5","source":{"kind":"arxiv","id":"2309.16671","version":6},"attestation_state":"computed","paper":{"title":"Demystifying CLIP Data","license":"http://creativecommons.org/licenses/by-nc-sa/4.0/","headline":"MetaCLIP balances CommonCrawl image-text pairs using CLIP-derived metadata to exceed original CLIP performance on zero-shot benchmarks.","cross_cats":["cs.CL"],"primary_cat":"cs.CV","authors_text":"Christoph Feichtenhofer, Gargi Ghosh, Hu Xu, Luke Zettlemoyer, Po-Yao Huang, Russell Howes, Saining Xie, Shang-Wen Li, Vasu Sharma, Xiaoqing Ellen Tan","submitted_at":"2023-09-28T17:59:56Z","abstract_excerpt":"Contrastive Language-Image Pre-training (CLIP) is an approach that has advanced research and applications in computer vision, fueling modern recognition systems and generative models. We believe that the main ingredient to the success of CLIP is its data and not the model architecture or pre-training objective. However, CLIP only provides very limited information about its data and how it has been collected, leading to works that aim to reproduce CLIP's data by filtering with its model parameters. In this work, we intend to reveal CLIP's data curation approach and in our pursuit of making it o"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":true,"formal_links_present":true},"canonical_record":{"source":{"id":"2309.16671","kind":"arxiv","version":6},"metadata":{"license":"http://creativecommons.org/licenses/by-nc-sa/4.0/","primary_cat":"cs.CV","submitted_at":"2023-09-28T17:59:56Z","cross_cats_sorted":["cs.CL"],"title_canon_sha256":"77371d4b9df8c37f41b4553938b8a1d5762fff9a02903ac4b560ddeca04e5b06","abstract_canon_sha256":"ebf6224ad45b03c9907c6c2a98803e7c9116e2a50f09fa513efa5dcd022d1323"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-05-17T23:38:48.379391Z","signature_b64":"igPPM9GWA+Lz96Kbq4pXGWKsIAZzNR/ShlPJ51kj3TzD/62M0f/B0VkGogAhaMX1cOxGXxH5uGdRYHHxvqixCw==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"7c48c4cf22e27f16ab8839fccf502d21084bf52b5499072da4555157a99911e5","last_reissued_at":"2026-05-17T23:38:48.378631Z","signature_status":"signed_v1","first_computed_at":"2026-05-17T23:38:48.378631Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"Demystifying CLIP Data","license":"http://creativecommons.org/licenses/by-nc-sa/4.0/","headline":"MetaCLIP balances CommonCrawl image-text pairs using CLIP-derived metadata to exceed original CLIP performance on zero-shot benchmarks.","cross_cats":["cs.CL"],"primary_cat":"cs.CV","authors_text":"Christoph Feichtenhofer, Gargi Ghosh, Hu Xu, Luke Zettlemoyer, Po-Yao Huang, Russell Howes, Saining Xie, Shang-Wen Li, Vasu Sharma, Xiaoqing Ellen Tan","submitted_at":"2023-09-28T17:59:56Z","abstract_excerpt":"Contrastive Language-Image Pre-training (CLIP) is an approach that has advanced research and applications in computer vision, fueling modern recognition systems and generative models. We believe that the main ingredient to the success of CLIP is its data and not the model architecture or pre-training objective. However, CLIP only provides very limited information about its data and how it has been collected, leading to works that aim to reproduce CLIP's data by filtering with its model parameters. In this work, we intend to reveal CLIP's data curation approach and in our pursuit of making it o"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"MetaCLIP applied to CommonCrawl with 400M image-text data pairs outperforms CLIP's data on multiple standard benchmarks. In zero-shot ImageNet classification, MetaCLIP achieves 70.8% accuracy, surpassing CLIP's 68.3% on ViT-B models. Scaling to 1B data attains 72.4%.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That metadata derived from CLIP's own concepts is sufficient to capture the key distributional properties that made CLIP data effective, and that explicit balancing over this metadata is the primary driver of the observed gains rather than other unmeasured factors in the raw pool.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"MetaCLIP curates balanced 400M-pair subsets from CommonCrawl that outperform CLIP data, reaching 70.8% zero-shot ImageNet accuracy on ViT-B versus CLIP's 68.3%.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"MetaCLIP balances CommonCrawl image-text pairs using CLIP-derived metadata to exceed original CLIP performance on zero-shot benchmarks.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"0d269e3a356b6884149179892cfac490ad24009d4a26d1eccc834cf3b5a3abf5"},"source":{"id":"2309.16671","kind":"arxiv","version":6},"verdict":{"id":"684ca178-8eed-4ae5-8192-451ab796bca5","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-16T09:16:41.611459Z","strongest_claim":"MetaCLIP applied to CommonCrawl with 400M image-text data pairs outperforms CLIP's data on multiple standard benchmarks. In zero-shot ImageNet classification, MetaCLIP achieves 70.8% accuracy, surpassing CLIP's 68.3% on ViT-B models. Scaling to 1B data attains 72.4%.","one_line_summary":"MetaCLIP curates balanced 400M-pair subsets from CommonCrawl that outperform CLIP data, reaching 70.8% zero-shot ImageNet accuracy on ViT-B versus CLIP's 68.3%.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That metadata derived from CLIP's own concepts is sufficient to capture the key distributional properties that made CLIP data effective, and that explicit balancing over this metadata is the primary driver of the observed gains rather than other unmeasured factors in the raw pool.","pith_extraction_headline":"MetaCLIP balances CommonCrawl image-text pairs using CLIP-derived metadata to exceed original CLIP performance on zero-shot benchmarks."},"references":{"count":179,"sample":[{"doi":"","year":2015,"title":"Coresets for nonparametric estimation-the case of dp-means","work_id":"2effacca-ca04-40bb-95d7-c6d934b4b7a8","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2020,"title":"An image is worth 16x16 words: Transformers for image recognition at scale","work_id":"282f8133-1b3d-4933-b68c-70ee9cdd289e","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2011,"title":"Scalable training of mixture models via coresets","work_id":"0087f78b-4086-46af-9166-7fc8265f8325","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2023,"title":"Datacomp: In search of the next generation of multimodal datasets, 2023","work_id":"6b216263-81b4-4644-bf69-bb67cde04476","ref_index":6,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2004,"title":"On coresets for k-means and k-median clustering","work_id":"c9a1491f-4ac1-4e2c-84bf-3c5b50a08fe6","ref_index":7,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":179,"snapshot_sha256":"8b35deeb41bf4fb875ca75dd7d03d92a854a79d01654aca3d5dd04ac9bc33aa5","internal_anchors":33},"formal_canon":{"evidence_count":2,"snapshot_sha256":"1b591dced2f862b76482ace20541da88ba62ce22ff3c9eaa323c0eac25eb8ae1"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2309.16671","created_at":"2026-05-17T23:38:48.378761+00:00"},{"alias_kind":"arxiv_version","alias_value":"2309.16671v6","created_at":"2026-05-17T23:38:48.378761+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2309.16671","created_at":"2026-05-17T23:38:48.378761+00:00"},{"alias_kind":"pith_short_12","alias_value":"PREMJTZC4J7R","created_at":"2026-05-18T12:33:37.589309+00:00"},{"alias_kind":"pith_short_16","alias_value":"PREMJTZC4J7RNK4I","created_at":"2026-05-18T12:33:37.589309+00:00"},{"alias_kind":"pith_short_8","alias_value":"PREMJTZC","created_at":"2026-05-18T12:33:37.589309+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":27,"internal_anchor_count":27,"sample":[{"citing_arxiv_id":"2502.06818","citing_title":"Rethinking the Global Knowledge of CLIP in Training-Free Open-Vocabulary Semantic Segmentation","ref_index":43,"is_internal_anchor":true},{"citing_arxiv_id":"2508.00756","citing_title":"LeakyCLIP: Extracting Training Data from CLIP","ref_index":50,"is_internal_anchor":true},{"citing_arxiv_id":"2605.16409","citing_title":"Multilingual OCR-Aware Fine-Tuning and Prompt-Guided Chain-of-Thought Reasoning for Multimodal Large Language Models","ref_index":11,"is_internal_anchor":true},{"citing_arxiv_id":"2506.08849","citing_title":"Adapting Vision-Language Foundation Model for Next Generation Medical Ultrasound Image Analysis","ref_index":34,"is_internal_anchor":true},{"citing_arxiv_id":"2507.21114","citing_title":"Page image classification for content-specific data processing","ref_index":20,"is_internal_anchor":true},{"citing_arxiv_id":"2509.18095","citing_title":"MetaEmbed: Scaling Multimodal Retrieval at Test-Time with Flexible Late Interaction","ref_index":65,"is_internal_anchor":true},{"citing_arxiv_id":"2511.16719","citing_title":"SAM 3: Segment Anything with Concepts","ref_index":144,"is_internal_anchor":true},{"citing_arxiv_id":"2506.20670","citing_title":"MMSearch-R1: Incentivizing LMMs to Search","ref_index":63,"is_internal_anchor":true},{"citing_arxiv_id":"2602.01738","citing_title":"Simplicity Prevails: The Emergence of Generalizable AIGI Detection in Visual Foundation Models","ref_index":31,"is_internal_anchor":true},{"citing_arxiv_id":"2602.22394","citing_title":"Vision Transformers Need More Than Registers","ref_index":39,"is_internal_anchor":true},{"citing_arxiv_id":"2605.11405","citing_title":"20/20 Vision Language Models: A Prescription for Better VLMs through Data Curation Alone","ref_index":37,"is_internal_anchor":true},{"citing_arxiv_id":"2603.27759","citing_title":"When Surfaces Lie: Exploiting Wrinkle-Induced Attention Shift to Attack Vision-Language Models","ref_index":35,"is_internal_anchor":true},{"citing_arxiv_id":"2604.03117","citing_title":"Revealing Physical-World Semantic Vulnerabilities: Universal Adversarial Patches for Infrared Vision-Language Models","ref_index":42,"is_internal_anchor":true},{"citing_arxiv_id":"2605.11107","citing_title":"Birds of a Feather Flock Together: Background-Invariant Representations via Linear Structure in VLMs","ref_index":43,"is_internal_anchor":true},{"citing_arxiv_id":"2605.11405","citing_title":"20/20 Vision Language Models: A Prescription for Better VLMs through Data Curation Alone","ref_index":37,"is_internal_anchor":true},{"citing_arxiv_id":"2605.08985","citing_title":"LLaVA-UHD v4: What Makes Efficient Visual Encoding in MLLMs?","ref_index":45,"is_internal_anchor":true},{"citing_arxiv_id":"2604.24997","citing_title":"DouC: Dual-Branch CLIP for Training-Free Open-Vocabulary Segmentation","ref_index":4,"is_internal_anchor":true},{"citing_arxiv_id":"2604.24642","citing_title":"Probing CLIP's Comprehension of 360-Degree Textual and Visual Semantics","ref_index":18,"is_internal_anchor":true},{"citing_arxiv_id":"2605.01325","citing_title":"Rethinking Model Selection in VLM Through the Lens of Gromov-Wasserstein Distance","ref_index":49,"is_internal_anchor":true},{"citing_arxiv_id":"2605.00809","citing_title":"Let ViT Speak: Generative Language-Image Pre-training","ref_index":74,"is_internal_anchor":true},{"citing_arxiv_id":"2410.13720","citing_title":"Movie Gen: A Cast of Media Foundation Models","ref_index":75,"is_internal_anchor":true},{"citing_arxiv_id":"2604.21786","citing_title":"From Codebooks to VLMs: Evaluating Automated Visual Discourse Analysis for Climate Change on Social Media","ref_index":59,"is_internal_anchor":true},{"citing_arxiv_id":"2604.13313","citing_title":"Concrete Jungle: Towards Concreteness Paved Contrastive Negative Mining for Compositional Understanding","ref_index":47,"is_internal_anchor":true},{"citing_arxiv_id":"2604.12081","citing_title":"Human-Inspired Context-Selective Multimodal Memory for Social Robots","ref_index":56,"is_internal_anchor":true},{"citing_arxiv_id":"2604.12833","citing_title":"Challenging Vision-Language Models with Physically Deployable Multimodal Semantic Lighting Attacks","ref_index":42,"is_internal_anchor":true}]},"formal_canon":{"evidence_count":2,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/PREMJTZC4J7RNK4IHH6M6UBNEE","json":"https://pith.science/pith/PREMJTZC4J7RNK4IHH6M6UBNEE.json","graph_json":"https://pith.science/api/pith-number/PREMJTZC4J7RNK4IHH6M6UBNEE/graph.json","events_json":"https://pith.science/api/pith-number/PREMJTZC4J7RNK4IHH6M6UBNEE/events.json","paper":"https://pith.science/paper/PREMJTZC"},"agent_actions":{"view_html":"https://pith.science/pith/PREMJTZC4J7RNK4IHH6M6UBNEE","download_json":"https://pith.science/pith/PREMJTZC4J7RNK4IHH6M6UBNEE.json","view_paper":"https://pith.science/paper/PREMJTZC","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2309.16671&json=true","fetch_graph":"https://pith.science/api/pith-number/PREMJTZC4J7RNK4IHH6M6UBNEE/graph.json","fetch_events":"https://pith.science/api/pith-number/PREMJTZC4J7RNK4IHH6M6UBNEE/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/PREMJTZC4J7RNK4IHH6M6UBNEE/action/timestamp_anchor","attest_storage":"https://pith.science/pith/PREMJTZC4J7RNK4IHH6M6UBNEE/action/storage_attestation","attest_author":"https://pith.science/pith/PREMJTZC4J7RNK4IHH6M6UBNEE/action/author_attestation","sign_citation":"https://pith.science/pith/PREMJTZC4J7RNK4IHH6M6UBNEE/action/citation_signature","submit_replication":"https://pith.science/pith/PREMJTZC4J7RNK4IHH6M6UBNEE/action/replication_record"}},"created_at":"2026-05-17T23:38:48.378761+00:00","updated_at":"2026-05-17T23:38:48.378761+00:00"}