{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2023:X5N46CMJETB4TB5ED6TJANC2FV","short_pith_number":"pith:X5N46CMJ","schema_version":"1.0","canonical_sha256":"bf5bcf098924c3c987a41fa690345a2d4c3f37b27b81749215a8edb78e20ed8c","source":{"kind":"arxiv","id":"2303.15343","version":4},"attestation_state":"computed","paper":{"title":"Sigmoid Loss for Language Image Pre-Training","license":"http://creativecommons.org/licenses/by/4.0/","headline":"A pairwise sigmoid loss for image-text pre-training achieves 84.5% zero-shot ImageNet accuracy using only four TPU chips in two days.","cross_cats":["cs.AI"],"primary_cat":"cs.CV","authors_text":"Alexander Kolesnikov, Basil Mustafa, Lucas Beyer, Xiaohua Zhai","submitted_at":"2023-03-27T15:53:01Z","abstract_excerpt":"We propose a simple pairwise Sigmoid loss for Language-Image Pre-training (SigLIP). Unlike standard contrastive learning with softmax normalization, the sigmoid loss operates solely on image-text pairs and does not require a global view of the pairwise similarities for normalization. The sigmoid loss simultaneously allows further scaling up the batch size, while also performing better at smaller batch sizes. Combined with Locked-image Tuning, with only four TPUv4 chips, we train a SigLiT model that achieves 84.5% ImageNet zero-shot accuracy in two days. The disentanglement of the batch size fr"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":true,"formal_links_present":false},"canonical_record":{"source":{"id":"2303.15343","kind":"arxiv","version":4},"metadata":{"license":"http://creativecommons.org/licenses/by/4.0/","primary_cat":"cs.CV","submitted_at":"2023-03-27T15:53:01Z","cross_cats_sorted":["cs.AI"],"title_canon_sha256":"7f4a056490916a9af113a28c02097641876e16d9ab2879231c7be248053b6bdb","abstract_canon_sha256":"cc0332e47d7937841046a3090d20ffdd67b32b1eb76a47acb7de21d23a708af6"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-05-17T23:38:47.786092Z","signature_b64":"nGnGFCYMhIikKsby995XPS6hdDYqbDY/tyy0U4q70ifbffFx5gp92nO8cvLfhYb2MgHtuP6TsDF70kBAW92MAw==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"bf5bcf098924c3c987a41fa690345a2d4c3f37b27b81749215a8edb78e20ed8c","last_reissued_at":"2026-05-17T23:38:47.785603Z","signature_status":"signed_v1","first_computed_at":"2026-05-17T23:38:47.785603Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"Sigmoid Loss for Language Image Pre-Training","license":"http://creativecommons.org/licenses/by/4.0/","headline":"A pairwise sigmoid loss for image-text pre-training achieves 84.5% zero-shot ImageNet accuracy using only four TPU chips in two days.","cross_cats":["cs.AI"],"primary_cat":"cs.CV","authors_text":"Alexander Kolesnikov, Basil Mustafa, Lucas Beyer, Xiaohua Zhai","submitted_at":"2023-03-27T15:53:01Z","abstract_excerpt":"We propose a simple pairwise Sigmoid loss for Language-Image Pre-training (SigLIP). Unlike standard contrastive learning with softmax normalization, the sigmoid loss operates solely on image-text pairs and does not require a global view of the pairwise similarities for normalization. The sigmoid loss simultaneously allows further scaling up the batch size, while also performing better at smaller batch sizes. Combined with Locked-image Tuning, with only four TPUv4 chips, we train a SigLiT model that achieves 84.5% ImageNet zero-shot accuracy in two days. The disentanglement of the batch size fr"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"Combined with Locked-image Tuning, with only four TPUv4 chips, we train a SigLiT model that achieves 84.5% ImageNet zero-shot accuracy in two days.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That the sigmoid loss, which forgoes global batch normalization, will continue to produce high-quality representations when scaled to new datasets or model sizes without additional hyper-parameter tuning.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"SigLIP replaces softmax-based contrastive loss with a simple pairwise sigmoid loss for vision-language pre-training, decoupling batch size from normalization and reaching strong zero-shot performance with limited compute.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"A pairwise sigmoid loss for image-text pre-training achieves 84.5% zero-shot ImageNet accuracy using only four TPU chips in two days.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"d80ef83f6df86463442678c0aff55eadcfaffab928244d2d897958b5ce0d08b4"},"source":{"id":"2303.15343","kind":"arxiv","version":4},"verdict":{"id":"7220d1d9-574d-4d8a-a61f-c6040846dd57","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-16T13:00:19.151560Z","strongest_claim":"Combined with Locked-image Tuning, with only four TPUv4 chips, we train a SigLiT model that achieves 84.5% ImageNet zero-shot accuracy in two days.","one_line_summary":"SigLIP replaces softmax-based contrastive loss with a simple pairwise sigmoid loss for vision-language pre-training, decoupling batch size from normalization and reaching strong zero-shot performance with limited compute.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That the sigmoid loss, which forgoes global batch normalization, will continue to produce high-quality representations when scaled to new datasets or model sizes without additional hyper-parameter tuning.","pith_extraction_headline":"A pairwise sigmoid loss for image-text pre-training achieves 84.5% zero-shot ImageNet accuracy using only four TPU chips in two days."},"references":{"count":60,"sample":[{"doi":"","year":2023,"title":"Getting vit in shape: Scaling laws for compute-optimal model design","work_id":"3d85a12f-9454-4f5c-bca9-b96d474ddde2","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2019,"title":"ObjectNet: A large-scale bias-controlled dataset for pushing the limits of object recognition models","work_id":"2f994755-ea11-439a-a510-79be7aa13443","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2006,"title":"Are we done with imagenet?","work_id":"9efae043-283b-44ae-8324-207d3747f93f","ref_index":3,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2022,"title":"Bet- ter plain vit baselines for imagenet-1k, 2022","work_id":"8b408975-f8cf-4010-ace0-d6cd6ac702ec","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2022,"title":"Lucas Beyer, Xiaohua Zhai, and Alexander Kolesnikov. Big vision. https://github.com/google-research/ big_vision, 2022. 10, 17","work_id":"96277c24-f45f-4e02-b1c2-2e713c7788c7","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":60,"snapshot_sha256":"beed77e7aaad0bc528e634f9078674503ab1cdd1b8cf0caad84708ba4148c8e0","internal_anchors":9},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2303.15343","created_at":"2026-05-17T23:38:47.785681+00:00"},{"alias_kind":"arxiv_version","alias_value":"2303.15343v4","created_at":"2026-05-17T23:38:47.785681+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2303.15343","created_at":"2026-05-17T23:38:47.785681+00:00"},{"alias_kind":"pith_short_12","alias_value":"X5N46CMJETB4","created_at":"2026-05-18T12:33:37.589309+00:00"},{"alias_kind":"pith_short_16","alias_value":"X5N46CMJETB4TB5E","created_at":"2026-05-18T12:33:37.589309+00:00"},{"alias_kind":"pith_short_8","alias_value":"X5N46CMJ","created_at":"2026-05-18T12:33:37.589309+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":31,"internal_anchor_count":31,"sample":[{"citing_arxiv_id":"2604.05157","citing_title":"IntentScore: Intent-Conditioned Action Evaluation for Computer-Use Agents","ref_index":19,"is_internal_anchor":true},{"citing_arxiv_id":"2605.23556","citing_title":"Is Dimensionality a Barrier for Retrieval Models?","ref_index":61,"is_internal_anchor":true},{"citing_arxiv_id":"2605.23136","citing_title":"A Gaia-linked High-purity QSO Candidate Catalog in Selected Fields with Extinction-binned Calibration and Spectrum-informed Training","ref_index":76,"is_internal_anchor":true},{"citing_arxiv_id":"2504.09925","citing_title":"FLARE: Fully Integration of Vision-Language Representations for Deep Cross-Modal Understanding","ref_index":76,"is_internal_anchor":true},{"citing_arxiv_id":"2505.13255","citing_title":"Policy Contrastive Decoding for Robotic Foundation Models","ref_index":22,"is_internal_anchor":true},{"citing_arxiv_id":"2507.01201","citing_title":"Escaping Plato's Cave: JAM for Aligning Independently Trained Vision and Language Models","ref_index":42,"is_internal_anchor":true},{"citing_arxiv_id":"2604.16503","citing_title":"Motif-Video 2B: Technical Report","ref_index":50,"is_internal_anchor":true},{"citing_arxiv_id":"2605.16165","citing_title":"Second-Order Multi-Level Variance Correction for Modality Competition in Multimodal Models","ref_index":27,"is_internal_anchor":true},{"citing_arxiv_id":"2605.19410","citing_title":"Vision Harnessing Agent for Open Ad-hoc Segmentation","ref_index":18,"is_internal_anchor":true},{"citing_arxiv_id":"2506.05831","citing_title":"HeartcareGPT: A Unified Multimodal ECG Suite for Dual Signal-Image Modeling and Understanding","ref_index":19,"is_internal_anchor":true},{"citing_arxiv_id":"2508.03583","citing_title":"OpenLifelogQA: An Open-Ended Multi-Modal Lifelog Question-Answering Dataset","ref_index":26,"is_internal_anchor":true},{"citing_arxiv_id":"2512.15977","citing_title":"Are vision-language models ready to zero-shot replace supervised classification models in agriculture?","ref_index":11,"is_internal_anchor":true},{"citing_arxiv_id":"2509.06951","citing_title":"F1: A Vision-Language-Action Model Bridging Understanding and Generation to Actions","ref_index":29,"is_internal_anchor":true},{"citing_arxiv_id":"2602.19577","citing_title":"Chasing Ghosts: A Simulation-to-Real Olfactory Navigation Stack with Optional Vision Augmentation","ref_index":27,"is_internal_anchor":true},{"citing_arxiv_id":"2605.14211","citing_title":"ASH: Agents that Self-Hone via Embodied Learning","ref_index":43,"is_internal_anchor":true},{"citing_arxiv_id":"2605.03259","citing_title":"CropVLM: A Domain-Adapted Vision-Language Model for Open-Set Crop Analysis","ref_index":38,"is_internal_anchor":true},{"citing_arxiv_id":"2604.24559","citing_title":"Aligned Multi-View Scripts for Universal Chart-to-Code Generation","ref_index":3,"is_internal_anchor":true},{"citing_arxiv_id":"2605.06481","citing_title":"OA-WAM: Object-Addressable World Action Model for Robust Robot Manipulation","ref_index":93,"is_internal_anchor":true},{"citing_arxiv_id":"2605.00663","citing_title":"Affordance Agent Harness: Verification-Gated Skill Orchestration","ref_index":83,"is_internal_anchor":true},{"citing_arxiv_id":"2604.12358","citing_title":"Why and When Visual Token Pruning Fails? A Study on Relevant Visual Information Shift in MLLMs Decoding","ref_index":51,"is_internal_anchor":true},{"citing_arxiv_id":"2604.16503","citing_title":"Motif-Video 2B: Technical Report","ref_index":50,"is_internal_anchor":true},{"citing_arxiv_id":"2604.07201","citing_title":"BRIDGE: Multimodal-to-Text Retrieval via Reinforcement-Learned Query Alignment","ref_index":44,"is_internal_anchor":true},{"citing_arxiv_id":"2604.09862","citing_title":"FF3R: Feedforward Feature 3D Reconstruction from Unconstrained views","ref_index":47,"is_internal_anchor":true},{"citing_arxiv_id":"2604.07079","citing_title":"MARVEL: Multimodal Adaptive Reasoning-intensiVe Expand-rerank and retrievaL","ref_index":36,"is_internal_anchor":true},{"citing_arxiv_id":"2605.00663","citing_title":"Affordance Agent Harness: Verification-Gated Skill Orchestration","ref_index":83,"is_internal_anchor":true}]},"formal_canon":{"evidence_count":0,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/X5N46CMJETB4TB5ED6TJANC2FV","json":"https://pith.science/pith/X5N46CMJETB4TB5ED6TJANC2FV.json","graph_json":"https://pith.science/api/pith-number/X5N46CMJETB4TB5ED6TJANC2FV/graph.json","events_json":"https://pith.science/api/pith-number/X5N46CMJETB4TB5ED6TJANC2FV/events.json","paper":"https://pith.science/paper/X5N46CMJ"},"agent_actions":{"view_html":"https://pith.science/pith/X5N46CMJETB4TB5ED6TJANC2FV","download_json":"https://pith.science/pith/X5N46CMJETB4TB5ED6TJANC2FV.json","view_paper":"https://pith.science/paper/X5N46CMJ","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2303.15343&json=true","fetch_graph":"https://pith.science/api/pith-number/X5N46CMJETB4TB5ED6TJANC2FV/graph.json","fetch_events":"https://pith.science/api/pith-number/X5N46CMJETB4TB5ED6TJANC2FV/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/X5N46CMJETB4TB5ED6TJANC2FV/action/timestamp_anchor","attest_storage":"https://pith.science/pith/X5N46CMJETB4TB5ED6TJANC2FV/action/storage_attestation","attest_author":"https://pith.science/pith/X5N46CMJETB4TB5ED6TJANC2FV/action/author_attestation","sign_citation":"https://pith.science/pith/X5N46CMJETB4TB5ED6TJANC2FV/action/citation_signature","submit_replication":"https://pith.science/pith/X5N46CMJETB4TB5ED6TJANC2FV/action/replication_record"}},"created_at":"2026-05-17T23:38:47.785681+00:00","updated_at":"2026-05-17T23:38:47.785681+00:00"}