{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2022:CWLACUSRWUFEKM6AMMMAN3MEI5","short_pith_number":"pith:CWLACUSR","schema_version":"1.0","canonical_sha256":"1596015251b50a4533c0631806ed8447695f4402afde97f760aa83815f3887d3","source":{"kind":"arxiv","id":"2205.01068","version":4},"attestation_state":"computed","paper":{"title":"OPT: Open Pre-trained Transformer Language Models","license":"http://creativecommons.org/licenses/by/4.0/","headline":"A suite of open decoder-only transformer models up to 175B parameters matches GPT-3 performance while using only one-seventh the carbon footprint.","cross_cats":["cs.LG"],"primary_cat":"cs.CL","authors_text":"Anjali Sridhar, Christopher Dewan, Daniel Simig, Kurt Shuster, Luke Zettlemoyer, Mikel Artetxe, Mona Diab, Moya Chen, Myle Ott, Naman Goyal, Punit Singh Koura, Sam Shleifer, Shuohui Chen, Stephen Roller, Susan Zhang, Tianlu Wang, Todor Mihaylov, Xian Li, Xi Victoria Lin","submitted_at":"2022-05-02T17:49:50Z","abstract_excerpt":"Large language models, which are often trained for hundreds of thousands of compute days, have shown remarkable capabilities for zero- and few-shot learning. Given their computational cost, these models are difficult to replicate without significant capital. For the few that are available through APIs, no access is granted to the full model weights, making them difficult to study. We present Open Pre-trained Transformers (OPT), a suite of decoder-only pre-trained transformers ranging from 125M to 175B parameters, which we aim to fully and responsibly share with interested researchers. We show "},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":true,"formal_links_present":true},"canonical_record":{"source":{"id":"2205.01068","kind":"arxiv","version":4},"metadata":{"license":"http://creativecommons.org/licenses/by/4.0/","primary_cat":"cs.CL","submitted_at":"2022-05-02T17:49:50Z","cross_cats_sorted":["cs.LG"],"title_canon_sha256":"45acb651556d7ed8eb487e5a4ae17192714946bac38d32ca93c68aa2f50bcf5c","abstract_canon_sha256":"d2107f53416b5be7f0bb6def77a89d29c846e0845f2cb1d2776e3de4e806af70"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-07-05T04:33:35.683132Z","signature_b64":"NfZm5M5fQwkZHcwkRaCLFAjRuoVo2zEUZJgfc4s5d9LzMj0q8O0lNsLnnX/j9ltkJSeTwDoRsLt5wjsZe4B7Dg==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"1596015251b50a4533c0631806ed8447695f4402afde97f760aa83815f3887d3","last_reissued_at":"2026-07-05T04:33:35.682666Z","signature_status":"signed_v1","first_computed_at":"2026-07-05T04:33:35.682666Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"OPT: Open Pre-trained Transformer Language Models","license":"http://creativecommons.org/licenses/by/4.0/","headline":"A suite of open decoder-only transformer models up to 175B parameters matches GPT-3 performance while using only one-seventh the carbon footprint.","cross_cats":["cs.LG"],"primary_cat":"cs.CL","authors_text":"Anjali Sridhar, Christopher Dewan, Daniel Simig, Kurt Shuster, Luke Zettlemoyer, Mikel Artetxe, Mona Diab, Moya Chen, Myle Ott, Naman Goyal, Punit Singh Koura, Sam Shleifer, Shuohui Chen, Stephen Roller, Susan Zhang, Tianlu Wang, Todor Mihaylov, Xian Li, Xi Victoria Lin","submitted_at":"2022-05-02T17:49:50Z","abstract_excerpt":"Large language models, which are often trained for hundreds of thousands of compute days, have shown remarkable capabilities for zero- and few-shot learning. Given their computational cost, these models are difficult to replicate without significant capital. For the few that are available through APIs, no access is granted to the full model weights, making them difficult to study. We present Open Pre-trained Transformers (OPT), a suite of decoder-only pre-trained transformers ranging from 125M to 175B parameters, which we aim to fully and responsibly share with interested researchers. We show "},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"We show that OPT-175B is comparable to GPT-3, while requiring only 1/7th the carbon footprint to develop.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"The benchmarks and evaluation protocols used to establish comparability between OPT-175B and GPT-3 are fair, comprehensive, and not affected by differences in training data or optimization details.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"OPT releases open decoder-only transformers up to 175B parameters that match GPT-3 performance at one-seventh the carbon cost, along with code and training logs.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"A suite of open decoder-only transformer models up to 175B parameters matches GPT-3 performance while using only one-seventh the carbon footprint.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"8ab40f7ee2e277ac9022bc1947c1d858bab27146fd55e6501d37bae04b54071d"},"source":{"id":"2205.01068","kind":"arxiv","version":4},"verdict":{"id":"1d6ba001-ceeb-4bc0-9e9c-dd75184cc129","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-10T20:48:21.628279Z","strongest_claim":"We show that OPT-175B is comparable to GPT-3, while requiring only 1/7th the carbon footprint to develop.","one_line_summary":"OPT releases open decoder-only transformers up to 175B parameters that match GPT-3 performance at one-seventh the carbon cost, along with code and training logs.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"The benchmarks and evaluation protocols used to establish comparability between OPT-175B and GPT-3 are fair, comprehensive, and not affected by differences in training data or optimization details.","pith_extraction_headline":"A suite of open decoder-only transformer models up to 175B parameters matches GPT-3 performance while using only one-seventh the carbon footprint."},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2205.01068/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":294,"sample":[{"doi":"","year":2021,"title":"Naman Goyal and Cynthia Gao and Vishrav Chaudhary and Peng. The. CoRR , volume =. 2021 , url =. 2106.03193 , timestamp =","work_id":"a8f99012-fb3a-463a-9161-3ecabb9042fd","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"10.1609/aaai.v34i05.6399","year":2020,"title":"Proceedings of the AAAI Conference on Artificial Intelligence , author=","work_id":"a80a62e7-1fb6-4805-a09c-3447c2c1145b","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"10.1609/aaai.v34i05.6239","year":2020,"title":"PIQA: Reasoning about physical commonsense in natural language","work_id":"d1a9f293-b596-42e3-b48c-112b9f9a06df","ref_index":3,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":null,"title":"Neural Network Ac- ceptability Judgments","work_id":"ee6536e2-986a-4e85-877b-cd6cf6b9219e","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":1975,"title":"Biochimica et Biophysica Acta (BBA)-Protein Structure , volume=","work_id":"41f4e658-9007-4c08-8088-398e658e9daf","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":294,"snapshot_sha256":"339c43b1702284963b3144d9586466b5e2b4f9c5b15dc5159827f659dc673aaa","internal_anchors":38},"formal_canon":{"evidence_count":2,"snapshot_sha256":"197a4fd29ffbcbb3208587b87b52f4e8a37221b88163fbf9847ecd55f7220af6"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2205.01068","created_at":"2026-07-05T04:33:35.682739+00:00"},{"alias_kind":"arxiv_version","alias_value":"2205.01068v4","created_at":"2026-07-05T04:33:35.682739+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2205.01068","created_at":"2026-07-05T04:33:35.682739+00:00"},{"alias_kind":"pith_short_12","alias_value":"CWLACUSRWUFE","created_at":"2026-07-05T04:33:35.682739+00:00"},{"alias_kind":"pith_short_16","alias_value":"CWLACUSRWUFEKM6A","created_at":"2026-07-05T04:33:35.682739+00:00"},{"alias_kind":"pith_short_8","alias_value":"CWLACUSR","created_at":"2026-07-05T04:33:35.682739+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":268,"internal_anchor_count":268,"sample":[{"citing_arxiv_id":"2607.05694","citing_title":"Beyond Heuristic Tuning: Power-Calibrated LLM Watermarking","ref_index":67,"is_internal_anchor":true},{"citing_arxiv_id":"2607.05708","citing_title":"Akashic: A Low-Overhead LLM Inference Service with MemAttention","ref_index":51,"is_internal_anchor":true},{"citing_arxiv_id":"2607.07527","citing_title":"A Unified Detection Framework for AI-Related Content and Artifacts","ref_index":32,"is_internal_anchor":true},{"citing_arxiv_id":"2604.17708","citing_title":"Co-evolving Agent Architectures and Interpretable Reasoning for Automated Optimization","ref_index":23,"is_internal_anchor":true},{"citing_arxiv_id":"2606.25285","citing_title":"EPTS: Elastic Post-Training Sparsity for Efficient Large Language Model Compression","ref_index":43,"is_internal_anchor":true},{"citing_arxiv_id":"2606.24163","citing_title":"CORE-BREW: LLR-Based Soft Decoding for Robust Multi-Bit LLM Watermarking","ref_index":29,"is_internal_anchor":true},{"citing_arxiv_id":"2606.23285","citing_title":"On the Effect of Segmentation Width and Cluster Size on Speech Resynthesis and Continuation in Generative Spoken Language Models","ref_index":35,"is_internal_anchor":true},{"citing_arxiv_id":"2606.22873","citing_title":"SingGuard: A Policy-Adaptive Multimodal LLM Guardrail with Dynamic Reasoning","ref_index":57,"is_internal_anchor":true},{"citing_arxiv_id":"2606.22189","citing_title":"L20-Edu-135M: An Auditable Single-GPU Study of Data-Efficient Small Language Modeling","ref_index":23,"is_internal_anchor":true},{"citing_arxiv_id":"2606.21851","citing_title":"TALAS: Teacher-Anchored Layer Alignment with Adaptive Sharpness-Aware Minimization for Embedding Distillation","ref_index":72,"is_internal_anchor":true},{"citing_arxiv_id":"2606.18105","citing_title":"OmniPlan: An Adaptive Framework for Timely and Near-Optimal Network Planning Optimization","ref_index":47,"is_internal_anchor":true},{"citing_arxiv_id":"2606.17781","citing_title":"MIVE: A Minimalist Integer Vector Engine for Softmax LayerNorm and RMSNorm Acceleration","ref_index":16,"is_internal_anchor":true},{"citing_arxiv_id":"2606.17110","citing_title":"Loss Landscape Poisoning: Targeted Extraction of Unseen Training Data from LLMs","ref_index":87,"is_internal_anchor":true},{"citing_arxiv_id":"2606.17030","citing_title":"Qwen-RobotWorld Technical Report: Unifying Embodied World Modeling through Language-Conditioned Video Generation","ref_index":74,"is_internal_anchor":true},{"citing_arxiv_id":"2606.14970","citing_title":"Zero-order Parameter-free Optimization for LMO-based Methods: Novel Approach for Efficient Fine-tuning","ref_index":91,"is_internal_anchor":true},{"citing_arxiv_id":"2607.01876","citing_title":"SAB-LVLM: Significance-Aware Binarization for Large Vision-Language Models","ref_index":2,"is_internal_anchor":true},{"citing_arxiv_id":"2606.11592","citing_title":"Defense Against Prompt Inversion Attacks: An Information-Theoretic Approach for LLM Collaborative Inference","ref_index":18,"is_internal_anchor":true},{"citing_arxiv_id":"2606.12234","citing_title":"On The Effectiveness-Fluency Trade-Off In LLM Conditioning: A Systematic Study","ref_index":43,"is_internal_anchor":true},{"citing_arxiv_id":"2606.12633","citing_title":"ECA: Efficient Continual Alignment for Open-Ended Image-to-Text Generation","ref_index":88,"is_internal_anchor":true},{"citing_arxiv_id":"2606.11853","citing_title":"Task-Aware Structured Memory for Dynamic Multi-modal In-Context Learning","ref_index":63,"is_internal_anchor":true},{"citing_arxiv_id":"2607.01617","citing_title":"3DLS: A 3D Logic-Stacked Architecture for Disaggregated LLM Serving","ref_index":8,"is_internal_anchor":true},{"citing_arxiv_id":"2606.10488","citing_title":"5% > 100%: Flatness Preference is All You Need for Multimodal Parameter-Efficient Fine-Tuning","ref_index":41,"is_internal_anchor":true},{"citing_arxiv_id":"2606.09132","citing_title":"Vision Language Model Helps Private Information De-Identification in Vision Data","ref_index":55,"is_internal_anchor":true},{"citing_arxiv_id":"2606.09125","citing_title":"Unveiling Privacy Risks in Multi-modal Large Language Models: Task-specific Vulnerabilities and Mitigation Challenges","ref_index":68,"is_internal_anchor":true},{"citing_arxiv_id":"2606.09401","citing_title":"Benchmarking Empirical Privacy Protection for Adaptations of Large Language Models","ref_index":181,"is_internal_anchor":true}]},"formal_canon":{"evidence_count":2,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/CWLACUSRWUFEKM6AMMMAN3MEI5","json":"https://pith.science/pith/CWLACUSRWUFEKM6AMMMAN3MEI5.json","graph_json":"https://pith.science/api/pith-number/CWLACUSRWUFEKM6AMMMAN3MEI5/graph.json","events_json":"https://pith.science/api/pith-number/CWLACUSRWUFEKM6AMMMAN3MEI5/events.json","paper":"https://pith.science/paper/CWLACUSR"},"agent_actions":{"view_html":"https://pith.science/pith/CWLACUSRWUFEKM6AMMMAN3MEI5","download_json":"https://pith.science/pith/CWLACUSRWUFEKM6AMMMAN3MEI5.json","view_paper":"https://pith.science/paper/CWLACUSR","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2205.01068&json=true","fetch_graph":"https://pith.science/api/pith-number/CWLACUSRWUFEKM6AMMMAN3MEI5/graph.json","fetch_events":"https://pith.science/api/pith-number/CWLACUSRWUFEKM6AMMMAN3MEI5/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/CWLACUSRWUFEKM6AMMMAN3MEI5/action/timestamp_anchor","attest_storage":"https://pith.science/pith/CWLACUSRWUFEKM6AMMMAN3MEI5/action/storage_attestation","attest_author":"https://pith.science/pith/CWLACUSRWUFEKM6AMMMAN3MEI5/action/author_attestation","sign_citation":"https://pith.science/pith/CWLACUSRWUFEKM6AMMMAN3MEI5/action/citation_signature","submit_replication":"https://pith.science/pith/CWLACUSRWUFEKM6AMMMAN3MEI5/action/replication_record"}},"created_at":"2026-07-05T04:33:35.682739+00:00","updated_at":"2026-07-05T04:33:35.682739+00:00"}