{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2023:D5P4DFSMRE2GY3GALFZAHJKPJS","short_pith_number":"pith:D5P4DFSM","schema_version":"1.0","canonical_sha256":"1f5fc1964c89346c6cc0597203a54f4ca3c5077e9cedad67ec1893384ad44ec7","source":{"kind":"arxiv","id":"2302.09419","version":3},"attestation_state":"computed","paper":{"title":"A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"","cross_cats":["cs.CL","cs.LG"],"primary_cat":"cs.AI","authors_text":"(2) Beihang University, (3) Lehigh University, (4) Macquarie University, (5) Nanyang Technological University, (6) University of California San Diego, (7) Salesforce AI Research, (8) Duke University, (9) University of Illinois at Chicago), Caiming Xiong (7), Ce Zhou (1), Cheng Ji (2), Chen Li (2), Guangjing Wang (1), Hao Peng (2), Jian Pei (8), Jianxin Li (2), Jia Wu (4), Jun Yu (3), Kai Zhang (3), Lichao Sun (3) ((1) Michigan State University, Lifang He (3), Pengtao Xie (6), Philip S. Yu (9), Qian Li (2), Qiben Yan (1), Yixin Liu (3), Ziwei Liu (5)","submitted_at":"2023-02-18T20:51:09Z","abstract_excerpt":"Pretrained Foundation Models (PFMs) are regarded as the foundation for various downstream tasks with different data modalities. A PFM (e.g., BERT, ChatGPT, and GPT-4) is trained on large-scale data which provides a reasonable parameter initialization for a wide range of downstream applications. BERT learns bidirectional encoder representations from Transformers, which are trained on large datasets as contextual language models. Similarly, the generative pretrained transformer (GPT) method employs Transformers as the feature extractor and is trained using an autoregressive paradigm on large dat"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":false,"formal_links_present":false},"canonical_record":{"source":{"id":"2302.09419","kind":"arxiv","version":3},"metadata":{"license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","primary_cat":"cs.AI","submitted_at":"2023-02-18T20:51:09Z","cross_cats_sorted":["cs.CL","cs.LG"],"title_canon_sha256":"1ad8681d1ff955e6d8996bbdbdd430edfa00a503969518b72a73efe69a837ba3","abstract_canon_sha256":"d7902c8a52cfc602e3e230254d4e9643f5b4f3324a872c4ac464b3d480d986ba"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-07-05T06:05:34.183434Z","signature_b64":"F0kY1PTZ1BSB2KK/BPtK5GBSv2V11ELbT5rLjJ+cwUeAe7e+3ZXsbROl4IGgiY3I6EDBsp7RMt8U3SMg9mYnAQ==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"1f5fc1964c89346c6cc0597203a54f4ca3c5077e9cedad67ec1893384ad44ec7","last_reissued_at":"2026-07-05T06:05:34.182940Z","signature_status":"signed_v1","first_computed_at":"2026-07-05T06:05:34.182940Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"","cross_cats":["cs.CL","cs.LG"],"primary_cat":"cs.AI","authors_text":"(2) Beihang University, (3) Lehigh University, (4) Macquarie University, (5) Nanyang Technological University, (6) University of California San Diego, (7) Salesforce AI Research, (8) Duke University, (9) University of Illinois at Chicago), Caiming Xiong (7), Ce Zhou (1), Cheng Ji (2), Chen Li (2), Guangjing Wang (1), Hao Peng (2), Jian Pei (8), Jianxin Li (2), Jia Wu (4), Jun Yu (3), Kai Zhang (3), Lichao Sun (3) ((1) Michigan State University, Lifang He (3), Pengtao Xie (6), Philip S. Yu (9), Qian Li (2), Qiben Yan (1), Yixin Liu (3), Ziwei Liu (5)","submitted_at":"2023-02-18T20:51:09Z","abstract_excerpt":"Pretrained Foundation Models (PFMs) are regarded as the foundation for various downstream tasks with different data modalities. A PFM (e.g., BERT, ChatGPT, and GPT-4) is trained on large-scale data which provides a reasonable parameter initialization for a wide range of downstream applications. BERT learns bidirectional encoder representations from Transformers, which are trained on large datasets as contextual language models. Similarly, the generative pretrained transformer (GPT) method employs Transformers as the feature extractor and is trained using an autoregressive paradigm on large dat"},"claims":{"count":0,"items":[],"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"source":{"id":"2302.09419","kind":"arxiv","version":3},"verdict":{"id":null,"model_set":{},"created_at":null,"strongest_claim":"","one_line_summary":"","pipeline_version":null,"weakest_assumption":"","pith_extraction_headline":""},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2302.09419/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2302.09419","created_at":"2026-07-05T06:05:34.182998+00:00"},{"alias_kind":"arxiv_version","alias_value":"2302.09419v3","created_at":"2026-07-05T06:05:34.182998+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2302.09419","created_at":"2026-07-05T06:05:34.182998+00:00"},{"alias_kind":"pith_short_12","alias_value":"D5P4DFSMRE2G","created_at":"2026-07-05T06:05:34.182998+00:00"},{"alias_kind":"pith_short_16","alias_value":"D5P4DFSMRE2GY3GA","created_at":"2026-07-05T06:05:34.182998+00:00"},{"alias_kind":"pith_short_8","alias_value":"D5P4DFSM","created_at":"2026-07-05T06:05:34.182998+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":9,"internal_anchor_count":0,"sample":[{"citing_arxiv_id":"2606.09085","citing_title":"Mixture-of-Experts Transformer for Automatic Modulation Recognition","ref_index":29,"is_internal_anchor":false},{"citing_arxiv_id":"2606.08303","citing_title":"GeoGNN: Time Series Geo-Localization using Two-Tower Graph Neural Networks","ref_index":68,"is_internal_anchor":false},{"citing_arxiv_id":"2501.05465","citing_title":"Small Language Models (SLMs) Can Still Pack a Punch: A survey (updated 2026)","ref_index":158,"is_internal_anchor":false},{"citing_arxiv_id":"2509.21820","citing_title":"Can LLMs Generate and Solve Linguistic Olympiad Puzzles?","ref_index":2,"is_internal_anchor":false},{"citing_arxiv_id":"2307.06435","citing_title":"A Comprehensive Overview of Large Language Models","ref_index":53,"is_internal_anchor":false},{"citing_arxiv_id":"2402.06196","citing_title":"Large Language Models: A Survey","ref_index":8,"is_internal_anchor":false},{"citing_arxiv_id":"2604.19015","citing_title":"FedProxy: Federated Fine-Tuning of LLMs via Proxy SLMs and Heterogeneity-Aware Fusion","ref_index":28,"is_internal_anchor":false},{"citing_arxiv_id":"2604.08398","citing_title":"ADAPTive Input Training for Many-to-One Pre-Training on Time-Series Classification","ref_index":18,"is_internal_anchor":false},{"citing_arxiv_id":"2303.18223","citing_title":"A Survey of Large Language Models","ref_index":39,"is_internal_anchor":false}]},"formal_canon":{"evidence_count":0,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/D5P4DFSMRE2GY3GALFZAHJKPJS","json":"https://pith.science/pith/D5P4DFSMRE2GY3GALFZAHJKPJS.json","graph_json":"https://pith.science/api/pith-number/D5P4DFSMRE2GY3GALFZAHJKPJS/graph.json","events_json":"https://pith.science/api/pith-number/D5P4DFSMRE2GY3GALFZAHJKPJS/events.json","paper":"https://pith.science/paper/D5P4DFSM"},"agent_actions":{"view_html":"https://pith.science/pith/D5P4DFSMRE2GY3GALFZAHJKPJS","download_json":"https://pith.science/pith/D5P4DFSMRE2GY3GALFZAHJKPJS.json","view_paper":"https://pith.science/paper/D5P4DFSM","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2302.09419&json=true","fetch_graph":"https://pith.science/api/pith-number/D5P4DFSMRE2GY3GALFZAHJKPJS/graph.json","fetch_events":"https://pith.science/api/pith-number/D5P4DFSMRE2GY3GALFZAHJKPJS/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/D5P4DFSMRE2GY3GALFZAHJKPJS/action/timestamp_anchor","attest_storage":"https://pith.science/pith/D5P4DFSMRE2GY3GALFZAHJKPJS/action/storage_attestation","attest_author":"https://pith.science/pith/D5P4DFSMRE2GY3GALFZAHJKPJS/action/author_attestation","sign_citation":"https://pith.science/pith/D5P4DFSMRE2GY3GALFZAHJKPJS/action/citation_signature","submit_replication":"https://pith.science/pith/D5P4DFSMRE2GY3GALFZAHJKPJS/action/replication_record"}},"created_at":"2026-07-05T06:05:34.182998+00:00","updated_at":"2026-07-05T06:05:34.182998+00:00"}