{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2025:4OPPEZYDPIZPUSILFDOQDEVLHT","short_pith_number":"pith:4OPPEZYD","schema_version":"1.0","canonical_sha256":"e39ef267037a32fa490b28dd0192ab3cc688c146dc2f6902336d3e5199910f14","source":{"kind":"arxiv","id":"2502.11458","version":1},"attestation_state":"computed","paper":{"title":"Towards Efficient Pre-training: Exploring FP4 Precision in Large Language Models","license":"http://creativecommons.org/licenses/by/4.0/","headline":"","cross_cats":["cs.AI"],"primary_cat":"cs.LG","authors_text":"Boni Hu, Ding Tang, Haoran Xu, Jiecheng Zhou, Liang Liu, Rong Fu, Weiming Zhang, Xingcheng Zhang, Yi Wang, Zhilin Pei, Zhongling Su","submitted_at":"2025-02-17T05:33:11Z","abstract_excerpt":"The burgeoning computational demands for training large language models (LLMs) necessitate efficient methods, including quantized training, which leverages low-bit arithmetic operations to reduce costs. While FP8 precision has shown potential, leveraging FP4 remains challenging due to inherent quantization errors and limited representation capability. Based on the Transformer architecture, we present an FP4 training scheme for LLMs, overcoming these obstacles through mixed-precision quantization strategies tailed for different modules and training stages. This allows us to apply the precision "},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":false,"formal_links_present":false},"canonical_record":{"source":{"id":"2502.11458","kind":"arxiv","version":1},"metadata":{"license":"http://creativecommons.org/licenses/by/4.0/","primary_cat":"cs.LG","submitted_at":"2025-02-17T05:33:11Z","cross_cats_sorted":["cs.AI"],"title_canon_sha256":"c1ee292fc10ff2c9ceb3431a5bda3ee0f9a074c684accee0e9660e313108398e","abstract_canon_sha256":"452f49e8efe7a26d34617aaa17cd0acf8677bdbb2c2ef3b74761886e2d7513ec"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-07-05T10:15:29.078261Z","signature_b64":"Vctjgaqu3OjHD2tuP+IJzTeTuCmm/RIO81vorib6+yknwa0oULqNQ0cRRLfQP6JB8Ghv2w/7C9Lxo48HXzYODQ==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"e39ef267037a32fa490b28dd0192ab3cc688c146dc2f6902336d3e5199910f14","last_reissued_at":"2026-07-05T10:15:29.077770Z","signature_status":"signed_v1","first_computed_at":"2026-07-05T10:15:29.077770Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"Towards Efficient Pre-training: Exploring FP4 Precision in Large Language Models","license":"http://creativecommons.org/licenses/by/4.0/","headline":"","cross_cats":["cs.AI"],"primary_cat":"cs.LG","authors_text":"Boni Hu, Ding Tang, Haoran Xu, Jiecheng Zhou, Liang Liu, Rong Fu, Weiming Zhang, Xingcheng Zhang, Yi Wang, Zhilin Pei, Zhongling Su","submitted_at":"2025-02-17T05:33:11Z","abstract_excerpt":"The burgeoning computational demands for training large language models (LLMs) necessitate efficient methods, including quantized training, which leverages low-bit arithmetic operations to reduce costs. While FP8 precision has shown potential, leveraging FP4 remains challenging due to inherent quantization errors and limited representation capability. Based on the Transformer architecture, we present an FP4 training scheme for LLMs, overcoming these obstacles through mixed-precision quantization strategies tailed for different modules and training stages. This allows us to apply the precision "},"claims":{"count":0,"items":[],"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"source":{"id":"2502.11458","kind":"arxiv","version":1},"verdict":{"id":null,"model_set":{},"created_at":null,"strongest_claim":"","one_line_summary":"","pipeline_version":null,"weakest_assumption":"","pith_extraction_headline":""},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2502.11458/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2502.11458","created_at":"2026-07-05T10:15:29.077829+00:00"},{"alias_kind":"arxiv_version","alias_value":"2502.11458v1","created_at":"2026-07-05T10:15:29.077829+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2502.11458","created_at":"2026-07-05T10:15:29.077829+00:00"},{"alias_kind":"pith_short_12","alias_value":"4OPPEZYDPIZP","created_at":"2026-07-05T10:15:29.077829+00:00"},{"alias_kind":"pith_short_16","alias_value":"4OPPEZYDPIZPUSIL","created_at":"2026-07-05T10:15:29.077829+00:00"},{"alias_kind":"pith_short_8","alias_value":"4OPPEZYD","created_at":"2026-07-05T10:15:29.077829+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":5,"internal_anchor_count":0,"sample":[{"citing_arxiv_id":"2510.04212","citing_title":"Why Low-Precision Transformer Training Fails: An Analysis on Flash Attention","ref_index":34,"is_internal_anchor":false},{"citing_arxiv_id":"2605.09825","citing_title":"Pretraining large language models with MXFP4 on Native FP4 Hardware","ref_index":8,"is_internal_anchor":false},{"citing_arxiv_id":"2605.09825","citing_title":"Pretraining large language models with MXFP4 on Native FP4 Hardware","ref_index":8,"is_internal_anchor":false},{"citing_arxiv_id":"2605.09825","citing_title":"Pretraining large language models with MXFP4 on Native FP4 Hardware","ref_index":8,"is_internal_anchor":false},{"citing_arxiv_id":"2604.08826","citing_title":"HiFloat4 Format for Language Model Pre-training on Ascend NPUs","ref_index":17,"is_internal_anchor":false}]},"formal_canon":{"evidence_count":0,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/4OPPEZYDPIZPUSILFDOQDEVLHT","json":"https://pith.science/pith/4OPPEZYDPIZPUSILFDOQDEVLHT.json","graph_json":"https://pith.science/api/pith-number/4OPPEZYDPIZPUSILFDOQDEVLHT/graph.json","events_json":"https://pith.science/api/pith-number/4OPPEZYDPIZPUSILFDOQDEVLHT/events.json","paper":"https://pith.science/paper/4OPPEZYD"},"agent_actions":{"view_html":"https://pith.science/pith/4OPPEZYDPIZPUSILFDOQDEVLHT","download_json":"https://pith.science/pith/4OPPEZYDPIZPUSILFDOQDEVLHT.json","view_paper":"https://pith.science/paper/4OPPEZYD","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2502.11458&json=true","fetch_graph":"https://pith.science/api/pith-number/4OPPEZYDPIZPUSILFDOQDEVLHT/graph.json","fetch_events":"https://pith.science/api/pith-number/4OPPEZYDPIZPUSILFDOQDEVLHT/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/4OPPEZYDPIZPUSILFDOQDEVLHT/action/timestamp_anchor","attest_storage":"https://pith.science/pith/4OPPEZYDPIZPUSILFDOQDEVLHT/action/storage_attestation","attest_author":"https://pith.science/pith/4OPPEZYDPIZPUSILFDOQDEVLHT/action/author_attestation","sign_citation":"https://pith.science/pith/4OPPEZYDPIZPUSILFDOQDEVLHT/action/citation_signature","submit_replication":"https://pith.science/pith/4OPPEZYDPIZPUSILFDOQDEVLHT/action/replication_record"}},"created_at":"2026-07-05T10:15:29.077829+00:00","updated_at":"2026-07-05T10:15:29.077829+00:00"}