{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2021:Y5ELGMT5Q3TJROR6WNM7BZD2I2","short_pith_number":"pith:Y5ELGMT5","schema_version":"1.0","canonical_sha256":"c748b3327d86e698ba3eb359f0e47a46aa13563cbd8f9f6323f0ffe466f92ede","source":{"kind":"arxiv","id":"2104.05158","version":7},"attestation_state":"computed","paper":{"title":"Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"","cross_cats":["cs.AI","cs.LG","cs.PF"],"primary_cat":"cs.DC","authors_text":"Aarti Basant, Adi Gangidi, Ajit Mathews, Andrew Tulloch, Assaf Eisenman, Avinash Nayak, Bharath Muthiah, Bill Jia, Ching-Hsiang Chu, Chonglin Sun, Dheevatsa Mudigere, Dimitry Melts, Dmytro Ivchenko, Ehsan K. Ardestani, Ellie Wen, Guoqiang Jerry Chen, Hong Li, Huayu Li, Jade Nie, Jianyu Huang, Jie Amy Yang, Jiyan Yang, Jiyuan Qian, Jongsoo Park, Junjie Yang, Kiran Kumar Matam, Krishna Dhulipala, Krishnakumar Nair, KR Kishore, Leon Gao, Liang Luo, Lin Qiao, Lin Yang, Mahmoud khorashadi, Manoj Krishnan, Maxim Naumov, Mikhail Smelyanskiy, Mustafa Ozdal, Pallab Bhattacharya, Petr Lapukhov, Rakesh Komuravelli, Serhat Yilmaz, Srinivas Sridharan, Tyler Graf, Vijay Rao, Whitney Zhao, Xiaodong Wang, Xing Liu, Yinbin Ma, Yuchen Hao, Yuxi Hu, Zhihao Jia, Zhuobo Feng","submitted_at":"2021-04-12T02:15:55Z","abstract_excerpt":"Deep learning recommendation models (DLRMs) are used across many business-critical services at Facebook and are the single largest AI application in terms of infrastructure demand in its data-centers. In this paper we discuss the SW/HW co-designed solution for high-performance distributed training of large-scale DLRMs. We introduce a high-performance scalable software stack based on PyTorch and pair it with the new evolution of Zion platform, namely ZionEX. We demonstrate the capability to train very large DLRMs with up to 12 Trillion parameters and show that we can attain 40X speedup in terms"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":false,"formal_links_present":false},"canonical_record":{"source":{"id":"2104.05158","kind":"arxiv","version":7},"metadata":{"license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","primary_cat":"cs.DC","submitted_at":"2021-04-12T02:15:55Z","cross_cats_sorted":["cs.AI","cs.LG","cs.PF"],"title_canon_sha256":"27d2f7f99027afa627306f7e7b54abfab6bea3cc01ab0f208a20fa4db7b77606","abstract_canon_sha256":"795b2f8a7d5d3740f4c10fd85fac0851197c22405dc87ad2d2767579bd0341b7"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-07-05T05:45:24.285505Z","signature_b64":"KKBBChekl4pQEy7u7KaIxJ+1nGZkwgsEPS8FonsCyEHprbvCgWi3CPLZ4jeli4MxMGIZ8orOP340JngpSKppDg==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"c748b3327d86e698ba3eb359f0e47a46aa13563cbd8f9f6323f0ffe466f92ede","last_reissued_at":"2026-07-05T05:45:24.284950Z","signature_status":"signed_v1","first_computed_at":"2026-07-05T05:45:24.284950Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"","cross_cats":["cs.AI","cs.LG","cs.PF"],"primary_cat":"cs.DC","authors_text":"Aarti Basant, Adi Gangidi, Ajit Mathews, Andrew Tulloch, Assaf Eisenman, Avinash Nayak, Bharath Muthiah, Bill Jia, Ching-Hsiang Chu, Chonglin Sun, Dheevatsa Mudigere, Dimitry Melts, Dmytro Ivchenko, Ehsan K. Ardestani, Ellie Wen, Guoqiang Jerry Chen, Hong Li, Huayu Li, Jade Nie, Jianyu Huang, Jie Amy Yang, Jiyan Yang, Jiyuan Qian, Jongsoo Park, Junjie Yang, Kiran Kumar Matam, Krishna Dhulipala, Krishnakumar Nair, KR Kishore, Leon Gao, Liang Luo, Lin Qiao, Lin Yang, Mahmoud khorashadi, Manoj Krishnan, Maxim Naumov, Mikhail Smelyanskiy, Mustafa Ozdal, Pallab Bhattacharya, Petr Lapukhov, Rakesh Komuravelli, Serhat Yilmaz, Srinivas Sridharan, Tyler Graf, Vijay Rao, Whitney Zhao, Xiaodong Wang, Xing Liu, Yinbin Ma, Yuchen Hao, Yuxi Hu, Zhihao Jia, Zhuobo Feng","submitted_at":"2021-04-12T02:15:55Z","abstract_excerpt":"Deep learning recommendation models (DLRMs) are used across many business-critical services at Facebook and are the single largest AI application in terms of infrastructure demand in its data-centers. In this paper we discuss the SW/HW co-designed solution for high-performance distributed training of large-scale DLRMs. We introduce a high-performance scalable software stack based on PyTorch and pair it with the new evolution of Zion platform, namely ZionEX. We demonstrate the capability to train very large DLRMs with up to 12 Trillion parameters and show that we can attain 40X speedup in terms"},"claims":{"count":0,"items":[],"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"source":{"id":"2104.05158","kind":"arxiv","version":7},"verdict":{"id":null,"model_set":{},"created_at":null,"strongest_claim":"","one_line_summary":"","pipeline_version":null,"weakest_assumption":"","pith_extraction_headline":""},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2104.05158/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2104.05158","created_at":"2026-07-05T05:45:24.285013+00:00"},{"alias_kind":"arxiv_version","alias_value":"2104.05158v7","created_at":"2026-07-05T05:45:24.285013+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2104.05158","created_at":"2026-07-05T05:45:24.285013+00:00"},{"alias_kind":"pith_short_12","alias_value":"Y5ELGMT5Q3TJ","created_at":"2026-07-05T05:45:24.285013+00:00"},{"alias_kind":"pith_short_16","alias_value":"Y5ELGMT5Q3TJROR6","created_at":"2026-07-05T05:45:24.285013+00:00"},{"alias_kind":"pith_short_8","alias_value":"Y5ELGMT5","created_at":"2026-07-05T05:45:24.285013+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":3,"internal_anchor_count":0,"sample":[{"citing_arxiv_id":"2605.10886","citing_title":"LoKA: Low-precision Kernel Applications for Recommendation Models At Scale","ref_index":53,"is_internal_anchor":false},{"citing_arxiv_id":"2605.10886","citing_title":"LoKA: Low-precision Kernel Applications for Recommendation Models At Scale","ref_index":53,"is_internal_anchor":false},{"citing_arxiv_id":"2304.11277","citing_title":"PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel","ref_index":20,"is_internal_anchor":false}]},"formal_canon":{"evidence_count":0,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/Y5ELGMT5Q3TJROR6WNM7BZD2I2","json":"https://pith.science/pith/Y5ELGMT5Q3TJROR6WNM7BZD2I2.json","graph_json":"https://pith.science/api/pith-number/Y5ELGMT5Q3TJROR6WNM7BZD2I2/graph.json","events_json":"https://pith.science/api/pith-number/Y5ELGMT5Q3TJROR6WNM7BZD2I2/events.json","paper":"https://pith.science/paper/Y5ELGMT5"},"agent_actions":{"view_html":"https://pith.science/pith/Y5ELGMT5Q3TJROR6WNM7BZD2I2","download_json":"https://pith.science/pith/Y5ELGMT5Q3TJROR6WNM7BZD2I2.json","view_paper":"https://pith.science/paper/Y5ELGMT5","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2104.05158&json=true","fetch_graph":"https://pith.science/api/pith-number/Y5ELGMT5Q3TJROR6WNM7BZD2I2/graph.json","fetch_events":"https://pith.science/api/pith-number/Y5ELGMT5Q3TJROR6WNM7BZD2I2/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/Y5ELGMT5Q3TJROR6WNM7BZD2I2/action/timestamp_anchor","attest_storage":"https://pith.science/pith/Y5ELGMT5Q3TJROR6WNM7BZD2I2/action/storage_attestation","attest_author":"https://pith.science/pith/Y5ELGMT5Q3TJROR6WNM7BZD2I2/action/author_attestation","sign_citation":"https://pith.science/pith/Y5ELGMT5Q3TJROR6WNM7BZD2I2/action/citation_signature","submit_replication":"https://pith.science/pith/Y5ELGMT5Q3TJROR6WNM7BZD2I2/action/replication_record"}},"created_at":"2026-07-05T05:45:24.285013+00:00","updated_at":"2026-07-05T05:45:24.285013+00:00"}