{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2025:IHKBZYRE6I4BPR5RJYSKX65EXM","short_pith_number":"pith:IHKBZYRE","schema_version":"1.0","canonical_sha256":"41d41ce224f23817c7b14e24abfba4bb05797d3e58378e29f2ba280e06cdaf51","source":{"kind":"arxiv","id":"2504.03624","version":4},"attestation_state":"computed","paper":{"title":"Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models","license":"http://creativecommons.org/licenses/by/4.0/","headline":"","cross_cats":["cs.AI","cs.LG"],"primary_cat":"cs.CL","authors_text":"Aarti Basant, Abhinav Khattar, Adithya Renduchintala, Akhiad Bercovich, Aleksander Ficek, Alexis Bjorlin, Ali Taghibakhshi, Amala Sanjay Deshmukh, Ameya Sunil Mahabaleshwarkar, Andrew Tao, Anna Shors, Ashwath Aithal, Ashwin Poojary, Ayush Dattagupta, Balaram Buddharaju, Bobby Chen, Boris Ginsburg, Boxin Wang, Brandon Norick, Brian Butterfield, Bryan Catanzaro, Carlo del Mundo, Chengyu Dong, Christine Harvey, Christopher Parisien, Daniel Korzekwa, Danny Yin, Dan Su, Daria Gitman, David Mosallanezhad, Deepak Narayanan, Denys Fridman, Dima Rekesh, Ding Ma, Dmytro Pykhtar, Dong Ahn, Duncan Riach, Dusan Stosic, Eileen Long, Elad Segal, Ellie Evans, Eric Chung, Erick Galinkin, Evelina Bakhturina, Ewa Dobrowolska, Fei Jia, Fuxiao Liu, Gargi Prasad, Gerald Shen, Guilin Liu, Guo Chen, Haifeng Qian, Helen Ngo, Hongbin Liu, Hui Li, Igor Gitman, Ilia Karmanov, Ivan Moshkov, Izik Golan, Jane Polak Scowcroft, Jan Kautz, Jared Casper, Jarno Seppanen, Jason Lu, Jason Sewall, Jiaqi Zeng, Jiaxuan You, Jimmy Zhang, Jing Zhang, Jining Huang, Jinze Xue, Jocelyn Huang, Joey Conway, John Kamalu, Jonathan Cohen, Jon Barker, Joseph Jennings, Jupinder Parmar, Karan Sapra, Kari Briski, Kateryna Chumachenko, Katherine Luna, Keshav Santhanam, Kezhi Kong, Kirthi Sivamani, Krzysztof Pawelec, Kumar Anik, Kunlun Li, Lawrence McAfee, Leon Derczynski, Lindsey Pavao, Luis Vega, Lukas Voegtle, Maciej Bala, Maer Rodrigues de Melo, Makesh Narsimhan Sreedhar, Marcin Chochowski, Markus Kliegl, Marta Stepniewska-Dziubinska, Matthieu Le, Matvei Novikov, Mehrzad Samadi, Michael Andersch, Michael Evans, Miguel Martinez, Mike Chrzanowski, Mike Ranzinger, Mikolaj Blaz, Misha Smelyanskiy, Mohamed Fawzy, Mohammad Shoeybi, Mostofa Patwary, Nayeon Lee, Nima Tajbakhsh, Ning Xu, NVIDIA: Aaron Blakeman, Oleg Rybakov, Oleksii Kuchaiev, Olivier Delalleau, Osvald Nitski, Parth Chadha, Pasha Shamis, Paulius Micikevicius, Pavlo Molchanov, Peter Dykas, Philipp Fischer, Pierre-Yves Aquilanti, Piotr Bialecki, Prasoon Varshney, Pritam Gundecha, Przemek Tredak, Rabeeh Karimi, Rahul Kandu, Ran El-Yaniv, Raviraj Joshi, Roger Waleffe, Ruoxi Zhang, Sabrina Kavanaugh, Sahil Jain, Samuel Kriman, Sangkug Lym, Sanjeev Satheesh, Saurav Muralidharan, Sean Narenthiran, Selvaraj Anandaraj, Seonmyeong Bak, Sergey Kashirsky, Seungju Han, Shantanu Acharya, Shaona Ghosh, Sharath Turuvekere Sreenivas, Sharon Clay, Shelby Thomas, Shrimai Prabhumoye, Shubham Pachori, Shubham Toshniwal, Shyamala Prayaga, Siddhartha Jain, Sirshak Das, Slawek Kierat, Somshubra Majumdar, Song Han, Soumye Singhal, Sriharsha Niverty, Stefania Alborghetti, Suseella Panguluri, Swetha Bhendigeri, Syeda Nahida Akter, Szymon Migacz, Tal Shiri, Terry Kong, Timo Roman, Tomer Ronen, Trisha Saar, Tugrul Konuk, Tuomas Rintamaki, Tyler Poon, Ushnish De, Vahid Noroozi, Varun Singh, Vijay Korthikanti, Vitaly Kurin, Wasi Uddin Ahmad, Wei Du, Wei Ping, Wenliang Dai, Wonmin Byeon, Xiaowei Ren, Yao Xu, Yejin Choi, Yian Zhang, Ying Lin, Yoshi Suhara, Zhiding Yu, Zhiqi Li, Zhiyu Li, Zhongbo Zhu, Zhuolin Yang, Zijia Chen","submitted_at":"2025-04-04T17:41:58Z","abstract_excerpt":"As inference-time scaling becomes critical for enhanced reasoning capabilities, it is increasingly becoming important to build models that are efficient to infer. We introduce Nemotron-H, a family of 8B and 56B/47B hybrid Mamba-Transformer models designed to reduce inference cost for a given accuracy level. To achieve this goal, we replace the majority of self-attention layers in the common Transformer model architecture with Mamba layers that perform constant computation and require constant memory per generated token. We show that Nemotron-H models offer either better or on-par accuracy comp"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":false,"formal_links_present":false},"canonical_record":{"source":{"id":"2504.03624","kind":"arxiv","version":4},"metadata":{"license":"http://creativecommons.org/licenses/by/4.0/","primary_cat":"cs.CL","submitted_at":"2025-04-04T17:41:58Z","cross_cats_sorted":["cs.AI","cs.LG"],"title_canon_sha256":"a55454648e4e75486e61c5b1c71f3d61c51c73ce96aa9dbdb0ff471354fc2290","abstract_canon_sha256":"b7c5b18c614c374f65d9f5d2b9b93cdd223883a2d78fd8dbdb9eaaf77c1ad7ee"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-07-05T12:05:42.624459Z","signature_b64":"PmCiyUfyl92IpYrFbKsi5pBExZGHErWfzcF7K+0d7dPkvSK45tjx+Uo6xhORpkPeaYZrnvFio6qAk/rof2WbCA==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"41d41ce224f23817c7b14e24abfba4bb05797d3e58378e29f2ba280e06cdaf51","last_reissued_at":"2026-07-05T12:05:42.623951Z","signature_status":"signed_v1","first_computed_at":"2026-07-05T12:05:42.623951Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models","license":"http://creativecommons.org/licenses/by/4.0/","headline":"","cross_cats":["cs.AI","cs.LG"],"primary_cat":"cs.CL","authors_text":"Aarti Basant, Abhinav Khattar, Adithya Renduchintala, Akhiad Bercovich, Aleksander Ficek, Alexis Bjorlin, Ali Taghibakhshi, Amala Sanjay Deshmukh, Ameya Sunil Mahabaleshwarkar, Andrew Tao, Anna Shors, Ashwath Aithal, Ashwin Poojary, Ayush Dattagupta, Balaram Buddharaju, Bobby Chen, Boris Ginsburg, Boxin Wang, Brandon Norick, Brian Butterfield, Bryan Catanzaro, Carlo del Mundo, Chengyu Dong, Christine Harvey, Christopher Parisien, Daniel Korzekwa, Danny Yin, Dan Su, Daria Gitman, David Mosallanezhad, Deepak Narayanan, Denys Fridman, Dima Rekesh, Ding Ma, Dmytro Pykhtar, Dong Ahn, Duncan Riach, Dusan Stosic, Eileen Long, Elad Segal, Ellie Evans, Eric Chung, Erick Galinkin, Evelina Bakhturina, Ewa Dobrowolska, Fei Jia, Fuxiao Liu, Gargi Prasad, Gerald Shen, Guilin Liu, Guo Chen, Haifeng Qian, Helen Ngo, Hongbin Liu, Hui Li, Igor Gitman, Ilia Karmanov, Ivan Moshkov, Izik Golan, Jane Polak Scowcroft, Jan Kautz, Jared Casper, Jarno Seppanen, Jason Lu, Jason Sewall, Jiaqi Zeng, Jiaxuan You, Jimmy Zhang, Jing Zhang, Jining Huang, Jinze Xue, Jocelyn Huang, Joey Conway, John Kamalu, Jonathan Cohen, Jon Barker, Joseph Jennings, Jupinder Parmar, Karan Sapra, Kari Briski, Kateryna Chumachenko, Katherine Luna, Keshav Santhanam, Kezhi Kong, Kirthi Sivamani, Krzysztof Pawelec, Kumar Anik, Kunlun Li, Lawrence McAfee, Leon Derczynski, Lindsey Pavao, Luis Vega, Lukas Voegtle, Maciej Bala, Maer Rodrigues de Melo, Makesh Narsimhan Sreedhar, Marcin Chochowski, Markus Kliegl, Marta Stepniewska-Dziubinska, Matthieu Le, Matvei Novikov, Mehrzad Samadi, Michael Andersch, Michael Evans, Miguel Martinez, Mike Chrzanowski, Mike Ranzinger, Mikolaj Blaz, Misha Smelyanskiy, Mohamed Fawzy, Mohammad Shoeybi, Mostofa Patwary, Nayeon Lee, Nima Tajbakhsh, Ning Xu, NVIDIA: Aaron Blakeman, Oleg Rybakov, Oleksii Kuchaiev, Olivier Delalleau, Osvald Nitski, Parth Chadha, Pasha Shamis, Paulius Micikevicius, Pavlo Molchanov, Peter Dykas, Philipp Fischer, Pierre-Yves Aquilanti, Piotr Bialecki, Prasoon Varshney, Pritam Gundecha, Przemek Tredak, Rabeeh Karimi, Rahul Kandu, Ran El-Yaniv, Raviraj Joshi, Roger Waleffe, Ruoxi Zhang, Sabrina Kavanaugh, Sahil Jain, Samuel Kriman, Sangkug Lym, Sanjeev Satheesh, Saurav Muralidharan, Sean Narenthiran, Selvaraj Anandaraj, Seonmyeong Bak, Sergey Kashirsky, Seungju Han, Shantanu Acharya, Shaona Ghosh, Sharath Turuvekere Sreenivas, Sharon Clay, Shelby Thomas, Shrimai Prabhumoye, Shubham Pachori, Shubham Toshniwal, Shyamala Prayaga, Siddhartha Jain, Sirshak Das, Slawek Kierat, Somshubra Majumdar, Song Han, Soumye Singhal, Sriharsha Niverty, Stefania Alborghetti, Suseella Panguluri, Swetha Bhendigeri, Syeda Nahida Akter, Szymon Migacz, Tal Shiri, Terry Kong, Timo Roman, Tomer Ronen, Trisha Saar, Tugrul Konuk, Tuomas Rintamaki, Tyler Poon, Ushnish De, Vahid Noroozi, Varun Singh, Vijay Korthikanti, Vitaly Kurin, Wasi Uddin Ahmad, Wei Du, Wei Ping, Wenliang Dai, Wonmin Byeon, Xiaowei Ren, Yao Xu, Yejin Choi, Yian Zhang, Ying Lin, Yoshi Suhara, Zhiding Yu, Zhiqi Li, Zhiyu Li, Zhongbo Zhu, Zhuolin Yang, Zijia Chen","submitted_at":"2025-04-04T17:41:58Z","abstract_excerpt":"As inference-time scaling becomes critical for enhanced reasoning capabilities, it is increasingly becoming important to build models that are efficient to infer. We introduce Nemotron-H, a family of 8B and 56B/47B hybrid Mamba-Transformer models designed to reduce inference cost for a given accuracy level. To achieve this goal, we replace the majority of self-attention layers in the common Transformer model architecture with Mamba layers that perform constant computation and require constant memory per generated token. We show that Nemotron-H models offer either better or on-par accuracy comp"},"claims":{"count":0,"items":[],"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"source":{"id":"2504.03624","kind":"arxiv","version":4},"verdict":{"id":null,"model_set":{},"created_at":null,"strongest_claim":"","one_line_summary":"","pipeline_version":null,"weakest_assumption":"","pith_extraction_headline":""},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2504.03624/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2504.03624","created_at":"2026-07-05T12:05:42.624016+00:00"},{"alias_kind":"arxiv_version","alias_value":"2504.03624v4","created_at":"2026-07-05T12:05:42.624016+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2504.03624","created_at":"2026-07-05T12:05:42.624016+00:00"},{"alias_kind":"pith_short_12","alias_value":"IHKBZYRE6I4B","created_at":"2026-07-05T12:05:42.624016+00:00"},{"alias_kind":"pith_short_16","alias_value":"IHKBZYRE6I4BPR5R","created_at":"2026-07-05T12:05:42.624016+00:00"},{"alias_kind":"pith_short_8","alias_value":"IHKBZYRE","created_at":"2026-07-05T12:05:42.624016+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":21,"internal_anchor_count":0,"sample":[{"citing_arxiv_id":"2606.11634","citing_title":"Architecture-Aware Reinforcement Learning Makes Sliding-Window Attention Competitive in Math Reasoning","ref_index":4,"is_internal_anchor":false},{"citing_arxiv_id":"2606.05362","citing_title":"MOSAIC: A Workload-Driven Simulation and Design-Space Exploration Framework for Heterogeneous NPUs","ref_index":32,"is_internal_anchor":false},{"citing_arxiv_id":"2606.02332","citing_title":"Forget Attention: Importance-Aware Attention Is All You Need","ref_index":23,"is_internal_anchor":false},{"citing_arxiv_id":"2606.30562","citing_title":"Morphing into Hybrid Attention Models","ref_index":7,"is_internal_anchor":false},{"citing_arxiv_id":"2606.03014","citing_title":"MOSAIC: Efficient Mixture-of-Agent Scheduling via Adaptive Aggregation and Inference Concurrency","ref_index":47,"is_internal_anchor":false},{"citing_arxiv_id":"2605.18826","citing_title":"The Routing and Filtering Structure of Attention","ref_index":2,"is_internal_anchor":false},{"citing_arxiv_id":"2605.17093","citing_title":"HEED: Density-Weighted Residual Alignment for Hybrid Vision-Language Model Distillation","ref_index":3,"is_internal_anchor":false},{"citing_arxiv_id":"2605.19150","citing_title":"Flash PD-SSM: Memory-Optimized Structured Sparse State-Space Models","ref_index":6,"is_internal_anchor":false},{"citing_arxiv_id":"2605.18753","citing_title":"DashAttention: Differentiable and Adaptive Sparse Hierarchical Attention","ref_index":69,"is_internal_anchor":false},{"citing_arxiv_id":"2509.24552","citing_title":"Short window attention enables long-term memorization","ref_index":25,"is_internal_anchor":false},{"citing_arxiv_id":"2512.20856","citing_title":"NVIDIA Nemotron 3: Efficient and Open Intelligence","ref_index":205,"is_internal_anchor":false},{"citing_arxiv_id":"2601.01972","citing_title":"Hidden State Poisoning Attacks against Mamba-based Language Models","ref_index":2,"is_internal_anchor":false},{"citing_arxiv_id":"2503.15558","citing_title":"Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning","ref_index":39,"is_internal_anchor":false},{"citing_arxiv_id":"2506.02153","citing_title":"Small Language Models are the Future of Agentic AI","ref_index":9,"is_internal_anchor":false},{"citing_arxiv_id":"2510.26692","citing_title":"Kimi Linear: An Expressive, Efficient Attention Architecture","ref_index":12,"is_internal_anchor":false},{"citing_arxiv_id":"2604.03829","citing_title":"Mambalaya: Einsum-Based Fusion Optimizations on State-Space Models","ref_index":9,"is_internal_anchor":false},{"citing_arxiv_id":"2605.06068","citing_title":"VibeServe: Can AI Agents Build Bespoke LLM Serving Systems?","ref_index":56,"is_internal_anchor":false},{"citing_arxiv_id":"2605.05066","citing_title":"The Impossibility Triangle of Long-Context Modeling","ref_index":5,"is_internal_anchor":false},{"citing_arxiv_id":"2604.08050","citing_title":"ABMAMBA: Multimodal Large Language Model with Aligned Hierarchical Bidirectional Scan for Efficient Video Captioning","ref_index":5,"is_internal_anchor":false},{"citing_arxiv_id":"2605.07182","citing_title":"Star Elastic: Many-in-One Reasoning LLMs with Efficient Budget Control","ref_index":14,"is_internal_anchor":false},{"citing_arxiv_id":"2605.07075","citing_title":"ModelLens: Finding the Best for Your Task from Myriads of Models","ref_index":48,"is_internal_anchor":false}]},"formal_canon":{"evidence_count":0,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/IHKBZYRE6I4BPR5RJYSKX65EXM","json":"https://pith.science/pith/IHKBZYRE6I4BPR5RJYSKX65EXM.json","graph_json":"https://pith.science/api/pith-number/IHKBZYRE6I4BPR5RJYSKX65EXM/graph.json","events_json":"https://pith.science/api/pith-number/IHKBZYRE6I4BPR5RJYSKX65EXM/events.json","paper":"https://pith.science/paper/IHKBZYRE"},"agent_actions":{"view_html":"https://pith.science/pith/IHKBZYRE6I4BPR5RJYSKX65EXM","download_json":"https://pith.science/pith/IHKBZYRE6I4BPR5RJYSKX65EXM.json","view_paper":"https://pith.science/paper/IHKBZYRE","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2504.03624&json=true","fetch_graph":"https://pith.science/api/pith-number/IHKBZYRE6I4BPR5RJYSKX65EXM/graph.json","fetch_events":"https://pith.science/api/pith-number/IHKBZYRE6I4BPR5RJYSKX65EXM/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/IHKBZYRE6I4BPR5RJYSKX65EXM/action/timestamp_anchor","attest_storage":"https://pith.science/pith/IHKBZYRE6I4BPR5RJYSKX65EXM/action/storage_attestation","attest_author":"https://pith.science/pith/IHKBZYRE6I4BPR5RJYSKX65EXM/action/author_attestation","sign_citation":"https://pith.science/pith/IHKBZYRE6I4BPR5RJYSKX65EXM/action/citation_signature","submit_replication":"https://pith.science/pith/IHKBZYRE6I4BPR5RJYSKX65EXM/action/replication_record"}},"created_at":"2026-07-05T12:05:42.624016+00:00","updated_at":"2026-07-05T12:05:42.624016+00:00"}