{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2025:ILET7EFWUV3TH2MPJNM5QJ3ZZY","short_pith_number":"pith:ILET7EFW","schema_version":"1.0","canonical_sha256":"42c93f90b6a57733e98f4b59d82779ce165ddd0ef8455c7af74200e6b895ed21","source":{"kind":"arxiv","id":"2510.26692","version":2},"attestation_state":"computed","paper":{"title":"Kimi Linear: An Expressive, Efficient Attention Architecture","license":"http://creativecommons.org/licenses/by-nc-nd/4.0/","headline":"Kimi Linear, a hybrid linear attention model, outperforms full attention across contexts while cutting KV cache by up to 75%.","cross_cats":["cs.LG"],"primary_cat":"cs.CL","authors_text":"Bohong Yin, Bo Pang, Chao Hong, Chengyin Liu, Chu Wei, Dehao Zhang, Enming Yuan, Enzhe Lu, Fanqing Meng, Feng Wang, Guanduo Chen, Guohong Fu, Guokun Lai, Haiming Wang, Huabin Zheng, Jiacheng You, Jianlin Su, Jiawen Tao, Jiaxi Hu, Jiezhong Qiu, Junjie Yan, Kimi Team: Yu Zhang, Longguang Zhong, Longhui Yu, Longyu Guan, Mengnan Dong, Shaowei Liu, Shengjun Fang, Siyuan Pan, Songlin Yang, T.Y. Liu, Weiran He, Weixiao Huang, Weixin Xu, Weizhou Liu, Wenhao Wu, Wentao Li, Xingcheng Yao, Xin Men, Xinran Xu, Xinyu Zhou, Yanru Chen, Yejie Wang, Yibo Liu, Yiwei Li, Yizhi Zhang, Yucheng Wang, Yu Fan, Yuhao Wu, Yulun Du, Yutian Chen, Yuxin Wu, Yuzhi Wang, Zhejun Jiang, Zhengtao Wang, Zheng Zhang, Zhilin Yang, Zhiyuan Li, Zongyu Lin","submitted_at":"2025-10-30T16:59:43Z","abstract_excerpt":"We introduce Kimi Linear, a hybrid linear attention architecture that, for the first time, outperforms full attention under fair comparisons across various scenarios -- including short-context, long-context, and reinforcement learning (RL) scaling regimes. At its core lies Kimi Delta Attention (KDA), an expressive linear attention module that extends Gated DeltaNet with a finer-grained gating mechanism, enabling more effective use of limited finite-state RNN memory. Our bespoke chunkwise algorithm achieves high hardware efficiency through a specialized variant of the Diagonal-Plus-Low-Rank (DP"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":true,"formal_links_present":true},"canonical_record":{"source":{"id":"2510.26692","kind":"arxiv","version":2},"metadata":{"license":"http://creativecommons.org/licenses/by-nc-nd/4.0/","primary_cat":"cs.CL","submitted_at":"2025-10-30T16:59:43Z","cross_cats_sorted":["cs.LG"],"title_canon_sha256":"a6b1cc72271b81e7d4adb408a744f99a473483890771f20696ae05573b790722","abstract_canon_sha256":"63aa04095f6dad59b03b0ada9b6acbc358796b216a3ac16471cfe03949676f15"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-05-18T03:35:40.170081Z","signature_b64":"bTgv07y+nzisDFTMEt6MIdGbHFn5yZcAL/GdVeM9rOnaTvGRA10wehHDMy6YQ1Cu+zX6Nq8/w+0xZCffSH87CQ==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"42c93f90b6a57733e98f4b59d82779ce165ddd0ef8455c7af74200e6b895ed21","last_reissued_at":"2026-05-18T03:35:40.169133Z","signature_status":"signed_v1","first_computed_at":"2026-05-18T03:35:40.169133Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"Kimi Linear: An Expressive, Efficient Attention Architecture","license":"http://creativecommons.org/licenses/by-nc-nd/4.0/","headline":"Kimi Linear, a hybrid linear attention model, outperforms full attention across contexts while cutting KV cache by up to 75%.","cross_cats":["cs.LG"],"primary_cat":"cs.CL","authors_text":"Bohong Yin, Bo Pang, Chao Hong, Chengyin Liu, Chu Wei, Dehao Zhang, Enming Yuan, Enzhe Lu, Fanqing Meng, Feng Wang, Guanduo Chen, Guohong Fu, Guokun Lai, Haiming Wang, Huabin Zheng, Jiacheng You, Jianlin Su, Jiawen Tao, Jiaxi Hu, Jiezhong Qiu, Junjie Yan, Kimi Team: Yu Zhang, Longguang Zhong, Longhui Yu, Longyu Guan, Mengnan Dong, Shaowei Liu, Shengjun Fang, Siyuan Pan, Songlin Yang, T.Y. Liu, Weiran He, Weixiao Huang, Weixin Xu, Weizhou Liu, Wenhao Wu, Wentao Li, Xingcheng Yao, Xin Men, Xinran Xu, Xinyu Zhou, Yanru Chen, Yejie Wang, Yibo Liu, Yiwei Li, Yizhi Zhang, Yucheng Wang, Yu Fan, Yuhao Wu, Yulun Du, Yutian Chen, Yuxin Wu, Yuzhi Wang, Zhejun Jiang, Zhengtao Wang, Zheng Zhang, Zhilin Yang, Zhiyuan Li, Zongyu Lin","submitted_at":"2025-10-30T16:59:43Z","abstract_excerpt":"We introduce Kimi Linear, a hybrid linear attention architecture that, for the first time, outperforms full attention under fair comparisons across various scenarios -- including short-context, long-context, and reinforcement learning (RL) scaling regimes. At its core lies Kimi Delta Attention (KDA), an expressive linear attention module that extends Gated DeltaNet with a finer-grained gating mechanism, enabling more effective use of limited finite-state RNN memory. Our bespoke chunkwise algorithm achieves high hardware efficiency through a specialized variant of the Diagonal-Plus-Low-Rank (DP"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"Kimi Linear outperforms full attention under fair comparisons across short-context, long-context, and RL scaling regimes while reducing KV cache usage by up to 75% and achieving up to 6 times decoding throughput for a 1M context.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"The performance gains come from the new finer-grained gating in KDA and the specialized DPLR variant rather than from differences in training data, hyperparameters, or evaluation setup.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"Kimi Linear hybridizes linear attention with a new KDA module to beat full attention on tasks while slashing KV cache by 75% and speeding decoding up to 6x.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Kimi Linear, a hybrid linear attention model, outperforms full attention across contexts while cutting KV cache by up to 75%.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"87d036fae9011c8fb89659f53247b5997710d4cddce56e8d9b76a70382d359a5"},"source":{"id":"2510.26692","kind":"arxiv","version":2},"verdict":{"id":"0c7784e7-1f72-4068-a0d6-891ba04e90e0","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-13T23:42:51.206799Z","strongest_claim":"Kimi Linear outperforms full attention under fair comparisons across short-context, long-context, and RL scaling regimes while reducing KV cache usage by up to 75% and achieving up to 6 times decoding throughput for a 1M context.","one_line_summary":"Kimi Linear hybridizes linear attention with a new KDA module to beat full attention on tasks while slashing KV cache by 75% and speeding decoding up to 6x.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"The performance gains come from the new finer-grained gating in KDA and the specialized DPLR variant rather than from differences in training data, hyperparameters, or evaluation setup.","pith_extraction_headline":"Kimi Linear, a hybrid linear attention model, outperforms full attention across contexts while cutting KV cache by up to 75%."},"references":{"count":129,"sample":[{"doi":"","year":2025,"title":"gpt-oss-120b & gpt-oss-20b Model Card","work_id":"178c1f7e-4f19-4392-a45d-45a6dfa88ead","ref_index":1,"cited_arxiv_id":"2508.10925","is_internal_anchor":true},{"doi":"","year":2023,"title":"Colt5: Faster long-range transformers with conditional computation","work_id":"ffe0489f-b4bb-4f0a-a69a-f7e79c161dd0","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"10.2139/ssrn.5240330.doi:10.2139/ssrn.5240330","year":2025,"title":"Physics of Language Models: Part 4.1, Architecture Design and the Magic of Canon Layers","work_id":"4c8a21d6-4ecc-40b1-b707-2e5d9d1b063a","ref_index":3,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2024,"title":"Simple linear attention language models balance the recall-throughput tradeoff","work_id":"f4cb9331-2572-41e4-8db1-75c0cf2eaf98","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2023,"title":"Zoology: Measuring and improving recall in efficient language models","work_id":"a51e0ddf-22e2-4d3c-9759-7201f7d9a699","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":129,"snapshot_sha256":"b35e0ccc5f1adb251335aac84691f357ddabdc23c2c44b69108f7ceceed9c129","internal_anchors":34},"formal_canon":{"evidence_count":2,"snapshot_sha256":"9de9ee8984985cbcc996f7c8a2581ca8bb796e9c891f5fac2f3225e98266eacb"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2510.26692","created_at":"2026-05-18T03:35:40.169302+00:00"},{"alias_kind":"arxiv_version","alias_value":"2510.26692v2","created_at":"2026-05-18T03:35:40.169302+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2510.26692","created_at":"2026-05-18T03:35:40.169302+00:00"},{"alias_kind":"pith_short_12","alias_value":"ILET7EFWUV3T","created_at":"2026-05-18T12:33:37.589309+00:00"},{"alias_kind":"pith_short_16","alias_value":"ILET7EFWUV3TH2MP","created_at":"2026-05-18T12:33:37.589309+00:00"},{"alias_kind":"pith_short_8","alias_value":"ILET7EFW","created_at":"2026-05-18T12:33:37.589309+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":27,"internal_anchor_count":27,"sample":[{"citing_arxiv_id":"2605.19660","citing_title":"OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond","ref_index":53,"is_internal_anchor":true},{"citing_arxiv_id":"2605.16928","citing_title":"Full Attention Strikes Back: Transferring Full Attention into Sparse within Hundred Training Steps","ref_index":22,"is_internal_anchor":true},{"citing_arxiv_id":"2602.14814","citing_title":"Learning State-Tracking from Code Using Linear RNNs","ref_index":15,"is_internal_anchor":true},{"citing_arxiv_id":"2602.21204","citing_title":"Test-Time Training with KV Binding Is Secretly Linear Attention","ref_index":19,"is_internal_anchor":true},{"citing_arxiv_id":"2605.15178","citing_title":"SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer","ref_index":80,"is_internal_anchor":true},{"citing_arxiv_id":"2604.03263","citing_title":"LPC-SM: Local Predictive Coding and Sparse Memory for Long-Context Language Modeling","ref_index":7,"is_internal_anchor":true},{"citing_arxiv_id":"2605.13473","citing_title":"OSDN: Improving Delta Rule with Provable Online Preconditioning in Linear Attention","ref_index":57,"is_internal_anchor":true},{"citing_arxiv_id":"2604.22782","citing_title":"Stochastic KV Routing: Enabling Adaptive Depth-Wise Cache Sharing","ref_index":11,"is_internal_anchor":true},{"citing_arxiv_id":"2605.12357","citing_title":"$\\delta$-mem: Efficient Online Memory for Large Language Models","ref_index":14,"is_internal_anchor":true},{"citing_arxiv_id":"2605.08587","citing_title":"Kaczmarz Linear Attention","ref_index":40,"is_internal_anchor":true},{"citing_arxiv_id":"2605.08696","citing_title":"Structured Recurrent Mixers for Massively Parallelized Sequence Generation","ref_index":56,"is_internal_anchor":true},{"citing_arxiv_id":"2605.10537","citing_title":"Mela: Test-Time Memory Consolidation based on Transformation Hypothesis","ref_index":20,"is_internal_anchor":true},{"citing_arxiv_id":"2604.24715","citing_title":"Long-Context Aware Upcycling: A New Frontier for Hybrid LLM Scaling","ref_index":44,"is_internal_anchor":true},{"citing_arxiv_id":"2605.05696","citing_title":"Irminsul: MLA-Native Position-Independent Caching for Agentic LLM Serving","ref_index":28,"is_internal_anchor":true},{"citing_arxiv_id":"2605.06221","citing_title":"UniPrefill: Universal Long-Context Prefill Acceleration via Block-wise Dynamic Sparsification","ref_index":41,"is_internal_anchor":true},{"citing_arxiv_id":"2604.22575","citing_title":"SpikingBrain2.0: Brain-Inspired Foundation Models for Efficient Long-Context and Cross-Platform Inference","ref_index":35,"is_internal_anchor":true},{"citing_arxiv_id":"2605.05838","citing_title":"MDN: Parallelizing Stepwise Momentum for Delta Linear Attention","ref_index":55,"is_internal_anchor":true},{"citing_arxiv_id":"2604.10103","citing_title":"Long-Horizon Streaming Video Generation via Hybrid Attention with Decoupled Distillation","ref_index":41,"is_internal_anchor":true},{"citing_arxiv_id":"2604.10098","citing_title":"Attention Sink in Transformers: A Survey on Utilization, Interpretation, and Mitigation","ref_index":21,"is_internal_anchor":true},{"citing_arxiv_id":"2605.07588","citing_title":"Revisiting Transformer Layer Parameterization Through Causal Energy Minimization","ref_index":11,"is_internal_anchor":true},{"citing_arxiv_id":"2604.07279","citing_title":"Mem3R: Streaming 3D Reconstruction with Hybrid Memory via Test-Time Training","ref_index":52,"is_internal_anchor":true},{"citing_arxiv_id":"2604.05688","citing_title":"Attention Editing: A Versatile Framework for Cross-Architecture Attention Conversion","ref_index":18,"is_internal_anchor":true},{"citing_arxiv_id":"2604.14922","citing_title":"LongAct: Harnessing Intrinsic Activation Patterns for Long-Context Reinforcement Learning","ref_index":25,"is_internal_anchor":true},{"citing_arxiv_id":"2604.15039","citing_title":"Prefill-as-a-Service: KVCache of Next-Generation Models Could Go Cross-Datacenter","ref_index":26,"is_internal_anchor":true},{"citing_arxiv_id":"2604.19241","citing_title":"UniEP: Unified Expert-Parallel MoE MegaKernel for LLM Training","ref_index":49,"is_internal_anchor":true}]},"formal_canon":{"evidence_count":2,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/ILET7EFWUV3TH2MPJNM5QJ3ZZY","json":"https://pith.science/pith/ILET7EFWUV3TH2MPJNM5QJ3ZZY.json","graph_json":"https://pith.science/api/pith-number/ILET7EFWUV3TH2MPJNM5QJ3ZZY/graph.json","events_json":"https://pith.science/api/pith-number/ILET7EFWUV3TH2MPJNM5QJ3ZZY/events.json","paper":"https://pith.science/paper/ILET7EFW"},"agent_actions":{"view_html":"https://pith.science/pith/ILET7EFWUV3TH2MPJNM5QJ3ZZY","download_json":"https://pith.science/pith/ILET7EFWUV3TH2MPJNM5QJ3ZZY.json","view_paper":"https://pith.science/paper/ILET7EFW","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2510.26692&json=true","fetch_graph":"https://pith.science/api/pith-number/ILET7EFWUV3TH2MPJNM5QJ3ZZY/graph.json","fetch_events":"https://pith.science/api/pith-number/ILET7EFWUV3TH2MPJNM5QJ3ZZY/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/ILET7EFWUV3TH2MPJNM5QJ3ZZY/action/timestamp_anchor","attest_storage":"https://pith.science/pith/ILET7EFWUV3TH2MPJNM5QJ3ZZY/action/storage_attestation","attest_author":"https://pith.science/pith/ILET7EFWUV3TH2MPJNM5QJ3ZZY/action/author_attestation","sign_citation":"https://pith.science/pith/ILET7EFWUV3TH2MPJNM5QJ3ZZY/action/citation_signature","submit_replication":"https://pith.science/pith/ILET7EFWUV3TH2MPJNM5QJ3ZZY/action/replication_record"}},"created_at":"2026-05-18T03:35:40.169302+00:00","updated_at":"2026-05-18T03:35:40.169302+00:00"}