Pith Number

pith:ILET7EFW

pith:2025:ILET7EFWUV3TH2MPJNM5QJ3ZZY

not attested not anchored not stored refs resolved

Kimi Linear: An Expressive, Efficient Attention Architecture

Bohong Yin, Bo Pang, Chao Hong, Chengyin Liu, Chu Wei, Dehao Zhang, Enming Yuan, Enzhe Lu, Fanqing Meng, Feng Wang, Guanduo Chen, Guohong Fu, Guokun Lai, Haiming Wang, Huabin Zheng, Jiacheng You, Jianlin Su, Jiawen Tao, Jiaxi Hu, Jiezhong Qiu, Junjie Yan, Kimi Team: Yu Zhang, Longguang Zhong, Longhui Yu, Longyu Guan, Mengnan Dong, Shaowei Liu, Shengjun Fang, Siyuan Pan, Songlin Yang, T.Y. Liu, Weiran He, Weixiao Huang, Weixin Xu, Weizhou Liu, Wenhao Wu, Wentao Li, Xingcheng Yao, Xin Men, Xinran Xu, Xinyu Zhou, Yanru Chen, Yejie Wang, Yibo Liu, Yiwei Li, Yizhi Zhang, Yucheng Wang, Yu Fan, Yuhao Wu, Yulun Du, Yutian Chen, Yuxin Wu, Yuzhi Wang, Zhejun Jiang, Zhengtao Wang, Zheng Zhang, Zhilin Yang, Zhiyuan Li, Zongyu Lin

Kimi Linear, a hybrid linear attention model, outperforms full attention across contexts while cutting KV cache by up to 75%.

arxiv:2510.26692 v2 · 2025-10-30 · cs.CL · cs.LG

Open paper page JSON Open Graph Bundle Merged state What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{ILET7EFWUV3TH2MPJNM5QJ3ZZY}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

Kimi Linear outperforms full attention under fair comparisons across short-context, long-context, and RL scaling regimes while reducing KV cache usage by up to 75% and achieving up to 6 times decoding throughput for a 1M context.

C2weakest assumption

The performance gains come from the new finer-grained gating in KDA and the specialized DPLR variant rather than from differences in training data, hyperparameters, or evaluation setup.

C3one line summary

Kimi Linear hybridizes linear attention with a new KDA module to beat full attention on tasks while slashing KV cache by 75% and speeding decoding up to 6x.

References

129 extracted · 129 resolved · 34 Pith anchors

[1] gpt-oss-120b & gpt-oss-20b Model Card 2025 · arXiv:2508.10925

[2] Colt5: Faster long-range transformers with conditional computation 2023

[3] Physics of Language Models: Part 4.1, Architecture Design and the Magic of Canon Layers 2025 · doi:10.2139/ssrn.5240330.doi:10.2139/ssrn.5240330

[4] Simple linear attention language models balance the recall-throughput tradeoff 2024

[5] Zoology: Measuring and improving recall in efficient language models 2023

Formal links

2 machine-checked theorem links

Cited by

26 papers in Pith

Full Attention Strikes Back: Transferring Full Attention into Sparse within Hundred Training Steps

Learning State-Tracking from Code Using Linear RNNs

Test-Time Training with KV Binding Is Secretly Linear Attention

SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer

LPC-SM: Local Predictive Coding and Sparse Memory for Long-Context Language Modeling

Receipt and verification

First computed	2026-05-18T03:35:40.169133Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

42c93f90b6a57733e98f4b59d82779ce165ddd0ef8455c7af74200e6b895ed21

Aliases

arxiv: 2510.26692 · arxiv_version: 2510.26692v2 · doi: 10.48550/arxiv.2510.26692 · pith_short_12: ILET7EFWUV3T · pith_short_16: ILET7EFWUV3TH2MP · pith_short_8: ILET7EFW

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/ILET7EFWUV3TH2MPJNM5QJ3ZZY \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 42c93f90b6a57733e98f4b59d82779ce165ddd0ef8455c7af74200e6b895ed21

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "63aa04095f6dad59b03b0ada9b6acbc358796b216a3ac16471cfe03949676f15",
    "cross_cats_sorted": [
      "cs.LG"
    ],
    "license": "http://creativecommons.org/licenses/by-nc-nd/4.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2025-10-30T16:59:43Z",
    "title_canon_sha256": "a6b1cc72271b81e7d4adb408a744f99a473483890771f20696ae05573b790722"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2510.26692",
    "kind": "arxiv",
    "version": 2
  }
}