{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2024:F3SQWA3GM44IXO5Y3EQ3SSF4O5","short_pith_number":"pith:F3SQWA3G","schema_version":"1.0","canonical_sha256":"2ee50b036667388bbbb8d921b948bc7750779dea0b22b98eabe92be28d7cfed6","source":{"kind":"arxiv","id":"2405.16406","version":4},"attestation_state":"computed","paper":{"title":"SpinQuant: LLM quantization with learned rotations","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"SpinQuant learns rotation matrices to quantize LLM weights, activations, and KV cache to 4 bits while keeping outputs identical in full precision.","cross_cats":["cs.AI","cs.CL","cs.CV"],"primary_cat":"cs.LG","authors_text":"Bilge Soran, Changsheng Zhao, Dhruv Choudhary, Igor Fedorov, Raghuraman Krishnamoorthi, Tijmen Blankevoort, Vikas Chandra, Yuandong Tian, Zechun Liu","submitted_at":"2024-05-26T02:15:49Z","abstract_excerpt":"Post-training quantization (PTQ) techniques applied to weights, activations, and the KV cache greatly reduce memory usage, latency, and power consumption of Large Language Models (LLMs), but may lead to large quantization errors when outliers are present. Rotating activation or weight matrices helps remove outliers and benefits quantization. In this work, we identify a collection of applicable rotation parameterizations that lead to identical outputs in full-precision Transformer architectures while enhancing quantization accuracy. In addition, we find that some random rotations lead to much b"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":true,"formal_links_present":true},"canonical_record":{"source":{"id":"2405.16406","kind":"arxiv","version":4},"metadata":{"license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","primary_cat":"cs.LG","submitted_at":"2024-05-26T02:15:49Z","cross_cats_sorted":["cs.AI","cs.CL","cs.CV"],"title_canon_sha256":"ba0aab9cac8a079304b5cac58e1b79301b39a03caf2c9306481eb82595e638bd","abstract_canon_sha256":"1538f48e6afe2d097304cf1c4b0d8a7c258b50ad60ddaf75d8de3fc142c5dd7a"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-05-17T23:38:51.013633Z","signature_b64":"snVszCMuXMBnphByK+gyedC3ZV/8v9rG8EsQiWPM9ehBm2CVVMfj6RGbGjGw0IQvPrheQyTlDmiGqUagHl+oAg==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"2ee50b036667388bbbb8d921b948bc7750779dea0b22b98eabe92be28d7cfed6","last_reissued_at":"2026-05-17T23:38:51.013172Z","signature_status":"signed_v1","first_computed_at":"2026-05-17T23:38:51.013172Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"SpinQuant: LLM quantization with learned rotations","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"SpinQuant learns rotation matrices to quantize LLM weights, activations, and KV cache to 4 bits while keeping outputs identical in full precision.","cross_cats":["cs.AI","cs.CL","cs.CV"],"primary_cat":"cs.LG","authors_text":"Bilge Soran, Changsheng Zhao, Dhruv Choudhary, Igor Fedorov, Raghuraman Krishnamoorthi, Tijmen Blankevoort, Vikas Chandra, Yuandong Tian, Zechun Liu","submitted_at":"2024-05-26T02:15:49Z","abstract_excerpt":"Post-training quantization (PTQ) techniques applied to weights, activations, and the KV cache greatly reduce memory usage, latency, and power consumption of Large Language Models (LLMs), but may lead to large quantization errors when outliers are present. Rotating activation or weight matrices helps remove outliers and benefits quantization. In this work, we identify a collection of applicable rotation parameterizations that lead to identical outputs in full-precision Transformer architectures while enhancing quantization accuracy. In addition, we find that some random rotations lead to much b"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"With 4-bit quantization of weight, activation, and KV-cache, SpinQuant narrows the accuracy gap on zero-shot reasoning tasks with full precision to merely 2.9 points on the LLaMA-2 7B model, surpassing LLM-QAT by 19.1 points and SmoothQuant by 25.0 points.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That learned rotation matrices found on calibration data will generalize to preserve full-precision outputs and improve quantization accuracy across diverse downstream tasks without introducing new errors.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"SpinQuant learns optimal rotations to enable accurate 4-bit quantization of LLM weights, activations, and KV cache, reducing the zero-shot gap to full precision to 2.9 points on LLaMA-2 7B.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"SpinQuant learns rotation matrices to quantize LLM weights, activations, and KV cache to 4 bits while keeping outputs identical in full precision.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"e00aafc4efd9bbf03e020b914683a8bf0efe6147b80be42e4c86bc5c5ec0197d"},"source":{"id":"2405.16406","kind":"arxiv","version":4},"verdict":{"id":"06614176-a0bf-4c20-a74b-f5dc696b89d2","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-15T15:47:00.210125Z","strongest_claim":"With 4-bit quantization of weight, activation, and KV-cache, SpinQuant narrows the accuracy gap on zero-shot reasoning tasks with full precision to merely 2.9 points on the LLaMA-2 7B model, surpassing LLM-QAT by 19.1 points and SmoothQuant by 25.0 points.","one_line_summary":"SpinQuant learns optimal rotations to enable accurate 4-bit quantization of LLM weights, activations, and KV cache, reducing the zero-shot gap to full precision to 2.9 points on LLaMA-2 7B.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That learned rotation matrices found on calibration data will generalize to preserve full-precision outputs and improve quantization accuracy across diverse downstream tasks without introducing new errors.","pith_extraction_headline":"SpinQuant learns rotation matrices to quantize LLM weights, activations, and KV cache to 4 bits while keeping outputs identical in full precision."},"references":{"count":33,"sample":[{"doi":"","year":null,"title":"GPT-4 Technical Report","work_id":"b928e041-6991-4c08-8c81-0359e4097c7b","ref_index":1,"cited_arxiv_id":"2303.08774","is_internal_anchor":true},{"doi":"","year":1905,"title":"BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions","work_id":"511eeb84-4b95-46d5-b14f-50da43f4f19f","ref_index":2,"cited_arxiv_id":"1905.10044","is_internal_anchor":true},{"doi":"","year":null,"title":"Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge","work_id":"28ea1282-d657-4c61-a83c-f1249be6d6b1","ref_index":3,"cited_arxiv_id":"1803.05457","is_internal_anchor":true},{"doi":"","year":null,"title":"Extreme compression of large language models via additive quantization","work_id":"38bf2888-df57-4a97-a9ca-5d4d524ea463","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":null,"title":"GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers","work_id":"19ed8c44-202a-48f6-8169-637d5a5f2408","ref_index":5,"cited_arxiv_id":"2210.17323","is_internal_anchor":true}],"resolved_work":33,"snapshot_sha256":"cd1c0503cc77f5067322272dc17db76c13f6d45f04cfcc7e269f9b607b26c5fa","internal_anchors":14},"formal_canon":{"evidence_count":3,"snapshot_sha256":"219c07d593f3ccf44032d91fdda8cd14bcca20b19d6c69bdece7b4d8bcedbf18"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2405.16406","created_at":"2026-05-17T23:38:51.013239+00:00"},{"alias_kind":"arxiv_version","alias_value":"2405.16406v4","created_at":"2026-05-17T23:38:51.013239+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2405.16406","created_at":"2026-05-17T23:38:51.013239+00:00"},{"alias_kind":"pith_short_12","alias_value":"F3SQWA3GM44I","created_at":"2026-05-18T12:33:37.589309+00:00"},{"alias_kind":"pith_short_16","alias_value":"F3SQWA3GM44IXO5Y","created_at":"2026-05-18T12:33:37.589309+00:00"},{"alias_kind":"pith_short_8","alias_value":"F3SQWA3G","created_at":"2026-05-18T12:33:37.589309+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":37,"internal_anchor_count":37,"sample":[{"citing_arxiv_id":"2605.15828","citing_title":"Not All Tasks Quantize Equally: Fisher-Guided Quantization for Visual Geometry Transformer","ref_index":5,"is_internal_anchor":true},{"citing_arxiv_id":"2412.14590","citing_title":"MixLLM: LLM Quantization with Global Mixed-precision between Output-features and Highly-efficient System Design","ref_index":28,"is_internal_anchor":true},{"citing_arxiv_id":"2511.06516","citing_title":"You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations","ref_index":41,"is_internal_anchor":true},{"citing_arxiv_id":"2605.18800","citing_title":"Theory-optimal Quantization Based on Flatness","ref_index":10,"is_internal_anchor":true},{"citing_arxiv_id":"2605.15828","citing_title":"Not All Tasks Quantize Equally: Fisher-Guided Quantization for Visual Geometry Transformer","ref_index":5,"is_internal_anchor":true},{"citing_arxiv_id":"2605.17997","citing_title":"MARR: Module-Adaptive Residual Reconstruction for Low-Bit Post-Training Quantization","ref_index":24,"is_internal_anchor":true},{"citing_arxiv_id":"2605.17757","citing_title":"OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization","ref_index":11,"is_internal_anchor":true},{"citing_arxiv_id":"2605.19218","citing_title":"Rotation-Aligned Key Channel Pruning for Efficient Vision-Language Model Inference","ref_index":39,"is_internal_anchor":true},{"citing_arxiv_id":"2605.19929","citing_title":"Breaking Modality Heterogeneity in Low-Bit Quantization for Large Vision-Language Models","ref_index":32,"is_internal_anchor":true},{"citing_arxiv_id":"2604.18556","citing_title":"GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling","ref_index":23,"is_internal_anchor":true},{"citing_arxiv_id":"2605.08894","citing_title":"Fitting Is Not Enough: Smoothness in Extremely Quantized LLMs","ref_index":25,"is_internal_anchor":true},{"citing_arxiv_id":"2506.12040","citing_title":"BTC-LLM: Efficient Sub-1-Bit LLM Quantization via Learnable Transformation and Binary Codebook","ref_index":20,"is_internal_anchor":true},{"citing_arxiv_id":"2511.11663","citing_title":"SpecQuant: Spectral Decomposition and Adaptive Truncation for Ultra-Low-Bit LLMs Quantization","ref_index":3,"is_internal_anchor":true},{"citing_arxiv_id":"2511.14582","citing_title":"OmniZip: Audio-Guided Dynamic Token Compression for Fast Omnimodal Large Language Models","ref_index":30,"is_internal_anchor":true},{"citing_arxiv_id":"2512.02010","citing_title":"Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling","ref_index":36,"is_internal_anchor":true},{"citing_arxiv_id":"2601.17187","citing_title":"High-Rate Quantized Matrix Multiplication I","ref_index":18,"is_internal_anchor":true},{"citing_arxiv_id":"2602.05902","citing_title":"CoreQ: Learning-Free Mismatch Correction and Successive Rounding for Quantization","ref_index":13,"is_internal_anchor":true},{"citing_arxiv_id":"2603.08899","citing_title":"ConFu: Contemplate the Future for Better Speculative Sampling","ref_index":13,"is_internal_anchor":true},{"citing_arxiv_id":"2404.14294","citing_title":"A Survey on Efficient Inference for Large Language Models","ref_index":218,"is_internal_anchor":true},{"citing_arxiv_id":"2604.02525","citing_title":"AdaHOP: Fast and Accurate Low-Precision Training via Outlier-Pattern-Aware Rotation","ref_index":23,"is_internal_anchor":true},{"citing_arxiv_id":"2604.26378","citing_title":"CoQuant: Joint Weight-Activation Subspace Projection for Mixed-Precision LLMs","ref_index":7,"is_internal_anchor":true},{"citing_arxiv_id":"2605.09375","citing_title":"31.1 A 14.08-to-135.69Token/s ReRAM-on-Logic Stacked Outlier-Free Large-Language-Model Accelerator with Block-Clustered Weight-Compression and Adaptive Parallel-Speculative-Decoding","ref_index":12,"is_internal_anchor":true},{"citing_arxiv_id":"2605.04738","citing_title":"OSAQ: Outlier Self-Absorption for Accurate Low-bit LLM Quantization","ref_index":12,"is_internal_anchor":true},{"citing_arxiv_id":"2605.08894","citing_title":"Fitting Is Not Enough: Smoothness in Extremely Quantized LLMs","ref_index":25,"is_internal_anchor":true},{"citing_arxiv_id":"2604.25098","citing_title":"Revisiting the Effectiveness of LLM Pruning for Test-Time Scaling","ref_index":6,"is_internal_anchor":true}]},"formal_canon":{"evidence_count":3,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/F3SQWA3GM44IXO5Y3EQ3SSF4O5","json":"https://pith.science/pith/F3SQWA3GM44IXO5Y3EQ3SSF4O5.json","graph_json":"https://pith.science/api/pith-number/F3SQWA3GM44IXO5Y3EQ3SSF4O5/graph.json","events_json":"https://pith.science/api/pith-number/F3SQWA3GM44IXO5Y3EQ3SSF4O5/events.json","paper":"https://pith.science/paper/F3SQWA3G"},"agent_actions":{"view_html":"https://pith.science/pith/F3SQWA3GM44IXO5Y3EQ3SSF4O5","download_json":"https://pith.science/pith/F3SQWA3GM44IXO5Y3EQ3SSF4O5.json","view_paper":"https://pith.science/paper/F3SQWA3G","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2405.16406&json=true","fetch_graph":"https://pith.science/api/pith-number/F3SQWA3GM44IXO5Y3EQ3SSF4O5/graph.json","fetch_events":"https://pith.science/api/pith-number/F3SQWA3GM44IXO5Y3EQ3SSF4O5/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/F3SQWA3GM44IXO5Y3EQ3SSF4O5/action/timestamp_anchor","attest_storage":"https://pith.science/pith/F3SQWA3GM44IXO5Y3EQ3SSF4O5/action/storage_attestation","attest_author":"https://pith.science/pith/F3SQWA3GM44IXO5Y3EQ3SSF4O5/action/author_attestation","sign_citation":"https://pith.science/pith/F3SQWA3GM44IXO5Y3EQ3SSF4O5/action/citation_signature","submit_replication":"https://pith.science/pith/F3SQWA3GM44IXO5Y3EQ3SSF4O5/action/replication_record"}},"created_at":"2026-05-17T23:38:51.013239+00:00","updated_at":"2026-05-17T23:38:51.013239+00:00"}