Pith Number

pith:FNPD4PXG

pith:2024:FNPD4PXGU4BFCGWCADRUSB7GTJ

not attested not anchored not stored refs resolved

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits

Furu Wei, Hongyu Wang, Jilong Xue, Lei Wang, Li Dong, Lingxiao Ma, Ruiping Wang, Shaohan Huang, Shuming Ma, Wenhui Wang

Ternary-weight LLMs achieve full-precision performance at far lower computational cost

arxiv:2402.17764 v1 · 2024-02-27 · cs.CL · cs.LG

Open paper page JSON What is a Pith Number?

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

Claims

C1strongest claim

It matches the full-precision (i.e., FP16 or BF16) Transformer LLM with the same model size and training tokens in terms of both perplexity and end-task performance, while being significantly more cost-effective in terms of latency, memory, throughput, and energy consumption.

C2weakest assumption

That the training procedure and scaling law developed for the 1.58-bit ternary setting will continue to produce competitive performance when model size or data volume increases beyond the scales tested.

C3one line summary

BitNet b1.58 shows that ternary 1.58-bit LLMs can match full-precision performance at substantially lower inference cost.

References

15 extracted · 15 resolved · 9 Pith anchors

[1] PIQA: Reasoning about Physical Commonsense in Natural Language 1911 · arXiv:1911.11641

[2] arXiv preprint arXiv:2307.13304 , year=

[3] BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions 1905 · arXiv:1905.10044

[4] 1.1 computing’s energy problem (and what we can do about it) 2014

[5] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration · arXiv:2306.00978

Formal links

1 machine-checked theorem link

Cited by

18 papers in Pith

Key and Value Weights Are Probably All You Need: On the Necessity of the Query, Key, Value weight Triplet in Self-Attention Transformers

Vec-LUT: Vector Table Lookup for Parallel Ultra-Low-Bit LLM Inference on Edge Devices

Locale-Conditioned Few-Shot Prompting Mitigates Demonstration Regurgitation in On-Device PII Substitution with Small Language Models

NativeTernary: A Self-Delimiting Binary Encoding with Unary Run-Length Hierarchy Markers for Ternary Neural Network Weights, Structured Data, and General Computing Infrastructure

A Composite Activation Function for Learning Stable Binary Representations

Receipt and verification

First computed	2026-05-17T23:38:13.210836Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

2b5e3e3ee6a702511ac200e34907e69a7fedfad8892125210de0673b00108196

Aliases

arxiv: 2402.17764 · arxiv_version: 2402.17764v1 · doi: 10.48550/arxiv.2402.17764

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/FNPD4PXGU4BFCGWCADRUSB7GTJ \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 2b5e3e3ee6a702511ac200e34907e69a7fedfad8892125210de0673b00108196

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "426827275ed354a9f956802377eb8139613353790a25470518f41f878e208786",
    "cross_cats_sorted": [
      "cs.LG"
    ],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2024-02-27T18:56:19Z",
    "title_canon_sha256": "07386fb3bc4c5e218ecb5dcf1df4250ca8801e5ca786069feec96a2427d82d23"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2402.17764",
    "kind": "arxiv",
    "version": 1
  }
}