Pith Number

pith:OLBY5FDU

pith:2025:OLBY5FDUE5HIQNCKA3E2AWZIIR

not attested not anchored not stored refs resolved

CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation

Hanqi Yan, Linhai Zhang, Yali Du, Yulan He, Zhanghao Hu, Zhenyi Shen

Self-distillation aligns one token's hidden state to transfer chain-of-thought reasoning into continuous space without accuracy loss.

arxiv:2502.21074 v3 · 2025-02-28 · cs.CL

Open paper page JSON Open Graph Bundle Merged state What is a Pith Number?

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

CODI is the first implicit CoT approach to match the performance of explicit CoT on GSM8k at the GPT-2 scale, achieving a 3.1x compression rate and outperforming the previous state-of-the-art by 28.2% in accuracy.

C2weakest assumption

That aligning the hidden states of a single designated token between the explicit teacher and implicit student is sufficient to transfer full reasoning capability without loss or distortion.

C3one line summary

CODI compresses explicit CoT into continuous space via self-distillation and is the first implicit method to match explicit CoT performance on GSM8k at GPT-2 scale with 3.1x compression and 28.2% higher accuracy than prior implicit approaches.

References

118 extracted · 118 resolved · 7 Pith anchors

[1] Training language models to follow instructions with human feedback , url =

[2] Aho and Jeffrey D 1972

[3] Publications Manual , year = "1983", publisher = 1983

[4] Chandra and Dexter C 1981 · doi:10.1145/322234.322243

[5] Scalable training of

Cited by

21 papers in Pith

Deep Thinking by Markov Chain of Continuous Thoughts

Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasoner

SpiralThinker: Latent Reasoning through an Iterative Process with Text-Latent Interleaving

Mull-Tokens: Modality-Agnostic Latent Thinking

ConFu: Contemplate the Future for Better Speculative Sampling

Receipt and verification

First computed	2026-05-17T23:38:12.664428Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

72c38e9474274e88344a06c9a05b28446923da29442b1a19d42667b405a9396f

Aliases

arxiv: 2502.21074 · arxiv_version: 2502.21074v3 · doi: 10.48550/arxiv.2502.21074 · pith_short_12: OLBY5FDUE5HI · pith_short_16: OLBY5FDUE5HIQNCK · pith_short_8: OLBY5FDU

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/OLBY5FDUE5HIQNCKA3E2AWZIIR \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 72c38e9474274e88344a06c9a05b28446923da29442b1a19d42667b405a9396f

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "b439973371a7088a8c826d686f23ee531eed1627abe2f31744fa6508a976f03b",
    "cross_cats_sorted": [],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2025-02-28T14:07:48Z",
    "title_canon_sha256": "fec4a20ac60bf55293b6e3d5812e45d9bb1ad1127781d285889ad7f64202510a"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2502.21074",
    "kind": "arxiv",
    "version": 3
  }
}