Pith Number

pith:KWFZ7LKC

pith:2026:KWFZ7LKCVDH2UJMDUDL7SIR6CL

not attested not anchored not stored refs resolved

MixSD: Mixed Contextual Self-Distillation for Knowledge Injection

Jiarui Liu, Lechen Zhang, Mona Diab, Weihao Xuan, Yingheng Wang, Yinghui He, Yongjin Yang, Zhijing Jin

Aligning fine-tuning targets with a language model's own generation distribution prevents catastrophic forgetting of pretrained capabilities.

arxiv:2605.16865 v1 · 2026-05-16 · cs.CL

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{KWFZ7LKCVDH2UJMDUDL7SIR6CL}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

aligning supervision with the model's native generation distribution is a simple and effective principle for knowledge injection that mitigates catastrophic forgetting.

C2weakest assumption

The mixed supervision sequences preserve the factual learning signal while remaining substantially closer to the base model's distribution, as constructed from the expert and naive conditionals.

C3one line summary

MixSD achieves superior memorization-retention trade-off in knowledge injection by using mixed self-generated supervision from the base model's conditionals, retaining up to 100% held-out capability versus 1% for standard SFT.

References

57 extracted · 57 resolved · 9 Pith anchors

[1] Measuring short-form factuality in large language models · arXiv:2411.04368

[2] arXiv preprint arXiv:2502.20377 , year=

[3] American Invitational Mathematics Examination (AIME) 2024 , author= 2024

[4] Measuring Mathematical Problem Solving With the MATH Dataset · arXiv:2103.03874

[5] Training Verifiers to Solve Math Word Problems · arXiv:2110.14168

Formal links

1 machine-checked theorem link

Receipt and verification

First computed	2026-05-20T00:03:27.143636Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

558b9fad42a8cfaa2583a0d7f9223e12fd89bae3676dcccdca66ce398399cc8a

Aliases

arxiv: 2605.16865 · arxiv_version: 2605.16865v1 · doi: 10.48550/arxiv.2605.16865 · pith_short_12: KWFZ7LKCVDH2 · pith_short_16: KWFZ7LKCVDH2UJMD · pith_short_8: KWFZ7LKC

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/KWFZ7LKCVDH2UJMDUDL7SIR6CL \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 558b9fad42a8cfaa2583a0d7f9223e12fd89bae3676dcccdca66ce398399cc8a

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "525996c2af35114b47f73523be53e16451c1d75fb47c025a9525adaa2e02f7bc",
    "cross_cats_sorted": [],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2026-05-16T07:57:09Z",
    "title_canon_sha256": "d91fa992df7169978a7e2800ca98bd851dbc95e62d7f4f7e1bb7ffd198ddadab"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2605.16865",
    "kind": "arxiv",
    "version": 1
  }
}