Pith Number

pith:KW7XYM5B

pith:2026:KW7XYM5B6RYRB4ECLKAOQB534H

not attested not anchored not stored refs pending

Learning to Foresee: Unveiling the Unlocking Efficiency of On-Policy Distillation

Chunxi Luo, Ding Cao, Guangzhong Sun, Guiquan Liu, Junfeng Fang, Kai Yang, Liang Lin, Saiyong Yang, Tianxiang Zhao, Weijie Liu, Xin Xu, Yuchen Cai

On-policy distillation locks onto a stable update trajectory toward the final model early in training.

arxiv:2605.11739 v3 · 2026-05-12 · cs.CL

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{KW7XYM5B6RYRB4ECLKAOQB534H}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

OPD's efficiency stems from a form of ``foresight'': it establishes a stable update trajectory toward the final model early in training. This manifests at the Module-Allocation Level by concentrating updates on critical modules and at the Update-Direction Level by stronger low-rank concentration aligning with the final subspace, enabling EffOPD to achieve an average 3x training acceleration.

C2weakest assumption

That the observed module utility patterns and low-rank alignment are causal drivers of efficiency rather than correlated side effects, and that adaptive extrapolation along the current direction generalizes without degrading final performance across diverse tasks and model scales.

C3one line summary

On-policy distillation gains efficiency from early foresight in module allocation and low-rank update directions, enabling EffOPD to accelerate training by 3x via adaptive extrapolation without extra modules or tuning.

Receipt and verification

First computed	2026-05-22T01:04:06.080374Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

55bf7c33a1f47110f0825a80e807bbe1df4d20a11d0fda0a615ccbf8dd5f3608

Aliases

arxiv: 2605.11739 · arxiv_version: 2605.11739v3 · doi: 10.48550/arxiv.2605.11739 · pith_short_12: KW7XYM5B6RYR · pith_short_16: KW7XYM5B6RYRB4EC · pith_short_8: KW7XYM5B

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/KW7XYM5B6RYRB4ECLKAOQB534H \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 55bf7c33a1f47110f0825a80e807bbe1df4d20a11d0fda0a615ccbf8dd5f3608

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "29be5cf5a4a7a59eae794005e1ab80404060a346f322416b0b85aff6eda66c38",
    "cross_cats_sorted": [],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2026-05-12T08:19:15Z",
    "title_canon_sha256": "24a4e720c67fa9bfcfb3d2ff0a86b4c0190ab5ee5944ed8d591d6b7e1b54d7b9"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2605.11739",
    "kind": "arxiv",
    "version": 3
  }
}