Pith Number

pith:WYOTTIM7

pith:2026:WYOTTIM7ROJ4XIBGSHJUUJL54H

not attested not anchored not stored refs resolved

Accelerating Zeroth-Order Spectral Optimization with Partial Orthogonalization from Power Iteration

Jiahe Chen, Ziye Ma

Partial orthogonalization via power iteration accelerates zeroth-order fine-tuning of large language models.

arxiv:2605.09034 v2 · 2026-05-09 · cs.LG

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{WYOTTIM7ROJ4XIBGSHJUUJL54H}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

Experiments on LLM fine-tuning show that our method can achieve from 1.5x to 4x the convergence speed of ZO-Muon, the current SOTA algorithm, across SuperGlue datasets in the OPT-13B model.

C2weakest assumption

The assumption that projecting the search space onto a momentum-derived subspace sufficiently lowers gradient variance to stabilize the streaming power-iteration procedure and enable effective partial orthogonalization in the zeroth-order regime (abstract, paragraph on streaming variant).

C3one line summary

ZO-MOPI accelerates zeroth-order LLM fine-tuning by applying partial spectral orthogonalization from power iteration inside a momentum-projected subspace to reduce variance and exploit dominant directions.

References

26 extracted · 26 resolved · 6 Pith anchors

[1] Dion: Distributed orthonormal- ized updates.arXiv preprint: 2504.05295

[2] Enhancing zeroth-order fine-tuning for language models with low-rank structures.arXiv preprint arXiv:2410.07698

[3] Boolq: Exploring the surprising difficulty of natural yes/no questions 2019

[4] Variance- reduced zeroth-order methods for fine-tuning language models.arXiv preprint arXiv:2404.08080

[5] ARO : A New Lens On Matrix Optimization For Large Models

Formal links

2 machine-checked theorem links

Receipt and verification

First computed	2026-05-20T00:01:43.050728Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

b61d39a19f8b93cba02691d34a257de1c5155204de04de9c2df55619a9e876da

Aliases

arxiv: 2605.09034 · arxiv_version: 2605.09034v2 · doi: 10.48550/arxiv.2605.09034 · pith_short_12: WYOTTIM7ROJ4 · pith_short_16: WYOTTIM7ROJ4XIBG · pith_short_8: WYOTTIM7

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/WYOTTIM7ROJ4XIBGSHJUUJL54H \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: b61d39a19f8b93cba02691d34a257de1c5155204de04de9c2df55619a9e876da

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "bb98591811764327e9dcc45e04ece45f1493c8f7f3df164546662746b696236a",
    "cross_cats_sorted": [],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.LG",
    "submitted_at": "2026-05-09T16:16:45Z",
    "title_canon_sha256": "99cf8b2f472b177bbce07c93deb1fc36c779117d99a8bd71f72d650e982582fe"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2605.09034",
    "kind": "arxiv",
    "version": 2
  }
}