Pith Number

pith:BYO6ZUZA

pith:2026:BYO6ZUZAKDHZTZFUY5RFGTNQCS

not attested not anchored not stored refs resolved

Always Learning, Always Mixing: Efficient and Simple Data Mixing All The Time

Apurva Gandhi, Kyunghyun Cho, Michael Y. Hu, Pratyusha Sharma, Tal Linzen

OP-Mix simulates candidate data mixtures by interpolating low-rank adapters trained on the current model, enabling efficient mixing across all phases of language model training.

arxiv:2605.15220 v1 · 2026-05-13 · cs.CL · cs.AI · cs.LG

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{BYO6ZUZAKDHZTZFUY5RFGTNQCS}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

OP-Mix consistently finds near-optimal mixtures while using a fraction of the compute of the baselines. In pretraining, OP-Mix improves upon training without mixing by 6.3% in average perplexity. For continual learning, OP-Mix matches the performance of both retraining and on-policy distillation while using 66% and 95% less overall compute, respectively.

C2weakest assumption

That interpolating between low-rank adapters trained directly on the current model accurately simulates the effect of different data mixtures on the full model's learning dynamics without requiring separate proxy models or fixed domain assumptions.

C3one line summary

OP-Mix is an on-policy data mixing method that uses low-rank adapter interpolation to find near-optimal data mixtures throughout language model training with reduced compute.

References

61 extracted · 61 resolved · 3 Pith anchors

[1] arXiv preprint arXiv:2602.12237 , year=

[2] The Thirteenth International Conference on Learning Representations , year=

[3] and Carbin, Michael , title = 2020

[4] doi:10.5281/zenodo.12608602 , url = · doi:10.5281/zenodo.12608602

[5] Proceedings of the 39th International Conference on Machine Learning , pages = 2022

Receipt and verification

First computed	2026-05-20T00:00:46.924763Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

0e1decd32050cf99e4b4c762534db0148411cc5d3cdbb5e35a9dae5a7b84a2cb

Aliases

arxiv: 2605.15220 · arxiv_version: 2605.15220v1 · doi: 10.48550/arxiv.2605.15220 · pith_short_12: BYO6ZUZAKDHZ · pith_short_16: BYO6ZUZAKDHZTZFU · pith_short_8: BYO6ZUZA

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/BYO6ZUZAKDHZTZFUY5RFGTNQCS \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 0e1decd32050cf99e4b4c762534db0148411cc5d3cdbb5e35a9dae5a7b84a2cb

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "b02916e3588c09fe7233174d7c1f1ad36ab5296ff41f7fdcf464af241eab45ab",
    "cross_cats_sorted": [
      "cs.AI",
      "cs.LG"
    ],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2026-05-13T02:29:19Z",
    "title_canon_sha256": "47ccfa359dc3ecc490055107686b0da1b7f1ea016203b4f3c159ab97bd172d75"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2605.15220",
    "kind": "arxiv",
    "version": 1
  }
}