Pith Number

pith:APCCNCY4

pith:2022:APCCNCY42HHIJXSKVRWKP2A7K3

not attested not anchored not stored refs pending

OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization

Asli Celikyilmaz, Brian O'Horo, Christopher Dewan, Daniel Simig, Gabriel Pereyra, Jeff Wang, Kurt Shuster, Luke Zettlemoyer, Ping Yu, Punit Singh Koura, Qing Liu, Ramakanth Pasunuru, Srinivasan Iyer, Tianlu Wang, Todor Mihaylov, Ves Stoyanov, Xian Li, Xi Victoria Lin

Instruction-tuning on a 2000-task benchmark produces models that generalize to held-out categories, tasks, and instances.

arxiv:2212.12017 v3 · 2022-12-22 · cs.CL

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{APCCNCY42HHIJXSKVRWKP2A7K3}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

OPT-IML demonstrates all three generalization abilities at both scales on four different evaluation benchmarks with diverse tasks and input formats -- PromptSource, FLAN, Super-NaturalInstructions, and UnifiedSKG.

C2weakest assumption

That the consolidation of tasks from eight existing benchmarks into 2000 tasks and the defined held-out category/task/instance splits provide a representative and unbiased measure of generalization to truly unseen NLP problems.

C3one line summary

OPT-IML 30B and 175B models, trained on a new 2000-task instruction benchmark, demonstrate generalization to held-out categories, tasks, and instances while outperforming base OPT and competing with benchmark-specific models.

Formal links

2 machine-checked theorem links

Cited by

27 papers in Pith

The Flan Collection: Designing Data and Methods for Effective Instruction Tuning

Functional-level Uncertainty Quantification for Calibrated Fine-tuning on LLMs

Optimus: A Robust Defense Framework for Mitigating Toxicity while Fine-Tuning Conversational AI

Multilingual and Multimodal LLMs in the Wild: Building for Low-Resource Languages

A Comprehensive Overview of Large Language Models

Receipt and verification

First computed	2026-05-17T23:38:14.863298Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

03c4268b1cd1ce84de4aac6ca7e81f56d23f69fada537f6a0c40966b36b8a18f

Aliases

arxiv: 2212.12017 · arxiv_version: 2212.12017v3 · doi: 10.48550/arxiv.2212.12017 · pith_short_12: APCCNCY42HHI · pith_short_16: APCCNCY42HHIJXSK · pith_short_8: APCCNCY4

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/APCCNCY42HHIJXSKVRWKP2A7K3 \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 03c4268b1cd1ce84de4aac6ca7e81f56d23f69fada537f6a0c40966b36b8a18f

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "025164ad055811875b0f90849498f9de497302ffe14bb73802b0234deb1bbe90",
    "cross_cats_sorted": [],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2022-12-22T19:56:09Z",
    "title_canon_sha256": "50f144c76cd8960a3a65c93eec8753ba5b9444987ea030fc5c004d82595e8a06"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2212.12017",
    "kind": "arxiv",
    "version": 3
  }
}