Pith Number

pith:AVENB3IB

pith:2026:AVENB3IBCVFZBCAJ4KISDJXH6T

not attested not anchored not stored refs pending

Efficiently Aligning Language Models with Online Natural Language Feedback

Christine Ye, Joe Benton

Natural language feedback builds proxy rewards that align language models with up to 50 times fewer expert samples.

arxiv:2605.04356 v2 · 2026-05-05 · cs.LG · cs.AI

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{AVENB3IBCVFZBCAJ4KISDJXH6T}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

For Qwen3-8B, ICL methods recover up to 35% of performance with 50x fewer expert samples, while fine-tuning methods recover 80% with up to 20x fewer samples and 100% with 3x fewer samples. For Haiku 4.5, ICL methods recover up to 35% of performance with 30x fewer samples, and fine-tuning methods recover 100% with 10x fewer samples.

C2weakest assumption

That proxy reward models constructed via ICL or fine-tuning on limited natural language feedback will continue to provide useful training signals without introducing systematic biases or being gamed in ways that degrade actual alignment quality.

C3one line summary

Online natural language feedback enables recovery of 35-100% of alignment performance in fuzzy domains using 3-50x fewer expert samples via iterative proxy reward updates with ICL and fine-tuning.

Receipt and verification

First computed	2026-06-04T00:06:43.647372Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

0548d0ed01154b908809e29121a6e7f4e302d2aa339eca96b748b69bce213116

Aliases

arxiv: 2605.04356 · arxiv_version: 2605.04356v2 · doi: 10.48550/arxiv.2605.04356 · pith_short_12: AVENB3IBCVFZ · pith_short_16: AVENB3IBCVFZBCAJ · pith_short_8: AVENB3IB

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/AVENB3IBCVFZBCAJ4KISDJXH6T \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 0548d0ed01154b908809e29121a6e7f4e302d2aa339eca96b748b69bce213116

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "1a53a2c1de0780da99fae1934fb22e6f32e79e449857b058d7a475c453baa00a",
    "cross_cats_sorted": [
      "cs.AI"
    ],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "cs.LG",
    "submitted_at": "2026-05-05T23:25:00Z",
    "title_canon_sha256": "c7e29bfac70de996b41da12ee86de15520cb004bdb92d8dd660faf343c407cb4"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2605.04356",
    "kind": "arxiv",
    "version": 2
  }
}