Pith Number

pith:AP2ERP4W

pith:2023:AP2ERP4WROMF6ADSYSWAL5U4KF

not attested not anchored not stored refs resolved

Large Language Models as Optimizers

Chengrun Yang, Denny Zhou, Hanxiao Liu, Quoc V. Le, Xinyun Chen, Xuezhi Wang, Yifeng Lu

Large language models can optimize solutions by iteratively generating new candidates from a prompt that lists all prior attempts together with their scores.

arxiv:2309.03409 v3 · 2023-09-07 · cs.LG · cs.AI · cs.CL

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{AP2ERP4WROMF6ADSYSWAL5U4KF}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

With a variety of LLMs, we demonstrate that the best prompts optimized by OPRO outperform human-designed prompts by up to 8% on GSM8K, and by up to 50% on Big-Bench Hard tasks.

C2weakest assumption

That an LLM, when shown a growing list of prior solutions and their numeric scores inside a prompt, will reliably generate new solutions that improve on the best previous score rather than plateau or regress.

C3one line summary

Large language models can optimize by being prompted with histories of past solutions and scores to propose better ones, producing prompts that raise accuracy up to 8% on GSM8K and 50% on Big-Bench Hard over human-designed baselines.

References

51 extracted · 51 resolved · 22 Pith anchors

[1] PaLM 2 Technical Report · arXiv:2305.10403

[2] Constitutional AI: Harmlessness from AI Feedback · arXiv:2212.08073

[3] arXiv preprint arXiv:2305.17126 , year=

[4] Dohan and David R

[5] Teaching Large Language Models to Self-Debug · arXiv:2304.05128

Formal links

2 machine-checked theorem links

Cited by

33 papers in Pith

PRISM: Prompt Reliability via Iterative Simulation and Monitoring for Enterprise Conversational AI

Contexting as Recommendation: Evolutionary Collaborative Filtering for Context Engineering

FORGE: Self-Evolving Agent Memory With No Weight Updates via Population Broadcast

Towards Robust Argumentative Essay Understanding via TIDE: An Interactive Framework with Trial and Debate

Prompt Optimization for LLM Code Generation via Reinforcement Learning

Receipt and verification

First computed	2026-05-17T23:39:19.769046Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

03f448bf968b985f0072c4ac05f69c515c7535edb675e2102f91ffe4f89aa05c

Aliases

arxiv: 2309.03409 · arxiv_version: 2309.03409v3 · doi: 10.48550/arxiv.2309.03409 · pith_short_12: AP2ERP4WROMF · pith_short_16: AP2ERP4WROMF6ADS · pith_short_8: AP2ERP4W

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/AP2ERP4WROMF6ADSYSWAL5U4KF \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 03f448bf968b985f0072c4ac05f69c515c7535edb675e2102f91ffe4f89aa05c

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "696a3ad040aa7c54bce5841a41ecbdc0ec1f364eb2796cfc81219edc4c0c3f43",
    "cross_cats_sorted": [
      "cs.AI",
      "cs.CL"
    ],
    "license": "http://creativecommons.org/publicdomain/zero/1.0/",
    "primary_cat": "cs.LG",
    "submitted_at": "2023-09-07T00:07:15Z",
    "title_canon_sha256": "c7267a9ba911c0155389c05e1a869f5061975eba6a5996cee9757c2e248dba5a"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2309.03409",
    "kind": "arxiv",
    "version": 3
  }
}