Pith Number

pith:V55XZYK2

pith:2026:V55XZYK2HASK3NDXNHQME2X2WL

not attested not anchored not stored refs resolved

When Should an AI Workflow Release? Always-Valid Inference for Black-Box Generate-Verify Systems

Will Wei Sun, Young Hyun Cho

A hard-negative reference pool of high-scoring failures gives finite-sample control over when black-box AI workflows release outputs on infeasible tasks.

arxiv:2605.12947 v1 · 2026-05-13 · stat.ML · cs.AI · cs.LG · stat.ME

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{V55XZYK2HASK3NDXNHQME2X2WL}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

a conservative reference pool yields finite-sample control of the probability of releasing on infeasible tasks, that is, tasks for which the given workflow is not capable of producing a reliable solution.

C2weakest assumption

That a hard-negative reference pool of high-scoring failures can be built that is sufficiently conservative to deliver finite-sample error control yet not so conservative that it blocks release on feasible tasks where moderate evidence accumulates.

C3one line summary

A wrapper for black-box generate-verify AI pipelines that uses a conservative hard-negative reference pool and e-processes to control the probability of releasing on infeasible tasks while permitting release on feasible ones.

References

34 extracted · 34 resolved · 9 Pith anchors

[1] Do As I Can, Not As I Say: Grounding Language in Robotic Affordances 2022 · arXiv:2204.01691

[2] Program Synthesis with Large Language Models 2021 · arXiv:2108.07732

[3] Barber, R. F., Candes, E. J., Ramdas, A. & Tibshirani, R. J. (2023), ‘Conformal prediction beyond exchangeability’,The Annals of Statistics51(2), 816–845 2023

[4] Training Verifiers to Solve Math Word Problems 2021 · arXiv:2110.14168

[5] Doob, J. L. (1939), ‘Jean ville, étude critique de la notion de collectif’. Grünwald, P., de Heide, R. & Koolen, W. M. (2020), Safe testing,in‘2020 Information theory and applications workshop (ITA)’, 1939

Formal links

2 machine-checked theorem links

Receipt and verification

First computed	2026-05-18T03:09:09.473696Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

af7b7ce15a3824adb47769e0c26afab2c733a8de9b0611a9b3cbb09193eb8ff5

Aliases

arxiv: 2605.12947 · arxiv_version: 2605.12947v1 · doi: 10.48550/arxiv.2605.12947 · pith_short_12: V55XZYK2HASK · pith_short_16: V55XZYK2HASK3NDX · pith_short_8: V55XZYK2

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/V55XZYK2HASK3NDXNHQME2X2WL \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: af7b7ce15a3824adb47769e0c26afab2c733a8de9b0611a9b3cbb09193eb8ff5

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "72766323ae5962a588ec581f64b33632f7ee70c50106b8915a017a7757009e8e",
    "cross_cats_sorted": [
      "cs.AI",
      "cs.LG",
      "stat.ME"
    ],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "stat.ML",
    "submitted_at": "2026-05-13T03:30:39Z",
    "title_canon_sha256": "01075e14f89e8ed7c06fe2ba3ff7657279947c0f3ac38e7d7ff1adb4693fd8cd"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2605.12947",
    "kind": "arxiv",
    "version": 1
  }
}