Pith Number

pith:WM7MTQMP

pith:2026:WM7MTQMPHRMRSQMXUN5KL77VAK

not attested not anchored not stored refs resolved

Heuristic Pathologies and Further Variance Reduction via Uncertainty Propagation in the AIVAT Family of Techniques

Juho Kim, Tuomas Sandholm

Fix the heuristic value function before seeing evaluation data to avoid setting AIVAT sample variance pathologically low or enabling p-hacking via gradient descent on the test statistic.

arxiv:2605.14261 v1 · 2026-05-14 · cs.AI · cs.GT

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{WM7MTQMPHRMRSQMXUN5KL77VAK}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

The heuristic value function should be fixed prior to observing the evaluation data to prevent setting sample variance pathologically low or p-hacking via gradient descent; uncertainty propagation then enables further variance reduction via inverse-variance weighted averaging, yielding a 43.0% reduction in samples needed on 10,000 poker hands.

C2weakest assumption

That the heuristic uncertainty can be quantified and propagated in a way that produces meaningful further variance reduction without introducing biases or errors that invalidate the overall estimator, and that the poker dataset and parameterization choices generalize beyond the specific experiments.

C3one line summary

AIVAT heuristics can be gamed for pathological low variance or p-hacking unless fixed before data observation, and uncertainty propagation yields additional variance reduction at possible cost to unbiasedness.

References

15 extracted · 15 resolved · 0 Pith anchors

[1] N. Bard, J. Hawkin, J. Rubin, and M. Zinkevich. The annual computer poker competition.AI Magazine, 34(2):112–114, 2013 2013

[2] D. Billings and M. Kan. A tool for the direct assessment of poker decisions.ICGA Journal, 29 (3):119–142, 2006 2006

[3] M. Bowling, M. Johanson, N. Burch, and D. Szafron. Strategy evaluation in extensive games with importance sampling. InProceedings of the International Conference on Machine Learning (ICML), 2008 2008

[4] N. Brown and T. Sandholm. Superhuman AI for heads-up no-limit poker: Libratus beats top professionals.Science, 359(6374):418–424, 2018 2018

[5] N. Brown and T. Sandholm. Superhuman AI for multiplayer poker.Science, 365(6456):885–890, 2019 2019

Formal links

1 machine-checked theorem link

Receipt and verification

First computed	2026-05-17T23:39:10.481164Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

b33ec9c18f3c59194197a37aa5fff502b2dbe2336ead81683e2a36d1d3f7f89e

Aliases

arxiv: 2605.14261 · arxiv_version: 2605.14261v1 · doi: 10.48550/arxiv.2605.14261 · pith_short_12: WM7MTQMPHRMR · pith_short_16: WM7MTQMPHRMRSQMX · pith_short_8: WM7MTQMP

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/WM7MTQMPHRMRSQMXUN5KL77VAK \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: b33ec9c18f3c59194197a37aa5fff502b2dbe2336ead81683e2a36d1d3f7f89e

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "c43d33b7d07dc3ab06000cedcc56c348c5204a29ce2b93b11efa12dc815c454f",
    "cross_cats_sorted": [
      "cs.GT"
    ],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.AI",
    "submitted_at": "2026-05-14T02:04:26Z",
    "title_canon_sha256": "ab6dfbb22e7ab7e2bdfe20d0f4886f2f5c33c566f624112e167eecd5912d6471"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2605.14261",
    "kind": "arxiv",
    "version": 1
  }
}