Pith Number

pith:IW3VIUTT

pith:2026:IW3VIUTTAZXJV5WNFJ5SN5FY4U

not attested not anchored not stored refs pending

Algorithmic Blindness in Large Language Models: A Calibration Study of Performance Prediction

Ashish Mahendran Kurapath, Sohan Venkatesh, Tejas Melkote

Large language models exhibit systematic failure at predicting algorithmic outcomes despite knowing algorithm facts.

arxiv:2602.21947 v5 · 2026-02-25 · cs.CL

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{IW3VIUTTAZXJV5WNFJ5SN5FY4U}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

We find systematic, near-total failure across models. The predicted ranges are far wider than true confidence intervals yet still fail to contain the true algorithmic mean in most cases. Most models perform worse than random guessing and the best model's marginal improvement is attributable to benchmark memorization rather than principled reasoning.

C2weakest assumption

That performance on causal discovery tasks constitutes a valid and generalizable test of algorithmic reasoning ability rather than a narrow or confounded proxy.

C3one line summary

LLMs exhibit algorithmic blindness, producing predictions whose ranges miss true algorithmic means in most cases and often perform worse than random guessing.

Receipt and verification

First computed	2026-07-28T01:22:30.388470Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

45b7545273066e9af6cd2a7b26f4b8e50fded2c25b5620de1dd6f293096be22c

Aliases

arxiv: 2602.21947 · arxiv_version: 2602.21947v5 · doi: 10.48550/arxiv.2602.21947 · pith_short_12: IW3VIUTTAZXJ · pith_short_16: IW3VIUTTAZXJV5WN · pith_short_8: IW3VIUTT

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/IW3VIUTTAZXJV5WNFJ5SN5FY4U \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 45b7545273066e9af6cd2a7b26f4b8e50fded2c25b5620de1dd6f293096be22c

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "8cba68210066087d5ea0ad0b49dae0ff41901c5d32fee6cfe3046bab57a2a2f1",
    "cross_cats_sorted": [],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2026-02-25T14:32:15Z",
    "title_canon_sha256": "438db4d5b341362553dfc592b579c836629a2fb5d41bd991285198e03575af59"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2602.21947",
    "kind": "arxiv",
    "version": 5
  }
}