pith:S5MHI7VN
Grading the Unspoken: Evaluating Tacit Reasoning in Quantum Field Theory and String Theory with LLMs
LLMs achieve near-ceiling scores on explicit quantum field theory derivations but degrade when required to reconstruct omitted reasoning steps or reorganize representations under global consistency constraints.
arxiv:2604.14188 v1 · 2026-04-01 · physics.comp-ph · cs.CL · hep-th · physics.comp-ph
Add to your LaTeX paper
\usepackage{pith}
\pithnumber{S5MHI7VNOGKWKQ2GDHQZLNB7JQ}
Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge
Record completeness
Claims
We observe near-ceiling performance on explicit derivations within stable conceptual frames, but systematic degradation when tasks require reconstruction of omitted reasoning steps or reorganization of representations under global consistency constraints.
The expert-curated set of twelve questions and the five-level rubric accurately isolate and measure tacit reasoning demands without introducing selection bias or rubric-specific artifacts.
LLMs reach near-ceiling performance on explicit QFT and string theory derivations but degrade when required to reconstruct omitted reasoning steps or resolve implicit tensions under global consistency constraints.
References
Receipt and verification
| First computed | 2026-07-08T14:19:00.148259Z |
|---|---|
| Builder | pith-number-builder-2026-05-17-v1 |
| Signature | Pith Ed25519
(pith-v1-2026-05) · public key |
| Schema | pith-number/v1.0 |
Canonical hash
9758747ead719565434619e195b43f4c28fe936585472d19155af5efd2f8d721
Aliases
· · · · ·Agent API
Verify this Pith Number yourself
curl -sH 'Accept: application/ld+json' https://pith.science/pith/S5MHI7VNOGKWKQ2GDHQZLNB7JQ \
| jq -c '.canonical_record' \
| python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 9758747ead719565434619e195b43f4c28fe936585472d19155af5efd2f8d721
Canonical record JSON
{
"metadata": {
"abstract_canon_sha256": "c80cacce1e74cd95a4fb45054680bccdca19723507ac2ee38f15dd67d59569cb",
"cross_cats_sorted": [
"cs.CL",
"hep-th",
"physics.comp-ph"
],
"license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
"primary_cat": "physics.comp-ph",
"submitted_at": "2026-04-01T02:03:15Z",
"title_canon_sha256": "249263378c57f1974d0f8a779ffd5d0f8d4c34afae65017298ac446495bb5110"
},
"schema_version": "1.0",
"source": {
"id": "2604.14188",
"kind": "arxiv",
"version": 1
}
}