pith:XWKVJIXS
The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
Large Reasoning Models exhibit complete accuracy collapse beyond certain complexities and reduce reasoning effort despite available compute.
arxiv:2506.06941 v3 · 2025-06-07 · cs.AI · cs.CL · cs.LG
Add to your LaTeX paper
\usepackage{pith}
\pithnumber{XWKVJIXSLVYNKHWXKRVTXQIOU2}
Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge
Record completeness
Claims
LRMs face a complete accuracy collapse beyond certain complexities. Moreover, they exhibit a counterintuitive scaling limit: their reasoning effort increases with problem complexity up to a point, then declines despite having remaining token budget.
That the chosen controllable puzzle environments provide an unbiased and generalizable measure of reasoning complexity without introducing artifacts that do not appear in other domains such as math or coding.
LRMs exhibit complete accuracy collapse beyond certain puzzle complexities, with reasoning effort rising then declining, outperforming standard LLMs only on medium-complexity tasks.
References
Formal links
Cited by
Receipt and verification
| First computed | 2026-05-17T23:38:50.945787Z |
|---|---|
| Builder | pith-number-builder-2026-05-17-v1 |
| Signature | Pith Ed25519
(pith-v1-2026-05) · public key |
| Schema | pith-number/v1.0 |
Canonical hash
bd9554a2f25d70d51ed7546b3bc10ea6987dd4cbd948aa53f779b964a512b7c5
Aliases
· · · · ·Agent API
Verify this Pith Number yourself
curl -sH 'Accept: application/ld+json' https://pith.science/pith/XWKVJIXSLVYNKHWXKRVTXQIOU2 \
| jq -c '.canonical_record' \
| python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: bd9554a2f25d70d51ed7546b3bc10ea6987dd4cbd948aa53f779b964a512b7c5
Canonical record JSON
{
"metadata": {
"abstract_canon_sha256": "8a45099f14d045accff594ca13ca08c77d46017efad9a353a561b48d2641f330",
"cross_cats_sorted": [
"cs.CL",
"cs.LG"
],
"license": "http://creativecommons.org/licenses/by/4.0/",
"primary_cat": "cs.AI",
"submitted_at": "2025-06-07T22:42:29Z",
"title_canon_sha256": "a0d32bd599754e05eb9948d06ed7aed1b2cdac8f3f64203a8c1b4e2a57a86a6c"
},
"schema_version": "1.0",
"source": {
"id": "2506.06941",
"kind": "arxiv",
"version": 3
}
}