Pith Number

pith:DRCNPBOL

pith:2026:DRCNPBOL4LFP4KPSBUUH2GHM3S

not attested not anchored not stored refs resolved

When Attention Closes: How LLMs Lose the Thread in Multi-Turn Interaction

Dilek Hakkani-T\"ur, Joseph Hsieh, Seunghyun Yoon, Trung Bui, Vardhan Dongre, Viet Dac Lai

LLMs lose track of instructions in multi-turn chats when attention to goal tokens fades, though residual streams may still encode the needed information.

arxiv:2605.12922 v1 · 2026-05-13 · cs.AI · cs.CL

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{DRCNPBOL4LFP4KPSBUUH2GHM3S}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

Across architectures and model scales, the gap between attention loss and residual decodability predicts whether goal-conditioned behavior survives channel closure.

C2weakest assumption

That the linear probes on residual streams recover causally relevant goal information rather than spurious correlations, and that the sliding-window ablations isolate the attention channel without confounding other mechanisms.

C3one line summary

Attention to goal tokens declines in multi-turn LLM interactions while residual representations often retain decodable goal information, and the gap between these predicts whether goal-conditioned behavior survives.

References

39 extracted · 39 resolved · 17 Pith anchors

[1] LLMs Get Lost In Multi-Turn Conversation · arXiv:2505.06120

[2] Multi-if: Benchmarking llms on multi-turn and multilingual instructions following

[3] Transactions of the association for computational linguistics , volume=

[4] Roformer: Enhanced transformer with rotary position embedding , author=. Neurocomputing , volume=. 2024 , publisher= 2024

[5] Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

Formal links

2 machine-checked theorem links

Receipt and verification

First computed	2026-05-18T03:09:10.199507Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

1c44d785cbe2cafe29f20d287d18ecdca6420aaac85dabee8dacae5215f2c33d

Aliases

arxiv: 2605.12922 · arxiv_version: 2605.12922v1 · doi: 10.48550/arxiv.2605.12922 · pith_short_12: DRCNPBOL4LFP · pith_short_16: DRCNPBOL4LFP4KPS · pith_short_8: DRCNPBOL

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/DRCNPBOL4LFP4KPSBUUH2GHM3S \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 1c44d785cbe2cafe29f20d287d18ecdca6420aaac85dabee8dacae5215f2c33d

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "6c8c5dbb6f4bc4438f88cdee9f6f6be59c17805b60a5aa72fe5788b4dc225d20",
    "cross_cats_sorted": [
      "cs.CL"
    ],
    "license": "http://creativecommons.org/licenses/by-sa/4.0/",
    "primary_cat": "cs.AI",
    "submitted_at": "2026-05-13T02:58:18Z",
    "title_canon_sha256": "2bbf3223d3da58d082e2aedb69c8f93e0a5c66f6ef707e03a30352b81963495e"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2605.12922",
    "kind": "arxiv",
    "version": 1
  }
}