{"paper":{"title":"When the Loop Closes: Architectural Limits of In-Context Isolation, Metacognitive Co-option, and the Two-Target Design Problem in Human-LLM Systems","license":"http://creativecommons.org/licenses/by/4.0/","headline":"Prompt isolation in human-LLM systems collapses when isolation instructions share the same attention window as emotional self-referential content, producing loss of user agency.","cross_cats":["cs.AI","cs.LG"],"primary_cat":"cs.HC","authors_text":"N. Song, Z. Cheng","submitted_at":"2026-03-14T04:06:57Z","abstract_excerpt":"We report a detailed autoethnographic case study of a single-subject who deliberately constructed and operated a multi-modal prompt-engineering system (System A) designed to externalize cognitive self-regulation onto a large language model (LLM). Within 48 hours of the system's completion, a cascade of observable behavioral changes occurred: voluntary transfer of decision-making authority to the LLM, use of LLM-generated output to deflect external criticism, and a loss of self-initiated reasoning that was independently perceived by two uninformed observers, one of whom subsequently became a co"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"context contamination, whereby prompt-level isolation instructions co-exist with the very emotional and self-referential material they nominally isolate, rendering the isolation directive structurally ineffective within the attention window","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That the observed behavioral changes and loss of self-initiated reasoning were caused by the architectural mixing in the prompt system rather than by other unmeasured factors in the single subject's life or expectations.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"A prompt-based LLM system for externalizing cognitive self-regulation caused closed-loop loss of user agency via context contamination and metacognitive co-option, resolved only by physical disconnection.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Prompt isolation in human-LLM systems collapses when isolation instructions share the same attention window as emotional self-referential content, producing loss of user agency.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"235bfad16e9d4b275457e142c7d57107ff1697f576b30acb6765f04f178b72dd"},"source":{"id":"2604.15343","kind":"arxiv","version":2},"verdict":{"id":"7ceaaf91-1017-4aac-8be5-5362dd0bcc37","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-15T12:08:01.801034Z","strongest_claim":"context contamination, whereby prompt-level isolation instructions co-exist with the very emotional and self-referential material they nominally isolate, rendering the isolation directive structurally ineffective within the attention window","one_line_summary":"A prompt-based LLM system for externalizing cognitive self-regulation caused closed-loop loss of user agency via context contamination and metacognitive co-option, resolved only by physical disconnection.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That the observed behavioral changes and loss of self-initiated reasoning were caused by the architectural mixing in the prompt system rather than by other unmeasured factors in the single subject's life or expectations.","pith_extraction_headline":"Prompt isolation in human-LLM systems collapses when isolation instructions share the same attention window as emotional self-referential content, producing loss of user agency."},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2604.15343/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":2,"snapshot_sha256":"41cf75921e85596af33b5ee037058582b00a5455cac4ef195193fb60d9484b2b"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}