{"paper":{"title":"Probing Spectrum-Like Organization of States of Mind in Transformer Representation Spaces","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"Transformer embeddings recover annotated cognitive energy scores and seven-tier labels above chance","cross_cats":["cs.LG"],"primary_cat":"cs.CL","authors_text":"Sophie Zhao","submitted_at":"2025-12-23T03:37:34Z","abstract_excerpt":"We investigate whether graded states of mind form spectrum-like structure in transformer representation spaces. To do so, we construct a dataset of 636 short natural-language sentences annotated with both a continuous score from $-5$ to $5$ and one of seven ordered tiers, ranging from collapsed or scarcity-driven expressions to more coherent, reflective, and integrative ones. We evaluate five frozen transformer representations: four sentence-embedding models and one decoder-only residual-stream representation. Across all representations, simple probes reliably recover both the continuous score"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"transformer embedding spaces exhibit statistically significant geometric organization aligned with the annotated cognitive structure.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"The human annotations of energy scores and seven-tier cognitive progression accurately capture stable, human-interpretable cognitive attributes rather than surface linguistic features or annotator bias.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"Transformer sentence embeddings linearly recover continuous energy scores and seven-tier cognitive labels from 480 annotated sentences, with UMAP showing a coherent low-to-high gradient.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Transformer embeddings recover annotated cognitive energy scores and seven-tier labels above chance","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"75e940c4dd323d2bd35de7eb144a7b88ac151d9ce3fd3659bbe3a410e462fb4e"},"source":{"id":"2512.22227","kind":"arxiv","version":3},"verdict":{"id":"7abbe9b2-5c22-46ba-a933-76a9af33a9c6","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-16T20:29:55.797425Z","strongest_claim":"transformer embedding spaces exhibit statistically significant geometric organization aligned with the annotated cognitive structure.","one_line_summary":"Transformer sentence embeddings linearly recover continuous energy scores and seven-tier cognitive labels from 480 annotated sentences, with UMAP showing a coherent low-to-high gradient.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"The human annotations of energy scores and seven-tier cognitive progression accurately capture stable, human-interpretable cognitive attributes rather than surface linguistic features or annotator bias.","pith_extraction_headline":"Transformer embeddings recover annotated cognitive energy scores and seven-tier labels above chance"},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2512.22227/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":2,"snapshot_sha256":"dac88a87c649b857c010dcaea2bcfed1e1a769a75c4f884d2d2f870e13aa1e3c"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}