{"paper":{"title":"Do Language Models Align with Brains? Prediction Scores Are Not Enough","license":"http://creativecommons.org/licenses/by/4.0/","headline":"Language-model representations fail L-PACT alignment gates once nuisance controls and brain-brain ceilings are applied.","cross_cats":["cs.AI"],"primary_cat":"q-bio.NC","authors_text":"Xiao Jia","submitted_at":"2026-05-13T18:37:17Z","abstract_excerpt":"Brain-language model comparisons often interpret neural prediction scores as evidence that model representations capture brain-relevant language computation. We asked whether language models align with brains, and whether prediction scores are enough to support that claim, using L-PACT, a source-audited framework that evaluates predictive, relational, mechanism-stripping, and reliability-bounded evidence. Across primary naturalistic language neural datasets and derived language-model representations, L-PACT compared real model features with nuisance baselines and severe controls, tested whethe"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"In the analyzed derived artifact set, the tested language-model representations do not satisfy L-PACT alignment gates; apparent positives are converted into an auditable control-explained taxonomy rather than treated as structural alignment.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That the chosen nuisance baselines, acoustic-envelope gates, and brain-brain ceilings fully capture all alternative explanations for observed model-to-brain prediction scores without excluding genuine alignment signals.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"Language model representations fail all L-PACT alignment gates once controls explain the apparent predictive and relational effects.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Language-model representations fail L-PACT alignment gates once nuisance controls and brain-brain ceilings are applied.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"2d18c7fb7011b5ece06e3108e9218317c6a19cbc435a0a54e9b22344e2f06ba2"},"source":{"id":"2605.14025","kind":"arxiv","version":1},"verdict":{"id":"17038d9b-5948-484f-b1d8-f021bbe94b98","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-15T05:49:38.470077Z","strongest_claim":"In the analyzed derived artifact set, the tested language-model representations do not satisfy L-PACT alignment gates; apparent positives are converted into an auditable control-explained taxonomy rather than treated as structural alignment.","one_line_summary":"Language model representations fail all L-PACT alignment gates once controls explain the apparent predictive and relational effects.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That the chosen nuisance baselines, acoustic-envelope gates, and brain-brain ceilings fully capture all alternative explanations for observed model-to-brain prediction scores without excluding genuine alignment signals.","pith_extraction_headline":"Language-model representations fail L-PACT alignment gates once nuisance controls and brain-brain ceilings are applied."},"references":{"count":64,"sample":[{"doi":"","year":2016,"title":"A. G. Huth, W. A. de Heer, T. L. Griffiths, F. E. Theunissen, J. L. Gallant, Natural speech reveals the semantic maps that tile human cerebral cortex.Nature532, 453–458 (2016)","work_id":"7d82120b-92cd-4a2f-8504-44aed23f9bd3","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2018,"title":"S. Jain, A. G. Huth, Incorporating context into language encoding models for fMRI.Advances in Neural Information Processing Systems31(2018)","work_id":"85e30981-7779-4af5-b71b-180b6f2e7401","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2021,"title":"Schrimpf et al., The neural architecture of language: Integrative modeling converges on predictive processing","work_id":"81316b8e-f32f-457c-a0aa-4f7cbf4af036","ref_index":3,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2022,"title":"C. Caucheteux, J.-R. King, Brains and algorithms partially converge in natural language processing.Communica- tions Biology5, 134 (2022)","work_id":"7e3a342e-a5bc-4544-83e2-ff45f07a8e97","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2022,"title":"Goldstein et al., Shared computational principles for language processing in humans and deep language models","work_id":"9ba281da-974c-420e-a8fc-1c2037a35b08","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":64,"snapshot_sha256":"beb8e2c5d521c86aa180eda74d8b5ff8f3d19a9265091709c590b56519b8f75f","internal_anchors":4},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}