{"paper":{"title":"Perceptrons and localization of attention's mean-field landscape","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"The perceptron block makes critical points of the mean-field attention energy atomic and localized on the sphere.","cross_cats":["math.OC"],"primary_cat":"cs.LG","authors_text":"Antonio \\'Alvarez-L\\'opez, Borjan Geshkovski, Dom\\`enec Ruiz-Balet","submitted_at":"2026-01-29T07:47:46Z","abstract_excerpt":"The forward pass of a Transformer can be seen as an interacting particle system on the unit sphere: time plays the role of layers, particles that of token embeddings, and the unit sphere idealizes layer normalization. In some weight settings the system can even be seen as a gradient flow for an explicit energy, and one can make sense of the infinite context length (mean-field) limit thanks to Wasserstein gradient flows. In this paper we study the effect of the perceptron block in this setting, and show that critical points are generically atomic and localized on subsets of the sphere."},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"critical points are generically atomic and localized on subsets of the sphere.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That in some weight settings the system can be seen as a gradient flow for an explicit energy, allowing the mean-field Wasserstein analysis to apply directly to the perceptron block.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"In the mean-field limit of attention with perceptron blocks, critical points of the energy landscape are generically atomic and localized on subsets of the unit sphere.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"The perceptron block makes critical points of the mean-field attention energy atomic and localized on the sphere.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"f6a100aa050adf9469d4f53b976c30ae098c66fe425c4727f538657bfd65c590"},"source":{"id":"2601.21366","kind":"arxiv","version":2},"verdict":{"id":"7357b214-c1fd-42d3-bd64-8f792d91747f","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-16T09:46:43.424606Z","strongest_claim":"critical points are generically atomic and localized on subsets of the sphere.","one_line_summary":"In the mean-field limit of attention with perceptron blocks, critical points of the energy landscape are generically atomic and localized on subsets of the unit sphere.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That in some weight settings the system can be seen as a gradient flow for an explicit energy, allowing the mean-field Wasserstein analysis to apply directly to the perceptron block.","pith_extraction_headline":"The perceptron block makes critical points of the mean-field attention energy atomic and localized on the sphere."},"references":{"count":17,"sample":[{"doi":"","year":null,"title":"Atten- tion’s forward pass and Frank-Wolfe.arXiv preprint arXiv:2508.09628,","work_id":"04dcc830-26c9-4d53-a03a-9610389db975","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":null,"title":"Bronstein and Petar Velickovic and Razvan Pascanu , title =","work_id":"298dab4e-c69c-4720-af0e-c20f50c58b39","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2025,"title":"Emer- gence of meta-stable clustering in mean-field transformer models","work_id":"6c010231-0766-4a85-82a5-2906c54dc3fd","ref_index":3,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2025,"title":"36 [BPA25b] Giuseppe Bruno, Federico Pasqualotto, and Andrea Agazzi","work_id":"3fa3ce2e-1fd7-4dcb-8118-01ebf7d772f7","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2025,"title":"A phase transition between positional and semantic learning in a solvable model of dot-product attention.Journal of Statistical Mechanics: Theory and Experiment, 2025(7):074001,","work_id":"8e435152-3add-47fb-a4b4-2781e3eb9b95","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":17,"snapshot_sha256":"36cad539574664ae0158bf6f9cfdb6a6aeb668c9895926f87ca0666a15bfa74f","internal_anchors":3},"formal_canon":{"evidence_count":2,"snapshot_sha256":"6f8a80f89988b4a1fcb41978fad33706027ad692cae3801a7c28ec6b8441d328"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}