Transformer activations show spectral anti-concentration for concepts in the tail while syntax prefers high-variance directions, forming a dual geometry.
Proceedings of the National Academy of Sciences , volume=
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
years
2026 2verdicts
UNVERDICTED 2representative citing papers
Hyperspherical Pooled Mahalanobis (HPM) normalizes frozen long-tailed features to the unit sphere and uses pooled ridge-regularized covariance to improve raw Mahalanobis OOD scoring, lifting AUROC from 46.49 to 85.67 on CIFAR-10-LT.
citing papers explorer
-
Concepts Whisper While Syntax Shouts: Spectral Anti-Concentration and the Dual Geometry of Transformer Representations
Transformer activations show spectral anti-concentration for concepts in the tail while syntax prefers high-variance directions, forming a dual geometry.
-
Is Complex Training Necessary for Long-Tailed OOD Detection? A Re-think from Feature Geometry
Hyperspherical Pooled Mahalanobis (HPM) normalizes frozen long-tailed features to the unit sphere and uses pooled ridge-regularized covariance to improve raw Mahalanobis OOD scoring, lifting AUROC from 46.49 to 85.67 on CIFAR-10-LT.