Transformers achieve approximation and generalization error bounds for noisy manifold regression that scale with the intrinsic dimension of the task-level manifold.
The shape of learning: Anisotropy and intrinsic dimensions in transformer-based models
2 Pith papers cite this work, alongside 1 external citations. Polarity classification is still indexing.
2
Pith papers citing it
1
external citations · external index
fields
cs.LG 2verdicts
UNVERDICTED 2representative citing papers
Contribution Weights combine attention, value magnitude, and directional alignment to measure token influence more faithfully than attention alone, and show attention sinks actively suppress information via a convex sink-rate to output-norm relationship.
citing papers explorer
-
Transformers for Learning on Noisy and Task-Level Manifolds: Approximation and Generalization Insights
Transformers achieve approximation and generalization error bounds for noisy manifold regression that scale with the intrinsic dimension of the task-level manifold.
-
Contribution Weights: A Geometrical Analysis of Self-Attention Transformers
Contribution Weights combine attention, value magnitude, and directional alignment to measure token influence more faithfully than attention alone, and show attention sinks actively suppress information via a convex sink-rate to output-norm relationship.