Transformers can be viewed as unrolled inference in a probabilistic Laplacian Eigenmaps model, and replacing the attention matrix by attention minus identity improves validation performance.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Transformers as Unrolled Inference in Probabilistic Laplacian Eigenmaps: An Interpretation and Potential Improvements
Transformers can be viewed as unrolled inference in a probabilistic Laplacian Eigenmaps model, and replacing the attention matrix by attention minus identity improves validation performance.