Attention mechanisms are classified by their relational inductive bias: self-attention assumes a complete graph, masked attention a total order, strided attention p-previous connections, encoder-decoder a bipartite graph, and graph attention data-dependent neighborhoods.
IEEE Signal Processing Magazine 34(4), 18–42 (2017)
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Relational inductive biases on attention mechanisms
Attention mechanisms are classified by their relational inductive bias: self-attention assumes a complete graph, masked attention a total order, strided attention p-previous connections, encoder-decoder a bipartite graph, and graph attention data-dependent neighborhoods.