Self-attention weights in Transformers are non-identifiable for long sequences, and the paper offers effective attention and Hidden Token Attribution as diagnostic tools.
Brown, Stephen A
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2019 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
On Identifiability in Transformers
Self-attention weights in Transformers are non-identifiable for long sequences, and the paper offers effective attention and Hidden Token Attribution as diagnostic tools.