A one-step belief propagation refinement with a repulsive Potts prior increases attention entropy and improves downstream accuracy of small Transformers, with GTD as a diagnostic for multi-hop attention.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Mitigating Attention Localization in Small Scale: Self-Attention Refinement via One-step Belief Propagation
A one-step belief propagation refinement with a repulsive Potts prior increases attention entropy and improves downstream accuracy of small Transformers, with GTD as a diagnostic for multi-hop attention.