After training, a 110M-parameter power-law attention model's learned scoring operator becomes nearly input-invariant, so inference can cache it; the paper proves this collapse conditionally, measures it at 1e-6 and below, and machine-checks selected proofs in Lean 4.
CoulGAT: An Experiment on Interpretability of Graph Attention Networks
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
We present an attention mechanism inspired from definition of screened Coulomb potential. This attention mechanism was used to interpret the Graph Attention (GAT) model layers and training dataset by using a flexible and scalable framework (CoulGAT) developed for this purpose. Using CoulGAT, a forest of plain and resnet models were trained and characterized using this attention mechanism against CHAMPS dataset. The learnable variables of the attention mechanism are used to extract node-node and node-feature interactions to define an empirical standard model for the graph structure and hidden layer. This representation of graph and hidden layers can be used as a tool to compare different models, optimize hidden layers and extract a compact definition of graph structure of the dataset.
fields
cs.LG 1years
2026 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Power law graph attention: exact generalization of scaled dot-product attention, empirical collapse at inference
After training, a 110M-parameter power-law attention model's learned scoring operator becomes nearly input-invariant, so inference can cache it; the paper proves this collapse conditionally, measures it at 1e-6 and below, and machine-checks selected proofs in Lean 4.