Averaging top-K attention heads selected on 10 labeled proteins recovers contacts in one forward pass and outperforms the multi-pass Categorical Jacobian on leakage-clean data for bidirectional protein LMs.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2026 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
Protein contacts are already in the attention: a single-forward-pass alternative to the Categorical Jacobian
Averaging top-K attention heads selected on 10 labeled proteins recovers contacts in one forward pass and outperforms the multi-pass Categorical Jacobian on leakage-clean data for bidirectional protein LMs.