Protein language models complete repeats by combining induction heads that copy the aligned residue from the other repeat copy with neurons encoding amino-acid similarity; the approximate-repeat circuit contains and generalizes the exact-repeat circuit.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2026 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Induction Meets Biology: Mechanisms of Repeat Detection in Protein Language Models
Protein language models complete repeats by combining induction heads that copy the aligned residue from the other repeat copy with neurons encoding amino-acid similarity; the approximate-repeat circuit contains and generalizes the exact-repeat circuit.