K-MSHC reports that Gemma-9B's grammar, arithmetic-verification, and word-problem abilities rely on largely separate, task-specific attention-head circuits with partial overlap at low-importance heads.
A structural probe for finding syntax in word representations
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
$K$-MSHC: Unmasking Minimally Sufficient Head Circuits in Large Language Models with Experiments on Syntactic Classification Tasks
K-MSHC reports that Gemma-9B's grammar, arithmetic-verification, and word-problem abilities rely on largely separate, task-specific attention-head circuits with partial overlap at low-importance heads.