Over 40 checkpoints of OLMo-7B, attention heads and FFNs shift from general-purpose to specialized roles for factual recall, with location-based facts learned earlier and more stably than name-based facts.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Time Course MechInterp: Analyzing the Evolution of Components and Knowledge in Large Language Models
Over 40 checkpoints of OLMo-7B, attention heads and FFNs shift from general-purpose to specialized roles for factual recall, with location-based facts learned earlier and more stably than name-based facts.