The paper defines a weight-matrix similarity index based on maximum absolute cosine values and Gumbel fitting, then uses it to show that neighboring transformer layers in open LLMs have similar weights and form clusters.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
DOCS: Quantifying Weight Similarity for Deeper Insights into Large Language Models
The paper defines a weight-matrix similarity index based on maximum absolute cosine values and Gumbel fitting, then uses it to show that neighboring transformer layers in open LLMs have similar weights and form clusters.