Using a log-linear model and Jacobian analysis, the paper claims isotropy in LLM hidden embeddings stabilizes the softmax partition function and improves time-series forecasting, with illustrative experiments on five models and 22 datasets.
After performing K-means clustering, each observation p (i.e., one of the J vector representations in V) is assigned to one of C clusters
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
citation-role summary
method 1
citation-polarity summary
fields
cs.CL 1years
2025 1verdicts
REJECT 1roles
method 1polarities
use method 1representative citing papers
citing papers explorer
-
When can isotropy help adapt LLMs' next word prediction to numerical domains?
Using a log-linear model and Jacobian analysis, the paper claims isotropy in LLM hidden embeddings stabilizes the softmax partition function and improves time-series forecasting, with illustrative experiments on five models and 22 datasets.