A low-rank Gaussian mixture model shows that training task diversity measured by non-overlapping subspace columns improves ICL generalization and shortens learning plateaus for linear attention, with empirical extension to nonlinear cases.
Patti and Jayson Lynch and Avi Shporer and Nakul Verma and Eugene Wu and Gilbert Strang , title =
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
stat.ML 1years
2026 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
The Effect of Training Task Diversity on In-Context Learning through the Lens of Low-Dimensional Subspaces
A low-rank Gaussian mixture model shows that training task diversity measured by non-overlapping subspace columns improves ICL generalization and shortens learning plateaus for linear attention, with empirical extension to nonlinear cases.