A low-rank Gaussian mixture model shows that training task diversity measured by non-overlapping subspace columns improves ICL generalization and shortens learning plateaus for linear attention, with empirical extension to nonlinear cases.
arXiv preprint arXiv:2305.16704 , year=
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
stat.ML 1years
2026 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
The Effect of Training Task Diversity on In-Context Learning through the Lens of Low-Dimensional Subspaces
A low-rank Gaussian mixture model shows that training task diversity measured by non-overlapping subspace columns improves ICL generalization and shortens learning plateaus for linear attention, with empirical extension to nonlinear cases.