TSVD framework maintains low-rank orthonormal weights during LLM pretraining via truncated SVD, adaptive spectral rank selection, and caching to reduce compute while matching baseline performance.
Cuttlefish: Low-rank model training without all the tuning.arXiv preprint arXiv:2305.02538, 2023
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2026 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
Efficient Pre-Training of LLMs through Truncated SVD Layers
TSVD framework maintains low-rank orthonormal weights during LLM pretraining via truncated SVD, adaptive spectral rank selection, and caching to reduce compute while matching baseline performance.