OSoRA fine-tunes LLMs by updating only singular values and one output-dimension vector, using frozen singular vectors from an SVD of the pretrained weights.
In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , pages 2381–2391, Brussels, Belgium
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
REJECT 1representative citing papers
citing papers explorer
-
OSoRA: Output-Dimension and Singular-Value Initialized Low-Rank Adaptation
OSoRA fine-tunes LLMs by updating only singular values and one output-dimension vector, using frozen singular vectors from an SVD of the pretrained weights.