TLoRA jointly optimizes LoRA initialization via task-data SVD and sensitivity-driven rank allocation, delivering stronger results than standard LoRA across NLU, reasoning, math, code, and chat tasks while using fewer trainable parameters.
arXiv preprint arXiv:2410.07170 , year=
4 Pith papers cite this work. Polarity classification is still indexing.
4
Pith papers citing it
citation-role summary
method 1
citation-polarity summary
years
2026 4verdicts
UNVERDICTED 4roles
method 1polarities
use method 1representative citing papers
Alpha in LoRA outperforms learning-rate scaling, follows a square-root law with rank, and enables a minimalist LoRA-alpha method that improves performance across tasks.
CERSA derives low-rank fine-tuning subspaces from SVD principal components that retain 90-95% spectral energy, delivering higher performance than LoRA and other PEFT baselines at substantially lower memory cost across vision, generation, and language tasks.
GiVA uses gradients to initialize vector adapters so they match LoRA performance at eight times lower rank while keeping extreme parameter efficiency.
citing papers explorer
-
TLoRA: Task-aware Low Rank Adaptation of Large Language Models
TLoRA jointly optimizes LoRA initialization via task-data SVD and sensitivity-driven rank allocation, delivering stronger results than standard LoRA across NLU, reasoning, math, code, and chat tasks while using fewer trainable parameters.
-
The Hidden Power of Scaling Factor in LoRA Optimization
Alpha in LoRA outperforms learning-rate scaling, follows a square-root law with rank, and enables a minimalist LoRA-alpha method that improves performance across tasks.
-
CERSA: Cumulative Energy-Retaining Subspace Adaptation for Memory-Efficient Fine-Tuning
CERSA derives low-rank fine-tuning subspaces from SVD principal components that retain 90-95% spectral energy, delivering higher performance than LoRA and other PEFT baselines at substantially lower memory cost across vision, generation, and language tasks.
-
GiVA: Gradient-Informed Bases for Vector-Based Adaptation
GiVA uses gradients to initialize vector adapters so they match LoRA performance at eight times lower rank while keeping extreme parameter efficiency.