REVIEW 2 cited by
Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Fine-tuning pre-trained generative language models to down-stream language generation tasks has shown promising results. However, this comes with the cost of having a single, large model for each task, which is not ideal in low-memory/power scenarios (e.g., mobile). In this paper, we propose an effective way to fine-tune multiple down-stream generation tasks simultaneously using a single, large pre-trained model. The experiments on five diverse language generation tasks show that by just using an additional 2-3% parameters for each task, our model can maintain or even improve the performance of fine-tuning the whole model.
Forward citations
Cited by 2 Pith papers
-
GeoRA: Geometry-Aware Low-Rank Adaptation for RLVR
GeoRA uses SVD to extract principal directions from the RL update subspace for low-rank adapter initialization and freezes residuals to preserve pre-trained structure during RLVR training.
-
UORA: Uniform Orthogonal Reinitialization Adaptation in Parameter-Efficient Fine-Tuning of Large Models
UORA is a LoRA/VeRA-style PEFT method that selectively reinitializes low-magnitude rows and columns of frozen random matrices, reaching LoRA-comparable performance with far fewer trainable parameters.
Discussion (0). Continue with ORCID to comment.