REVIEW 4 cited by
Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Prompt tuning, in which a base pretrained model is adapted to each task via conditioning on learned prompt vectors, has emerged as a promising approach for efficiently adapting large language models to multiple downstream tasks. However, existing methods typically learn soft prompt vectors from scratch, and it has not been clear how to exploit the rich cross-task knowledge with prompt vectors in a multitask learning setting. We propose multitask prompt tuning (MPT), which first learns a single transferable prompt by distilling knowledge from multiple task-specific source prompts. We then learn multiplicative low rank updates to this shared prompt to efficiently adapt it to each downstream target task. Extensive experiments on 23 NLP datasets demonstrate that our proposed approach outperforms the state-of-the-art methods, including the full finetuning baseline in some cases, despite only tuning 0.035% as many task-specific parameters.
Forward citations
Cited by 4 Pith papers
-
Towards Anytime Retrieval: A Benchmark for Anytime Person Re-Identification
AT-USTC, a 403k-image RGB/IR dataset covering six time-based ReID scenarios, and Uni-AT, a multi-scenario model, are proposed, with Uni-AT achieving 55.8% any-time Rank-1 on the new benchmark.
-
Modeling Code: Is Text All You Need?
A GNN-encoded LLVM IR graph, prepended as soft prompts to a frozen code LLM, improves accuracy on device mapping, algorithm classification, vulnerability detection, and code translation tasks.
-
Stabilizing Black-Box Prompt Optimization with Textual Regularization and Signal Aggregation
TRAS adds success-based textual regularization and Monte Carlo signal aggregation to black-box prompt optimization, improving accuracy and reducing instruction loss when moving prompts across models.
-
SOS-LoRA: Static Orthogonal-Subspace Low-Rank Adaptation with Fixed Multi-Scale Scaling
A LoRA update split into several fixed, differently-scaled low-rank experts with orthogonal input directions improves fine-tuning accuracy at the same parameter count.
Discussion (0). Sign in to comment.