REVIEW 4 cited by
Simple, Scalable Adaptation for Neural Machine Translation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Fine-tuning pre-trained Neural Machine Translation (NMT) models is the dominant approach for adapting to new languages and domains. However, fine-tuning requires adapting and maintaining a separate model for each target task. We propose a simple yet efficient approach for adaptation in NMT. Our proposed approach consists of injecting tiny task specific adapter layers into a pre-trained model. These lightweight adapters, with just a small fraction of the original model size, adapt the model to multiple individual tasks simultaneously. We evaluate our approach on two tasks: (i) Domain Adaptation and (ii) Massively Multilingual NMT. Experiments on domain adaptation demonstrate that our proposed approach is on par with full fine-tuning on various domains, dataset sizes and model capacities. On a massively multilingual dataset of 103 languages, our adaptation approach bridges the gap between individual bilingual models and one massively multilingual model for most language pairs, paving the way towards universal machine translation.
Forward citations
Cited by 4 Pith papers
-
MOON: Multi-Objective OrthoNormalized Updates for Multitask Learning
MOON applies spectral-nuclear-norm geometry to multi-objective gradient manipulation and uses polar-factor updates, with O(T^-1/2) deterministic and O(T^-1/4) stochastic convergence to Pareto stationarity.
-
Mettle: Meta-Token Learning for Memory-Efficient Audio-Visual Adaptation
Mettle distills frozen transformer layer features into compact meta-tokens via parallel cross-attention and linear projection, cutting training memory dramatically while retaining competitive accuracy on three audio-v...
-
LKA: Large Kernel Adapter for Enhanced Medical Image Classification
A 7x7 depthwise convolution inside a parameter-efficient adapter improves fine-tuned medical image classification by a few percent over standard PEFT baselines.
-
PEFT A2Z: Parameter-Efficient Fine-Tuning Survey for Large Language and Vision Models
A survey that organizes PEFT methods into additive, selective, reparameterized, hybrid, and unified families, but with no new method or verified experiments.
Discussion (0). Continue with ORCID to comment.