Using a development set for early stopping can change reported accuracy for low-resource NLP models, with per-language differences up to 18 percentage points compared to tuning training length on other languages.
Diverse Few-Shot Text Classification with Multiple Metrics
1 Pith paper cite this work. Polarity classification is still indexing.
abstract
We study few-shot learning in natural language domains. Compared to many existing works that apply either metric-based or optimization-based meta-learning to image domain with low inter-task variance, we consider a more realistic setting, where tasks are diverse. However, it imposes tremendous difficulties to existing state-of-the-art metric-based algorithms since a single metric is insufficient to capture complex task variations in natural language domain. To alleviate the problem, we propose an adaptive metric learning approach that automatically determines the best weighted combination from a set of metrics obtained from meta-training tasks for a newly seen few-shot task. Extensive quantitative evaluations on real-world sentiment analysis and dialog intent classification datasets demonstrate that the proposed method performs favorably against state-of-the-art few shot learning algorithms in terms of predictive accuracy. We make our code and data available for further study.
fields
cs.CL 1years
2019 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Towards Realistic Practices In Low-Resource Natural Language Processing: The Development Set
Using a development set for early stopping can change reported accuracy for low-resource NLP models, with per-language differences up to 18 percentage points compared to tuning training length on other languages.