REVIEW 2 cited by
Universal Language Model Fine-Tuning with Subword Tokenization for Polish
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Universal Language Model for Fine-tuning [arXiv:1801.06146] (ULMFiT) is one of the first NLP methods for efficient inductive transfer learning. Unsupervised pretraining results in improvements on many NLP tasks for English. In this paper, we describe a new method that uses subword tokenization to adapt ULMFiT to languages with high inflection. Our approach results in a new state-of-the-art for the Polish language, taking first place in Task 3 of PolEval'18. After further training, our final model outperformed the second best model by 35%. We have open-sourced our pretrained models and code.
Forward citations
Cited by 2 Pith papers
-
Meta-aware Learning in text-to-SQL Large Language Model
Combining schema, chain-of-thought, metadata knowledge, and tokenized prompt structures during fine-tuning improves text-to-SQL execution accuracy on private business databases compared to schema-only fine-tuning.
-
Applying a Pre-trained Language Model to Spanish Twitter Humor Prediction
A Spanish Twitter language model trained from scratch with label smoothing placed 3rd and 2nd in the HAHA 2019 humor classification and regression tasks.
Discussion (0). Continue with ORCID to comment.