REVIEW 3 cited by
Does the Order of Training Samples Matter? Improving Neural Data-to-Text Generation with Curriculum Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Recent advancements in data-to-text generation largely take on the form of neural end-to-end systems. Efforts have been dedicated to improving text generation systems by changing the order of training samples in a process known as curriculum learning. Past research on sequence-to-sequence learning showed that curriculum learning helps to improve both the performance and convergence speed. In this work, we delve into the same idea surrounding the training samples consisting of structured data and text pairs, where at each update, the curriculum framework selects training samples based on the model's competence. Specifically, we experiment with various difficulty metrics and put forward a soft edit distance metric for ranking training samples. Our benchmarks show faster convergence speed where training time is reduced by 38.7% and performance is boosted by 4.84 BLEU.
Forward citations
Cited by 3 Pith papers
-
Learning from Impairment: Leveraging Insights from Clinical Linguistics in Language Modelling Research
Aphasia treatment protocols like CATE offer complexity hierarchies that the paper proposes to reuse for language model evaluation and curriculum learning, without providing empirical evidence.
-
Data Efficacy for Language Model Training
Ordering training data by a gradient-based score, using a folding scheme that interleaves multiple curriculum passes, improves small-scale LM accuracy by roughly 1.5 to 2 points on average benchmarks.
-
GIT-CXR: End-to-End Transformer for Chest X-Ray Report Generation
Length-based curriculum learning improves an end-to-end GIT transformer for chest X-ray report generation, yielding high METEOR and clinical F1 scores; the state-of-the-art claim is however weakened by inconsistent ev...
Discussion (0). Continue with ORCID to comment.