REVIEW 2 cited by
Data Distillation for Text Classification
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Deep learning techniques have achieved great success in many fields, while at the same time deep learning models are getting more complex and expensive to compute. It severely hinders the wide applications of these models. In order to alleviate this problem, model distillation emerges as an effective means to compress a large model into a smaller one without a significant drop in accuracy. In this paper, we study a related but orthogonal issue, data distillation, which aims to distill the knowledge from a large training dataset down to a smaller and synthetic one. It has the potential to address the large and growing neural network training problem based on the small dataset. We develop a novel data distillation method for text classification. We evaluate our method on eight benchmark datasets. The results that the distilled data with the size of 0.1% of the original text data achieves approximately 90% performance of the original is rather impressive.
Forward citations
Cited by 2 Pith papers
-
Semi-Supervised Text-Attributed Graph Distillation
STAD distills large text-attributed graphs into tiny human-readable graphs that match or beat full-graph semi-supervised node classification.
-
Simple yet Effective Graph Distillation via Clustering
ClustGDD distills large graphs by clustering node embeddings and refining synthetic attributes, achieving state-of-the-art node classification accuracy at orders of magnitude lower time cost.
Discussion (0). Sign in to comment.