Reconstruction of Word Embeddings from Sub-Word Parameters

Karl Stratos

REVIEW

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1707.06957 v1 pith:BFHWU4WG submitted 2017-07-21 cs.CL

Reconstruction of Word Embeddings from Sub-Word Parameters

Karl Stratos This is my paper

classification cs.CL

keywords wordembeddingscostmodelparameterspre-trainedsub-wordanalogy

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

0 comments

read the original abstract

Pre-trained word embeddings improve the performance of a neural model at the cost of increasing the model size. We propose to benefit from this resource without paying the cost by operating strictly at the sub-lexical level. Our approach is quite simple: before task-specific training, we first optimize sub-word parameters to reconstruct pre-trained word embeddings using various distance measures. We report interesting results on a variety of tasks: word similarity, word analogy, and part-of-speech tagging.

Reconstruction of Word Embeddings from Sub-Word Parameters

discussion (0)