pith. sign in

arxiv: 1305.0328 · v2 · pith:RT65OWT2new · submitted 2013-05-02 · 📊 stat.ME · math.PR

General Type Token Distribution

classification 📊 stat.ME math.PR
keywords numbertypesasymptoticcorpusderivedistributiondistributionsestimator
0
0 comments X
read the original abstract

We consider the problem of estimating the number of types in a corpus using the number of types observed in a sample of tokens from that corpus. We derive exact and asymptotic distributions for the number of observed types, conditioned upon the number of tokens and the latent type distribution. We use the asymptotic distributions to derive an estimator of the latent number of types and we validate this estimator numerically.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.