Next-token prediction estimates a marginal text law that is useful only under ergodicity assumptions and when observed prefixes carry low residual mutual information about omitted latent circumstances.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2026 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
When Is Next-Token Prediction Useful? Marginalization, Ergodicity, Mixture Identifiability, Local Sufficiency, RAG, Tools, and Programming
Next-token prediction estimates a marginal text law that is useful only under ergodicity assumptions and when observed prefixes carry low residual mutual information about omitted latent circumstances.