Cross-tokenizer likelihood scoring via BPE recursion allows exact subset-vocab distillation with O(1) evaluations per token and general-case approximations, improving memory efficiency and accuracy in LM distillation.
exclude the last token, obtained either from Algorithm 2 or from previous sampling call (to be shown)
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
Cross-Tokenizer Likelihood Scoring Algorithms for Language Model Distillation
Cross-tokenizer likelihood scoring via BPE recursion allows exact subset-vocab distillation with O(1) evaluations per token and general-case approximations, improving memory efficiency and accuracy in LM distillation.