BlockBPE parallelizes BPE tokenization on GPUs and reports speedups over CPU tokenizers at high batch sizes, but with a large accuracy drop on math benchmarks.
Run state of the art nlp workloads at scale with rapids, huggingface, and dask
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
REJECT 1representative citing papers
citing papers explorer
-
BlockBPE: Parallel BPE Tokenization
BlockBPE parallelizes BPE tokenization on GPUs and reports speedups over CPU tokenizers at high batch sizes, but with a large accuracy drop on math benchmarks.