Indic NER with SentencePiece tokenization beats BPE in zero-shot cross-lingual transfer, but the paper's evidence is limited by internal contradictions and undisclosed implementation details.
When every token counts: Optimal segmentation for low-resource language models,
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
REJECT 1representative citing papers
citing papers explorer
-
Tokenization Matters: Improving Zero-Shot NER for Indic Languages
Indic NER with SentencePiece tokenization beats BPE in zero-shot cross-lingual transfer, but the paper's evidence is limited by internal contradictions and undisclosed implementation details.