Subword tokenization impairs phonological knowledge encoding in LMs, but an IPA-based fine-tuning method restores it with minimal impact on other capabilities.
LLM-Powered Grapheme-to- Phoneme Conversion: Benchmark and Case Study
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
fields
cs.CL 3representative citing papers
Hybrid OLaPh framework outperforms prior G2P baselines on WikiPron while enabling synthetic data for an LLM that generalizes well on out-of-vocabulary terms.
Larger and Japanese-specialized LLMs outperform conventional morphological analyzers on Japanese G2P conversion, with parse-mode prompting yielding the lowest character error rates below 0.52%.
citing papers explorer
-
How Tokenization Limits Phonological Knowledge Representation in Language Models and How to Improve Them
Subword tokenization impairs phonological knowledge encoding in LMs, but an IPA-based fine-tuning method restores it with minimal impact on other capabilities.
-
OLaPh: Optimal Language Phonemizer
Hybrid OLaPh framework outperforms prior G2P baselines on WikiPron while enabling synthetic data for an LLM that generalizes well on out-of-vocabulary terms.
-
Benchmarking Large Language Models for Grapheme-to-Phoneme Conversion: A Japanese Case Study
Larger and Japanese-specialized LLMs outperform conventional morphological analyzers on Japanese G2P conversion, with parse-mode prompting yielding the lowest character error rates below 0.52%.