REVIEW 3 major objections 4 minor 63 references
Unsupervised Stemming based Language Model for Telugu Broadcast News Transcription
T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Splitting Telugu words into stems and suffixes before language modeling reduces ASR word errors.
desk verdict The reported 0.76%/0.94% WER gains come from a test-set-leaked evaluation, so the paper's central claim does not hold up as stated. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is a split-token vocabulary: before language-model training, each inflected Telugu word is represented as a root or stem plus its suffix or prefix, and the n-gram model assigns probabilities to those pieces as separate tokens. Supervised splitting follows explicit Telugu grammar rules, such as sixteen nominal case-marker inflections and twenty-one suffixes that attach to verbs ending in the "uu" sound. Unsupervised splitting, adapted from an existing Hindi stemmer, builds a bipartite graph of all possible prefix and suffix splits in the corpus, keeps only prefixes and suffixes that occur with enough distinct partners, and iteratively prunes low-frequency vertices until a stable stem-suffix inventory remains. The smoothing method completes the mechanism by giving nonzero probability to unseen stem-suffix combinations; Witten-Bell is best for unsplit text and Kneser-Ney is best after splitting.
What would settle it
Decode the same 5-hour-30-minute test set under all three vocabulary conditions and score word error twice: once on raw decoder tokens and once after stem-suffix recombination, with test transcripts excluded from language-model training. If the split vocabulary does not lower the raw token error rate, the reported gains come from the unmeasured recombination step rather than from stemming itself.
Extended reading notes
Core claim
The paper finds that for Telugu broadcast-news ASR, training the n-gram language model on stem and suffix tokens instead of whole inflected words reduces out-of-vocabulary words and improves recognition accuracy. In experiments with a subspace Gaussian mixture model acoustic model, word error rate falls from 18.69% with no stemming to 17.93% after grammar-based supervised splitting and to 16.97% after unsupervised statistical splitting. Among five smoothing techniques, Witten-Bell works best on unsplit text, while Kneser-Ney works best on pre-processed, split text. The authors attribute the improvements to the lower out-of-vocabulary rate that split vocabularies produce, and they also report further gains when unsplit text is added alongside the unsupervised-stemmed corpus.
Load-bearing premise
The entire reported gain rests on the assumption that the decoder's separate stem and suffix tokens can be recombined into the intended surface word, a step the paper describes only in passing in Section I and never measures.
Editorial extensions
If this is right
- Telugu broadcast-news ASR can be improved without new speech data by reusing an existing text corpus and a statistical stemmer.
- Splitting the vocabulary shrinks the language model's unique-word count, reducing the computational burden caused by Telugu's many morphological variants.
- The unsupervised stemmer transfers from Hindi to Telugu, suggesting that the approach may generalize to other suffix-heavy Indic languages.
- Kneser-Ney smoothing should be preferred once the training text is stem-split, while Witten-Bell remains the better choice for unsplit text.
- The per-stage accuracy gains compound: supervised stemming and then unsupervised stemming together reduce word error by roughly 1.7 percentage points in the paper's main comparison.
Reading between the lines
- A natural extension, not tested in the paper, is to vary the stem-frequency and suffix-frequency thresholds of the unsupervised stemmer; the reported 0.94-point gain is likely threshold-dependent and might be larger on other corpora.
- The paper's row showing 16.42% WER for "without splitting + unsupervised stemming" suggests that a hybrid vocabulary—unsplit words plus discovered stem-suffix units—may be the most practical configuration, though the paper does not analyze this combination in depth.
- If the unmeasured recombination step can be made explicit and robust, the same split-vocabulary recipe should transfer to Kannada, Tamil, and Malayalam, which share Telugu's agglutinative suffix patterns.
- A direct way to test the mechanism would be to hold the acoustic model fixed and compare perplexity on held-out Telugu text for split versus unsplit vocabularies, isolating whether the gains come from language-model coverage rather than decoder behavior.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper addresses out-of-vocabulary (OOV) words in Telugu broadcast-news ASR by building language models from stemmed text. The authors collect 65 hours of Telugu broadcast audio, transcribe it, build HMM and SGMM acoustic models with Kaldi, and evaluate Witten-Bell, Kneser-Ney, and other smoothing techniques. They then apply a supervised stemming method using Telugu grammar rules and an unsupervised stemming algorithm adapted from a Hindi stemmer, and report that the unsupervised method improves ASR word accuracy by 0.94% over the supervised method and by 0.76% over the unstemmed baseline. The paper also reports perplexity and OOV statistics under several training conditions and claims that Kneser-Ney smoothing performs best on pre-processed data.
Significance. If the reported results were obtained from a valid held-out evaluation, the paper would be a useful practical contribution: it provides a new Telugu broadcast-news corpus, a phoneme set, and a systematic comparison of smoothing techniques for Telugu, and it demonstrates that morphological segmentation can reduce OOVs for a morphologically rich language. The use of standard tools (Kaldi, SRILM, cmuclmtk) and the availability of detailed corpus statistics are strengths. However, the central empirical claim is undermined by the evaluation protocol described below, so the significance currently lies more in the assembled resources and baseline comparisons than in the headline accuracy improvement.
major comments (3)
- [Section V and Table 10 vs. Table 8] The headline improvements of 0.76% and 0.94% in Section V are derived from the 'with test data' condition, not from a held-out evaluation. The WERs in Table 10 (18.69, 17.93, 16.97) exactly match the SGMM/Kneser-Ney 'with test data' columns in Table 8. In this protocol the LM training text includes the transcripts of the test set, so OOVs are artificially zero and test n-grams are directly scored; such a comparison cannot measure how stemming helps recognize unseen words. Under the valid 'without test data' condition in Table 8, the SGMM/Kneser-Ney WERs are 53.21, 49.98, and 43.21, giving improvements of 3.23 and 6.77 percentage points. The abstract and conclusions must be rewritten to report the held-out numbers, and the 'with test data' condition should be presented only as an oracle or removed.
- [Section I and WER computation] The manuscript never specifies how recognized root-word and suffix/prefix tokens are recombined into surface words before WER is computed. Section I says only that the recognized text 'is combined to form a meaningful new word.' If WER is scored on split tokens, the baseline and stemmed systems are compared on different word units, making the WER difference meaningless; if recombination is performed, its errors and the recombination algorithm must be described. Without this information, the measured WER improvement cannot be interpreted.
- [Section IV.B, unsupervised stemmer thresholds] The unsupervised stemmer is described as requiring a 'stem frequency threshold' and a 'suffix frequency threshold,' but these values are never reported. The segmentation produced by the algorithm depends directly on these thresholds, so the experimental result is not reproducible without them. If the thresholds were chosen by inspecting the test set, the comparison would be circular; even if not, the sensitivity of the downstream WER to these parameters is unknown. The authors should report the threshold values and, ideally, a small sensitivity analysis.
minor comments (4)
- [Section III vs. Table 8] The test-set duration is inconsistent: Tables 2, 3, and 7 state '2-HOUR 30 MIN' of test data, while Section IV and Table 8 refer to '5 hours 30 minutes' of test speech. Please clarify the actual test set size.
- [Section V] The baseline ASR accuracy is reported as 81.01% in Section V, but Table 10 gives a WER of 18.69% for the unstemmed condition, which implies an accuracy of 81.31%. These numbers should be reconciled.
- [Throughout] Several cross-references appear as 'Error! Reference source not found.' (e.g., before Tables 2 and 7), and Figure 1 is cited but not shown. These placeholders should be fixed before publication.
- [Table 8 and Section IV] There are typos such as 'knser-ney' and 'Kneser -ney' in Table 8 and 'un supervised' in the Section IV heading; a careful proofread is needed.
Circularity Check
No significant circularity: the headline WER gains are measured quantities, not fitted outputs; the test-set-inclusion issue is an evaluation-protocol flaw, not a constructional circularity.
full rationale
The paper's claimed improvements (0.76% and 0.94%) are empirical ASR word-error rates from Table 10, not quantities derived from the same data by definition. The supervised split uses grammatical/morphological rules, and the unsupervised stemmer is explicitly adapted from prior Hindi work (references [3] and [36]); neither is defined in terms of the headline result. No equation in the paper identifies a predicted value with an input, and no parameter is fitted to the reported WER. The most serious concern is that Table 10's values (18.69, 17.93, 16.97) coincide with Table 8's 'with test data' SGMM/Kneser-Ney columns, meaning the LM training corpus included the test transcripts; also, the stem+suffix recombination step is never specified. These are methodological and reproducibility risks, not circularity: the WER numbers are still external measurements rather than quantities forced by construction, and the possible leakage is not a self-citation or renamed fit. Because the central claim rests on measurements rather than on a circular derivation, the circularity score is 0.
Assumptions & free parameters
free parameters (2)
- stem frequency threshold =
unknown
- suffix frequency threshold =
unknown
assumptions (4)
- domain assumption The unsupervised Hindi stemmer from [36] transfers to Telugu with 'few changes'.
- domain assumption Splitting Telugu words at morphological boundaries preserves enough phonetic information for the ASR decoder, and the split tokens are valid language model units.
- domain assumption The 65-hour broadcast news corpus and the 1,60,271-sentence text corpus are representative of the test conditions.
- domain assumption Witten-Bell and Kneser-Ney smoothing are appropriate back-off models for Telugu agglutinative morphology.
Cite this review
Pith. "Pith review of Unsupervised Stemming based Language Model for Telugu Broadcast News Transcription." pith.science (2026). https://pith.science/paper/FBQVE5IP
@misc{pith2026190803734,
author = {Pith},
title = {Pith review of: Unsupervised Stemming based Language Model for Telugu Broadcast News Transcription},
year = {2026},
howpublished = {\url{https://pith.science/paper/FBQVE5IP}},
note = {Machine review of arXiv:1908.03734}
}
read the original abstract
In Indian Languages , native speakers are able to understand new words formed by either combining or modifying root words with tense and / or gender. Due to data insufficiency, Automatic Speech Recognition system (ASR) may not accommodate all the words in the language model irrespective of the size of the text corpus. It also becomes computationally challenging if the volume of the data increases exponentially due to morphological changes to the root word. In this paper a new unsupervised method is proposed for a Indian language: Telugu, based on the unsupervised method for Hindi, to generate the Out of Vocabulary (OOV) words in the language model. By using techniques like smoothing and interpolation of pre-processed data with supervised and unsupervised stemming, different issues in language model for Indian language: Telugu has been addressed. We observe that the smoothing techniques Witten-Bell and Kneser-Ney perform well when compared to other techniques on pre-processed data from supervised learning. The ASRs accuracy is improved by 0.76% and 0.94% with supervised and unsupervised stemming respectively.
Reference graph
Works this paper leans on
-
[36]
https://www.iitm.ac.in/donlab/tts/downloads/cls/cls_v2.1.6.pdf
“https://www.iitm.ac.in/donlab/tts/downloads/cls/cls_v2.1.6.pdf.”
-
[1]
చద వ చ డ (Caduvu cunnāḍu) చద వ (caduvu) చ డ (Cunnāḍu)
-
[2]
చద వ చ న (Caduvu cunnadi చద వ (caduvu) చ న (cunnadi)
-
[3]
చద వ చ వ (caduvu cunnāvu) చద వ (caduvu) చ వ (cunnāvu)
-
[4]
చద వ చ న (caduvu cunnānu) చద వ (caduvu) చ న (Cunnānu)
-
[5]
చద వ చ ర (caduvu cunnāru) చద వ (caduvu) చ ర (cunnāru)
-
[6]
చద వ చ న (caduvu cunnavi) చద వ (caduvu) చ న (cunnavi)
-
[7]
చద వ చ మ (caduvu cunnāmu) చద వ (caduvu) చ మ (Cunnāmu)
Show all 63 references
-
[8]
చద వ క డ (caduvu kunnāḍu) చద వ (caduvu) క డ (kunnāḍu)
-
[9]
చద వ క న (caduvu kunnadi) చద వ (caduvu) క న (kunnadi)
-
[10]
చద వ క వ (caduvu kunnāvu) చద వ (caduvu) క వ (kunnāvu)
-
[11]
చద వ క న (caduvu kunnānu) చద వ (caduvu) క న (kunnānu)
-
[12]
చద వ క ర (caduvu kunnāru) చద వ (caduvu) క ర (kunnāru)
-
[13]
చద వ క న (caduvu kunnavi) చద వ (caduvu) క న (kunnavi)
-
[14]
చద వ క మ (caduvu kunnāmu) చద వ (caduvu) క మ (kunnāmu)
-
[15]
చద వ డ (caduvu tunnāḍu) చద వ (caduvu) డ (tunnāḍu)
-
[16]
చద వ న (caduvu tunnadi) చద వ (caduvu) న (tunnadi)
-
[17]
చద వ వ (caduvu tunnāvu) చద వ (caduvu) వ (tunnāvu)
-
[18]
చద వ న (caduvu tunnānu) చద వ (caduvu) న (tunnānu)
-
[19]
చద వ ర (Caduvu tunnāru) చద వ (caduvu) ర (tunnāru) 9
-
[20]
చద వ న (caduvu tunnavi) చద వ (caduvu) న (tunnavi)
-
[21]
చద వ మ (caduvu tunnāmu) చద వ (caduvu) మ (tunnāmu)
-
[22]
చద వ ట(caduvuṭa) చద వ (caduvu) ట (ṭa)
-
[23]
చద వ న (caduvutānu) చద వ (caduvu) న (tānu)
-
[24]
చద వ వ (Caduvutāvu) చద వ (caduvu) వ (tāvu)
-
[25]
చద వ మ (caduvumu) చద వ (caduvu) మ (mu )
-
[26]
చద వ మ (caduvutāmu) చద వ (caduvu) మ (tāmu)
-
[27]
sandhi” or “samaasamu
చద వ ర (caduvutāru) చద వ (caduvu) ర (tāru) Let us consider the inflected words chan'dan'gaa , mukhyan'gaa , akhilapakshhan'gaa, kein'draman'trigaa , kein'dran'gaa. These four words have different base -words with a fixed suffix. Similarly, some inflected words will have same b...
1902
-
[28]
DNN-HMM Acoustic Modeling for Large Vocabulary Telugu Speech Recognition,
V. V. R. Vegesna, K. Gurugubelli, H. K. Vydana, B. Pulugandla, M. Shrivastava, and A. K. Vuppala, “DNN-HMM Acoustic Modeling for Large Vocabulary Telugu Speech Recognition,” in Mining Intelligence and Knowledge Exploration. MIKE 2017. , Lecture No., P. R. Ghosh A., Pal R., Ed....
2017
-
[29]
Hierarchical Hybrid Language models for Open Vocabulary Continuous Speech Recognition using WFST,
M. A. Basha Shaik, D. Rybach, S. Hahn, R. Schlüter, and H. Ney, “Hierarchical Hybrid Language models for Open Vocabulary Continuous Speech Recognition using WFST,” Proc. Work. Stat. Percept. Audit. (SAPA - SCALE), pp. 46–51, 2012
2012
-
[30]
Improved Hindi broadcast ASR by adapting the language model and pronunciation model using a priori syntactic and morphophonemic knowledge,
P. Jyothi and M. Hase gawa-Johnson, “Improved Hindi broadcast ASR by adapting the language model and pronunciation model using a priori syntactic and morphophonemic knowledge,” in Proceedings of the Annual Conference of the International Speech Communication Association, INTER...
2015
-
[31]
Applications and explanations of Zipf’s law,
D. M. W. Powers, “Applications and explanations of Zipf’s law,” in Proceedings of the joint conferences on new methods in language processing and computational natural language learning, 1998, pp. 151–160
1998
-
[32]
Evaluation metrics for language models,
S. Chen, D. Beeferman, and R. Rosenfeld, “Evaluation metrics for language models,” Proc. DARPA Broadcast News Transcr. Underst. Work., pp. 275– 280, 1998
1998
-
[33]
https://www.youtube.com/
“https://www.youtube.com/.”
-
[34]
https://ffmpeg.org/
“https://ffmpeg.org/.”
-
[35]
https://baraha.com/v10/index.php
“https://baraha.com/v10/index.php.”
-
[37]
Development of Indian Language Speech Databases for Large Vocabulary Speech Recognition Systems,
G. Anumanchipalli et al. , “Development of Indian Language Speech Databases for Large Vocabulary Speech Recognition Systems,” Proc. SPECOM, 2005
2005
-
[38]
Tools for the Development of a Hindi Speech Synthesis System.,
A. . R. Kalika Bali, P artha Pratim Talukdar, N Sridhar Krishna, “Tools for the Development of a Hindi Speech Synthesis System.,” in 5th ISCA Speech Synthesis Workshop, 2004, pp. 109–114
2004
-
[39]
A tutorial on hidden Markov models and selected applications in speech recognition,
L. R. Rabiner, “A tutorial on hidden Markov models and selected applications in speech recognition,” Proc. IEEE, vol. 77, no. 2, pp. 257–286, 1989. 14
1989
-
[40]
The subspace Gaussian mixture model - A structured model for speech recognition,
D. Povey et al. , “The subspace Gaussian mixture model - A structured model for speech recognition,” Comput. Speech Lang., vol. 25, no. 2, pp. 404–439, Apr. 2011
2011
-
[41]
Context -dependent pre -trained deep neural networks for large-vocabulary speech recognition,
G. E. Dahl, D. Yu, L. Deng, and A. Acero, “Context -dependent pre -trained deep neural networks for large-vocabulary speech recognition,” IEEE Trans. Audio, Speech Lang. Process. , vol. 20, no. 1, pp. 30–42, 2012
2012
-
[42]
The Kaldi Speech Recognition Toolkit,
D. Povey et al., “The Kaldi Speech Recognition Toolkit,” Proc. ASRU, 2011
2011
-
[43]
Statistical Language Modeling Using Leaving -One-Out,
H. Ney, S. Martin, and F. Wessel, “Statistical Language Modeling Using Leaving -One-Out,” in Corpus-Based Methods in Language and Speech Processing. , Text, Spee., B. G. Young S., Ed. Springer, 1997, pp. 174–207
1997
-
[44]
An empirical study of smoothing techniques for language modeling (Tech. Rep. No. TR-10-98)
S. Chen and J. Goodman, “An empirical study of smoothing techniques for language modeling (Tech. Rep. No. TR-10-98).”
-
[45]
Evaluation metrics for language models,
S. F. Chen, D. Beeferman, and R. Rosenfeld, “Evaluation metrics for language models,” Proc. DARPA Broadcast News Transcr. Underst. Work., pp. 275– 280, 1998
1998
-
[46]
Interpolated estimation of Markov source parameters from sparse data.,
R. L. Jelinek, F. & Mercer, “Interpolated estimation of Markov source parameters from sparse data.,” in Proceedings of the Workshop on Pattern Recognition in Practice, North -Holland, Amsterdam, The Netherlands, 1980, pp. 381–397
1980
-
[47]
Estimation of Probabilities from Sparse Data for the Language Model Component of a Speech Recognizer,
S. M. Katz, “Estimation of Probabilities from Sparse Data for the Language Model Component of a Speech Recognizer,” 1987
1987
-
[48]
I. H. W. Timothy C Bell, John G Cleary, Text compression . Prentice -Hall, Inc. Upper Saddle River, NJ, USA , 1990
1990
-
[49]
Improved backing -off for M -gram language modeling,
R. Kneser and H. Ney, “Improved backing -off for M -gram language modeling,” in 1995 International Conference on Acoustics, Speech, and Signal Processing , 2002, vol. 1, pp. 181 – 184
1995
-
[50]
Generalized linear interpolation of language model s,
Bo-June (Paul) Hsu, “Generalized linear interpolation of language model s,” in 2007 IEEE Workshop on Automatic Speech Recognition & Understanding (ASRU), 2007, pp. 136–140
2007
-
[51]
A comparison of the enhanced Good -Turing and deleted estimation methods for estimating probabilities of English bigrams,
K. W. Church and W. A. Gale, “A comparison of the enhanced Good -Turing and deleted estimation methods for estimating probabilities of English bigrams,” 1991
1991
-
[52]
On structuring probabilistic dependences in stochastic language modelling,
H. Ney, U. Essen, and R. Kneser, “On structuring probabilistic dependences in stochastic language modelling,” Comput. Speech Lang., vol. 8, no. 1, pp. 1–38, Jan. 1994
1994
-
[53]
The Zero-Frequency Problem: Estimating the Probabilities of Novel Events in Adaptive Text Compression,
I. H. Witten and T. C. Bell, “The Zero-Frequency Problem: Estimating the Probabilities of Novel Events in Adaptive Text Compression,” IEEE Trans. Inf. Theory , vol. 37, no. 4, pp. 1085 –1094, 1991
1991
-
[54]
https://cmusphinx.github.io/wiki/tutoriallm/#training-an-arpa-model-with-cmuclmtk
“https://cmusphinx.github.io/wiki/tutoriallm/#training-an-arpa-model-with-cmuclmtk.”
-
[55]
SRILM AN EXT ENSIBLE LANGUAGE MODELING TOOLKIT,
A. Stolcke, “SRILM AN EXT ENSIBLE LANGUAGE MODELING TOOLKIT,” ICSLP 2002 , pp. 901 – 904, 2002
2002
-
[56]
An estimate of an upper bound for the entropy of English,
P. F. Brown, V. J. Della Pietra, R. L. Mercer, S. A. Della Pietra, and J. C. Lai, “An estimate of an upper bound for the entropy of English,” Comput. Linguist., vol. 18, no. 1, pp. 31–40, 1992
1992
-
[57]
TelMore: Morphological Generator for Telugu Nouns and verbs,
M. Ganapathiraju and L. Levin, “TelMore: Morphological Generator for Telugu Nouns and verbs,” in Proceedings of the Second International Conference on Universal Digital Library, 15 Alexandria, Egypt November 17-19, 2006, 2006, pp. 1–7
2006
-
[58]
Hyderabad: Andhrapradesh Sahitya Acadamy, 1976
Dr.Divakarla venkatawdhani, telugu in thirty days . Hyderabad: Andhrapradesh Sahitya Acadamy, 1976
1976
-
[59]
B.Purushottam; R Srihari Shastri; D VenkataRama, Vyakarana padakosamu Sastra nighantuvu
-
[60]
Krishnamurti and J
B. Krishnamurti and J. P. L. Gwynn, A Grammar of Modern Tel ugu. Oxford: Oxford University Press, 1985
1985
-
[61]
Krishnamurti, Telugu Verbal Bases
B. Krishnamurti, Telugu Verbal Bases. Motilal Banarsidass Publishers Pvt. Limited, 2009
2009
-
[62]
TelStem:An Unsupervised Telugu Stemmer with Heuristic Improvements and Normalized Signatures,
A. P. Siva Kumar, P. Premchand, and A. Govardhan, “TelStem:An Unsupervised Telugu Stemmer with Heuristic Improvements and Normalized Signatures,” 2011
2011
-
[63]
An unsupervised Hindi stemmer with heuristic improvements,
A. K. Pandey and T. J. Siddiqui, “An unsupervised Hindi stemmer with heuristic improvements,” in Proceedings of the second workshop on Analytics for noisy unstructured text data - AND ’08, 2008, pp. 99–105
2008
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.