REVIEW 4 major objections 4 minor 18 references
A Breadth-First Catalog of Text Processing, Speech Processing and Multimodal Research in South Asian Languages
T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read A broad LLM-assisted survey maps 2022–2024 South Asian text, speech, and multimodal research and ranks low-resource languages by activity.
desk verdict A useful, honest breadth-first survey of South Asian language work, but the headline counts rest on an unvalidated classifier and one clear citation error, so treat the numbers as approximate and send it to review with a request for reproducibility. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the paper's multi-stage curation pipeline: Google Scholar queries generated from a base prompt with GPT-4o, metadata compiled through Publish or Perish; a hand-labeled sample of 20 titles used as in-context examples for GPT-4o relevance classification of all titles; irrelevant or pre-2022 papers excluded; the O1 model groups the relevant papers into themes, after BERTopic produced less coherent clusters; and for the 21-language spotlight, literal string matching is used instead of the classifier because the classifier's precision (0.33) and recall (0.7) were too low. This pipeline is what turns a large, noisy literature into the paper's tables, counts, and trend claims.
What would settle it
Manually label a random sample of the papers the pipeline classified as relevant and irrelevant (for example, 200 from the 369 'relevant' language-model papers and 200 from the excluded set), then compare the labels with the classifier's decisions; if precision or recall on that sample is far below what the survey's counts assume, the field-level counts and the spotlight orderings would need to be revised. A simpler check: rerun the same search queries for the 21 spotlight languages using only exact language-name strings and see whether the counts reproduce.
Extended reading notes
Core claim
The paper claims to provide a systematic, breadth-first catalog of recent South Asian language technology research, covering text-based language models, multimodal models, and speech processing separately, and then focusing on 21 low-resource languages that are often overshadowed in multilingual work. It identifies major thematic clusters—machine translation and cross-lingual transfer, hate speech and sentiment, bias and fairness, LLM adaptation, datasets and benchmarks, image captioning, multimodal machine translation and hate speech, code-mixed speech, automatic speech recognition, and low-resource speech—and reports paper counts for each field. The spotlight section asserts that Nepali (63 papers), Assamese (58), and Telugu (57) were the most studied low-resource languages, while Kannada shows the most advanced speech-related work, and that no papers were found for Dari, Rohingya, Kurukh, and Santali. The accompanying trends and challenges sections argue that the field is growing quickly but that publication venues are fragmented, benchmarks are under-collected in multilingual LLM evaluations, code-mixing comparisons across languages are missing, and LLM-based meta-analysis still needs better precision and coherence.
Load-bearing premise
The survey's counts and trends rest on the assumption that a GPT-4o relevance classifier trained on only 20 hand-labeled titles decides correctly which papers belong in the survey; the paper itself reports precision of 0.33 and recall of 0.7 for the spotlight classifier and gives no accuracy measures for the main field-level classifications.
Editorial extensions
If this is right
- If the counts are right, the field's center of gravity is text-based language modeling: 369 relevant papers, versus 84 multimodal and 52 speech-processing papers.
- Researchers looking for low-resource languages with enough literature to build on should start with Nepali, Assamese, and Telugu (more than 50 papers each), followed by Kannada (49), which the survey identifies as the most speech-advanced.
- Dari, Rohingya, Kurukh, and Santali appear to have no dedicated papers in this window, signaling near-empty research space.
- Aggregating the scattered South Asian benchmarks into a single hosted benchmark would help because, per the paper, many are left out of multilingual LLM evaluations.
- LLM-based survey methods need better precision and recall and more coherent topics before their counts can be treated as stable.
Reading between the lines
- My inference: because the spotlight counts rely on literal string matching and the paper found the classifier imprecise, the reported per-language counts are best read as lower bounds; work indexed under alternative names (the paper itself tried 'Rakhine' for Rohingya) could be missed.
- My inference: the sharp gap between text papers (369) and speech or multimodal papers (84 and 52) may partly reflect how easily each type of work is discovered by title-based queries, so the true gap could be smaller than the counts suggest.
- My inference: a direct replication using abstracts instead of titles, or using a larger hand-labeled sample, would test whether the relevance classifier's low precision changes the field-level trends; the paper's own limitation section invites this.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents a breadth-first survey of NLP research in South Asian languages published between January 2022 and October 2024, covering text processing, multimodal models, and speech processing. The methodology uses Google Scholar searches, a GPT-4o-based relevance classifier trained on 20 hand-labeled titles, and an O1-based topic grouping, followed by a low-resource language spotlight on 21 languages using literal string matching. The paper reports overall relevant-paper counts (369 text, 84 multimodal, 52 speech) in Table 1 and per-language counts in Section 7 (e.g., Nepali 63, Assamese 58, Telugu 57), and identifies research themes, trends, challenges, and future directions.
Significance. If the quantitative findings were reliable, this survey would be a useful entry point for NLP researchers working on South Asian languages, particularly the low-resource spotlight that covers many under-served languages. The paper is transparent about its methodology and includes a Limitations section that acknowledges low precision/recall and topic cohesion issues, which is commendable. Its breadth-first perspective complements depth-first reviews such as Sankalp et al. (2024). However, the central counts and trends rest on an LLM relevance classifier whose accuracy is not reported for the main corpus, and the only reported metrics (precision 0.33, recall 0.7, in Section 7) are explicitly described as significantly low by the authors themselves, leading them to abandon the classifier for the spotlight section. The paper also contains a clear mis-citation in Section 5.2. These issues undermine confidence in the paper's main quantitative claims as they currently stand, although the survey's structure and curated bibliography retain value if the counts can be validated or appropriately caveated.
major comments (4)
- [Section 2 / Table 1 / Section 7] The main counts in Table 1 (369/84/52 relevant papers) and the theme-level trends in Sections 3–5 are produced by a GPT-4o relevance classifier, but no precision/recall or any other accuracy metric is reported for that classifier on the main corpus. The only numerical evaluation in the paper is in Section 7, where the spotlight relevance classifier is reported to have precision 0.33 and recall 0.7, a level the authors themselves call 'significantly low' and which motivated them to switch to literal string matching. Because the same classification approach underlies Table 1, the central quantitative claims are not established. The authors should provide per-category validation on a held-out labeled set (or at least a random sample) for the text, multimodal, and speech classifiers, or re-frame the counts as unverified with appropriate caveats and error bounds.
- [Section 5.2 / References] The paper cites Spiesberger et al. (2023) as an example of 'speech-to-speech translation for English → {Hindi, Telugu, Gujarati, Marathi, Punjabi}' in Section 5.2. The cited reference, 'Abusive speech detection in indic languages using acoustic features' (INTERSPEECH 2023), is a paper about abusive speech detection, not speech-to-speech translation. This is a load-bearing error because it inflates the speech-to-speech translation theme and indicates that the relevance/topic classification pipeline can mis-assign papers to incorrect themes. The citation should be corrected, and the other papers listed in that theme should be manually re-verified.
- [Section 2] The survey methodology is not reproducible. The Google Scholar queries, the 20 hand-labeled titles, and the per-paper relevance predictions from GPT-4o are not released, so the counts in Table 1 and Section 7 cannot be independently checked. The authors should provide these artifacts as supplementary material or a public repository. At a minimum, they should report the exact queries used for each language and category and the labeled examples used for in-context learning.
- [Section 7] The language-specific counts in Section 7 (e.g., 'Nepali, Assamese, and Telugu were the most popular languages, with more than 50 papers each since 2022') are based on literal string matching rather than the LLM classifier, which is an improvement, but the string-matching step is not validated against an independent search or manual review. Given that some language names (e.g., 'Burmese', 'Sindhi') can appear in non-linguistic or unrelated contexts, a manual verification of a sample, or a comparison with an alternative search strategy, is needed to support the 'most popular languages' claim with any confidence.
minor comments (4)
- [References] The reference list contains a duplicate entry for Majhi and Saha (2024): entries '2024a' and '2024b' have identical titles ('An automatic speech recognition system in odia language using attention mechanism and data augmentation') and appear to be the same paper. Please merge them and cite consistently, as both Section 6 and Section 7.7 cite these entries.
- [References / Section 4.3] The reference 'Singh et al., 2024b' (MIMIC: Misogyny identification in multimodal internet content) is listed twice in the reference list, once as 2024a and once as 2024b, with identical titles. Additionally, Section 4.3 cites this same work as both 'multimodal misogyny identification' and 'multimodal hate speech detection'; if the paper is specifically about misogyny, the hate-speech attribution should be corrected or clarified.
- [Section 7.17] The heading 'Chattisgarhi, Dhivehi, Sairaki, Rajasthani, and Balochi' contains two spelling errors: 'Chattisgarhi' should be 'Chhattisgarhi' and 'Sairaki' should be 'Saraiki', matching the spellings used in the abstract and elsewhere in the paper.
- [Section 7.7] There is a typo in Section 7.7: 'We alsio found work' should read 'We also found work'.
Circularity Check
No circularity: the survey's claims are empirical aggregations of external literature, not derivations from its own assumptions.
full rationale
The paper is a literature survey. Its main quantitative outputs—the relevant-paper counts in Table 1 and the language-specific counts in Section 7—are summaries of papers found through Google Scholar searches and filtered by an LLM-based relevance classifier or, for Section 7, literal string matching. These are empirical claims about an external literature, not results derived from the paper's own definitions or fitted parameters. The classifier is trained on 20 hand-labeled titles, but the counts are not predictions of those labels; they are inferences about unseen papers, and the paper openly reports the spotlight classifier's low precision/recall (0.33 precision, 0.7 recall) and concedes in the Limitations that 'low precision/recall persist.' This is a validity and robustness concern, not circularity: no equation or construction forces the reported counts to equal the inputs. The paper's statements that 'Nepali, Assamese, and Telugu were the most popular languages' and the per-language counts in Section 7 are based on string matching, and while their accuracy is unvalidated, nothing in the paper defines those counts into existence. Self-citations are not load-bearing: the cited prior work by the author's organization or coauthors is not used to justify the survey's central classifications or counts. No step in the derivation chain reduces to its own input by construction, so the correct finding is no significant circularity.
Assumptions & free parameters
assumptions (3)
- domain assumption Google Scholar, queried with LLM-generated prompts, surfaces the relevant literature.
- ad hoc to paper GPT-4o, using 20 hand-labeled titles as in-context examples, classifies relevance accurately enough for counting.
- domain assumption O1-generated topic clusters are coherent enough to define the paper's themes.
Cite this review
Pith. "Pith review of A Breadth-First Catalog of Text Processing, Speech Processing and Multimodal Research in South Asian Languages." pith.science (2026). https://pith.science/paper/SATFN2H7
@misc{pith2026250100029,
author = {Pith},
title = {Pith review of: A Breadth-First Catalog of Text Processing, Speech Processing and Multimodal Research in South Asian Languages},
year = {2026},
howpublished = {\url{https://pith.science/paper/SATFN2H7}},
note = {Machine review of arXiv:2501.00029}
}
read the original abstract
We review the recent literature (January 2022- October 2024) in South Asian languages on text-based language processing, multimodal models, and speech processing, and provide a spotlight analysis focused on 21 low-resource South Asian languages, namely Saraiki, Assamese, Balochi, Bhojpuri, Bodo, Burmese, Chhattisgarhi, Dhivehi, Gujarati, Kannada, Kashmiri, Konkani, Khasi, Malayalam, Meitei, Nepali, Odia, Pashto, Rajasthani, Sindhi, and Telugu. We identify trends, challenges, and future research directions, using a step-wise approach that incorporates relevance classification and clustering based on large language models (LLMs). Our goal is to provide a breadth-first overview of the recent developments in South Asian language technologies to NLP researchers interested in working with South Asian languages.
Figures
Reference graph
Works this paper leans on
-
[2]
In 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 1–7
Dialect adaptation and data augmentation for low-resource asr: Taltech systems for the madasr 2023 challenge. In 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 1–7. MB Alvi, NA Mahoto, MSA Reshan, and M Unar. 2023. Count me too: Sentiment analysis of roman sindhi script. SAGE . . .. HM Anagha, K Sairam, and J Mahesh. 2023....
work page 2023
-
[4]
Efficient assamese word recognition for so- cietal empowerment: A comparative feature-based analysis. IEEE . . .. R Budhathoki and S Timilsina. 2023. Image captioning in nepali using cnn and transformer decoder. Journal of Engineering and Sciences. SS Burramsetty and NP Gonugunta. 2022. Event ex- traction from telugu-english code mixed social media text. ...
work page 2023
-
[5]
A Comprehensive Study of the Current State-of-the-Art in Nepali Automatic Speech Recognition Systems
Relevance of language-specific training on image caption synthesis for low resource assamese language. In 2023 International Conference on Asian Language Processing (IALP), pages 13–18. CoRover.ai. 2024. BharatGPT: Bridging the Multilin- gual Gap. https://www.corover.ai/. T Dalai, TK Mishra, and PK Sa. 2023. Part-of-speech tagging of odia language using s...
work page Pith review arXiv 2023
-
[8]
Pashto handwritten invariant character tra- jectory prediction using a customized deep learning technique. Sensors. Alvi Khan, Fida Kamal, Mohammad Abrar Chowdhury, Tasnim Ahmed, Md Tahmid Rahman Laskar, and Sabbir Ahmed. 2023. BanglaCHQ-summ: An ab- stractive summarization dataset for medical queries in Bangla conversational speech. In Proceedings of the...
work page 2023
-
[9]
Evaluating Telugu Proficiency in Large Language Models_ A Comparative Analysis of ChatGPT and Gemini
Evaluating the diversity, equity, and inclu- sion of NLP technology: A case study for Indian languages. In Findings of the Association for Compu- tational Linguistics: EACL 2023, pages 1763–1777, Dubrovnik, Croatia. Association for Computational Linguistics. Vanlalmuansangi Khenglawt, Sahinur Rahman Laskar, Riyanka Manna, Partha Pakray, and Ajoy Kumar Kha...
work page Pith review arXiv 2023
-
[10]
Multilingual Tourist Assistance using ChatGPT: Comparing Capabilities in Hindi, Telugu, and Kannada
Building a llama2-finetuned llm for odia lan- guage utilizing domain knowledge instruction set. . . . Conference on AI-ML . . .. S Kolar and R Kumar. 2023. Multilingual tourist assistance using chatgpt: Comparing capabilities in hindi, telugu, and kannada. arXiv preprint arXiv:2307.15376. N Koppula, J Pradeep Kumar, and K Srinivas Rao. 2022. Word sense di...
work page Pith review arXiv 2023
-
[11]
Data and knowledge-driven approaches for multilingual training to improve the performance of speech recognition systems of indian languages. MK Majhi and SK Saha. 2024a. An automatic speech recognition system in odia language using attention mechanism and data augmentation. International Journal of Speech Technology. MK Majhi and SK Saha. 2024b. An automa...
work page Pith review arXiv 2024
-
[12]
AxomiyaBERTa: A Phonologically-aware Transformer Model for Assamese
Development of a dataset and a deep learn- ing baseline named entity recognizer for three low resource languages: Bhojpuri, maithili, and magahi. ACM Transactions on . . .. Rajesh Kumar Mundotiya, Swasti Mishra, and Anil Ku- mar Singh. 2022. Hierarchical self attention based sequential labelling model for bhojpuri, maithili and magahi languages. Journal o...
work page Pith review arXiv 2023
Show all 18 references
-
[13]
Data Engineering and
Grapheme to phoneme conversion for malay- alam speech using encoder-decoder architecture. . . . Data Engineering and . . .. S Pudasaini, S Ghimire, P Ale, and A Shakya. 2024. Ap- plication of nepali large language models to improve sentiment analysis. Proceedings of the . . .....
2024
-
[15]
Procedia Com- puter
Phonogram-based automatic typo correction in malayalam social media comments. Procedia Com- puter . . .. F Rauf, R Irfan, L Mushtaq, and M Ashraf. 2022. Fake news detection in urdu using deep learning. VFAST Transactions on Software Engineering. P Ravva. 2023. Systems and reso...
2022
-
[16]
Acoustics
Visual speech recognition for kannada lan- guage using vgg16 convolutional neural network. Acoustics. F Rynjah, B Syiem, and LJ Singh. 2022. Investigating khasi speech recognition systems using a recurrent neural network-based language model. Int J Eng Trends Technol. Jayanta ...
2022 arXiv
-
[17]
FIRE (Working
Sinhala and gujarati hate speech detection. FIRE (Working . . .. Arghyadeep Sen, Shantipriya Parida, Ketan Kotwal, Subhadarshi Panda, Ondˇrej Bojar, and Satya Ranjan Dash. 2022. Bengali visual genome: A multimodal dataset for machine translation and image captioning. In Intell...
2022 arXiv
-
[18]
Intelligent systems and
Speech intelligibility quality in telugu speech patterns using a wavelet-based hybrid threshold trans- form method. Intelligent systems and . . .. Tushar Verma, Atul Shree, and Ashutosh Modi. 2023. Asr for low resource and multilingual noisy code- mixed speech. Interspeech. V ...
2023
-
[2020]
ICT . . .. Gowtham Ramesh, Sumanth Doddapaneni, Aravinth Bheemaraj, Mayank Jobanputra, Raghavan AK, Ajitesh Sharma, Sujit Sahoo, Harshita Diddee, Ma- halakshmi J, Divyanshu Kakwani, Navneet Kumar, Aswin Pradeep, Srihari Nagaraj, Kumar Deepak, Vivek Raghavan, Anoop Kunchukuttan...
2022
-
[2021]
Rest of World
ViTA: Visual-linguistic translation by aligning object tags. In Proceedings of the 8th Workshop on Asian Translation (WAT2021) , pages 166–173, Online. Association for Computational Linguistics. Smita Gupta and Atreyo Banerjee. 2024. Ai, justice, and the ecosystem approach – n...
2024 arXiv
-
[2022]
Code-mixed Kannada . . .. PP Bansod. 2023. Hate speech detection in hindi. San Jose State University. F Barakzai, S Bhatti, and S Saddar. 2022. Sentiment analysis of sindhi news articles using deep learning. 2022 IEEE 17th International . . .. H Baruah, SR Singh, and P Sarmah....
2023
-
[2023]
In 2023 14th International Conference on Computing Communica- tion and Networking Technologies (ICCCNT), pages 1–6
Nlp based model to convert english speech to gujarati text for deaf and dumb people. In 2023 14th International Conference on Computing Communica- tion and Networking Technologies (ICCCNT), pages 1–6. A M Abirami, Wei Qi Leong, Hamsawardhini Rengara- jan, D Anitha, R Suganya, ...
2023
-
[2024]
International Journal on Cyber- netics & Informatics (IJCI), 13(2)
Direct punjabi to english speech translation using discrete units. International Journal on Cyber- netics & Informatics (IJCI), 13(2). R Kevat and S Degadwala. 2024. Developing gujarati article summarization utilizing improved page-rank system. International Journal of Scienti...
2024
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.