REVIEW 5 major objections 5 minor 9 references
Topic Modeling in Marathi
T0 review · 5 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash
Pith's one-line read BERTopic with Indic BERT embeddings beats LDA for Marathi topics
desk verdict Useful benchmark direction, but the numbers are unauditable as written; send to review only with a demand for full configuration details. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is BERTopic's pipeline: sentence-transformers turn Marathi sentences into dense embeddings; UMAP reduces dimensionality; HDBSCAN clusters semantically similar documents; and a class-based TF-IDF (c-TF-IDF) scores words within each cluster to form topics. The comparison metric is topic coherence, which measures how related the top words of a topic are. The BERT models supply the semantic representations, and their language-specific versus multilingual training is the variable that distinguishes model performance.
What would settle it
Rerun both pipelines on the same three Marathi datasets with a single specified coherence metric (e.g., c_v) and with LDA's topic count and BERTopic's hyperparameters grid-searched; if LDA's best coherence meets or exceeds BERTopic's, or the gap falls below the numbers reported here, the central claim fails.
Extended reading notes
Core claim
The paper claims that BERTopic, a neural topic model that clusters sentence embeddings and derives topics with class-based TF-IDF, consistently outperforms LDA on topic coherence for Marathi. On three in-house datasets of long (LDC), medium (LPC), and short (SHC) documents, the best BERTopic configurations reach coherence between 0.71 and 0.82 per dataset, with monolingual MahaBERT-V2 performing best across all three, while LDA's coherence ranges from 0.34 to 0.55. The paper attributes the gap to pre-trained language models capturing semantic relations that LDA's bag-of-words statistics miss.
Load-bearing premise
The load-bearing assumption is that the reported coherence scores for BERTopic and LDA are directly comparable; the paper never states the coherence metric, topic count, hyperparameters, or dataset sizes, so the gap could shift or vanish under matched settings.
Editorial extensions
If this is right
- For Marathi topic modeling, the paper advises choosing BERTopic over LDA when coherence is the goal, with monolingual Marathi BERT models as the first choice.
- On short documents (SHC), sentence-BERT models outperform BERT models, while on long documents (LDC) BERT models have the edge, suggesting the optimal embedding model depends on document length.
- The success of pre-trained embeddings on Marathi indicates that low-resource languages can inherit effective topic models from transfer learning rather than requiring large topic-specific corpora.
- MahaBERT-V2 outperforming the multilingual MURIL on all three datasets suggests that a language-matched monolingual model is worth training even when a strong multilingual model is available.
Reading between the lines
- The reported superiority would be on firmer ground if the paper specified which coherence metric (c_v, c_npmi, or u_mass) was used, since absolute scores differ across metrics; the 0.63–0.82 versus 0.34–0.55 gap may not reproduce under a different metric.
- With three datasets differing only in document length, the authors leave untested how the BERTopic advantage scales with dataset size, vocabulary, or domain; a natural next experiment is varying topic count and measuring topic diversity, which the paper says it used but never reports.
- Because IndicSBERT and MURIL are cross-lingual, the same BERTopic configuration could be applied with minimal modification to Hindi, Tamil, or other Indic languages, making the recipe a testable template beyond Marathi.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper compares BERTopic combined with several Marathi/Indic BERT and Sentence-BERT models (MahaSBERT-STS, MahaSBERT, MahaBERT-V2, IndicSBERT, IndicSBERT-STS, MuRIL) against Gensim LDA for topic modeling on three Marathi datasets referred to as LPC, LDC, and SHC. The headline claim is that BERTopic yields substantially higher topic coherence (0.63–0.82) than LDA (0.34–0.55) on all three datasets. The paper also asserts that MahaBERT-V2 performs consistently better than the other BERT models. The evaluation uses topic coherence as the main metric, with a brief discussion of topic diversity, but no topic diversity results are reported.
Significance. If the comparison were fully specified and reproducible, a demonstration that BERTopic with Indic-language embeddings consistently outperforms LDA on a low-resource language like Marathi would be a useful empirical contribution to the growing literature on neural topic modeling for Indic languages. The direction of the reported results is plausible and consistent with findings in other languages, and the use of publicly available L3Cube models is a strength. However, in its current form the evidence is not auditable: the coherence metric, model hyperparameters, topic counts, and dataset characteristics are missing, so the central quantitative claim cannot be independently verified or interpreted as a general property of the methods rather than of a particular unspecified configuration.
major comments (5)
- [Section IV, Tables 1 and 2] The coherence metric is never named. Coherence scores are not comparable across metrics: c_v, c_npmi, and u_mass have different scales and different sensitivities to topic count and corpus statistics. If BERTopic scores were computed with one metric and LDA with another, the numerical gap would be meaningless. Please state the exact metric, the implementation (e.g., Gensim CoherenceModel), the top-word count used, and the reference corpus for computing co-occurrence statistics.
- [Section IV, Tables 1 and 2] The LDA topic count k is not reported, while BERTopic's topic count is determined automatically by HDBSCAN. Coherence is highly sensitive to the number of topics, so if LDA was run at a topic count that was not tuned per dataset, the apparent BERTopic advantage could be a configuration artifact. Report k for each dataset, the LDA hyperparameters (passes, chunksize, alpha, eta, random seed), and whether k was selected by a coherence search or fixed a priori.
- [Section IV, Table 1 and text] The sentence "In BERTopic, monolingual bert model MahaBERT-V2 performs consistently better than other bert models in the 3 datasets" is contradicted by the table the authors themselves provide. On the LPC dataset, IndicSBERT and IndicSBERT-STS score 0.72 while MahaBERT-V2 scores 0.71; on the SHC dataset, IndicSBERT-STS scores 0.81, the highest of all models. Please correct either the claim or the analysis, and discuss the actual ordering of model performance across datasets.
- [Section IV, Tables 1 and 2] All reported scores are single numbers with no variance, no number of runs, and no random seeds. The differences among BERTopic variants are small (e.g., 0.70 vs. 0.71 on LPC), and BERTopic's UMAP and HDBSCAN steps are stochastic; a single run does not establish the stability of the ranking. Report results across multiple seeds (or set and document deterministic parameters) and, if possible, provide confidence intervals or standard deviations.
- [Section II and Section IV] The three datasets LPC, LDC, and SHC are never described: their sizes, sources, document length distributions, and preprocessing details (beyond removal of stop words, URLs, and non-Devanagari words) are absent. This matters both for assessing the authors' short/medium/long document distinctions and for judging whether the results generalize. Please add a data description table or clear citations and preprocessing details.
minor comments (5)
- [Abstract and Section III] The abstract and methodology mention topic diversity as an evaluation metric, but no topic diversity results appear in Section IV. Either report those numbers or remove the claim.
- [Section II] "Maarten Gr." should be "Maarten Grootendorst".
- [References] Several references are incomplete or inconsistently formatted (e.g., [5] has a stray "1."; [8] lacks venue and page information; [9] contains a fragment of a URL). Please normalize all entries.
- [Section IV] The sentence "[4] MahaBERT-V2 is outperforming MuRIL in our evaluations, demonstrating superior performance across the tasks" appears to reference a different study and is not supported by the results in Table 1, where MuRIL scores are comparable to or higher than MahaBERT-V2 on the LDC dataset. Please clarify the source of this claim.
- [Section III] The sentence "When topics are shown to users, each topic t is typically shown as a list of the M = 5,..., 20 most likely words" is vague but the paper never states the actual M used for coherence computation. If M is not reported in the main text, this should be part of the evaluation configuration.
Circularity Check
No significant circularity: the comparison is empirical, with external LDA as baseline and no quantity defined as a fitted version of its own output.
full rationale
This paper makes an empirical performance comparison between BERTopic and LDA on Marathi datasets, reporting topic coherence scores in Tables 1 and 2. No result is derived from a definition that already contains the conclusion: BERTopic's coherence is computed from its topic-word lists, LDA's coherence is computed from its own topics, and the claim that BERTopic outperforms LDA is an observational statement about measured scores, not an equation that reduces to itself. The authors do cite their own L3Cube models and MahaCorpus (references [4], [12], [13]), but these citations introduce artifact models used as inputs to the experiment; they are not invoked as proof of the headline claim, and the comparison baseline (LDA) is external to the authors' prior work. The paper does not fit any parameter to a subset of the reported coherence values and then rename the fit as a prediction, nor does it import a uniqueness theorem from the authors' prior publications. The main weaknesses are reproducibility-related: the coherence metric, LDA topic count, BERTopic hyperparameters, dataset sizes, and variance are not reported, so the comparison cannot be audited. That is a correctness and reporting concern, not a circularity concern, because nothing in the argument is true by construction or by self-citation alone.
Assumptions & free parameters
free parameters (3)
- LDA topic count (k) =
Not reported
- BERTopic hyperparameters (min_topic_size, UMAP and HDBSCAN settings) =
Not reported
- Coherence metric and top-word count =
Not reported
assumptions (3)
- domain assumption Topic coherence is an adequate proxy for topic model quality
- domain assumption The LPC, LDC, and SHC datasets are representative and large enough for topic modeling
- domain assumption The Gensim LDA and BERTopic implementations behave as documented by default
Cite this review
Pith. "Pith review of Topic Modeling in Marathi." pith.science (2026). https://pith.science/paper/STMMUB2C
@misc{pith2026250202100,
author = {Pith},
title = {Pith review of: Topic Modeling in Marathi},
year = {2026},
howpublished = {\url{https://pith.science/paper/STMMUB2C}},
note = {Machine review of arXiv:2502.02100}
}
read the original abstract
While topic modeling in English has become a prevalent and well-explored area, venturing into topic modeling for Indic languages remains relatively rare. The limited availability of resources, diverse linguistic structures, and unique challenges posed by Indic languages contribute to the scarcity of research and applications in this domain. Despite the growing interest in natural language processing and machine learning, there exists a noticeable gap in the comprehensive exploration of topic modeling methodologies tailored specifically for languages such as Hindi, Marathi, Tamil, and others. In this paper, we examine several topic modeling approaches applied to the Marathi language. Specifically, we compare various BERT and non-BERT approaches, including multilingual and monolingual BERT models, using topic coherence and topic diversity as evaluation metrics. Our analysis provides insights into the performance of these approaches for Marathi language topic modeling. The key finding of the paper is that BERTopic, when combined with BERT models trained on Indic languages, outperforms LDA in terms of topic modeling performance.
Figures
Reference graph
Works this paper leans on
-
[1]
Empirical Study of Topic Modeling in Twitter
Liangjie Hong and Brian D. Daviso. “Empirical Study of Topic Modeling in Twitter”,SOMA '10: Proceedings of the First Workshop on Social Media Analytics, June 2010
work page 2010
-
[3]
Latent Dirichlet allocation (LDA) and topic modeling: models, applications, a survey
Hamed Jelodar, Yongli Wang, Chi Yuan, Xia Feng, Xiahui Jiang, Yanchao Li1, Liang Zhao. "Latent Dirichlet allocation (LDA) and topic modeling: models, applications, a survey" . Springer Link, Volume 78, pages 15169–15211, (2019)
work page 2019
-
[4]
Raviraj Joshi, “L3Cube-MahaCorpus and MahaBERT: Marathi Monolingual Corpus, Marathi BERT Language Models, and Resources”, https://arxiv.org/abs/2202.01159
-
[6]
Santosh Kumar Raya , Amir Ahmadb , and Ch. Aswani Kumar, Review and Implementation of Topic Modeling in Hindi, https://www.tandfonline.com/doi/epdf/10.1080/08839514.2019.1661576?needAccess=true
-
[7]
BERTopic: Neural topic modeling with a class-based TF-IDF procedure
Maarten Grootendorst, "BERTopic: Neural topic modeling with a class-based TF-IDF procedure.", https://arxiv.org/abs/2203.05794
- [8]
-
[9]
Jatinderkumar R. Saini and Prafulla B. Bafna, MaTop: An Evaluative Topic Model for Marathi, https://link.springer.com/chapter/10.1007/978-981-16-4538-9_14
-
[12]
Raviraj Joshi, “L3Cube-MahaSBERT and HindSBERT: Sentence BERT Models and Benchmarking BERT Sentence Representations for Hindi and Marathi”, https://arxiv.org/abs/2211.11187
Show all 9 references
-
[13]
L3Cube-IndicSBERT: A simple approach for learning cross-lingual sentence representations using multilingual BERT
Raviraj Joshi, “L3Cube-IndicSBERT: A simple approach for learning cross-lingual sentence representations using multilingual BERT”, https://arxiv.org/abs/2304.11434
Reviewed August 9, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.