Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Analyzing Emotions in Bangla Social Media Comments Using Machine Learning and LIME

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read On 22,698 Bangla social media comments, a decision tree boosted with AdaBoost reaches a macro F1-score of 0.7860 for six-emotion classification, the best among the classical models tested and within reach of a much larger pretrained…

desk verdict A cleanly written but flawed benchmark: the headline F1 comparison mixes averaging schemes, and the multi-label handling is unstated. read the letter →

arxiv 2506.10154 v1 pith:K6JY3NII submitted 2025-06-11 cs.CL cs.LG

Bidyarthi Paul , SM Musfiqur Rahman , Dipta Biswas , Md. Ziaul Hasan , Md. Zahid Hossain This is my paper · ORCID
classification cs.CLcs.LG
keywords emotiondetectionBanglatextclassificationlow-resourcelanguagesentimentanalysisTF-IDFAdaBoostLIMEEmoNoBadataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a classical, interpretable machine-learning pipeline can do serious emotion detection in Bangla social media text. Using 22,698 comments from the EmoNoBa dataset labeled with six emotions, the authors compare TF-IDF n-gram models (Linear SVM, KNN, Random Forest), a decision tree with and without AdaBoost, and a BiLSTM. Their best model, a decision tree boosted with AdaBoost, reaches a macro F1-score of 0.7860, well above the 0.4281 lexical-feature baseline and close to but below the 0.8273 reported for BanglaBERT-large. They also show that PCA consistently hurts performance and that LIME can expose which words drive the AdaBoost decisions. If these results hold, they suggest low-cost, transparent models remain a viable route for emotion analysis in low-resource languages.

What carries the argument

The central mechanism is a TF-IDF plus n-gram feature representation feeding a decision-tree classifier that is boosted with AdaBoost: TF-IDF encodes word-importance weights for unigrams, bigrams, and trigrams; the decision tree supplies an interpretable rule structure; and AdaBoost reweights hard examples so successive weak trees focus on misclassified comments. LIME, a local surrogate explainer, then attributes each prediction to the words that most influenced it. The paper's negative result is equally part of the machinery: PCA applied to the TF-IDF matrix compresses the feature space but consistently reduces F1, indicating that for this task the discarded dimensions carried signal.

What would settle it

Re-run the Decision Tree plus AdaBoost setup on EmoNoBa with an explicit multi-label handling rule, or on the subset of single-emotion comments only, and compare the macro F1 to 0.7860; a large drop would show the headline score depends on the unresolved multi-label transformation.

Watch

Extended reading notes

Core claim

The paper's central claim is that on the EmoNoBa corpus of 22,698 noisy Bangla comments, a decision tree classifier combined with AdaBoost on TF-IDF unigram features gives the best overall emotion-classification result among the models tested, with a macro F1-score of 0.7860. This beats the other classical models (Linear SVM 0.63, KNN 0.57, Random Forest 0.57), the unboosted decision tree (0.7799), and the deep BiLSTM (0.3869), and it substantially outperforms the 0.4281 lexical-feature benchmark from the dataset paper. The same experiments show that adding PCA lowers F1 scores across all three n-gram classifiers, and that unigram features outperform bigrams and trigrams. The authors present LIME explanations of the boosted decision tree as evidence that the model's predictions are interpretable, and they position these results as a step toward benchmark emotion detection for Bangla.

Load-bearing premise

The weakest load-bearing assumption is that every comment has exactly one of the six emotion labels, even though the paper reports that 15.3% of comments express more than one emotion and never describes how those multi-label comments were assigned to a single class before training and scoring.

Editorial extensions

If this is right

  • For Bangla emotion detection, classical feature-based models remain a reasonable default: the boosted decision tree beats the lexical baseline and all other non-transformer models in this study.
  • Unigram TF-IDF appears sufficient; bigram and trigram features add complexity without improving F1, so future low-resource pipelines can start with unigrams only.
  • PCA should not be assumed to help in this setting; the paper's results show it repeatedly lowers F1, so dimensionality reduction needs its own validation on each text-classification task.
  • Because the best model is a decision tree, its per-prediction reasoning can be shown with LIME word-level explanations, which is harder with large pretrained transformers.
  • A simple interpretable model can come within roughly four F1 points of the reported BanglaBERT-large score, suggesting a cost-performance tradeoff for low-resource deployment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct extension would be to re-run the same pipeline on a multi-label formulation of EmoNoBa, using all emotion labels present in the 15.3% multi-emotion comments; the paper's single-label F1 of 0.7860 would likely shift once those overlaps are counted explicitly.
  • The comparison suggests a testable hypothesis: a compact distilled transformer or a feature-based model with confidence thresholds could close the 0.0413 gap to BanglaBERT while keeping LIME-style explanations, a direction not explored here.
  • Since LIME explanations are shown only as examples, a quantitative next step is to have Bangla-speaking annotators judge whether the highlighted words are actually emotion-bearing; that would test the interpretability claim directly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents an empirical comparison of classical machine-learning models (Linear SVM, KNN, Random Forest, Decision Tree with and without AdaBoost) and a BiLSTM for fine-grained emotion classification on the EmoNoBa Bangla social-media dataset (22,698 comments, six emotion labels). The authors also apply TF-IDF with n-grams, PCA, and LIME explanations for the best model. The headline claim is that Decision Tree with AdaBoost achieves the highest overall F1-score of 0.7860, outperforming the other models and the lexical-feature baseline of Islam et al. (0.4281) while remaining below BanglaBERT-based results (0.7330 and 0.8273). The manuscript is written as a standard benchmark study, with a clear methodology diagram, multiple evaluation metrics, and a short limitations section.

Significance. If the reported results are robust, the paper would provide a useful classical machine-learning baseline for Bangla emotion detection, demonstrating that an explainable boosting ensemble can outperform lexical-feature baselines and approach transformer performance on EmoNoBa. Strengths include the use of a public dataset, comparison with prior published results, inclusion of LIME explanations, and explicit acknowledgment of data imbalance and scope limitations. However, the contribution is currently undercut by unresolved reporting issues: the F1-score averaging scheme is inconsistent across tables, the treatment of multi-label instances is undocumented, no variance estimates or cross-validation are reported, and hyperparameters are largely absent. These issues directly affect the validity of the central 'highest overall F1' claim, so the paper's significance cannot be fully assessed in its present form.

major comments (4)
  1. [§4.3, Table 3, Table 5] The headline comparison mixes different F1 averaging schemes. Section 4.3 states that macro, micro, and weighted averages were all computed, and Table 3 explicitly labels the Decision Tree results as 'Macro average F1-score' (0.7860 and 0.7799). Tables 2, 4, and 5 report only 'F1-score' without specifying the averaging. Given the strong class imbalance visible in Figure 2 (Joy and Sadness much more frequent than Fear and Surprise), macro and micro/weighted F1 can differ substantially. The central claim in Section 5.1 that 'The Decision Tree, when combined with AdaBoost, recorded the highest overall F1-score of 0.7860' is therefore unsupported unless every compared value in Tables 2 and 5 is also macro-averaged. The authors must state the averaging scheme for every reported F1 value and, ideally, report all three variants.
  2. [§3.3, §4.2] The dataset objective in Section 3.2 states that the aim is to identify all emotions expressed in a text, and Section 3.3 reports that 15.3% of entries express more than one emotion. Nevertheless, the classifiers described in Section 4.2 are standard single-label models, and the manuscript nowhere explains how multi-label instances are resolved before training and evaluation. Possible resolutions—dropping such instances, assigning a single dominant label, or creating multiple single-label training examples—would change the training distribution, test labels, and every F1 value in Tables 2–5. This is a load-bearing omission: without a stated and justified multi-label handling rule, the reported scores are not well-defined and cannot be compared with the EmoNoBa baselines in Table 6.
  3. [§3.4, §5.1] No error bars, cross-validation, or significance tests are reported. The evaluation rests on a single stratified split (80% train, 15% test, 5% validation) with no random seed, and the claimed improvement from AdaBoost is 0.7799 to 0.7860—a difference of 0.0061 that is likely within the noise of a single split on a dataset of this size. The statement in Section 5.1 that boosting 'highlight[s] the effectiveness' of the approach needs support from repeated splits or k-fold evaluation with standard deviations, and ideally a paired significance test. Without this, the superiority of the AdaBoost model over the plain Decision Tree is not established.
  4. [Table 6, Section 5.1] The comparison with prior EmoNoBa results is not yet interpretable. Table 6 lists F1 scores from Islam et al. (0.4281), Chakma et al. (0.7330), and Kabir et al. (0.8273), but the manuscript does not state whether those studies report macro, micro, or weighted F1, nor does it describe differences in data preprocessing (especially multi-label handling) and evaluation splits. Merely placing numbers side by side in Table 6 does not establish comparability. In addition, the citation markers in the text are inconsistent with the reference list: the text refers to 'Kabir et al. (2024) [18]' and 'Chakma et al. (2023) [17]', but reference [17] is Sadhu et al. and reference [18] is Chakma et al. The authors should verify all cited values and protocols.
minor comments (5)
  1. [§5] There is a typo in the first paragraph: 'Additionaly' should be 'Additionally'.
  2. [Figures 11 and 12] The two confusion matrices are introduced as 'a sample' for the best-performing model, but no analysis is provided; the captions do not state whether the matrices are normalized, and the reader cannot tell which classes or splits they refer to.
  3. [§4.2, References] Reference [14] is malformed: it begins 'A comparative study Shah, Kanish, et al.' and appears to merge two different works. The authors should correct all reference entries to their complete bibliographic details.
  4. [§4.4 and Tables 2–5] The experimental setup lists the hardware and Python versions but omits all model hyperparameters: SVM regularization C, number of neighbors K, Random Forest tree count, PCA component count, AdaBoost learning rate and estimator count, and the BiLSTM architecture (hidden size, layers, dropout, embedding dimension, training epochs). These details are essential for reproducibility.
  5. [§5, LIME explanation] The LIME discussion (Figure 10) describes predictions such as 'Not Fear' and 'Not Sadness' without explaining how LIME scores were aggregated or whether the features identified are consistent across instances. A quantitative evaluation of the explanations (e.g., faithfulness or agreement with human intuition) would strengthen the XAI contribution.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical benchmark whose claims are evaluated against an externally sourced dataset and external baseline results.

full rationale

The paper's derivation chain is self-contained in the sense required by the circularity analysis. It trains standard classifiers (Linear SVM, KNN, Random Forest, Decision Tree, AdaBoost, BiLSTM) on the externally published EmoNoBa dataset, using conventional preprocessing, TF-IDF with n-grams, PCA, and Word2Vec. No parameter is fitted to the target result, and no 'prediction' is defined in terms of a previously fitted value. The headline claim in Section 5.1 that Decision Tree with AdaBoost 'recorded the highest overall F1-score of 0.7860' is an empirical comparison against other models evaluated on the same test split and against external prior work (Islam et al. 2022, Chakma et al. 2023, Kabir et al. 2024). Those external comparisons are not self-citations by the present authors, and the cited dataset is a distinct prior publication. Potential concerns such as inconsistent averaging schemes across tables or the unresolved treatment of the 15.3% multi-label instances are correctness and reproducibility issues, not circular reasoning. There is no reduction, by construction or by self-citation chain, of the reported results to the paper's own inputs.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests entirely on empirical measurements with no formal derivation. The main unstated choices are the handling of multi-label comments, the hyperparameters of each model, and the averaging method behind reported F1 scores. No new theoretical entities are introduced.

free parameters (6)
  • PCA component count = not reported
    PCA is applied to TF-IDF features; the number of retained components is not given, though it directly sets information loss and performance.
  • n-gram range and TF-IDF settings = not reported
    The paper tests unigram, bigram, and trigram but does not give min_df, max_features, or other TF-IDF parameters.
  • SVM regularization C = not reported
    Linear SVM is used without reporting C, which controls overfitting in high-dimensional TF-IDF spaces.
  • K in KNN = not reported
    No value for the number of neighbors is provided.
  • AdaBoost learning rate and estimator count = not reported
    AdaBoost is applied to a decision tree, but tree depth, learning rate, and number of estimators are not specified.
  • BiLSTM architecture hyperparameters = not reported
    Embedding size, hidden dimension, epochs, batch size, and optimizer are not reported; the low F1 of 0.3869 suggests these choices matter.
assumptions (4)
  • domain assumption EmoNoBa labels are gold-standard ground truth for the six emotions.
    The paper uses the dataset as provided without any label review or noise analysis, yet label quality directly determines all F1 scores.
  • ad hoc to paper Multi-label instances can be treated as single-label without affecting the evaluation.
    Section 3.3 reports 15.3% of comments have more than one emotion, but the classification setup and evaluation are single-label; no resolution strategy is stated.
  • domain assumption TF-IDF with n-grams provides features sufficient to distinguish the six emotions.
    The whole machine learning pipeline assumes lexical signals survive punctuation removal and TF-IDF encoding; the paper's own results show BiLSTM struggles, so feature sufficiency is not established.
  • domain assumption Macro F1 is an appropriate summary for this imbalanced dataset.
    The class distribution is skewed (Fear and Surprise are rare), and macro F1 treats all classes equally; this is a reasonable choice but is not justified or compared with weighted F1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Analyzing Emotions in Bangla Social Media Comments Using Machine Learning and LIME." pith.science (2026). https://pith.science/paper/K6JY3NII

@misc{pith2026250610154,
  author       = {Pith},
  title        = {Pith review of: Analyzing Emotions in Bangla Social Media Comments Using Machine Learning and LIME},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K6JY3NII}},
  note         = {Machine review of arXiv:2506.10154}
}
read the original abstract

Research on understanding emotions in written language continues to expand, especially for understudied languages with distinctive regional expressions and cultural features, such as Bangla. This study examines emotion analysis using 22,698 social media comments from the EmoNoBa dataset. For language analysis, we employ machine learning models: Linear SVM, KNN, and Random Forest with n-gram data from a TF-IDF vectorizer. We additionally investigated how PCA affects the reduction of dimensionality. Moreover, we utilized a BiLSTM model and AdaBoost to improve decision trees. To make our machine learning models easier to understand, we used LIME to explain the predictions of the AdaBoost classifier, which uses decision trees. With the goal of advancing sentiment analysis in languages with limited resources, our work examines various techniques to find efficient techniques for emotion identification in Bangla.

Figures

Figures reproduced from arXiv: 2506.10154 by the authors.

Figure 1
Figure 1. Sample Dataset 3.2 Dataset Objective: For six basic emotion categories, the aim is to figure out all the feelings expressed in a piece of text. 3.3 Dataset Statistics and Analysis: Our dataset consists of 22,698 entries. On average, each entry is about 1.36 ± 0.82 sentences long. The typical sentence length is roughly 11.70 ± 10.70 words. The majority of the data, 77.28% of the instances are sourced from YouTube, wi… view at source ↗
Figure 2
Figure 2. Number of samples in each class [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Average Length of the Documents [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: represents a word cloud which is a visual representation of the most frequently occurring words within the dataset used for our research. Each word’s size is proportional to its frequency: the larger the word, the more often it appears in the dataset. By visually repre…
Figure 6
Figure 6. Figure 6: Methodology 4.1 Dataset Preprocessing For preprocessing our dataset, we focused on cleaning the text to ensure consis￾tency and accuracy for analysis. This involved removing all punctuation marks and emojis, as these elements could interfere with our emotion analysis a…
Figure 7
Figure 7. Figure 7: Flow Chart of Machine Learning Approach (PCA and n-gram) [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Flow Chart of Machine Learning Approach (AdaBoost and n-gram) [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Flow Chart of Deep Learning Approach 4.3 Evaluation To evaluate the performance of our classification model, we deployed a compre￾hensive set of metrics, including accuracy, precision, recall, and the F1 score. These metrics were calculated in three distinct forms to p…
Figure 10
Figure 10. Figure 10: Sample of LIME Explanations for all emotions [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]
Figure 11
Figure 11. Figure 11: Confusion Matrix of Love [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. BIDWESH: A Bangla Regional Based Hate Speech Detection Dataset

    cs.CL 2025-07 conditional novelty 5.0 of 10

    BIDWESH is a new 9,183-instance Bangla dialectal hate speech corpus covering Barishal, Noakhali, and Chittagong.

Reference graph

Works this paper leans on

18 extracted references · 15 canonical work pages · cited by 1 Pith paper

  1. [18]

    LowResource at BLP-2023 Task 2: Leveraging BanglaBert for Low Resource Sentiment Analysis of Bangla Language

    Chakma, Aunabil, and Masum Hasan. "LowResource at BLP-2023 Task 2: Lever- aging BanglaBert for Low Resource Sentiment Analysis of Bangla Language." arXiv preprint arXiv:2311.12735 (2023)

    work page Pith review arXiv 2023

  2. [17]

    An Empirical Study of Gendered Stereotypes in Emotional Attributes for Bangla in Multilingual Large Language Models

    Sadhu, Jayanta, Maneesha Rani Saha, and Rifat Shahriyar. "An Empirical Study of Gendered Stereotypes in Emotional Attributes for Bangla in Multilingual Large Language Models." arXiv preprint arXiv:2407.06432 (2024)

    arXiv 2024

  3. [1]

    A hybrid model for automatic emotion recognition in suicide notes

    Yang, Hui, et al. "A hybrid model for automatic emotion recognition in suicide notes." Biomedical informatics insights 5 (2012): BII-S8948

    work page 2012

  4. [2]

    Automatic detection of insulting sentences in conversation

    Allouch, Merav, et al. "Automatic detection of insulting sentences in conversation." 2018 IEEE International Conference on the Science of Electrical Engineering in Israel (ICSEE). IEEE, 2018

    work page 2018

  5. [3]

    Canceremo: A dataset for fine-grained emo- tion detection

    Sosea, Tiberiu, and Cornelia Caragea. "Canceremo: A dataset for fine-grained emo- tion detection." Proceedings of the 2020 Conference on Empirical Methods in Nat- ural Language Processing (EMNLP). 2020

    work page 2020

  6. [4]

    ProceedingsoftheFourth International Workshop on Semantic Evaluations (SemEval-2007)

    Agirre,Eneko,LluísMárquez,andRichardWicentowski."ProceedingsoftheFourth International Workshop on Semantic Evaluations (SemEval-2007)." Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007). 2007

    work page 2007

  7. [5]

    Semeval-2018 task 1: Affect in tweets

    Mohammad, Saif, et al. "Semeval-2018 task 1: Affect in tweets." Proceedings of the 12th international workshop on semantic evaluation. 2018

    work page 2018

  8. [6]

    GoEmotions: A dataset of fine-grained emotions

    Demszky, Dorottya, et al. "GoEmotions: A dataset of fine-grained emotions." arXiv preprint arXiv:2005.00547 (2020)

    arXiv 2020

Show all 18 references
  1. [7]

    Sazzad Reza Basunia. 2022. E-commerce in rural bangladesh: The missing dots. The Business Standard

    2022

  2. [8]

    Emonoba: A dataset for analyzing fine-grained emotions on noisy bangla texts

    Islam, Khondoker Ittehadul, et al. "Emonoba: A dataset for analyzing fine-grained emotions on noisy bangla texts." Proceedings of the 2nd Conference of the Asia- Pacific Chapter of the Association for Computational Linguistics and the 12th In- ternational Joint Conference on N...

    2022

  3. [9]

    Comparison of classical ma- chine learning approaches on bangla textual emotion analysis

    Rahman, Md Ataur, and Md Hanif Seddiqui. "Comparison of classical ma- chine learning approaches on bangla textual emotion analysis." arXiv preprint arXiv:1907.07826 (2019)

    2019 arXiv

  4. [10]

    Long short-term memory

    Hochreiter, Sepp, and Jürgen Schmidhuber. "Long short-term memory." Neural computation 9.8 (1997): 1735-1780

    1997

  5. [11]

    Neural machine trans- lation by jointly learning to align and translate

    Bahdanau, Dzmitry, Kyunghyun Cho, and Yoshua Bengio. "Neural machine trans- lation by jointly learning to align and translate." arXiv preprint arXiv:1409.0473 (2014). Title Suppressed Due to Excessive Length 15

    2014 arXiv

  6. [12]

    A study of fasttext word embedding effects in document classification in bangla language

    Mojumder, Pritom, et al. "A study of fasttext word embedding effects in document classification in bangla language." Cyber Security and Computer Science: Second EAI International Conference, ICONCS 2020, Dhaka, Bangladesh, February 15-16, 2020, Proceedings 2. Springer Internat...

    2020

  7. [13]

    A survey of opinion mining and sentiment analysis

    Liu, Bing, and Lei Zhang. "A survey of opinion mining and sentiment analysis." Mining text data. Springer, Boston, MA, 2012. 415-463

    2012

  8. [14]

    A comparative study Shah, Kanish, et al

    Sohrawardi, Saniat Javid, Iftekhar Azam, and Shazzad Hosain. "A comparative study Shah, Kanish, et al. "A comparative analysis of logistic regression, random forest and KNN models for the text classification." Augmented Human Research 5.1 (2020): 12

    2020

  9. [15]

    News articles classification using random forests and weighted multimodal features

    Liparas, Dimitris, et al. "News articles classification using random forests and weighted multimodal features." Multidisciplinary Information Retrieval: 7th Infor- mationRetrievalFacilityConference,IRFC2014,Copenhagen,Denmark,November 10-12, 2014, Proceedings 7. Springer Inter...

    2014

  10. [16]

    Detecting ambiguities in requirements documents using inspections

    Kamsties, Erik, et al. "Detecting ambiguities in requirements documents using inspections." Proceedings of the first workshop on inspection in software engineering (WISE’01). Vol. 13. 2001

    2001

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.