REVIEW 4 major objections 4 minor 32 references
Cyberbullying Detection in Hinglish Text Using MURIL and Explainable AI
T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read This paper argues that fine-tuning MURIL, a multilingual transformer pretrained on 17 Indian languages and code-mixed data, gives a Hinglish cyberbullying detector that beats RoBERTa and IndicBERT by 1.36 to 13.07 percentage points on…
desk verdict A useful but unverifiable empirical comparison: MURIL beats RoBERTa and IndicBERT on six Hinglish datasets, but the missing protocol details and an internal freezing contradiction mean the headline margins are not yet credible. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing machinery is MURIL (Multilingual Representations for Indian Languages), a BERT-style transformer pretrained on 17 Indian languages and on code-mixed and transliterated data, making its tokenizer and representations suited to romanized Hindi-English text. The paper couples it with a selective freezing schedule (embedding plus transformer layer 1 frozen, layers 2-12 fine-tuned) and a classification head consisting of dropout at p=0.16 followed by three linear hidden layers with LayerNorm, ReLU, and dropout, then a two-class softmax over the [CLS] vector. This combination is what the paper credits for the accuracy gains; the same backbone run with all layers frozen, fewer head layers, or basic preprocessing scores lower in the ablation.
What would settle it
Run a matched replication on the same six datasets, giving MURIL, RoBERTa, and IndicBERT equal hyperparameter search budgets, seeds, and cross-validation folds; if RoBERTa or IndicBERT matches or beats MURIL on most datasets, the central superiority claim would be falsified.
Extended reading notes
Core claim
The central discovery is that task-specific fine-tuning of MURIL, with the embedding layer and first transformer layer frozen while layers 2-12 are adapted, yields a Hinglish cyberbullying classifier that outperforms RoBERTa- and IndicBERT-based systems on every one of six datasets tested. The reported accuracies are 86.97% on Bohra et al., 84.62% on BullyExplain, 86.03% on BullySentemo, 75.41% on Kumar et al., 83.92% on HASOC 2021, and 94.63% on Mendeley Indo-HateSpeech, with corresponding gains of 1.36 to 13.07 percentage points over the multilingual baselines. The paper further claims that the model's gradient-based word attributions align with culturally salient cues such as Hindi insults, intensifiers, and caste-related terms, and that ablation experiments identify the best configuration: freeze the lower layers, use three hidden layers of sizes 512, 256, and 128 in the head, and apply the full preprocessing pipeline including language identification, transliteration normalization, and emoji standardization.
Load-bearing premise
The reported MURIL advantage assumes the RoBERTa and IndicBERT baselines were tuned and evaluated with comparable effort and identical data splits, because the paper does not report their learning rates, batch sizes, epochs, seeds, or fold assignments.
Editorial extensions
If this is right
- Fine-tuned MURIL can serve as a stronger baseline than RoBERTa or IndicBERT for code-mixed Hindi-English abuse detection, with gains of 1.36 to 13.07 percentage points.
- The reported configuration—freezing the embedding and first transformer layer, using three hidden layers in the head, and applying the full preprocessing stack—should be the default starting point for similar Hinglish classification tasks.
- The explainability layer identifies cross-linguistic patterns and culturally specific cues such as 'reservation' and 'backward' that drive predictions, which can help moderators understand and audit decisions.
- Failure analysis shows that context-dependent insults, cultural subtext, and cross-linguistic sarcasm remain unsolved, indicating where future data collection and modeling should focus.
Reading between the lines
- I would expect the same MURIL recipe to transfer to other code-mixed Indic languages covered by MURIL's pretraining, such as Bengali-English or Tamil-English, though the paper does not test this.
- The reported gains may be partly an artifact of tuning effort; a matched-hyperparameter replication on the same six datasets would test whether the advantage is intrinsic to MURIL's code-mixed pretraining.
- The word-attribution method could be turned into a practical moderation aid by flagging high-attribution cultural terms for human review, an application the paper motivates but does not build.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a MURIL-based classifier for detecting cyberbullying in code-mixed Hindi-English (Hinglish) text, with a classification head composed of three hidden layers, selective layer freezing, and a preprocessing pipeline tuned for code-mixed content. The authors evaluate the model on six Hinglish/hate-speech datasets (Bohra et al., BullyExplain, BullySentemo, Kumar et al., HASOC 2021, and Mendeley Indo-HateSpeech) and report that it outperforms RoBERTa- and IndicBERT-based baselines by 1.36 to 13.07 percentage points in accuracy, with additional ablation, attribution, confidence-calibration, and failure analyses on the BullyExplain dataset. The central claim is empirical: a fine-tuned MURIL pipeline is a stronger model than RoBERTa or IndicBERT for this task.
Significance. If the reported results are reproducible, the paper would provide a useful practical result: a code-mixed-language-pretrained model with a modest classification head and selective freezing beats general multilingual baselines across six diverse datasets, and the ablation study gives concrete guidance on layer freezing, head depth, and preprocessing. The paper also includes genuine qualitative analyses (attribution, confidence distributions, failure cases) that go beyond a bare accuracy table. However, the significance is currently undercut by the absence of any verifiable experimental protocol: no hyperparameter values, no splits, no seeds, no error bars, no significance tests, and no code. The internal inconsistency between the freezing configuration described in Section 3 and the ablation-optimal configuration in Table 14 means the reader cannot even determine which model produced the headline numbers.
major comments (4)
- [Section 4.2 and Algorithm 1] The experimental protocol is unspecified at every load-bearing point. Algorithm 1 names hyperparameters theta (learning rate, batch size, epochs, weight decay, dropout, patience, and number of folds K), but Section 4.2 never instantiates any of them, and no per-dataset train/validation/test split sizes, random seeds, or numbers of repeated runs are reported. Tables 1-6 therefore present single point estimates from an unreported protocol, and the claimed margins (as small as 1.36 percentage points on BullySentemo) could be explained by split or seed variation. This is a load-bearing gap for the paper's central comparative claim.
- [Section 3 vs. Section 6.3, Table 14] There is an internal contradiction about the freezing configuration. Section 3 states that 'the Embedding Layer and Transformer Layer 1 remain frozen' and that 'Transformer Layers 2-12 undergo fine-tuning,' whereas Table 14 and the Section 6.3 discussion identify 'Embedding + Layers 1-2 Frozen' (row 3) as the optimal configuration. The manuscript never states which configuration produced Tables 1-6. If the main results used the ablation-selected configuration, the model description in Section 3 is incorrect; if they used the Section 3 configuration, then the main results are not aligned with the ablation-optimal setting. This must be resolved before the reported numbers can be interpreted.
- [Tables 1-6] No measure of variance or statistical significance accompanies any result. There are no standard deviations, confidence intervals, or significance tests for the accuracy/F1 differences between MURIL and the baselines. Because several reported improvements are small (e.g., 1.44 points over IndicBERT on BullyExplain in Table 2, 1.36 points on BullySentemo in Table 3), the claim that MURIL 'outperforms' the baselines requires either error bars over multiple runs/seeds or a significance test. Without this, the headline margins are not established.
- [Section 4.2 / Reproducibility] The paper provides no code, no data splits, and no model configuration files, despite claiming in Section 4.2 that the design 'prioritizes reproducibility through standardized testing environments and consistent evaluation protocols.' The only concrete details are hardware (NVIDIA L40S), software versions, and max length 128. Independent replication of Tables 1-6 is therefore impossible, which is a general but decisive weakness for an empirical comparison paper.
minor comments (4)
- [Section 7] The conclusion states gains of '2.52 to 13.07 percentage points,' which contradicts the abstract's '1.36 to 13.07 percentage points'; also, 'five-data-set overall comprehensive evaluation' is inconsistent with the six datasets used throughout.
- [Section 6.4.1] The phrase 'demonstrating the complicity of confidence estimation' appears to be a typo for 'complexity,' and the prose elsewhere in this section ('the summit confidence accuracy') is awkward and should be rewritten for clarity.
- [References] Reference [22] is a duplicate of [3], and reference [23] refers to the same Mendeley dataset as [19] but with incomplete metadata; these should be consolidated or disambiguated.
- [Section 2] The claim that MURIL is 'the only model pre-trained both on transliterated and code-mixed data' is stated without a citation or comparison to other code-mixed pretrained models; a supporting reference or a softened phrasing would be appropriate.
Circularity Check
No circularity: the MURIL-vs-baseline comparison is an external benchmark evaluation, not a derivation that reduces to its own inputs.
full rationale
The paper's central claim is an empirical benchmark comparison, not a derivation: fine-tuned MURIL accuracy/F1 values in Tables 1-6 are measured on six externally published datasets and compared with external published baselines plus three independently implemented neural baselines. No equation is derived from a definition of the result, no fitted parameter is renamed as a prediction, and no load-bearing premise is justified by a self-citation (the single author has no prior work in the bibliography; MURIL pretraining is cited to Khanuja et al., an external source). The reported advantage over RoBERTa and IndicBERT therefore does not reduce by construction to anything the paper itself fitted or defined. The important caveats in this manuscript are experimental rather than circular: hyperparameters, train/validation/test splits, seeds, and baseline tuning budgets are not reported, and there is an unaddressed inconsistency between Section 3, which describes freezing the Embedding Layer and Transformer Layer 1, and Section 6.3/Table 14, whose ablation selects Embedding + Layers 1-2 frozen as optimal on BullyExplain. If the ablation-selected BullyExplain configuration was then used to produce the BullyExplain headline in Table 2, that number would not be a clean held-out evaluation; but the paper never states which configuration produced Tables 1-6, so this is a verifiability and selection-protocol risk, not a circular reduction that can be exhibited quote-by-quote. No circular step meeting the required reduction-by-construction evidence bar is present.
Assumptions & free parameters
free parameters (6)
- Number of frozen transformer layers =
Embedding + layers 1-2 frozen
- Classification head hidden layer sizes =
[512, 256, 128]
- Dropout probability =
0.16
- Maximum sequence length =
128
- Learning rate and weight decay
- Early stopping patience and number of folds
assumptions (4)
- domain assumption Annotated labels in all six datasets are reliable ground truth for cyberbullying.
- domain assumption Fine-tuning each transformer with the same generic protocol yields a fair comparison.
- domain assumption Gradient-based attribution scores are meaningful explanations of model decisions.
- domain assumption K-fold cross-validation splits avoid data leakage and are consistent across models.
Cite this review
Pith. "Pith review of Cyberbullying Detection in Hinglish Text Using MURIL and Explainable AI." pith.science (2026). https://pith.science/paper/BNCTZSDN
@misc{pith2026250616066,
author = {Pith},
title = {Pith review of: Cyberbullying Detection in Hinglish Text Using MURIL and Explainable AI},
year = {2026},
howpublished = {\url{https://pith.science/paper/BNCTZSDN}},
note = {Machine review of arXiv:2506.16066}
}
read the original abstract
The growth of digital communication platforms has led to increased cyberbullying incidents worldwide, creating a need for automated detection systems to protect users. The rise of code-mixed Hindi-English (Hinglish) communication on digital platforms poses challenges for existing cyberbullying detection systems, which were designed primarily for monolingual text. This paper presents a framework for cyberbullying detection in Hinglish text using the Multilingual Representations for Indian Languages (MURIL) architecture to address limitations in current approaches. Evaluation across six benchmark datasets -- Bohra \textit{et al.}, BullyExplain, BullySentemo, Kumar \textit{et al.}, HASOC 2021, and Mendeley Indo-HateSpeech -- shows that the MURIL-based approach outperforms existing multilingual models including RoBERTa and IndicBERT, with improvements of 1.36 to 13.07 percentage points and accuracies of 86.97\% on Bohra, 84.62\% on BullyExplain, 86.03\% on BullySentemo, 75.41\% on Kumar datasets, 83.92\% on HASOC 2021, and 94.63\% on Mendeley dataset. The framework includes explainability features through attribution analysis and cross-linguistic pattern recognition. Ablation studies show that selective layer freezing, appropriate classification head design, and specialized preprocessing for code-mixed content improve detection performance, while failure analysis identifies challenges including context-dependent interpretation, cultural understanding, and cross-linguistic sarcasm detection, providing directions for future research in multilingual cyberbullying detection.
Figures
Reference graph
Works this paper leans on
-
[1]
Badjatiya, S
P. Badjatiya, S. Gupta, M. Gupta, and V. Varma, ”Deep learning for hate speech detection in tweets,” in Proceedings of the 26th International Conference on World Wide Web Companion, 2017, pp. 759–760
2017
-
[2]
K. Bali, J. Sharma, M. Choudhury, and Y. Vyas, ”“i am borrowing ya mixing?” an analysis of english-hindi code mixing in facebook,” inProceedings of the First Workshop on Computational Approaches to Code Switching, 2014, pp. 116–126
work page 2014
-
[4]
T. Davidson, D. Warmsley, M. Macy, and I. Weber, ”Automated hate speech detection and the problem of offensive language,” inProceedings of the 11th International AAAI Conference on Web and Social Media, 2017, pp. 512–515
work page 2017
- [5]
- [6]
-
[7]
A. Joshi, A. Prabhu, M. Shrivastava, and V. Varma, ”Towards sub-word level compositions for sentiment analysis of hindi-english code mixed text,” inProceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, 2016, pp. 2482–2491
work page 2016
-
[8]
Myers-Scotton,Dueling languages: Grammatical structure in code-switching
C. Myers-Scotton,Dueling languages: Grammatical structure in code-switching. Oxford University Press, 1993
work page 1993
-
[9]
M. H. Obaid, S. K. Guirguis, and S. M. Elkaffas, ”Cyberbullying detection and severity determination model,” IEEE Access, vol. 11, pp. 97391–97399, 2023
work page 2023
Show all 32 references
-
[10]
Schmidt and M
A. Schmidt and M. Wiegand, ”A survey on hate speech detection using natural language processing,” in Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media, 2017, pp. 1–10
2017
-
[11]
Sharma, S
A. Sharma, S. Gupta, R. Motlani, P. Bansal, M. Srivastava, R. Mamidi, and D. M. Sharma, ”Shallow parsing pipeline for hindi-english code-mixed social media text,” inProceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistic...
2016
-
[12]
Y. Vyas, S. Gella, J. Sharma, K. Bali, and M. Choudhury, ”Pos tagging of english-hindi code-mixed social media content,” inProceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014, pp. 974–979
2014
-
[13]
Kumar, A
R. Kumar, A. K. Ojha, S. Malmasi, and M. Zampieri, ”Benchmarking aggression identification in social media,” inProceedings of the first workshop on trolling, aggression and cyberbullying (TRAC-2018), 2018, pp. 1–11
2018
-
[14]
Khanuja, D
S. Khanuja, D. Bansal, S. Mehtani, S. Khosla, A. Dey, B. Gopalan, D. K. Margam, P. Aggarwal, R. T. Nagipogu, S. Dave, S. Gupta, S. C. B. Gali, V. Subramanian, and P. Talukdar, ”MuRIL: Multilingual Representations for Indian Languages,”arXiv preprint arXiv:2103.10730, 2021. 17
2021 arXiv
-
[15]
Y. Liu, M. Ott, N. Goyal, J. Du, M. Ott, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, ”RoBERTa: A Robustly Optimized BERT Pretraining Approach,”arXiv preprint arXiv:1907.11692, 2019
1907 arXiv
-
[16]
Kakwani, A
D. Kakwani, A. Kunchukuttan, S. Golla, N. C. Gokul, A. Bhattacharyya, M. M. Kanojia, and P. Kumar, ”IndicNLPSuite: Monolingual Corpora, Evaluation Benchmarks and Pre-trained Multilingual Language Models for Indian Languages,” inFindings of the Association for Computational Lin...
2020
-
[17]
Kumar, A
R. Kumar, A. K. Ojha, S. Malmasi, and M. Zampieri, ”Aggression-annotated Corpus of Hindi-English Code-mixed Data,” inProceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), 2018
2018
-
[18]
Mandl, S
T. Mandl, S. Modha, G. K. Shahi, H. Madhu, S. Satapara, P. Majumder,et al., ”Overview of the HASOC sub- track at FIRE 2021: Hate speech and offensive content identification in English and Indo-Aryan languages,” arXiv preprint arXiv:2112.09301, 2021
2021 arXiv
-
[19]
Kaware, ”Indo-HateSpeech,” Mendeley Data, V1, 2024
P. Kaware, ”Indo-HateSpeech,” Mendeley Data, V1, 2024. [Online]. Available: https://doi.org/10.17632/snc7mxpj6t.1
2024 doi
-
[20]
Fortuna and S
P. Fortuna and S. Nunes, ”A survey on automatic detection of hate speech in text,”ACM Computing Surveys, vol. 51, no. 4, pp. 1–30, 2018
2018
-
[21]
Maurya, T
C. Maurya, T. Muhammad, P. Dhillon, S. Chauhan, S. Sakshi, S. Srivastava, R. Singh, T. Muhammad, and R. Uniyal, ”The effects of cyberbullying victimization on depression and suicidal ideation among adolescents and young adults: a three year cohort study from India,”BMC Psychia...
2022
-
[22]
Bohra, D
A. Bohra, D. Vijay, V. Singh, S. S. Akhtar, and M. Shrivastava, ”A dataset of hindi-english code-mixed social media text for hate speech detection,” inProceedings of the Second Workshop on Computational Modeling of People’s Opinions, Personality, and Emotions in Social Media, ...
2018
-
[23]
Kaware, ”Indo-hatespeech,” Mendeley Data, 2024
P. Kaware, ”Indo-hatespeech,” Mendeley Data, 2024
2024
-
[24]
Kamble and A
S. Kamble and A. Joshi, ”Hate speech detection from code-mixed hindi-english tweets using deep learning models,”arXiv preprint arXiv:1811.05145, 2018
2018 arXiv
-
[25]
Joshi, A
A. Joshi, A. Prabhu, M. Shrivastava, and V. Varma, ”Towards sub-word level compositions for sentiment analysis of hindi-english code mixed text,” inProceedings of COLING 2016, the 26th international conference on computational linguistics: technical papers, 2016, pp. 2482–2491
2016
-
[26]
Santosh and K
T. Santosh and K. Aravind, ”Hate speech detection in hindi-english code-mixed social media text,” in Proceedings of the ACM india joint international conference on data science and management of data, 2019, pp. 310–313
2019
-
[27]
Sharma, A
A. Sharma, A. Kabra, and M. Jain, ”Ceasing hate with moh: Hate speech detection in hindi–english code- switched language,”Information Processing & Management, vol. 59, no. 1, p. 102760, 2022
2022
-
[28]
Modhaet al., ”Detecting and visualizing hate speech in social media: A cyber watchdog for surveillance,” Expert Systems with Applications, vol
S. Modhaet al., ”Detecting and visualizing hate speech in social media: A cyber watchdog for surveillance,” Expert Systems with Applications, vol. 161, p. 113725, 2020
2020
-
[29]
Raiyani, T
K. Raiyani, T. Gonc ¸alves, P. Quaresma, and V. B. Nogueira, ”Fully connected neural network with advance preprocessor to identify aggression over facebook and twitter,” inProceedings of the first workshop on trolling, aggression and cyberbullying (TRAC-2018), 2018, pp. 28–41. 18
2018
-
[30]
Sinha, P
M. Sinha, P. Balamurugan, and S. Swamynathan, ”Cyber-bullying detection in english and hindi corpus with parental control,” inAIP Conference Proceedings, vol. 3075, AIP Publishing, 2024
2024
-
[31]
Kadamet al., ”Battling hateful content in indic languages hasoc’21,”arXiv preprint arXiv:2110.12780, 2021
A. Kadamet al., ”Battling hateful content in indic languages hasoc’21,”arXiv preprint arXiv:2110.12780, 2021
2021 arXiv
-
[32]
Maity, S
K. Maity, S. Saha, and P. Bhattacharyya, ”Emoji, sentiment and emotion aided cyberbullying detection in hinglish,”IEEE Transactions on Computational Social Systems, vol. 10, no. 5, pp. 2411–2420, Oct. 2023
2023
-
[33]
Maity and S
K. Maity and S. Saha, ”A multi-task model for sentiment aided cyberbullying detection in code-mixed Indian languages,” inNeural Information Processing: 28th International Conference, ICONIP 2021, Sanur, Bali, Indonesia, December 8–12, 2021, Proceedings, Part IV 28, Springer, 2...
2021
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.