REVIEW 4 major objections 6 minor 27 references
Attention based Bidirectional GRU hybrid model for inappropriate content detection in Urdu language
T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read An attention-based bidirectional GRU detects inappropriate Urdu text with 84% accuracy, beating four deep-learning baselines without pre-trained word embeddings.
desk verdict A sincere applied DL benchmark for Urdu content moderation whose main accuracy claim holds, but whose headline claim that attention improves efficiency is untested by the experiment design. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the BiGRU-A architecture: a bidirectional Gated Recurrent Unit reads the token sequence forward and backward, producing a hidden state per position, and an attention layer then assigns each hidden state a weight and aggregates them into a single context vector used for binary classification. This attention weighting is what the paper credits with handling long and variable-length sentences and capturing the words that matter for detecting inappropriate content. The experiments keep the same hyperparameters (sigmoid output, binary cross-entropy loss, Adam optimizer, dropout 0.5, learning rate 0.001) across all models, so the architecture difference is the primary variable.
What would settle it
Retrain all five models on the same corpus after manually correcting or re-annotating the machine-converted Roman Urdu portion; if the BiGRU-A advantage over LSTM shrinks or disappears, the reported 84% is an artifact of transliteration noise rather than the attention layer.
Extended reading notes
Core claim
The central discovery, as the paper states it, is that the hybrid of a bidirectional GRU encoder with an attention layer is the best of the five architectures tested for this task. On the larger UrduInAlarge dataset without word2Vec, BiGRU-A obtains 84.2% test accuracy; LSTM obtains 82.7%, Bi-LSTM 82.5%, GRU 81.0%, and TCN 80.7%. BiGRU-A also leads on the smaller UrduInAsmall dataset with 78.9% accuracy. The paper interprets the attention layer as the reason, because it weights each token's hidden state by its contribution to classification, and it interprets the word2Vec results as evidence that fixed pre-trained embeddings do not cover the slang and swear words that dominate the inappropriate class.
Load-bearing premise
In Section 6.3 the paper acknowledges that the machine conversion of misspelled Roman Urdu words produced incorrect translations; the whole comparison rests on that converted corpus being labeled accurately enough to train on.
Editorial extensions
If this is right
- On the larger 14,946-tweet corpus every model scores higher than on the smaller 5,734-tweet corpus, so the paper's setup converts additional data into accuracy gains.
- Adding pre-trained Urdu word2Vec embeddings lowers accuracy for all five models, so fixed generic embeddings should not be assumed helpful for profanity-heavy Urdu text.
- BiGRU-A leads the four baselines on both datasets in most evaluation metrics, so the attention layer's contribution is not limited to one dataset size.
- Because the winning configuration uses no feature selection or pre-trained embeddings, the approach is cheaper to deploy than embedding-dependent alternatives.
Reading between the lines
- The paper does not test fine-tuned or character-level embeddings; fine-tuning word2Vec on the training corpus might reverse the reported embedding penalty, since the missing swear words would then receive vectors.
- The test sets come from the same three source corpora as training; measuring on fresh, out-of-domain social media posts would show whether 84% generalizes beyond the collected sources.
- If transliteration errors are systematic, models may be learning conversion artifacts rather than Urdu profanity; comparing against a manually corrected version of the Roman Urdu subset would isolate this.
- The dataset-size finding suggests the field has not yet reached the scale at which embeddings pay off for Urdu; at substantially larger corpora the model ranking could change.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes an attention-based bidirectional GRU (BiGRU-A) for detecting inappropriate content in Urdu script. It combines three public datasets, one of which is machine-converted from Roman Urdu to Urdu script, and evaluates BiGRU-A against LSTM, Bi-LSTM, GRU, and TCN on two dataset sizes, with and without pre-trained Word2Vec embeddings. The reported best result is 84.2% accuracy on the larger dataset without an embedding layer. The paper claims that the attention layer improves model efficiency and that pre-trained Word2Vec hinders performance on this task.
Significance. If the empirical claims were properly supported, the work would be a useful contribution to low-resource Urdu NLP: it assembles a relatively large corpus, systematically compares several deep learning baselines, and studies the effect of dataset size and word embeddings. The authors are transparent about some limitations, notably the conversion challenges with the Roman Urdu corpus. However, the experimental protocol has gaps that currently undermine the central claims: the train/validation/test split is unspecified, hyperparameters appear to be tuned on test performance, no variance or significance testing is reported, and the attention-efficiency claim lacks a necessary ablation. These issues prevent the results from being taken as reliable evidence for the stated conclusions.
major comments (4)
- [§5.2, §6.1] The paper never specifies the train/validation/test split, and the text says hyperparameters were selected 'after repeated experiments' (§5.2) and that experiments were 'ran multiple times by altering the optimization parameters in order to achieve the best results from each model' (§6.1). This indicates that test-set performance was used to select models and hyperparameters, which makes the reported accuracies optimistic and the comparisons between models not statistically valid. Please provide a clear data split, use a held-out validation set for tuning, and report final performance on a separate test set.
- [Abstract, §6.2, §7; Tables 5–8] The claim that 'the attention layer improves the model's efficiency' is not supported by the experimental design. There is no BiGRU-without-attention baseline in Tables 5–8; the only comparison is between BiGRU-A and a unidirectional GRU, which differ in both bidirectionality and attention. Consequently, the higher accuracy of BiGRU-A over GRU (e.g., 84.2% vs. 81.0% in Table 7) cannot be attributed to attention. An ablation removing only the attention layer from BiGRU-A is required to support this load-bearing claim.
- [§3.1, §6.3] The third dataset is machine-converted from Roman Urdu to Urdu script using an online tool, and the authors acknowledge in §6.3 that 'incorrect translation of misspelled words' was a challenge. Because no manual verification or post-processing of the converted text is described, the UrduInAlarge corpus may contain systematic label and text noise. Since all accuracy numbers for UrduInAlarge depend on this corpus, please provide an analysis of conversion errors (e.g., a manual sample evaluation, a post-correction step, or a robustness experiment) to assess how much the reported results are affected.
- [Tables 5–8] All reported metrics appear to come from a single experimental run. Differences such as the 0.842 vs. 0.827 accuracy between BiGRU-A and LSTM in Table 7 may be within stochastic variation for deep learning models. Please report mean and standard deviation over multiple random seeds and, where possible, a significance test (e.g., paired bootstrap) to support the claim that BiGRU-A outperforms the baselines.
minor comments (6)
- [§5.2, Table 4] The hyperparameter table is missing architecture details essential for reproducibility: number of GRU layers, number of hidden units, attention dimension, maximum sequence length, vocabulary size, and the source and dimension of the pre-trained Word2Vec embeddings are not reported.
- [§4, Figure 2] The attention mechanism is described only qualitatively. Please specify the attention score computation (e.g., additive, dot-product, or bilinear) and how the context vector is formed and combined with the BiGRU output for classification.
- [Abstract and §7] The abstract states '84% accuracy' while Table 7 reports 0.842; consider reporting the same precision throughout to avoid ambiguity.
- [Throughout] There are several typographical and grammatical errors (e.g., 'yeilds', 'Tunning', 'word2eclayer', 'Its can be measured') that should be corrected during revision.
- [§3.1] The combined dataset is described as a contribution, but no URL or availability statement is given. If the corpus is to be a contribution, please make it publicly available or explain the restrictions.
- [Eq. (1)] Equation (1) is presented without context or definition of all symbols; it is unclear how the skip-gram probability is used in the embedding layer. Please either remove the equation or integrate it properly into the description.
Circularity Check
No significant circularity: the empirical claims are tested against external baselines and datasets, with no derivation chain that reduces to its inputs.
full rationale
The paper is an empirical comparison study, not a derivation from first principles. The central claim that BiGRU-A outperforms LSTM, Bi-LSTM, GRU, and TCN is supported by test-set accuracy tables (Tables 5-8) computed against external, publicly available Urdu datasets. No fitted parameter is renamed as a prediction, and no result is defined in terms of another result by construction. The word2vec performance degradation is explained post hoc by out-of-vocabulary swear words, citing external work (Khan et al., 2021), which is an interpretation rather than a circular step. The paper's conclusion that 'the attention layer improves the model's efficiency' is not supported by an ablation of BiGRU without attention, so the causal attribution is confounded with bidirectionality; however, confounding is a validity threat, not a circularity. There are no self-citations by the present authors, no imported uniqueness theorems, and no ansatz smuggled via citation. Accordingly, the circularity score is 0.
Assumptions & free parameters
free parameters (3)
- Model hyperparameters (dropout, learning rate, optimizer, activation) =
dropout 0.5, learning rate 0.001, adam, sigmoid
- Architecture dimensions (number of GRU layers, hidden units, attention size) =
not reported
- Pre-trained word2vec embedding source and dimension =
not specified
assumptions (3)
- domain assumption The combined dataset constructed from three public sources, including machine transliteration from Roman Urdu to Urdu script, has accurate class labels.
- domain assumption The unspecified train/test split (ratio, stratification, seed) yields independent and representative evaluation.
- domain assumption Standard deep learning assumptions: i.i.d. samples, fixed vocabulary, and hyperparameters chosen without using test set labels.
Cite this review
Pith. "Pith review of Attention based Bidirectional GRU hybrid model for inappropriate content detection in Urdu language." pith.science (2026). https://pith.science/paper/CUZLZK2C
@misc{pith2026250109722,
author = {Pith},
title = {Pith review of: Attention based Bidirectional GRU hybrid model for inappropriate content detection in Urdu language},
year = {2026},
howpublished = {\url{https://pith.science/paper/CUZLZK2C}},
note = {Machine review of arXiv:2501.09722}
}
read the original abstract
With the increased use of the internet and social networks for online discussions, the spread of toxic and inappropriate content on social networking sites has also increased. Several studies have been conducted in different languages. However, there is less work done for South Asian languages for inappropriate content identification using deep learning techniques. In Urdu language, the spellings are not unique, and people write different common spellings for the same word, while mixing it other languages, like English in the text makes it more challenging, and limited research work is available to process such language with the finest algorithms. The use of attention layer with a deep learning model can help handling the long-term dependencies and increase its efficiency . To explore the effects of the attention layer, this study proposes attention-based Bidirectional GRU hybrid model for identifying inappropriate content in Urdu Unicode text language. Four different baseline deep learning models; LSTM, Bi-LSTM, GRU, and TCN, are used to compare the performance of the proposed model. The results of these models were compared based on evaluation metrics, dataset size, and impact of the word embedding layer. The pre-trained Urdu word2Vec embeddings were utilized for our case. Our proposed model BiGRU-A outperformed all other baseline models by yielding 84\% accuracy without using pre-trained word2Vec layer. From our experiments, we have established that the attention layer improves the model's efficiency, and pre-trained word2Vec embedding does not work well with an inappropriate content dataset.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
author Akhter, M.P. , author Jiangbin, Z. , author Naqvi, I.R. , author Abdelmajeed, M. , author Sadiq, M.T. , year 2020 . title Automatic detection of offensive language for urdu and roman urdu . journal IEEE Access volume 8 , pages 91213--91226
work page 2020
-
[2]
author Alakrot, A. , author Murray, L. , author Nikolov, N.S. , year 2018 . title Towards accurate detection of offensive language in online communication in arabic , in: booktitle ACLING
work page 2018
-
[3]
author Ali, R. , author Farooq, U. , author Arshad, U. , author Shahzad, W. , author Beg, M.O. , year 2022 . title Hate speech detection on twitter using transfer learning . journal Computer Speech & Language volume 74 , pages 101365
work page 2022
-
[4]
author Amjad, M. , author Ashraf, N. , author Zhila, A. , author Sidorov, G. , author Zubiaga, A. , author Gelbukh, A. , year 2021 . title Threatening language detection and target identification in urdu tweets . journal IEEE Access volume 9 , pages 128302--128313
work page 2021
-
[5]
author Andrew, J.J. , year 2021 . title Judithjeyafreedaandrew@ dravidianlangtech-eacl2021: offensive language detection for dravidian code-mixed youtube comments , in: booktitle Proceedings of the First Workshop on Speech and Language Technologies for Dravidian Languages , pp. pages 169--174
work page 2021
-
[6]
author Balakrishnan, V. , author Khan, S. , author Fernandez, T. , author Arabnia, H.R. , year 2019 . title Cyberbullying detection on twitter using big five and dark triad features . journal Personality and Individual Differences
work page 2019
-
[7]
author Chakraborty, P. , author Seddiqui, M.H. , year 2019 . title Threat and abusive language detection on social media in bengali language , in: booktitle 2019 1st International Conference on Advances in Science, Engineering and Robotics Technology (ICASERT) , organization IEEE . pp. pages 1--6
work page 2019
-
[8]
author Collobert, R. , author Weston, J. , author Bottou, L. , author Karlen, M. , author Kavukcuoglu, K. , author Kuksa, P. , year 2011 . title Natural language processing (almost) from scratch . journal Journal of machine learning research volume 12 , pages 2493--2537
work page 2011
Show all 27 references
-
[9]
, author Malekzadeh, M
author Hajibabaee, P. , author Malekzadeh, M. , author Ahmadi, M. , author Heidari, M. , author Esmaeilzadeh, A. , author Abdolazimi, R. , author James Jr, H. , year 2022 . title Offensive language detection on social media based on text classification , in: booktitle 2022 IEE...
2022
-
[10]
, author Weiwei, K
author Jinbao, T. , author Weiwei, K. , author Yidan, C. , author Qiaoxin, T. , author Chenyuan, S. , author Long, L. , year 2021 . title Text classification method based on bigru-attention and cnn hybrid model , in: booktitle 2021 4th International Conference on Artificial In...
2021
-
[11]
, author Amjad, A
author Khan, L. , author Amjad, A. , author Ashraf, N. , author Chang, H.T. , author Gelbukh, A. , year 2021 . title Urdu sentiment analysis with deep learning methods . journal IEEE Access volume 9 , pages 97803--97812
2021
-
[12]
, author Kirmani, M.M
author Kumhar, S.H. , author Kirmani, M.M. , author Sheetlani, J. , author Hassan, M. , year 2021 . title Word embedding generation for urdu language using word2vec model . journal Materials Today: Proceedings
2021
-
[13]
, author Lee, H.R
author Lee, H.S. , author Lee, H.R. , author Park, J.U. , author Han, Y.S. , year 2018 a. title An abusive text detection system based on enhanced abusive and non-abusive word lists . journal Decis. Support Syst. volume 113 , pages 22--31
2018
-
[14]
, author Yoon, S
author Lee, Y. , author Yoon, S. , author Jung, K. , year 2018 b. title Comparative studies of detecting abusive language on twitter . journal arXiv preprint arXiv:1808.10245
2018 arXiv
-
[15]
, author Modha, S
author Mandl, T. , author Modha, S. , author Majumder, P. , author Patel, D. , author Dave, M. , author Mandlia, C. , author Patel, A. , year 2019 . title Overview of the hasoc track at fire 2019: Hate speech and offensive content identification in indo-european languages , in...
2019
-
[16]
, author Amudha, P
author Mathew, A. , author Amudha, P. , author Sivakumari, S. , year 2020 . title Deep learning techniques: an overview , in: booktitle International conference on advanced machine learning technologies and applications , organization Springer . pp. pages 599--608
2020
-
[17]
, author Chen, K
author Mikolov, T. , author Chen, K. , author Corrado, G. , author Dean, J. , year 2013 . title Efficient estimation of word representations in vector space . journal arXiv preprint arXiv:1301.3781
2013 arXiv
-
[18]
, author Ellis, D.P
author Raffel, C. , author Ellis, D.P. , year 2015 . title Feed-forward networks with attention can solve some long-term memory problems . journal arXiv preprint arXiv:1512.08756
2015 arXiv
-
[19]
, author Ojha, A.K
author Rani, P. , author Ojha, A.K. , year 2019 . title Kmi-coling at semeval-2019 task 6: exploring n-grams for offensive language detection , in: booktitle Proceedings of the 13th International Workshop on Semantic Evaluation , pp. pages 668--671
2019
-
[20]
, author Shakeel, M.H
author Rizwan, H. , author Shakeel, M.H. , author Karim, A. , year 2020 . title Hate-speech and offensive language detection in roman urdu , in: booktitle Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pp. pages 2512--2522
2020
-
[21]
, author Hassan, M
author Sajid, T. , author Hassan, M. , author Ali, M. , author Gillani, R. , year 2020 . title Roman urdu multi-class offensive text detection using hybrid features and svm , in: booktitle 2020 IEEE 23rd International Multitopic Conference (INMIC) , pp. pages 1--5
2020
-
[22]
, author Derczynski, L
author Sigurbergsson, G.I. , author Derczynski, L. , year 2020 . title Offensive language and hate speech detection for danish , in: booktitle LREC
2020
-
[23]
, author Michalska, S
author Subramani, S. , author Michalska, S. , author Wang, H. , author Du, J. , author Zhang, Y. , author Shakeel, H. , year 2019 . title Deep learning for multi-class identification from domestic violence online posts . journal IEEE Access volume 7 , pages 46210--46224
2019
-
[24]
, author Bajaj, P
author Yadav, Y. , author Bajaj, P. , author Gupta, R.K. , author Sinha, R. , year 2021 . title A comparative study of deep learning methods for hate speech and offensive language detection in textual data , in: booktitle 2021 IEEE 18th India Council International Conference (...
2021
-
[25]
, author Jhanwar, A
author Yenala, H. , author Jhanwar, A. , author Chinnakotla, M.K. , author Goyal, J. , year 2018 . title Deep learning for detecting inappropriate content in text . journal International Journal of Data Science and Analytics volume 6 , pages 273--286
2018
-
[26]
, author Kruschwitz, U
author Zimmerman, S. , author Kruschwitz, U. , author Fox, C. , year 2018 . title Improving hate speech detection with deep learning ensembles , in: booktitle Proceedings of the eleventh international conference on language resources and evaluation (LREC 2018)
2018
-
[27]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.