Pith. sign in

REVIEW 4 major objections 6 minor 27 references

Attention based Bidirectional GRU hybrid model for inappropriate content detection in Urdu language

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Pith's one-line read An attention-based bidirectional GRU detects inappropriate Urdu text with 84% accuracy, beating four deep-learning baselines without pre-trained word embeddings.

desk verdict A sincere applied DL benchmark for Urdu content moderation whose main accuracy claim holds, but whose headline claim that attention improves efficiency is untested by the experiment design. read the letter →

arxiv 2501.09722 v1 pith:CUZLZK2C submitted 2025-01-16 cs.CL cs.LG

classification cs.CLcs.LG
keywords UrduinappropriatecontentdetectionhatespeechBidirectionalGRUattentionmechanismdeeplearningword2vecsocialmediatext
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper aims to establish that an attention-based bidirectional GRU (BiGRU-A) can detect inappropriate Urdu text more accurately than four standard deep-learning baselines, and that this holds without pre-trained word embeddings. The payoff is practical: Urdu is under-resourced, spellings vary widely, and most prior work uses classical machine learning or smaller datasets. The authors assemble a combined corpus of about 15,000 tweets, train LSTM, Bi-LSTM, GRU, TCN, and BiGRU-A on it, and report that BiGRU-A reaches 84% accuracy while the next-best model reaches 82.7%. They also report that adding pre-trained Urdu word2Vec vectors lowers accuracy for every model, which they attribute to missing profanity in the embedding vocabulary.

What carries the argument

The load-bearing mechanism is the BiGRU-A architecture: a bidirectional Gated Recurrent Unit reads the token sequence forward and backward, producing a hidden state per position, and an attention layer then assigns each hidden state a weight and aggregates them into a single context vector used for binary classification. This attention weighting is what the paper credits with handling long and variable-length sentences and capturing the words that matter for detecting inappropriate content. The experiments keep the same hyperparameters (sigmoid output, binary cross-entropy loss, Adam optimizer, dropout 0.5, learning rate 0.001) across all models, so the architecture difference is the primary variable.

What would settle it

Retrain all five models on the same corpus after manually correcting or re-annotating the machine-converted Roman Urdu portion; if the BiGRU-A advantage over LSTM shrinks or disappears, the reported 84% is an artifact of transliteration noise rather than the attention layer.

Watch

Extended reading notes

Core claim

The central discovery, as the paper states it, is that the hybrid of a bidirectional GRU encoder with an attention layer is the best of the five architectures tested for this task. On the larger UrduInAlarge dataset without word2Vec, BiGRU-A obtains 84.2% test accuracy; LSTM obtains 82.7%, Bi-LSTM 82.5%, GRU 81.0%, and TCN 80.7%. BiGRU-A also leads on the smaller UrduInAsmall dataset with 78.9% accuracy. The paper interprets the attention layer as the reason, because it weights each token's hidden state by its contribution to classification, and it interprets the word2Vec results as evidence that fixed pre-trained embeddings do not cover the slang and swear words that dominate the inappropriate class.

Load-bearing premise

In Section 6.3 the paper acknowledges that the machine conversion of misspelled Roman Urdu words produced incorrect translations; the whole comparison rests on that converted corpus being labeled accurately enough to train on.

Editorial extensions

If this is right

  • On the larger 14,946-tweet corpus every model scores higher than on the smaller 5,734-tweet corpus, so the paper's setup converts additional data into accuracy gains.
  • Adding pre-trained Urdu word2Vec embeddings lowers accuracy for all five models, so fixed generic embeddings should not be assumed helpful for profanity-heavy Urdu text.
  • BiGRU-A leads the four baselines on both datasets in most evaluation metrics, so the attention layer's contribution is not limited to one dataset size.
  • Because the winning configuration uses no feature selection or pre-trained embeddings, the approach is cheaper to deploy than embedding-dependent alternatives.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test fine-tuned or character-level embeddings; fine-tuning word2Vec on the training corpus might reverse the reported embedding penalty, since the missing swear words would then receive vectors.
  • The test sets come from the same three source corpora as training; measuring on fresh, out-of-domain social media posts would show whether 84% generalizes beyond the collected sources.
  • If transliteration errors are systematic, models may be learning conversion artifacts rather than Urdu profanity; comparing against a manually corrected version of the Roman Urdu subset would isolate this.
  • The dataset-size finding suggests the field has not yet reached the scale at which embeddings pay off for Urdu; at substantially larger corpora the model ranking could change.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes an attention-based bidirectional GRU (BiGRU-A) for detecting inappropriate content in Urdu script. It combines three public datasets, one of which is machine-converted from Roman Urdu to Urdu script, and evaluates BiGRU-A against LSTM, Bi-LSTM, GRU, and TCN on two dataset sizes, with and without pre-trained Word2Vec embeddings. The reported best result is 84.2% accuracy on the larger dataset without an embedding layer. The paper claims that the attention layer improves model efficiency and that pre-trained Word2Vec hinders performance on this task.

Significance. If the empirical claims were properly supported, the work would be a useful contribution to low-resource Urdu NLP: it assembles a relatively large corpus, systematically compares several deep learning baselines, and studies the effect of dataset size and word embeddings. The authors are transparent about some limitations, notably the conversion challenges with the Roman Urdu corpus. However, the experimental protocol has gaps that currently undermine the central claims: the train/validation/test split is unspecified, hyperparameters appear to be tuned on test performance, no variance or significance testing is reported, and the attention-efficiency claim lacks a necessary ablation. These issues prevent the results from being taken as reliable evidence for the stated conclusions.

major comments (4)
  1. [§5.2, §6.1] The paper never specifies the train/validation/test split, and the text says hyperparameters were selected 'after repeated experiments' (§5.2) and that experiments were 'ran multiple times by altering the optimization parameters in order to achieve the best results from each model' (§6.1). This indicates that test-set performance was used to select models and hyperparameters, which makes the reported accuracies optimistic and the comparisons between models not statistically valid. Please provide a clear data split, use a held-out validation set for tuning, and report final performance on a separate test set.
  2. [Abstract, §6.2, §7; Tables 5–8] The claim that 'the attention layer improves the model's efficiency' is not supported by the experimental design. There is no BiGRU-without-attention baseline in Tables 5–8; the only comparison is between BiGRU-A and a unidirectional GRU, which differ in both bidirectionality and attention. Consequently, the higher accuracy of BiGRU-A over GRU (e.g., 84.2% vs. 81.0% in Table 7) cannot be attributed to attention. An ablation removing only the attention layer from BiGRU-A is required to support this load-bearing claim.
  3. [§3.1, §6.3] The third dataset is machine-converted from Roman Urdu to Urdu script using an online tool, and the authors acknowledge in §6.3 that 'incorrect translation of misspelled words' was a challenge. Because no manual verification or post-processing of the converted text is described, the UrduInAlarge corpus may contain systematic label and text noise. Since all accuracy numbers for UrduInAlarge depend on this corpus, please provide an analysis of conversion errors (e.g., a manual sample evaluation, a post-correction step, or a robustness experiment) to assess how much the reported results are affected.
  4. [Tables 5–8] All reported metrics appear to come from a single experimental run. Differences such as the 0.842 vs. 0.827 accuracy between BiGRU-A and LSTM in Table 7 may be within stochastic variation for deep learning models. Please report mean and standard deviation over multiple random seeds and, where possible, a significance test (e.g., paired bootstrap) to support the claim that BiGRU-A outperforms the baselines.
minor comments (6)
  1. [§5.2, Table 4] The hyperparameter table is missing architecture details essential for reproducibility: number of GRU layers, number of hidden units, attention dimension, maximum sequence length, vocabulary size, and the source and dimension of the pre-trained Word2Vec embeddings are not reported.
  2. [§4, Figure 2] The attention mechanism is described only qualitatively. Please specify the attention score computation (e.g., additive, dot-product, or bilinear) and how the context vector is formed and combined with the BiGRU output for classification.
  3. [Abstract and §7] The abstract states '84% accuracy' while Table 7 reports 0.842; consider reporting the same precision throughout to avoid ambiguity.
  4. [Throughout] There are several typographical and grammatical errors (e.g., 'yeilds', 'Tunning', 'word2eclayer', 'Its can be measured') that should be corrected during revision.
  5. [§3.1] The combined dataset is described as a contribution, but no URL or availability statement is given. If the corpus is to be a contribution, please make it publicly available or explain the restrictions.
  6. [Eq. (1)] Equation (1) is presented without context or definition of all symbols; it is unclear how the skip-gram probability is used in the embedding layer. Please either remove the equation or integrate it properly into the description.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the empirical claims are tested against external baselines and datasets, with no derivation chain that reduces to its inputs.

full rationale

The paper is an empirical comparison study, not a derivation from first principles. The central claim that BiGRU-A outperforms LSTM, Bi-LSTM, GRU, and TCN is supported by test-set accuracy tables (Tables 5-8) computed against external, publicly available Urdu datasets. No fitted parameter is renamed as a prediction, and no result is defined in terms of another result by construction. The word2vec performance degradation is explained post hoc by out-of-vocabulary swear words, citing external work (Khan et al., 2021), which is an interpretation rather than a circular step. The paper's conclusion that 'the attention layer improves the model's efficiency' is not supported by an ablation of BiGRU without attention, so the causal attribution is confounded with bidirectionality; however, confounding is a validity threat, not a circularity. There are no self-citations by the present authors, no imported uniqueness theorems, and no ansatz smuggled via citation. Accordingly, the circularity score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central empirical comparison rests on three unstated assumptions: the converted corpora are correctly labeled, the train/test evaluation is unbiased, and the hyperparameters were not tuned to the test set. These are domain assumptions, not formal axioms, and all are questionable given the paper's own text.

free parameters (3)
  • Model hyperparameters (dropout, learning rate, optimizer, activation) = dropout 0.5, learning rate 0.001, adam, sigmoid
    Table 4 lists values chosen by hand after 'repeated experiments'; no validation-based selection or sensitivity analysis is reported.
  • Architecture dimensions (number of GRU layers, hidden units, attention size) = not reported
    The paper never states the number of layers, hidden units, or attention configuration, making these effectively free and unspecified.
  • Pre-trained word2vec embedding source and dimension = not specified
    Section 4.3 mentions using Urdu word2vec skip-gram embeddings from Kumhar et al. (2021) but does not give the vector dimension or training corpus details.
assumptions (3)
  • domain assumption The combined dataset constructed from three public sources, including machine transliteration from Roman Urdu to Urdu script, has accurate class labels.
    Section 3.1 describes merging datasets and converting Roman Urdu via ijunoon. Section 6.3 acknowledges incorrect translation of misspelled words, which could mislabel examples.
  • domain assumption The unspecified train/test split (ratio, stratification, seed) yields independent and representative evaluation.
    The paper reports test accuracy in Tables 5-8 but never describes the split procedure, so evaluation unbiasedness is assumed.
  • domain assumption Standard deep learning assumptions: i.i.d. samples, fixed vocabulary, and hyperparameters chosen without using test set labels.
    Section 5.2 states parameters were chosen after repeated experiments to achieve best accuracy, suggesting test-set influence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Attention based Bidirectional GRU hybrid model for inappropriate content detection in Urdu language." pith.science (2026). https://pith.science/paper/CUZLZK2C

@misc{pith2026250109722,
  author       = {Pith},
  title        = {Pith review of: Attention based Bidirectional GRU hybrid model for inappropriate content detection in Urdu language},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CUZLZK2C}},
  note         = {Machine review of arXiv:2501.09722}
}
read the original abstract

With the increased use of the internet and social networks for online discussions, the spread of toxic and inappropriate content on social networking sites has also increased. Several studies have been conducted in different languages. However, there is less work done for South Asian languages for inappropriate content identification using deep learning techniques. In Urdu language, the spellings are not unique, and people write different common spellings for the same word, while mixing it other languages, like English in the text makes it more challenging, and limited research work is available to process such language with the finest algorithms. The use of attention layer with a deep learning model can help handling the long-term dependencies and increase its efficiency . To explore the effects of the attention layer, this study proposes attention-based Bidirectional GRU hybrid model for identifying inappropriate content in Urdu Unicode text language. Four different baseline deep learning models; LSTM, Bi-LSTM, GRU, and TCN, are used to compare the performance of the proposed model. The results of these models were compared based on evaluation metrics, dataset size, and impact of the word embedding layer. The pre-trained Urdu word2Vec embeddings were utilized for our case. Our proposed model BiGRU-A outperformed all other baseline models by yielding 84\% accuracy without using pre-trained word2Vec layer. From our experiments, we have established that the attention layer improves the model's efficiency, and pre-trained word2Vec embedding does not work well with an inappropriate content dataset.

Figures

Figures reproduced from arXiv: 2501.09722 by the authors.

Figure 1
Figure 1. Sample of Dataset [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Attention-based Bidirectional Gated Recurrent Unit BiGRU - A Proposed model 4. Methodology In this section we briefly discuss the methodology of our proposed model for identification of inappropriate content detection. DL has yet to be fully investigated for detection of inappropriate content in Urdu script. By using a hybrid DL strategy, the use of our suggested architecture and comparison with basic DL models tend… view at source ↗
Figure 3
Figure 3. Evaluation Metrics Comparison of both data sets with or without Word2Vec [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Accuracy Comparison of both data sets with Word2Vec however, there is a limit to what can be detected by automatic systems. Another primary drawback is the potential for each model’s training to proceed slowly. Each model must be trained once for each potential set of …
Figure 5
Figure 5. Figure 5: Accuracy Comparison of both data sets without Word2Vec 7. Conclusion and Future work Many studies have explored the field of automatic inappropriate content detection in European or English language. But very few studies have considered investigating the inappropriate …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 24 canonical work pages

  1. [1]

    , author Jiangbin, Z

    author Akhter, M.P. , author Jiangbin, Z. , author Naqvi, I.R. , author Abdelmajeed, M. , author Sadiq, M.T. , year 2020 . title Automatic detection of offensive language for urdu and roman urdu . journal IEEE Access volume 8 , pages 91213--91226

  2. [2]

    , author Murray, L

    author Alakrot, A. , author Murray, L. , author Nikolov, N.S. , year 2018 . title Towards accurate detection of offensive language in online communication in arabic , in: booktitle ACLING

  3. [3]

    , author Farooq, U

    author Ali, R. , author Farooq, U. , author Arshad, U. , author Shahzad, W. , author Beg, M.O. , year 2022 . title Hate speech detection on twitter using transfer learning . journal Computer Speech & Language volume 74 , pages 101365

  4. [4]

    , author Ashraf, N

    author Amjad, M. , author Ashraf, N. , author Zhila, A. , author Sidorov, G. , author Zubiaga, A. , author Gelbukh, A. , year 2021 . title Threatening language detection and target identification in urdu tweets . journal IEEE Access volume 9 , pages 128302--128313

  5. [5]

    , year 2021

    author Andrew, J.J. , year 2021 . title Judithjeyafreedaandrew@ dravidianlangtech-eacl2021: offensive language detection for dravidian code-mixed youtube comments , in: booktitle Proceedings of the First Workshop on Speech and Language Technologies for Dravidian Languages , pp. pages 169--174

  6. [6]

    , author Khan, S

    author Balakrishnan, V. , author Khan, S. , author Fernandez, T. , author Arabnia, H.R. , year 2019 . title Cyberbullying detection on twitter using big five and dark triad features . journal Personality and Individual Differences

  7. [7]

    , author Seddiqui, M.H

    author Chakraborty, P. , author Seddiqui, M.H. , year 2019 . title Threat and abusive language detection on social media in bengali language , in: booktitle 2019 1st International Conference on Advances in Science, Engineering and Robotics Technology (ICASERT) , organization IEEE . pp. pages 1--6

  8. [8]

    , author Weston, J

    author Collobert, R. , author Weston, J. , author Bottou, L. , author Karlen, M. , author Kavukcuoglu, K. , author Kuksa, P. , year 2011 . title Natural language processing (almost) from scratch . journal Journal of machine learning research volume 12 , pages 2493--2537

Show all 27 references
  1. [9]

    , author Malekzadeh, M

    author Hajibabaee, P. , author Malekzadeh, M. , author Ahmadi, M. , author Heidari, M. , author Esmaeilzadeh, A. , author Abdolazimi, R. , author James Jr, H. , year 2022 . title Offensive language detection on social media based on text classification , in: booktitle 2022 IEE...

  2. [10]

    , author Weiwei, K

    author Jinbao, T. , author Weiwei, K. , author Yidan, C. , author Qiaoxin, T. , author Chenyuan, S. , author Long, L. , year 2021 . title Text classification method based on bigru-attention and cnn hybrid model , in: booktitle 2021 4th International Conference on Artificial In...

  3. [11]

    , author Amjad, A

    author Khan, L. , author Amjad, A. , author Ashraf, N. , author Chang, H.T. , author Gelbukh, A. , year 2021 . title Urdu sentiment analysis with deep learning methods . journal IEEE Access volume 9 , pages 97803--97812

  4. [12]

    , author Kirmani, M.M

    author Kumhar, S.H. , author Kirmani, M.M. , author Sheetlani, J. , author Hassan, M. , year 2021 . title Word embedding generation for urdu language using word2vec model . journal Materials Today: Proceedings

  5. [13]

    , author Lee, H.R

    author Lee, H.S. , author Lee, H.R. , author Park, J.U. , author Han, Y.S. , year 2018 a. title An abusive text detection system based on enhanced abusive and non-abusive word lists . journal Decis. Support Syst. volume 113 , pages 22--31

  6. [14]

    , author Yoon, S

    author Lee, Y. , author Yoon, S. , author Jung, K. , year 2018 b. title Comparative studies of detecting abusive language on twitter . journal arXiv preprint arXiv:1808.10245

  7. [15]

    , author Modha, S

    author Mandl, T. , author Modha, S. , author Majumder, P. , author Patel, D. , author Dave, M. , author Mandlia, C. , author Patel, A. , year 2019 . title Overview of the hasoc track at fire 2019: Hate speech and offensive content identification in indo-european languages , in...

  8. [16]

    , author Amudha, P

    author Mathew, A. , author Amudha, P. , author Sivakumari, S. , year 2020 . title Deep learning techniques: an overview , in: booktitle International conference on advanced machine learning technologies and applications , organization Springer . pp. pages 599--608

  9. [17]

    , author Chen, K

    author Mikolov, T. , author Chen, K. , author Corrado, G. , author Dean, J. , year 2013 . title Efficient estimation of word representations in vector space . journal arXiv preprint arXiv:1301.3781

  10. [18]

    , author Ellis, D.P

    author Raffel, C. , author Ellis, D.P. , year 2015 . title Feed-forward networks with attention can solve some long-term memory problems . journal arXiv preprint arXiv:1512.08756

  11. [19]

    , author Ojha, A.K

    author Rani, P. , author Ojha, A.K. , year 2019 . title Kmi-coling at semeval-2019 task 6: exploring n-grams for offensive language detection , in: booktitle Proceedings of the 13th International Workshop on Semantic Evaluation , pp. pages 668--671

  12. [20]

    , author Shakeel, M.H

    author Rizwan, H. , author Shakeel, M.H. , author Karim, A. , year 2020 . title Hate-speech and offensive language detection in roman urdu , in: booktitle Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pp. pages 2512--2522

  13. [21]

    , author Hassan, M

    author Sajid, T. , author Hassan, M. , author Ali, M. , author Gillani, R. , year 2020 . title Roman urdu multi-class offensive text detection using hybrid features and svm , in: booktitle 2020 IEEE 23rd International Multitopic Conference (INMIC) , pp. pages 1--5

  14. [22]

    , author Derczynski, L

    author Sigurbergsson, G.I. , author Derczynski, L. , year 2020 . title Offensive language and hate speech detection for danish , in: booktitle LREC

  15. [23]

    , author Michalska, S

    author Subramani, S. , author Michalska, S. , author Wang, H. , author Du, J. , author Zhang, Y. , author Shakeel, H. , year 2019 . title Deep learning for multi-class identification from domestic violence online posts . journal IEEE Access volume 7 , pages 46210--46224

  16. [24]

    , author Bajaj, P

    author Yadav, Y. , author Bajaj, P. , author Gupta, R.K. , author Sinha, R. , year 2021 . title A comparative study of deep learning methods for hate speech and offensive language detection in textual data , in: booktitle 2021 IEEE 18th India Council International Conference (...

  17. [25]

    , author Jhanwar, A

    author Yenala, H. , author Jhanwar, A. , author Chinnakotla, M.K. , author Goyal, J. , year 2018 . title Deep learning for detecting inappropriate content in text . journal International Journal of Data Science and Analytics volume 6 , pages 273--286

  18. [26]

    , author Kruschwitz, U

    author Zimmerman, S. , author Kruschwitz, U. , author Fox, C. , year 2018 . title Improving hate speech detection with deep learning ensembles , in: booktitle Proceedings of the eleventh international conference on language resources and evaluation (LREC 2018)

  19. [27]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.